Customers trust AI hallucinations over workers’ safety advice
Customer service workers across multiple industries are encountering customers who trust AI chatbot hallucinations over expert warnings. Server Madison in New York has experienced diners dismissing allergy alerts because ChatGPT contradicts her, whilst others request wines that don't exist. This trend reflects a concerning pattern in which people increasingly defer to AI tools, even when trained professionals possess critical safety information.
The problem spans restaurants, national parks, retail shops and schools. Park rangers report visitors arriving with geographically impossible itineraries, baristas face requests for nonexistent products, and teachers note parents using AI for childcare advice that proves inadequate. Whilst some staff diplomatically reframe hallucinations by describing nonexistent wines as items the restaurant simply doesn't stock, many customers remain sceptical of human expertise even when corrected by rangers, printed maps and guidebooks. Meta's upcoming Muse AI agent is expected to exacerbate the issue further.
- Customers increasingly trust AI hallucinations over service workers' expert warnings
- False product recommendations and hallucinated itineraries affecting restaurants, retail and parks
- Safety concerns as diners ignore allergy information because AI contradicts staff
New here? Start with this
AI chatbots sometimes generate false information—a phenomenon known as "hallucination"—that sounds plausible but isn't true. When customers encounter this false information, they increasingly trust it over the advice of trained professionals who are actually helping them. This mismatch between artificial intelligence and real-world expertise is creating problems across multiple industries.
The issue appears in restaurants, where servers' allergy warnings are dismissed because ChatGPT suggested a different answer, and customers request wines that don't exist. It happens in national parks, where visitors arrive with impossible travel plans based on AI suggestions, and in shops and schools, where staff encounter customers preferring AI-generated advice to human knowledge. Even when corrected by rangers with maps or by experienced professionals, many customers remain sceptical of the human expert.
This matters because some of these decisions involve genuine safety risks. Allergy guidance, hiking routes and childcare advice all carry real consequences if they're wrong. The trend suggests people are increasingly deferring to AI tools even when the person in front of them has professional knowledge and a responsibility for their wellbeing.
Read the full article at the source →
Originally published by The Verge as “AI hallucinations are making entitled customers even worse”.