AI Customer Service Hallucination: Real Examples and Fixes
A hallucination in customer service is not an abstract model problem. It is a reply that promises a refund nobody approved, quotes a policy that does not exist, or tells a customer to send a video they already sent. Below are six we caught in a production system that answers customer email and chat for an e-bike brand in eight languages, what caused each one, and the change that stopped it. In almost every case the cause was the input, not the model.
1. The model contradicted a promise buried in a long thread
A ticket had 25 back-and-forth messages. Somewhere in the middle, the team had promised a full refund once the return reached the warehouse. The model writing the next reply did not see it and wrote something that contradicted the promise. A stronger model given the same thread got it right, which told us what was missing: the model had all the material and could not find what mattered in it.
Fix: no model receives a raw thread any more. Code and a dedicated extraction step turn the history into a short list of facts, each with source and date: what the customer wants now, what we promised, what has been done, what the customer has already sent, what is missing, and what conflicts. On one order this cut the input from 37,604 tokens to 4,722 and every item in the reply came out right.
2. Rules that fought each other produced invented shop names
On a content pipeline, the prompt said "always name the place in Chinese and English", "always give a number when you mention a cost" and "never invent a business name". When the source had no shop names or prices, the first two rules forced the model to write something and the third forbade it. The model picked the first two and invented names.
Fix: rewrite the forcing rules to "write it when the material has it; otherwise leave it out". The invented names disappeared. The general lesson: when a model invents, look for a rule that demands output the material cannot supply.
3. A wrong answer about a promotion, caused by a missing date
A customer asked whether a promotion still applied. The AI answered wrongly. When we asked ourselves how we knew it was wrong, the answer was simple: we knew today's date and the model did not. Giving the model the current date fixed it without a single new rule.
Fix: after every mistake, write down the fact you used to spot it, then make sure the model receives that fact. This beats adding "be careful about promotions" to the prompt.
4. A police report number became an order number
A customer included a police report reference in their email. The order-number extractor matched the first part of it as an order number and linked the ticket to another customer's order. The AI then answered with someone else's shipping status.
Fix: tighten the pattern, then scan production emails for every string with the same shape before shipping the new pattern. The same scan found a second problem: a full-width colon used in Chinese text made one extractor drop an entire email. Extractors now accept both full-width and half-width punctuation.
5. "We have no information" when the information existed
In a review of 400 drafts that agents edited before sending, 10% of the edits replaced "we do not have this information" with an answer the knowledge base already contained. The model was not lying about the world; it had not been shown the right entry.
Fix: the system now searches the knowledge base before every draft, one short query per question in the email, and passes the hits to the model. Retrieval quality depended mostly on the query: a short, single-topic question put the right FAQ entry first.
6. An automatic send reused an old judgement
When a customer replied to an older thread, an automatic send went out with a draft written for the previous message. The reply was fluent and wrong for the new email.
Fix: anything sent automatically is generated fresh for the email it answers. Earlier drafts and earlier cases can inform the judgement; they are never sent as they are.
What these have in common
- Give the model distilled facts with sources instead of raw material.
- Remove rules that force output the material cannot support.
- Supply the fact a person used to judge, such as today's date.
- Test extractors against real production text before release.
- Retrieve before every draft, with short questions.
- Generate automatic replies fresh, every time.
The same system and its numbers are in our customer service case study. For the wider pattern of agents breaking in production, see why AI agents fail in production. If you want this applied to your own support desk, the free plan maps it to your work in two working days.