Dingbo AI
Dingbo AI · Field notes

CSAT for AI Customer Service: What Low Scores Meant

Updated 2026-11-19 · Shenzhen Dingbo Intelligent Technology Co., Ltd.

Dingbo AI builds and runs AI agents for customer service. You pay per completed task.

Want AI to take over a repetitive job? Get a free assessment

Email only. We reply within 24 hours, free of charge.

When AI starts sending replies, the first question from management is whether customers like it less. The usual answer compares one average score before and after. That hides what is going on. We read every rating that came in over two days, from midday on 17 November 2026, for an e-bike brand's email support: 217 ratings, 52 of them low, meaning two points or fewer out of five. Then we grouped each rating by the message the customer was actually rating.

The four groups

What the groups say

The AI's own replies had the lowest share of low scores. This is not because the AI writes better than people. It is because of what it is allowed to send: simple answers with no pending action. The customer asked, got the answer, and was done. Agents handle the hard cases, where a refund or a part is involved, and those rate lower whoever writes the reply.

The third group is the finding. Six in ten ratings were low when the survey arrived after seven days of silence. Reading them one by one, the customers were not rating the reply. They were still waiting for a part, a refund or a follow-up, and the survey reached them in the middle of that wait. The score measured an unfinished job.

Read this way, most of the 52 low ratings were about two things: a survey sent before the matter was finished, and real delays. The wording of AI replies was a small part.

Rule 1: survey when the matter is finished

Our survey has two triggers: when a ticket closes, and when a ticket is still open and the customer has not replied for seven days. One exception was added after this review. When a reply promises that a colleague will do something, such as sending an invoice separately, issuing a voucher or shipping a part, the survey waits the full seven days even if the ticket is closed, so that the promised thing can arrive first.

Two further changes follow from the numbers: do not survey a customer about a notice they never asked for, and do not survey after seven days if we still owe the customer something.

Rule 2: group by the message that was rated

A rating belongs to a specific reply. Record which reply the survey followed, who sent it, AI or agent, and how long after the reply the survey went out. Without those three fields, the average mixes easy questions with hard ones and blames or credits the wrong sender.

Rule 3: read the low ones

A low score is a customer telling you something specific. Reading these 52 found one defect no dashboard showed: a reply had gone out in Vietnamese to a customer who wrote in Italian. The message was short and in capital letters, and the language detector misread it. We compared detection on 2,704 recent emails before and after the fix: 20 changed, all of them corrections.

Other low ratings pointed at replies that said a colleague would follow up "separately", where the follow-up was slow. The reply was accurate. The wait behind it was the problem.

Rule 4: do not judge AI by complaints alone

Agents and managers remember the bad AI reply someone forwarded. Count instead. In our checks in November 2026 the low share for AI-sent replies was 10 to 13 percent, and for agent replies 17 to 20 percent. A fair comparison also looks at what each side handles, because the AI gets the simple cases by design.

Rule 5: watch what the customer does next

Most customers never answer a survey. Every customer who is unhappy with a reply does something: writes again, repeats the question, or opens a chat. For replies the AI sends on its own, the system looks at the customer's next message and judges whether the reply settled the matter. To work, that check has to see the AI's reply and the customer's messages before and after it. A monitor that returns the same verdict for every case is not seeing what it is supposed to judge.

How to run this review yourself

Who this fits

Any team that sends a satisfaction survey and is about to let AI send replies, or already has. The review needs no new tooling beyond the three fields in Rule 2.

Related: how to measure AI customer service quality and after-hours customer service with AI. The free plan includes how we would measure quality on your tickets, within two working days.

Which job eats the most hours on your team? Tell us in one line and we will sketch a plan for free

We connect to the systems and documents you already use, the AI works through your process, and your people review and approve. You see real numbers from a pilot before you pay for anything, then you pay for completed work.

Or email us directly: yzh317179958@163.com

Free assessmentReply within 24h