Dingbo AI
Dingbo AI · Field notes

Human-in-the-Loop AI Agents: Who Approves What

Updated 2026-11-17 · Shenzhen Dingbo Intelligent Technology Co., Ltd.

Dingbo AI builds and runs AI agents for customer service. You pay per completed task.

Want AI to take over a repetitive job? Get a free assessment

Email only. We reply within 24 hours, free of charge.

"Human in the loop" is easy to say and easy to get wrong. Put the person on every step and the AI saves nobody any time. Put the person nowhere and the first wrong refund lands on your bank statement. The useful question is narrower: which decisions does a person own, and what does the AI hand them at that moment? This is how we split it on a live customer service system that handled 36,294 customer emails and 15,445 chat conversations in the last 30 days.

Draw the line at consequences, not at difficulty

Teams often hand the "easy" questions to the AI and keep the "hard" ones for people. That is the wrong axis. A hard troubleshooting question with ten steps is safe for the AI to answer, because a wrong step costs the customer one more message. A one-line "yes, we will refund you" is dangerous, because it moves money and creates a promise the company has to keep.

So we split by what the action changes in the world:

With that split, the AI does the reading, checking and writing on every case, and the person spends their time on the one decision that carries weight.

Give the approver a decision, not a pile of text

An approval step only works if the approver can decide in seconds. If they have to reread a twenty-message thread to check the AI, the loop becomes the bottleneck and people start clicking "approve" without reading. That is worse than no loop.

Our approver sees three things: the reply the AI wrote, the solution it proposes (for example "reship the charger, no refund"), and the short list of facts it used, each with its source: what the customer asked this time, what was promised before, what has been done, the verified order and shipping status, and what is missing. The approver then does one of three things: send as is, edit and send, or change the decision. In the last 30 days, 3,463 AI drafts went out without a single edit and 19,357 were edited first. The edited ones are the most valuable signal we have, because each edit shows where the AI's picture of the business differs from the team's.

Chat and email need different loops

An email draft waits for a person. A chat reply goes out the moment it is written and cannot be taken back. So the same "AI proposes, person decides" pattern cannot be copied into chat.

Our chat AI answers facts and published policy only. When a customer asks for anything that needs a person to act, such as an invoice, a refund, compensation, a reshipment, an exchange, a cancellation or an address change, the chat AI does not agree to it and does not quote an amount. It opens a ticket, gives the customer the ticket number and the follow-up email address, and the email team handles it with the approval loop above. Agents stepped into the chat window zero times in the last 30 days.

Record only what the person finally approved

This is the mistake that costs real money, and it is invisible until someone reconciles the books. When the AI drafts "we will refund 30 euros" and the agent changes it to "we will send a replacement part", every downstream record has to follow the agent's version: the execution ledger, the task card sent to the warehouse team, and the promise to the customer.

We learned this the hard way. For a stretch of days the compensation ledger took its amounts from the AI's proposal instead of the agent's final reply. When we compared AI draft against final sent text over 14 days, 38 of 276 cases did not match: 21 final replies gave no money at all, and 13 changed the amount or the remedy. The fix was a rule that now applies everywhere: registration, execution and promises use only the version a person confirmed. When the system cannot tell what the person changed, it marks the record "to check" and hands it to a person; it never fills the gap with the AI's value.

Let the second check catch what the first one waved through

Some cases the AI resolves fully on its own: a reply plus a solution such as reshipping a small part. Those replies go out, but the solution does not execute yet. It waits in an "AI review" queue, sorted by type (partial refund or part reship, return pickup, logistics check, cancellation before delivery, other). A reviewer approves or corrects each one, and only then does it reach the execution ledger and the team's group chat. Pure replies with nothing to execute are archived directly.

This keeps the speed where it is cheap, in the reply, and the human check where it matters, in the action.

When the AI is wrong, give it the fact you used

The reflex after a bad answer is to add a rule. Ask a different question first: how did I know it was wrong? Once the AI answered a question about a promotion incorrectly. We only knew the right answer because we knew today's date, and the model had not been given today's date. Giving it the date fixed the problem without a single new rule. Most approval loops improve fastest this way: each correction a person makes points to a fact the AI was missing, and connecting that fact removes the need for the correction next time. For the opposite failure, rules piling up until they contradict each other, see why AI agents fail in production.

A checklist for your own loop

The full numbers and the flow of one email through this system are in our customer service case study. If you want the same split on your own operation, the free plan maps it to your work in two working days.

Which job eats the most hours on your team? Tell us in one line and we will sketch a plan for free

We connect to the systems and documents you already use, the AI works through your process, and your people review and approve. You see real numbers from a pilot before you pay for anything, then you pay for completed work.

Or email us directly: yzh317179958@163.com

Free assessmentReply within 24h