Dingbo AI
Dingbo AI · Field notes

AI Agent Pilot: Running a Proof of Concept on Real Work

Updated 2026-11-19 · Shenzhen Dingbo Intelligent Technology Co., Ltd.

Dingbo AI builds and runs AI agents for customer service. You pay per completed task.

Want AI to take over a repetitive job? Get a free assessment

Email only. We reply within 24 hours, free of charge.

An AI agent demo proves that the agent can answer questions someone chose for it. A pilot has to prove something harder: that the agent can do one of your jobs, on your data, at a quality your team accepts. Every client of ours starts with a 2 to 4 week pilot on real work before paying anything. This is how we run it.

Pick one job, not a platform

A pilot that tries to cover "customer service" or "operations" ends with a slide deck. Pick one job with a clear start and finish: replying to order status emails, answering pre-sales chat, generating certificates from a spreadsheet. The job should happen every day, follow rules that can be written down, and leave a record in a system. A list of jobs that meet these conditions is in AI agent use cases running in production.

Record the baseline first

Before the agent touches anything, write down four numbers for the job: cases handled per day, average handling time per case, the size of the backlog, and time to first reply. Without them, the pilot ends in opinions. With them, the same four numbers measured the same way after the pilot give the answer.

Connect read-only

In a pilot the agent reads your systems and writes nothing into them. It looks up orders, shipments, tickets and documents. It does not edit an order, issue a refund or send anything to a customer without a person. This keeps the pilot safe to run on live work, and it is also how the finished system treats lookups: reading is the agent's job, changing things is a person's decision.

Each lookup must return one of four results: found, confirmed that none exists, found part of it, or could not check. An agent that cannot tell "could not check" from "does not exist" gives confident wrong answers, and a pilot is the place to find that out.

Replay real history before going live

Take a few hundred real cases from the last months, including the long threads and the angry ones, and run them through the agent. Compare each output with what your team actually sent. Test questions written for the pilot hide the real problems. History shows them.

One example from our own replays: a ticket with 25 back-and-forth messages, fed to the model as raw text, produced a reply that contradicted a refund the company had already promised. Distilling the thread into structured facts before the model saw it fixed the contradiction and cut the input from 37,604 tokens to 4,722. No invented test case would have found that.

Then run live, with a person reviewing everything

After the replay, the agent works on incoming cases and your team reviews every output before it goes anywhere. Nothing is sent automatically during the pilot. Reviewers do one of three things with each output: send it as it is, edit it and send it, or discard it. Those three actions are the pilot's raw data, so the review screen must record them.

The three numbers that decide

Fix the largest group, replay the same history, and confirm that the group shrank and nothing else got worse. A pilot that goes through this loop two or three times shows a trend, and a trend is worth more than one good week.

Set the go or no-go rule on day one

Agree before the pilot starts what result means "go". A rule written on the first day, in terms of the three numbers, cannot be bent later by enthusiasm or by disappointment. Write the stop rule as well: the result at which both sides agree the job is not ready for an agent.

What a pilot should end with

Why 2 to 4 weeks

Two weeks is the shortest period that covers a replay, a live run and one round of fixes. Four weeks covers a second round and a full monthly cycle of the business, such as a promotion or a billing run. A pilot that needs longer than that is a job that was scoped too wide.

The first step is free: describe the job in one line, and within two working days you receive a written plan naming the systems to connect, what the AI handles and what stays with your people.

Which job eats the most hours on your team? Tell us in one line and we will sketch a plan for free

We connect to the systems and documents you already use, the AI works through your process, and your people review and approve. You see real numbers from a pilot before you pay for anything, then you pay for completed work.

Or email us directly: yzh317179958@163.com

Free assessmentReply within 24h