A hallucination rate is a budget

Two assistants I own answer roughly 180–190K people a month. About 4% of sampled answers are wrong. People hear that number and ask how we plan to get it to zero, which is the wrong question, and asking it is usually the first sign that a team is about to spend a year on the wrong work.

Zero is a different product

An assistant that is never wrong is an assistant that answers almost nothing. You can have that. Restrict it to balance and tariff lookups against authoritative data, refuse everything else, and your error rate collapses — along with the 85% of conversations that currently finish without a human. Every point of accuracy past a certain line is bought with scope, and scope is the thing customers actually came for.

So the number is a budget. The question is not how to eliminate it but where to spend it.

Where we spend it

We spend it where being wrong is cheap and visible. A slightly wrong explanation of a roaming package costs a follow-up question. A wrong number on a bill costs trust, a complaint, and eventually a regulator asking how the answer was produced. Those two are not the same 4%, and any team reporting a single accuracy figure across both has already lost the ability to make this decision.

Three rules do most of the work:

1. A refusal is a correct answer. Escalating to a human with the context already gathered scores as a success in our rubric. This costs us self-service percentage points on paper and buys the error budget back where it matters.
2. The unresolved case goes to a person, not to a guess. Stated once, in writing, before anyone argued about it.
3. The regression set runs weekly, whether or not anything shipped. Models drift under you. So do the systems they read from.

The part nobody reports

The interesting number is not 85% self-service. It is what happens to the other 15%. For a while we could report what the assistant did and not what happened to the person afterwards — and the second one is the number the contact centre actually lives with. If you are standing up an assistant this year, instrument the handover before you instrument the assistant.

A 4% error rate that you chose, measured weekly, spent deliberately, and can explain to a regulator is a better product than a 1% error rate nobody can account for.