Menu
← Blog

When should AI hand a decision to a human?

Innokenty Bodrov ·

“Send complex cases to an operator” sounds reasonable until an engineer asks how the system will recognise one. In a pilot, I walk through several of those cases and write down what triggers the handoff, who receives it, and what they need to continue.

Define a condition you can test

Imagine a service that answers questions about application status. In this example, an automatic reply can go out when the service finds exactly one application, confirms it belongs to the customer, and retrieves a status from the approved list. The system can send the wording approved for that status.

When several applications match, the system keeps the reply in draft. The operator sees the application IDs and the reason for the handoff. An unavailable status source is a separate case. The operator needs to know what could not be retrieved so they can decide where to check and whether to reply now.

What happenedWhat the system doesWhat to verify
One confirmed application has a known statusSends the replyThe customer receives wording approved for the status
No application is confirmed, or several matchHands the case to an operatorThe handoff reason is visible. No automatic reply was sent
The status is unknown or its source is unavailableHands off with the reason recordedThe operator can see what could not be confirmed. The draft contains no invented status
The operator queue is unavailableSaves the case as awaiting handoffOnce the queue recovers, one task appears for the operator

Check whether the operator can pick up the case

Open one handed-off case with an operator. Can they reach the application from the case record? Can they see what status was available at handoff? Can they tell whether the customer received a message? If the operator has to reconstruct the case from a conversation thread, measure that time. It tells you how much work the automation is shifting to manual handling.

During the pilot, I would track the handoff rate, time to first action, and cases stuck awaiting assignment. I would also review a sample of automated replies. A low handoff rate can mean the rule is missing complex cases. The sample helps identify cases the system should have passed to an operator.

Explore my approach to AI implementation

Back to top