Menu
← Blog

The AI agent finished the task. Here’s what I check before accepting it

Innokenty Bodrov ·

Suppose I ask an agent to compare two versions of an API contract. One line stands out in its report: “customer_id is now required.” An engineer may read that as an instruction to change every request. If the rule applies to only one operation, the team could change more than it needs to.

I start the review with the decision someone will make from the report. Here, the team needs to decide which requests and tests to change. I want the contract version, the exact place in the schema, and an example that shows where the rule applies.

Give the reviewer a way to disprove the claim

In this teaching example, the new contract requires customer_id when creating a payment request. Looking up an existing request still requires only its request ID. If the agent says the field is mandatory across the API, I ask it to point to the rule for the status operation. Then I check both cases.

ScenarioExpected under the new contract
Create a request without customer_idRejected, with a response identifying the missing field
Look up the status by request ID without customer_idAccepted if the rest of the request is valid

For a contract comparison, check that the table accurately reflects both schemas. If the task includes verifying the implementation, collect the actual outcomes of both requests in a test environment. Record which check you performed so the next person knows whether to use the result to draft a task or make a release decision.

When the sources disagree

Suppose the new schema marks the field as required, while the migration guide says it can be omitted for another month. I keep both references and ask the API owner whether the guide describes an exception for existing clients and which ones qualify. The transition rule stays open until the owner confirms it.

Claim — customer_id is required when creating a request under v2.
Check — reviewed the creation and status schemas. The migration guide describes an exception.
Decision — the standard rule is confirmed. The API owner will confirm which clients qualify before implementation is accepted.

I applied this approach to requirements in AnalystCraft. Coworker helps trace a claim to its source and keep disagreements visible in the draft. A human can check both documents and see what decision still needs to be made.

Read the AnalystCraft case study

Back to top