← Back to all postssystems /

Design AI for Correction, Not Just Confirmation

A human review step only helps when people can challenge the evidence, change the recommendation, and stop the work before it causes harm.

aireviewworkflow

This is part five of Human Systems for AI Work. The previous article, What You Should Never Delegate to AI, separates assistance from responsibility. This article is about making that responsibility possible to exercise.

“Have a human review it” sounds like a safeguard. It can also mean sending someone a finished answer and asking whether it looks good.

Those are not the same thing.

A useful review process gives people enough context to identify a problem, enough authority to change the outcome, and enough time to do both. Without those conditions, human review becomes confirmation of work that is already treated as settled.

I want AI-assisted work designed around correction. The question is not just whether someone can approve the output. It is whether they can find and fix a meaningful error before others rely on it.

Start with a plausible mistake

Imagine a team using AI to summarize customer interviews before deciding what to work on next.

The summary says that customers want more advanced reporting. It is clear, well organized, and accompanied by several quotations. A reviewer reads it, recognizes familiar complaints, and approves it.

But suppose the interviews came mostly from experienced users. New customers who could not get started were barely represented. The quotations accurately support the stated theme. The theme still may not justify the proposed priority.

A spelling check will not catch this. Neither will verifying that each quotation exists. The issue is the connection between the evidence and the decision.

This is a useful way to plan review: name the mistake that would matter. Then ask what the reviewer would need to see to detect it.

For the interview summary, they need to know who participated, who did not, how themes were selected, and which observations do not fit the recommendation. They do not need every intermediate draft.

Review the right object

An answer can be correct in one sense and unsuitable in another.

A summary may accurately reflect its source while leaving out the evidence needed for the decision. A recommendation may be well supported but outside the team's authority. A message may be factually accurate and still make a commitment nobody can fulfill.

I separate these questions rather than asking for a general review:

  • Does the evidence support the important claims?
  • Does the recommendation follow from the evidence and our stated priorities?
  • Are we authorized and prepared to act on it?

Different people may be best placed to answer each question. The researcher can check the interview interpretation. The team lead can assess the priority. The person responsible for delivery can evaluate the commitment.

One reviewer can sometimes cover all three. The distinction still helps them notice what they have and have not checked.

“Looks good” is not a useful record if nobody knows which question it answered.

Show uncertainty where it changes the action

A general warning that AI can make mistakes does little to guide a particular review.

Useful uncertainty is attached to a claim or decision. In the interview example, a note might say: the reporting theme appears in six experienced-user interviews; the current sample does not establish whether it matters more than onboarding problems.

That tells the reviewer what is supported and what remains open. It also suggests an appropriate next step: inspect the missing perspective before treating the theme as a broad priority.

I prefer this to an unexplained confidence score. A number does not tell me whether the weakness is an outdated source, a small sample, ambiguous language, or an unsupported inference. Those weaknesses require different responses.

Ask for the limitation in plain language and its consequence for the work. If the uncertainty would not affect the decision, keep it brief. If it could reverse the recommendation, put it beside the recommendation rather than at the end of a long appendix.

Make disagreement specific

“Be critical” is a weak review instruction. It does not say what would count as a useful objection.

Before reading the recommendation, the reviewer can write down the conditions it must satisfy. For the customer research, a priority recommendation might need evidence from the affected customer group and a clear distinction between repeated complaints and isolated requests.

These criteria give the reviewer something to test beyond whether the narrative feels convincing.

Useful challenges are concrete: show the strongest observation that does not fit this theme; identify where the recommendation depends on a guess; explain what new information would change the conclusion.

AI can help generate those questions, but an AI-produced critique is not independent evidence. Asking the same system to argue against itself may uncover a weakness. It does not establish that the resulting answer has survived a genuinely different perspective.

For a consequential decision, bring in someone with relevant knowledge who did not help build the recommendation. Give them the underlying evidence, not only the polished summary.

Allow correction smaller than rejection

If the only choices are approve and reject, people may approve an imperfect result because rejecting it means starting over.

A useful process allows narrower responses. The reviewer can accept a finding while rejecting its generalization. They can keep the recommendation but reduce its scope. They can mark one claim as unresolved without holding up unrelated work.

In the interview example, the team might accept that experienced customers need better reporting while declining to call it the highest priority for all customers. That is a meaningful correction, not a failed review.

The next version should make the change visible. It should not quietly restore the broader claim in a new introduction or conclusion.

I want a short account of what changed and what still needs attention. This keeps the reviewer from having to compare entire drafts just to confirm that one objection was addressed.

Correction should preserve useful work while removing the part that cannot be defended.

Put review before commitment

Timing changes what a review can accomplish.

Reviewing a priority memo before the planning meeting allows the team to gather missing evidence. Reviewing it after the roadmap announcement leaves the team deciding whether to disappoint people who have already made plans.

Both reviews may be formally allowed to change the answer. They do not have the same practical freedom.

Identify the first moment when another person will rely on the output. That might be an email, an assignment, an appointment change, or a public statement. Place the consequential check before that moment.

Not every draft needs a checkpoint. Too many checkpoints consume the attention needed for the important one. Routine wording changes can happen freely; a change in scope, evidence, or commitment should return to the relevant owner.

The objective is to preserve a real opportunity to change course, not to maximize the number of approvals.

Give the reviewer a usable stop

A reviewer needs more than permission to leave comments. They need a clear way to pause the work when the problem is not resolvable inside the current task.

For the research summary, “we do not have evidence from new customers” may be a valid stopping point. The reviewer should not have to manufacture a competing recommendation to justify withholding approval.

The response can be a bounded next step: gather several relevant interviews, narrow the decision to existing customers, or ask the owner whether the missing evidence is an acceptable uncertainty.

Stopping also needs organizational support. A team that asks for careful review but treats every delay as poor performance is giving contradictory instructions. The person who pauses should be expected to state the problem and its consequence, not to pretend the answer is known.

This does not make objections unlimited. The owner can decide to proceed under uncertainty. If so, record that choice rather than relabeling the unresolved issue as verified.

Follow corrections beyond the document

Some errors are discovered after people have acted. A correction process needs to address that situation too.

Changing the original memo may not reach the people who copied its recommendation into a plan. A corrected date in a message may not update the arrangements somebody made after reading it.

When a meaningful error is found, ask who relied on it and what action needs to change. Contact those people directly when the consequence warrants it. A silent edit is not enough if the original claim has already affected a decision.

Separate the corrected fact from its implications. The finding might change while the chosen action remains reasonable for other reasons. Or a small factual correction might invalidate the action entirely.

The goal is not to issue a dramatic announcement for every typo. It is to repair the actual consequences rather than stopping at a cleaner document.

Match the effort to the risk

Correction-friendly work has costs. Evidence takes time to inspect. Independent reviewers may be scarce. More questions can delay a useful action without changing it.

I would not apply the same process to a meeting agenda and an eligibility decision. For a low-consequence draft, a quick check by the sender may be enough. For a decision affecting someone's access, money, or work, the review needs stronger evidence and a meaningful route to challenge the result.

Review can also introduce new mistakes. A confident human may remove a correct qualification or prefer an unsupported interpretation. Keep objections connected to evidence and authority rather than seniority alone.

Nor can review guarantee a correct outcome. Some information is unavailable, some effects emerge later, and reasonable people can disagree. The aim is to make errors easier to notice and repair, not to claim they have been eliminated.

If review repeatedly finds the same problem, change how the work is prepared. Do not make the reviewer fix it forever.

A practical closing

Take one recurring AI-assisted task and name a plausible consequential error. Check whether the reviewer would have the evidence, time, and authority to catch it.

Then make one change: expose the missing context, move review earlier, or give the reviewer a clear way to pause and narrow the work.

A review process is useful when a correction changes what happens next. Approval alone does not demonstrate that.