08. Correction and repair¶

Handoff is the exit when the AI cannot help. Correction is the repair when the AI was wrong but the user wants the AI to keep going. The discipline is to make pushback cheap, the AI's response to pushback honest, and the captured signal usable for iteration.

A platform engineer at a Chennai HR-tech company audits the AI résumé-screening tool. The AI scores candidates; recruiters override roughly 15% of scores. The overrides are not captured anywhere. The recruiter clicks "override," the score changes for that candidate, and the signal evaporates. Months later the team wonders why the model drift is invisible — the very thing they need to detect drift is being thrown away. The team rewrites correction as a structured signal: the override carries a reason code, a free-text note, the original score, and the corrected score. The data flows into the eval pipeline. Two months later the team has a calibration dataset large enough to retrain. The model improves where it was worst.

This chapter is the correction discipline.

What correction and repair is¶

Correction and repair is the UX and data discipline of letting users push back on AI output — visibly, with low friction — and capturing that pushback as a signal the system can act on, both immediately and for iteration.

Two timescales:

Timescale	What happens	UX surface
Immediate	The current response is repaired	Edit, retry, override; the user gets a better answer now
Iteration	The correction trains the system	Eval set, fine-tune data, prompt change; the AI gets better next time

Correction UX without iteration is a placebo. Iteration without correction UX has nothing to train on.

The cheap-pushback rule¶

If pushing back costs the user more than accepting the wrong answer, they accept it. Friction kills the signal.

Examples of expensive pushback:

"Tell us what went wrong" with a multi-field form.
A retry button that re-runs the same prompt and produces the same answer.
An edit affordance that requires the user to learn markdown.

Examples of cheap pushback:

A thumbs-down with one optional category select.
An inline edit on the answer text itself; the AI re-runs with the edit as context.
"This isn't right" → AI asks one short clarifying question → re-runs.

The discipline is to make the cost of correcting close to zero. Then the signal flows.

Three correction patterns¶

Pattern 1 — Edit in place¶

The user edits the AI's output directly. The edited version becomes the new state. The AI sees the edit and adjusts the next turn.

Use when: the AI is a co-author — drafting emails, generating code, writing copy. The edit is the user's signal.

Pattern 2 — Push back, re-run¶

The user signals "wrong" without specifying. The AI tries again, possibly with a clarifying question. The new attempt replaces the old.

Use when: the AI's answer is structurally wrong (not just a phrasing nit). The user knows it is wrong but does not want to write the correct version.

Pattern 3 — Override with reason¶

The user supplies the corrected answer (numeric, categorical, structured). The AI's answer is replaced. The reason is captured.

Use when: the AI is making a decision the user will sign off on — risk scores, categorisations, classifications. The corrected value matters more than the AI's narrative.

Most products use a mix. A drafting tool needs edit-in-place; a scoring tool needs override-with-reason; a conversational assistant needs push-back-and-re-run.

The signal capture¶

Every correction event carries a payload:

The original AI output.
The corrected output (or the signal that the original was wrong).
The reason if available (free text, category, or both).
The context: model version, prompt version, retrieval set, tool calls.
The user identity at the granularity required (per-user, per-role, per-tenant).
The timestamp and session ID.

This payload flows into telemetry and feedback loops (see 02_telemetry_feedback_loops). Without it, the correction is local repair only.

A common mistake: capturing the correction in the application database but not in the eval pipeline. The signal is technically present and operationally invisible.

How the AI responds to correction in-conversation¶

When the user pushes back mid-conversation, the AI's response shapes the next interaction. Three good responses:

Acknowledge the correction. "Got it — you wanted the breakdown by region, not by product. Here it is."
Ask a clarifying question if ambiguous. "When you say wrong, do you mean the number or the categorisation?"
Update internal state. The next AI turn reflects the correction, not the original assumption.

Three bad responses:

Defensiveness. "I gave the right answer based on the data."
Over-apology. "I'm so sorry, you're absolutely right, I should never have..."
Same-answer-restated. The AI re-emits the original answer in different words; the user gives up.

The AI's tone is brand-specific, but the substance is universal: acknowledge, clarify if needed, adjust.

The wrong-correction problem¶

Sometimes the user is wrong about the AI being wrong. The user insists the AI gave a wrong tax rate; the AI's answer was correct; the user's expectation was off. The UX has to handle this without picking a side prematurely.

The right pattern:

The AI does not capitulate immediately. It re-states the answer with the supporting source: "Here's where I got that number — [source]."
The AI offers escalation if the user still disputes: "If this doesn't match your records, let's get someone to verify."
The correction is still captured, with the disposition flagged "disputed — user maintained original was wrong."

This protects the eval pipeline from poisoning. A user who is wrong about being right corrupts the training signal if the system blindly accepts every correction as truth. The disposition layer separates raw correction signal from confirmed correction signal.

When correction becomes adversarial¶

A small fraction of users will deliberately corrupt the signal — bored, malicious, or testing the system. Defences:

Per-user weighting. A user with a 90% disagreement rate is weighted less than a user with a 5% rate.
Cross-user agreement. A correction confirmed by independent users carries more weight.
Audit and review for high-stakes corrections that change the system materially.

Adversarial correction is a security problem; treat it like one.

Multi-turn correction¶

Conversations accumulate corrections. The user corrected turn 3; turn 6 should reflect that correction, not regress. State management matters:

The conversation state carries the corrected facts forward.
New turns are evaluated against the corrected state, not the original.
If the system regresses (forgets a correction), it has lost the user's trust for the rest of the session.

This is a memory problem (see 11_long_term_memory_state) as much as a UX problem. The UX surfaces the regression — "I see I'm using the old number; let me update" — when the system catches it, and avoids it when it does not.

Common mistakes¶

Discarding the signal. The correction repairs the immediate response and disappears.

High friction. The user has to fill a form to say "wrong"; few users do.

Defensive AI. The AI argues with the correction; the user disengages.

Blind capitulation. The AI accepts every correction as truth; the eval pipeline gets poisoned.

No clarifying questions. The AI re-tries with no new information and produces the same wrong answer.

Regressive memory. The AI forgets the correction within the same conversation.

No escalation path. The user has corrected three times and still has the wrong answer; no way to reach a human.

Interview Q&A¶

Q1. The team's AI captures overrides in the application database but not in the eval pipeline. What is the cost? The team has the signal operationally but cannot use it for iteration. Drift is invisible because the signal that would detect it lives in the wrong system. The fix is to route correction events into telemetry and feedback loops the eval pipeline reads. Then the corrected data accumulates as a calibration set the team can use for retraining, prompt iteration, or eval-set growth. Wrong-answer note: "we'll set it up later" treats correction signal as optional; it is the most valuable training data the product generates.

Q2. The team is debating whether the AI should ever push back when a user says it was wrong. What is your view? Yes, when the AI has a verifiable source and the user's claim looks like a misunderstanding. The AI re-states the answer with the source and offers escalation if the user still disputes. Blind capitulation produces two harms: the user gets a worse answer next time (the AI defers when it should not), and the eval pipeline gets poisoned by user assertions that were not actually corrections. The correction is still logged, but with a disputed flag. Wrong-answer note: "the user is always right" loses the AI's calibration; "the AI is always right" destroys trust.

Q3. Walk through cheap-pushback UX for a chatbot. A thumbs-down on each response. Click → optional one-tap category ("Wrong," "Missed context," "Tone off"). Click → optional one-line note. Each step is skippable. The skip is the signal "wrong, no detail." A small number of users give detail; most just tap thumbs-down. Both are useful — the volume of unspecified thumbs-down is itself a calibration signal. Wrong-answer note: "ask for detailed reasons every time" produces low completion rates and biased capture.

Q4. The AI forgets a correction mid-conversation. How serious is the damage? Severe. The user gave the AI a signal; the AI lost it. Future corrections will be lower-effort because the user has updated their expectation downward. The fix is conversation-state management: corrections become facts the AI's next-turn prompt sees as authoritative. The UX recovery if regression happens is to surface the catch — "I had been using the wrong figure earlier; updating now." — but the engineering fix is to prevent regression. Wrong-answer note: "we'll just remind the user to re-state" externalises the system's failure onto the user.

Q5. How do you defend the eval pipeline from adversarial corrections? Per-user weighting based on historical agreement rates. Cross-user agreement for raw correction signal to become confirmed signal. Manual review queue for high-stakes corrections — those that would materially change model behaviour. Audit log on every captured correction so the team can replay if the signal is suspected. Treat the correction pipeline as a security surface, not just a UX surface. Wrong-answer note: "trust all corrections equally" lets a small number of bad actors poison the system.

What to do differently after reading this¶

Make pushback cheap — thumbs-down with optional category, edit-in-place where possible.
Capture every correction with full payload: original, corrected, reason, context, version.
Route corrections into the eval pipeline, not just the application database.
Have the AI acknowledge corrections in-conversation and update its state.
Do not let the AI capitulate blindly; verifiable sources should be re-stated with escalation offered.
Defend the pipeline from adversarial corrections with per-user weighting and cross-user agreement.

Bridge. Correction repairs individual interactions and feeds iteration. Onboarding shapes the user's mental model before the first interaction. The next chapter is the discipline of teaching users what the AI can and cannot do — so corrections are about accuracy, not unmet expectations. → 09-onboarding-and-mental-models.md