Skip to content

03. Uncertainty surfacing

Latency UX manages the wait. Uncertainty surfacing helps users calibrate trust. The AI is not always right; the UX is how the user knows when to verify and when to act on the response.

A platform engineer at a Chennai SaaS company observes a pattern: users complain when the AI is confidently wrong; users do not complain when the AI says "I'm not sure but here's my best guess." The same level of correctness in both; the difference is the uncertainty signal. The team adds explicit uncertainty in the AI's responses — a "high confidence", "moderate", "low" indicator; a "verify this with your records" line for low-confidence cases. Complaints about wrong answers drop substantially even though the model's accuracy is unchanged. The trust dynamics shifted from "the AI is wrong" to "the AI is honest about not knowing."

This chapter is the uncertainty discipline.

What uncertainty surfacing is

Uncertainty surfacing is the UX practice of communicating the AI's confidence in its response, so users can calibrate trust per response rather than uniformly.

The AI is more confident in some cases than others; the UX exposes the difference.

Three forms of uncertainty signal

Explicit confidence labels. "High confidence", "moderate", "low" or a percentage. Most direct; can be visually rendered.

Hedging language. "I'm fairly sure...", "I'm not certain but...", "This is my best guess..." — natural-language signals.

Verification prompts. "You may want to verify this with..."; "Double-check the date with the original document". Action-oriented; tells the user what to do.

Each fits different contexts; most platforms use a combination.

When to surface uncertainty

Not every response needs a confidence signal. Surface when:

  • The AI's confidence is meaningfully below 1.0 (the threshold depends on platform; commonly <0.85).
  • The cost of being wrong is material (financial decisions, medical information, legal advice).
  • The user's downstream action depends on the AI's correctness.
  • The AI explicitly does not know (refusal or "I don't know" should be visible).

Skip surfacing for:

  • Routine high-confidence responses (would add noise).
  • Casual conversation where uncertainty is implicit.
  • Operations where the AI's confidence is not measurable (most generation tasks).

Where the uncertainty comes from

For the UX to surface uncertainty, the AI needs to provide it. Sources:

  • Model-reported confidence. Some models can be prompted to report confidence; the reliability varies.
  • Retrieval-grounded confidence. A response based on retrieved documents that match well is higher confidence than one based on weak retrieval.
  • Self-consistency. Asking the model multiple times; agreement is high confidence; disagreement is low.
  • Domain-specific signals. A medical or legal AI may compute confidence from domain heuristics.

The confidence is approximate; it is more useful for "low/moderate/high" buckets than for precise percentages.

What good uncertainty UX looks like

A response with surfaced uncertainty:

[AI response with the answer]

ⓘ Moderate confidence
The information about the policy effective date is based on our most
recent document; verify with your customer-success rep if the
agreement was recently updated.

The signal is visible; the verification prompt is specific; the user has actionable guidance.

A response with low confidence:

[AI response with caveats]

⚠ Low confidence
I couldn't find clear information about this in our knowledge base.
Consider reaching out to our team for confirmation:
[Connect with support]

The low signal triggers the escalation path; the user is not left guessing.

What bad uncertainty UX looks like

  • Uniform high-confidence presentation. Every response looks the same regardless of confidence; users cannot calibrate.
  • Hedging on everything. Constant "I think..." for routine high-confidence responses; users tune out the signal.
  • Confidence percentages without context. "73% confidence" is meaningless without knowing what 73% means.
  • Disclaimers in fine print. Legal-looking text that users do not read.
  • Buried uncertainty signal. The signal is technically present but visually de-emphasised.

The honesty principle

Surfacing uncertainty is honest communication; under-surfacing produces over-confidence; over-surfacing produces under-trust. The discipline is honest calibration:

  • Confidence labels should reflect actual reliability.
  • Users should be able to verify that "high confidence" responses are usually right.
  • Patterns of mislabelled confidence (wrong responses claimed high; right responses claimed low) should be caught by feedback.

The judge-calibration discipline (chapter 06 of 02_telemetry_feedback_loops) applies: the AI's self-reported confidence should be calibrated against actual correctness.

Specific patterns

For factual queries. Show source citations with the response; confidence is the strength of the source match.

For inferences and recommendations. Show the chain of reasoning briefly; confidence is the strength of the reasoning.

For predictions. Show the prediction with its confidence interval; the interval is the uncertainty.

For multi-step tasks. Show intermediate steps; the user can verify each step before the AI proceeds.

Each pattern fits its context.

What uncertainty does not solve

  • Wrong-but-confident answers. If the model is confidently wrong, the UX has nothing to surface; the model is the issue.
  • Subjective questions. Some queries have no factual "right" answer; uncertainty does not apply in the same way.
  • User dismissal. Users who do not trust uncertainty signals will dismiss them; the design can do its best, but user behaviour is not entirely controllable.

Common mistakes

No uncertainty surfacing. The chapter-opening confidence pattern; trust loss on first error.

Uniform surfacing. Every response hedges; the signal is meaningless.

Confidence presented without context. Numbers without meaning.

Uncertainty in fine print. The signal is invisible.

Confidence not calibrated. The AI claims high confidence on wrong answers; the signal is harmful.

Interview Q&A

Q1. Why does surfacing uncertainty reduce trust loss? Because users distinguish "AI was honest about not knowing" from "AI was confidently wrong." The first is reasonable; the second damages trust. The same level of correctness with explicit uncertainty produces materially different user experience. Users learn the AI's pattern (high confidence often right; low confidence might need verification); their trust is calibrated rather than binary. Wrong-answer notes: "reduce errors instead" misses that errors will occur; the UX manages how they are received.

Q2. When should you surface uncertainty? When confidence is meaningfully below 1.0; when cost of being wrong is material; when user's downstream action depends on correctness; when AI does not know. Skip for routine high-confidence responses where surfacing would add noise. The discipline is selective: surface where it helps calibration, not on every response. Wrong-answer notes: "always" produces hedging fatigue; "never" produces the confidently-wrong failure.

Q3. Walk through what good uncertainty UX looks like. Visible signal (label, icon, colour) appropriate to the confidence level. Actionable verification prompt for low confidence ("verify with...", "consider reaching out to..."). For low confidence, may trigger an escalation path (handoff to human, chapter 07). The signal is in normal visual hierarchy, not fine print; users see it. The text is specific to what to verify, not generic ("this might be wrong" is unhelpful; "verify the policy date with..." is actionable). Wrong-answer notes: generic warnings without specificity produce user fatigue.

Q4. The team uses percentage confidence ("73%"). What is the concern? Numbers without context are meaningless. What does 73% mean? Better than 50%? Same as 70%? Different from 80%? Users do not have a calibrated interpretation. Better to use 3-bucket categorical (low/moderate/high) with the buckets calibrated against actual reliability. The bucket is interpretable; the percentage is precise but uninterpretable. Some platforms use both (bucket prominent; percentage available on hover); the bucket is the primary signal. Wrong-answer notes: "percentages are more precise" misses that interpretation matters more than precision.

What to do differently after reading this

  • Surface uncertainty for responses where it informs user action.
  • Use interpretable categories (low/moderate/high) over raw percentages.
  • Make verification prompts specific and actionable.
  • Calibrate confidence labels against actual reliability via feedback.
  • Reserve uncertainty surfacing for where it adds signal, not everywhere.

Bridge. Uncertainty surfacing tells the user when to be careful. Explainability tells the user why the AI gave the answer it did. The next chapter is the explainability discipline. → 04-explainability-and-citations.md