03. Implicit signals¶

Explicit feedback is direct; only 1-5% of users provide it. Implicit signals — every user's behaviour reveals something about satisfaction. The discipline is to capture, interpret, and aggregate the implicit signals so the team sees the broader picture.

A platform engineer at a Mumbai SaaS company looks at her explicit feedback dashboard: 2% response rate; weekly volume of 1,400 thumbs; 73% positive. The team is pleased. She adds implicit-signal tracking: how often users abandon a conversation mid-flow, how often they ask the same question multiple times in a session, how often they copy the response to their clipboard (a proxy for "useful enough to keep"). The implicit signals reveal patterns the explicit feedback hid: abandonment is up 12% over the past month for users with mobile devices; repeat-ask rate is highest in the late-evening hours; copy-rate has dropped for users on the new pricing flow. None of these would have surfaced from thumbs alone. The team's iteration shifts from "the explicit feedback is fine" to "we have specific patterns to investigate."

This chapter is the discipline. The signals are abundant; the interpretation is the work.

The six common implicit signals¶

Signal	What it captures
Engagement length	How much the user continues the conversation
Abandonment	The user leaves without resolving the task
Repeat-ask	The user asks the same or similar question multiple times
Follow-up clarification	The user asks for elaboration or correction
Copy / share	The user copies the response or shares it
Escalation to human	The user requests a human after the AI's response

Each is interpreted with care; each has noise; none alone is a verdict.

Engagement length¶

The duration or number of turns the user continues interacting after a response. Two interpretations:

Long engagement = good. The user is finding value; the conversation is productive.

Long engagement = bad. The user is struggling; each new turn is the system failing to resolve the original need.

The interpretation depends on the task. For exploratory tasks (research, brainstorming), longer is better. For transactional tasks (find my order, book a meeting), shorter resolves faster — longer means the system did not understand.

The team tags each task type; engagement length is interpreted per type. A long conversation for a transactional task is a flag; for an exploratory task it is positive.

Abandonment¶

The user starts a conversation, receives one or more responses, then leaves without explicit closure. Captured by:

User closes the chat window
User does not respond within a timeout (e.g., 5 minutes)
User starts a new conversation without finishing the previous

Abandonment is rarely binary — some abandonments are users finding their answer and moving on; others are users giving up. The interpretation requires care:

If the previous AI response was a successful resolution, abandonment is closure.
If the conversation was mid-flow (the user had a follow-up implied), abandonment is the user giving up.

The team often computes "abandonment-after-incomplete-response" — abandonments where the conversational state suggested more was expected. This is a sharper signal than raw abandonment.

Repeat-ask¶

The user asks the same or semantically similar question more than once in a session. Indicates:

The system's first response did not satisfy
The user's question was ambiguous and the system's clarification request did not help
The system gave conflicting answers across turns

Detection: semantic similarity between the user's messages within a session. An embedding-based comparison flags messages with high similarity to a prior message in the same session.

A repeat-ask rate above baseline for a specific feature is a strong signal of a usability problem. The investigation looks at the cases: what did the user ask, what did the system say, why did the user not accept the answer.

Follow-up clarification¶

The user's next message contains words like "no, I meant," "wait," "actually," "but you said," "that's not right." These indicate the user is correcting or clarifying the system's previous turn.

Detection: simple keyword or sentiment classifier on the user's reply. The signal is per-response — for which responses did the user immediately push back?

The rate of follow-up clarification is a leading indicator of misunderstanding rate. Useful as a per-prompt-version metric; a new prompt that increases follow-up clarification is producing more misunderstandings.

The user copies the AI's response to their clipboard or shares it (in apps that support sharing). Strong positive signal: the response was useful enough to extract.

Caveats:

Some copies are about further refining (the user pastes elsewhere to edit); not pure positive.
Some users do not copy even useful responses (they just remember or screenshot).
Mobile platforms may not capture copy reliably.

Despite caveats, copy-rate is a useful indicator. A drop in copy-rate for a specific feature is a signal worth investigating.

Escalation to human¶

The user explicitly asks for a human, or the platform routes to a human after the AI's response. Strong signal: the AI did not resolve.

Two interpretations:

The AI tried and failed (negative signal for the AI).
The user always wanted a human; the AI was a gating step (neutral; depends on platform policy).

The team distinguishes: was the user given an option to engage AI first, or routed to AI without choice? The signal is sharper for the second case.

A rising escalation rate is a high-priority investigation. The cases preceded by escalation are candidates for the eval set (chapter 05).

Interpreting signals with bias awareness¶

Implicit signals are behavioural, which means they include selection bias from who uses the platform, when, and on what device. A drop in copy-rate may be a real signal or a shift in user demographics; the team should slice by segment (chapter 04 of 01_dataset_golden_set_operations stratification applies).

The discipline:

Slice implicit signals by segment (user tier, device, region, feature).
Compare to baselines per segment, not just overall.
Triangulate with explicit feedback when available.
Look at the cases for signals that move; the aggregate is the alarm, the cases are the diagnosis.

What to capture per implicit signal¶

Each signal is a structured event:

implicit_signal_id: imp_01HNF...
ts: 2026-05-25T11:14:02Z
user_id_hash: <hashed>
session_id: sess_...
response_id: resp_...        # the response the signal is about
signal_type: repeat_ask       # one of the six types
signal_value:                 # signal-specific payload
  similar_to_message_id: msg_...
  similarity: 0.92
context:
  feature: support_agent
  device: mobile

The schema parallels explicit feedback (chapter 02). The join key is response_id; analysis correlates implicit signals with the system's outputs.

What implicit signals do not solve¶

Direct user voice. Implicit infers; explicit states. The two complement; neither alone is sufficient.
Diagnosis at the case level. Implicit signals reveal patterns; explanation requires reading the actual cases.
Long-tail rare cases. Implicit signals require aggregation; rare cases produce too few signals to surface.

Common mistakes¶

Tracking signals without interpretation. The data is collected; no one assigns meaning to the patterns.

Single-signal interpretation. Abandonment alone is ambiguous; combined with engagement-length and explicit feedback, it tells more.

Ignoring segment differences. Aggregate signals hide segment-level patterns.

No baseline. Without per-segment baselines, "high abandonment" is undefined.

Privacy ignored. Implicit signals are user behaviour; the same privacy discipline applies as explicit feedback.

Interview Q&A¶

Q1. The team's explicit feedback is 73% positive. They are confident. What might they be missing? Explicit feedback comes from 1-5% of users; the silent 95% are not represented. Implicit signals cover the broader population: abandonment, repeat-ask, follow-up clarification, copy-rate, escalation. The team might find that abandonment is up, repeat-ask is high on a specific feature, copy-rate has dropped — patterns the thumbs do not surface. The aggregate explicit signal can be 73% positive while the broader user experience degrades silently. Wrong-answer notes: "the thumbs are sufficient" misses the silent majority.

Q2. Walk through how you would interpret a sudden rise in repeat-ask rate. Look at the cases. Pull a sample of sessions where repeat-ask occurred. Read the user's two (or more) messages and the system's responses. Common patterns: the system answered a different question than the user asked (intent classification problem); the system gave a vague answer the user kept probing; the system contradicted itself across turns. Each is a different remediation. The rate is the alarm; the cases are the diagnosis. Add the diagnosed cases to the eval set per chapter 05. Wrong-answer notes: "fix the model" without diagnosing the pattern wastes the investigation.

Q3. Should engagement length be optimised up or down? Depends on the task. Exploratory tasks (research, brainstorming, learning) want longer engagement; the conversation is the value. Transactional tasks (find my order, book a meeting) want shorter engagement; the system should resolve faster. Tag tasks by type; interpret engagement length per type. A platform that uses one metric across both types produces misleading signals. Wrong-answer notes: "longer is always better" produces optimisation against the wrong target.

Q4. Implicit signals show abandonment up; explicit feedback shows 73% positive. How do you reconcile? They are signals from different populations. Explicit is from the 1-5% who clicked thumbs (and disproportionately when they had strong reactions). Implicit is from all users. The two diverging means the silent population is showing dissatisfaction the explicit minority does not. The right response is to investigate the implicit signal — which segments are abandoning? what cases? — and to enrich the eval set with their patterns. The 73% positive is true for who responded; the implicit signal is true for the broader user base. Wrong-answer notes: "trust the explicit since it's direct" misses the selection bias.

What to do differently after reading this¶

Capture all six implicit signals as structured events.
Tag tasks by type; interpret engagement length per type.
Compute baselines per segment; alarm on deviations.
Triangulate implicit and explicit signals; agreement is confidence, disagreement is investigation.
Always look at the cases when an implicit signal moves.

Bridge. Capture is one discipline. Storage and schema are how the captured signals become queryable. The next chapter is the storage discipline that supports the pipeline. → 04-feedback-storage-and-schema.md