03. Content Moderation — Interview Q&A

~15 min read · Part 4 of 4 (Overview → HLD → LLD → Q&A)

These are the questions an interviewer actually asks once the funnel is on the board. Each answer is the strong version, followed by the wrong answer that quietly sinks candidates.

Q1. Why not just pick one model threshold and auto-act on everything? Because the two errors cost wildly different amounts and one threshold can't make both small. Set the threshold low to catch more harmful content and you remove piles of legitimate posts (false positives, a censorship problem); set it high to protect legitimate content and harmful posts slip through (false negatives, a safety problem). So you pick two thresholds and cut the score into three bands — confidently safe auto-allows, confidently harmful auto-blocks, and an ambiguous middle that goes to humans. The model handles the easy 97% cheaply so people handle only the hard 3%. Common wrong answer to avoid: "Tune the threshold to maximize accuracy/F1." A single operating point still forces one error rate up when you push the other down, and aggregate accuracy hides which side you're failing on.

Q2. Walk me through what happens to 1,000,000 uploads in an hour. The synchronous hash/rules pass blocks known-bad items in under 200 ms without a GPU. Everything else is scored asynchronously and banded: ~95% (950,000) score below the low threshold and auto-allow, ~2% (20,000) score above the high threshold and auto-remove, and the ambiguous ~3% (30,000, about 8.3/s) route to the human queue. Draining that queue at a 60-second average handling time needs 8.3 × 60 ≈ 500 concurrent reviewers by Little's Law, and appeals run at ~5% of removals (~1,000/hour), of which ~15% overturn. The whole system is a funnel that widens human capacity only for the 3% it must. Common wrong answer to avoid: "We score everything and a human checks the flagged ones." Vague on the split and the numbers — the interviewer wants to hear the banding and the Little's Law staffing math, because that's the part that decides whether the system stays up.

Q3. Do you block the upload synchronously or review it after it's published? Tiered, by the cost of being wrong for that content type. Cheap deterministic checks — a perceptual-hash match against known-bad media — run synchronously at upload and block instantly, because a hash hit has effectively zero false positives. Deep ML scoring is too slow for a 1,000/s upload path, so it runs asynchronously. For high-severity categories you hold the item in pending until scored (fail closed); for low-risk categories you publish optimistically and remove if the async score comes back bad (fail open). The sync/async decision is per-category, not one global switch. Common wrong answer to avoid: "Block everything until the model finishes." At 1,000/s and multi-second inference you either stall every upload or build an unbounded queue on the user-facing path; deep scoring belongs off the synchronous path.

Q4. The review queue is backing up — reviewers can't keep pace. What's happening and what do you do? Arrival rate has crossed above service rate. Concretely: 8.3/s borderline items arriving, but only 400 reviewers online giving 400/60 ≈ 6.7/s of service, so the backlog grows at 1.6/s ≈ 96/minute and after an hour you're ~5,760 items behind with the oldest-item age past SLA. You can't hire reviewers in minutes, so you shed by priority: serve highest severity and exposure first, let low-risk borderline items auto-resolve with conservative thresholds, temporarily widen the auto-allow band, and surge on-call reviewers. Overload must degrade the least-risky items first — never the child-safety ones. Common wrong answer to avoid: "Add more servers" or "scale the queue." The bottleneck is human throughput, not machine capacity; more queue capacity just stores a bigger backlog. This is a queue-capacity problem, not a compute problem.

Q5. Raising recall to catch a new abuse type — what does that cost you? It costs reviewer headcount, directly. Tightening thresholds to catch more harmful content grows the borderline band: going from 3% to 5% of a million pushes queue arrivals from 8.3/s to 13.9/s, and required reviewers from 500 to 13.9 × 60 ≈ 830. Similarly, raising the auto-block threshold from 0.95 to 0.98 to cut false positives slides ~8,000 items/hour from auto-block into review, lifting staffing from 500 toward 630. Every threshold nudge is a staffing decision, which is why you plan them ahead and don't ship them on a Friday. Common wrong answer to avoid: "Just retrain for higher recall." Recall gains that widen the ambiguous band land on the human queue; if you don't staff for them, you've traded a safety metric for a growing backlog and blown SLAs.

Q6. The ML scoring service goes down. Now what? Items pile in the durable scoring queue unscored — at 278/s arriving and throughput dropped to 150/s, the backlog grows ~128/s ≈ 7,680/minute, so a ten-minute outage leaves ~77,000 items unscored and potentially live. The queue is durable, so nothing is lost; the safety comes from the fail-direction policy. High-severity categories hold in pending (fail closed) so nothing dangerous publishes unscored, while low-risk content publishes optimistically (fail open) and is rescored on recovery. The missing-score path is a deliberate policy choice per category, not an exception that silently allows everything. Common wrong answer to avoid: "Allow everything until the model is back" or "block everything." A single global fail direction is wrong — blanket-allow ships harmful content, blanket-block takes down the whole platform. Fail closed only where the cost of a false negative justifies it.

Q7. How do you make sure two reviewers don't grab the same item — and that an abandoned item isn't lost? The claim is an atomic dequeue-and-lease: SELECT ... FOR UPDATE SKIP LOCKED (or ZPOPMAX plus a lease key in Redis) selects the highest-priority queued item and leases it to one reviewer indivisibly, so 500 reviewers pull concurrently without ever getting the same row. The lease carries a visibility timeout; a redrive sweep returns any item whose lease expired back to the queue, so a reviewer who crashes mid-item doesn't strand it. Verdicts carry an idempotency token so a redrive-plus-retry can't remove content twice. Common wrong answer to avoid: "Read the next item, then mark it taken." That read-then-write is a race — two reviewers both read the same front item before either marks it — and with no lease timeout a crashed reviewer strands the item forever.

Q8. A user appeals a removal. How does that flow, and why does it matter beyond the one user? The appeal reopens the resolved decision, moves the item to appealed state (which blocks model-driven transitions so a rescore can't clobber it), and routes it back through review at a higher priority — usually to a senior reviewer. On overturn, the action executor reinstates the content and the system writes a corrected label. That corrected label is the point: appeals are the highest-quality error signal you have, so they feed training and are a live measurement of auto-block precision (~15% overturn on ~1,000 appeals/hour means ~150 wrongful removals you can now learn from). A spike in appeal volume is also an early rollback signal after a model deploy. Common wrong answer to avoid: "Appeals are a customer-support ticket." Treating appeals as a side channel wastes your best label source and loses the precision signal that catches a bad model deploy early.

Q9. Hashing/rules versus the ML model — why keep both? Each covers the other's blind spot. Hash and rule matches give precision on known bad content: an exact hash hit is certain and a curated rule won't wrongly flag novel posts, so they block known-bad cheaply and synchronously with near-zero false positives — but they're blind to anything new. The ML model gives recall on novel bad content the rules have never seen, at the cost of probabilistic errors. Stacking deterministic-first, ML-second means the hash pass sheds known-bad before it costs a GPU cycle, and the model only works on what's genuinely new. Common wrong answer to avoid: "The ML model makes rules obsolete." A model is probabilistic and can't guarantee zero false positives on a known-bad hash; you lose certain, cheap blocks and the audit-clean determinism regulators expect.

Q10. How do you prevent the feedback loop from being poisoned? Reviewer verdicts and overturned appeals feed the next model, so an adversary who can influence those labels can teach the model to allow their content. Defenses: weight and audit label sources rather than trusting all equally, require senior review on appeal overturns for high-severity categories, and hold out a trusted golden set to detect drift after each retrain. You also shadow-score and canary new models against the live one and auto-rollback when canary precision or golden-set recall drops below a floor. Common wrong answer to avoid: "Retrain on all reviewer and appeal decisions automatically." Uncritical feedback is exactly the attack surface — coordinated bad actors farm overturns to shift the model's boundary in their favor.

Q11. Where do you store the decision record, and why separate current state from history? Current state lives in a point-lookup row (content_decision, keyed by content_id) so the producer's polling and the reviewer's context load are fast; the full history lives in an append-only decision_event log that is never mutated. They're separated because they have opposite access patterns — one is read-hot and mutable, the other is write-only and read for audit — and because the immutable log is what you hand a regulator to prove why an item was removed, complete with the model_version and policy_version that decided it. That versioning is what makes a bad deploy diagnosable weeks later. Common wrong answer to avoid: "One row per item, updated in place." Overwriting state destroys the audit trail; you can't answer "why was this removed and by which model" after the fact, which fails both regulators and incident review.

Q12. Aggregate model accuracy is 0.98 and holding, but harmful content is getting through. How? A new abuse category is a rounding error in the aggregate — the model can be 0.98 accurate overall while its recall on an emerging category has collapsed, because that category is a tiny slice of traffic. Aggregate accuracy/AUC is the misleading metric here; you watch per-category recall, the auto-action mix, and the review queue's category composition. The leading operational signal isn't accuracy at all — it's the queue's oldest-item age and the arrival-versus-service overlay, because those move before the aggregate does. Common wrong answer to avoid: "Accuracy is high, so the model is fine." Global accuracy hides per-category failure and says nothing about the false-negative rate on the categories that matter most.

Deeper follow-ups

• How would you calibrate model scores so a 0.9 genuinely means ~90% likely violating, and why does the decision engine depend on that?
• How would you canary a new model against the live one on the same traffic without acting on the canary's decisions?
• A coordinated campaign uploads thousands of near-duplicate borderline items to flood the review queue — how do you detect and collapse them before they exhaust reviewers?
• How would you route items to reviewers by language, region, and category expertise without breaking the single-priority-queue model?
• What consistency guarantees does the appeal path need so a reinstatement can't race with a periodic rescore?
• How would you measure and bound the end-to-end time from upload to decision separately for the sync, async-auto, and human-review paths?

How this round is scored

Interviewers use content moderation to see whether you treat the humans as part of the system, not an afterthought. The strong signal is recognizing early that the model is the easy part and the scarce, slow, expensive resource is reviewer capacity — then sizing it with Little's Law and connecting every threshold decision back to that headcount. Seniority shows in the tradeoff discussions — precision versus recall, sync block versus async review, fail-open versus fail-closed, rules versus ML — where you name both sides, attach numbers, and pick per-category rather than globally. The failure-mode section (model outage, queue overload, bad deploy, poisoned feedback) separates people who have operated a Trust & Safety pipeline from those who have only drawn a classifier. Doing the arrival-versus-service math out loud, and using it to justify staffing and threshold choices rather than as decoration, is what pushes an answer from "correct" to "senior."