Skip to content

03. Distributed Cache — Interview Q&A

~16 min read · Part 4 of 4 (Overview → HLD → LLD → Q&A)

These are the questions an interviewer actually asks once the ring is on the board. Each answer is the strong version, followed by the wrong answer that quietly sinks candidates.

Q1. Why consistent hashing instead of hash(key) % N? Because the node count changes and modulo hashing reshuffles almost everything when it does. Going from 100 nodes to 99 changes hash(key) % 100 to hash(key) % 99, and those disagree for about 99% of keys — so a single node loss invalidates the whole cache at once and the miss storm buries the origin. Consistent hashing places nodes and keys on a ring; a key belongs to the first node clockwise, so removing a node hands off only the 1/N ≈ 1% arc it owned and every other key stays put. Add virtual nodes so that 1% spreads evenly across all survivors instead of dumping on one neighbor. Common wrong answer to avoid: "Just use hash(key) % N, it's simpler and evenly distributed." Even distribution is not the problem; stability under N changing is, and modulo fails exactly there.

Q2. A 100-node cluster doing 10M ops/second loses a node at peak. Walk me through what happens. Only node 42's ~1% of keys need a new owner; the other 99% are untouched, so the cache side is a non-event — its 100k ops/second redistributes to 99 survivors as ~1,010 extra ops/second each, a 1% bump. The real risk is the origin: those 1% of keys would land cold and 100k misses/second would hit a database already at its 100k baseline, pushing it to 200k past its 150k ceiling. The fix is warm replica promotion — node 42's replica already holds those keys, so they are hits from the first request and origin QPS never moves. Any keys written but not yet replicated when it died are caught by single-flight coalescing, so at most one origin read happens per distinct cold key. That is how "1% moves, no stampede" holds. Common wrong answer to avoid: "The load balancer routes around the dead node and we're fine." A cache has no interchangeable backends — each node owns specific keys, so a dead node means those keys are gone until a replica takes over, not just fewer servers.

Q3. Why virtual nodes? Isn't one ring position per node enough? One position per node makes each node's arc as wide as the random gap to its neighbor, so some nodes own 3× the keys and load of others, and a dead node dumps its entire arc onto a single neighbor — doubling that neighbor's load and often toppling it into a cascade. With ~160 virtual points per node, each node's share is the sum of 160 small random arcs, which is close to 1/100 for everyone, and a dead node's 160 arcs hand off to ~160 different neighbors — so its 1% spreads across all 99 survivors as ~1,010 ops/second each. Virtual nodes convert a failover cliff into a gentle ramp. Common wrong answer to avoid: "Virtual nodes are just an implementation detail." They are the difference between a smooth failover and a cascading overload of one neighbor.

Q4. Replication factor 2 doubles your RAM cost. Justify it. RF=2 takes the cluster from 1 TB to 2 TB of RAM — a real doubling of the memory bill. It buys the one thing that keeps the origin flat on node loss: a warm copy that a failover promotes with the keys already in memory, so the 100k ops/second from a dead node's keys are hits, not misses that would spike the database to 200k. It is worth stating plainly that this replication buys availability, not durability — the origin is the source of truth, so we are paying for zero-cold-spot failover, not for preventing data loss. If the origin could absorb a full node's miss load with margin, RF=1 plus coalescing might be acceptable; here it can't, so RF=2 is the cheaper choice than over-provisioning the database. Common wrong answer to avoid: "Replicate for durability so we never lose cached data." A cache doesn't need durability — the data is reconstructible from origin. Framing replication as durability leads to expensive synchronous replication that breaks the latency budget for no benefit.

Q5. Synchronous or asynchronous replication? Asynchronous. A SET returns as soon as the primary applies it, and the replica catches up within a bounded lag, typically well under a second. Synchronous replication would add a full network round-trip — roughly 0.5–1 ms — to every write, which alone breaks the sub-millisecond target the cache exists to hit. The cost of async is that an unplanned failover loses the last sub-second of writes (the primary-to-replica lag window), and that is acceptable precisely because the origin is authoritative and those writes reappear on the next read-through. Monitor the replication offset gap so the risk is a visible number, not a surprise. Common wrong answer to avoid: "Synchronous, so we never lose a write." You are protecting data that's already safe in the origin, and paying a per-write round-trip that defeats the whole point of the cache.

Q6. LRU or LFU for eviction — and does it matter? It matters under skew and scans. LRU evicts the least-recently-used entry, which is cheap and usually right, but a single sweep — a nightly job that reads every user once — walks the entire dataset through the cache and evicts the genuinely hot keys, collapsing the hit ratio from 99% to something that spikes origin load. LFU evicts the least-frequently-used entry, so a once-touched scan key never out-ranks a key hit thousands of times, protecting the hot set from that flush. LFU costs a per-key frequency counter and adapts more slowly when what's hot genuinely changes. Default to LFU for skewed, scan-prone workloads; plain LRU is fine for uniform access. Common wrong answer to avoid: "LRU is standard, just use it." It is a reasonable default, but naming it without knowing the scan-flush failure signals you haven't run a cache through a batch job.

Q7. How can 8 bits of frequency counter track a key hit millions of times? It doesn't count linearly — it's a logarithmic, probabilistic counter. Each access increments freq only with a probability that shrinks as freq grows, so the stored value tracks roughly the logarithm of the true access count; 8 bits then distinguish "hit ten times" from "hit ten million times" without overflowing. A separate decay term reduces freq over time so a formerly hot key ages out once it stops being accessed — without decay, LFU would pin yesterday's hot keys forever and never adapt to today's. Common wrong answer to avoid: "Use a 64-bit exact counter per key." Exact counters cost more memory than many values, never stop growing, and — without decay — permanently favor keys that were hot once and never again.

Q8. One key is getting 1M requests/second. Consistent hashing spread it, right? No, and this is the trap. Consistent hashing distributes distinct keys across nodes; a hot key is one key, so it hashes to exactly one node and saturates that node's single thread and NIC while the other 99 sit idle. Adding nodes does nothing. The fix is fan-out, not sharding: a small client-side L1 cache serves the hottest keys locally with a short TTL, cutting most of the 1M before it leaves the app boxes, and key-splitting writes the value under k#0..k#7 so reads pick a random copy and fan across 8 nodes — turning 1M/second on one node into ~125k/second on each of eight. A hot key is not a sharding problem; it's a fan-out problem. Common wrong answer to avoid: "Add more cache nodes." More nodes spread more distinct keys and leave the one hot key exactly where it was, on one overloaded machine.

Q9. A hot key's TTL expires and thousands of reads miss at once. How do you avoid crushing the origin? Two layers. Single-flight coalescing ensures that when N concurrent requests miss on the same key, exactly one reads origin and the rest wait for its result — so origin sees one read, not N. Better still, avoid the expiry entirely with probabilistic early recomputation: as a hot key nears its TTL, each read refreshes it with a rising probability, so one early reader repopulates it before it actually expires and the herd never forms. Serving the stale value during the refresh keeps latency flat. This is the same lease mechanism Facebook's memcache tier uses at fleet scale. Common wrong answer to avoid: "Set a longer TTL so it expires less often." That reduces frequency but not severity — when it does expire, the full herd still hits at once, and longer TTLs make staleness worse in the meantime.

Q10. Look-aside or write-through caching? Look-aside (the app reads cache, falls back to origin on miss, and populates) is the default: the cache is optional, so a cache outage degrades to slower reads rather than errors, and you cache only what's actually read. Its cost is a stale-write race — a reader can populate an old value just after a writer invalidated it — bounded by short TTLs and delete-on-write. Write-through (writes go through the cache to origin) keeps the cache always populated and consistent-on-write but couples the write path to cache availability and caches data that may never be read. Pick look-aside for read-heavy general caching; write-through when the read-after-write hit rate must be perfect and the write volume is modest. Common wrong answer to avoid: "Write-through so the cache is always consistent." It isn't free consistency — it puts the cache on the critical write path, so a cache hiccup now fails writes, and it wastes memory on write-only data.

Q11. How do you prevent split-brain when a node is partitioned rather than dead? Ownership must stay single-valued, so gate promotion on a quorum: a majority of primaries must vote a node dead before its replica is promoted. A partitioned primary on the minority side finds it cannot reach a quorum and stops serving its contested slots rather than continuing to accept writes that would diverge from the promoted replica on the majority side. This sacrifices availability on the minority partition to guarantee that exactly one node owns each slot. For a cache that's the right trade — a stalled slot self-heals from origin, but two primaries accepting conflicting writes for one slot corrupts. Common wrong answer to avoid: "Both sides keep serving and we reconcile later." Two primaries for one slot means conflicting values with no authority to reconcile them — a cache has no vector clocks or merge function, so this is silent corruption.

Q12. How much does the origin see at steady state, and why does that number drive the design? At 10M ops/second and a 99% hit ratio, misses are 1% × 10M = 100,000 QPS to the origin, which is why it's provisioned around a 150k QPS ceiling. That thin margin is the entire reason replication and coalescing exist: a single node's worth of keys going cold adds another 100k QPS, which would blow past 150k and brown out the database. Every failure-mode decision — warm replicas, single-flight, early recomputation — is ultimately about keeping that origin number flat through disruption, not about cache throughput, which was never the bottleneck. Common wrong answer to avoid: "Cache throughput is what we optimize." The cache easily does 10M ops/second; the scarce resource is origin QPS, and missing that inverts the whole design's priorities.

Deeper follow-ups

  • How would you migrate the cluster from RF=2 to RF=3 across three AZs without a cold-cache window during the transition?
  • If reads may be served from replicas to spread load, how do you bound and communicate the staleness that introduces, and when is it not worth it?
  • How would you detect a hot key automatically, on the node or in the client, and promote it to the L1 cache before it saturates its owner?
  • Redis Cluster uses 16,384 fixed slots rather than a continuous ring — what does bucketing the ring into fixed slots buy you, and what does it cost?
  • How would you resize the cluster from 100 to 150 nodes at peak while keeping key movement bounded and the hit ratio from dipping?
  • What changes if values are 100 KB instead of 2 KB — where does the bottleneck move, and how does that change node sizing and the hot-key defense?

How this round is scored

Interviewers use the distributed cache to see whether you reason about a system whose topology is always changing rather than a static hash map. The strongest early signal is naming consistent hashing and why — that modulo hashing reshuffles 99% of keys on a single node loss — because that one fact separates candidates who have operated a cache from those who have only used one. Seniority shows in the tradeoff discussions: replication for availability-not-durability, async-over-sync to protect the latency budget, LFU-over-LRU under scans, and fan-out-not-sharding for hot keys — each stated with both sides and a numeric reason. The threaded scenario is the discriminator: a candidate who computes that a dead node moves 1% of keys but threatens the origin with a 100k→200k QPS spike, and then closes that gap with warm replicas plus coalescing, is demonstrating exactly the failure-thinking the round is built to surface. Doing the miss-QPS math out loud, and using it to justify the design rather than as decoration, is what pushes an answer from "correct" to "senior."