01. WhatsApp / Messenger — High-Level Design¶

~15 min read · Part 2 of 4 (Overview → HLD → LLD → Q&A)

Architecture¶

   Alice ⇅ ┌──────────────┐        ┌──────────────┐ ⇅ Bob
           │ Gateway node │        │ Gateway node │
           │ (holds socket)│       │ (holds socket)│
           └──────┬───────┘        └──────▲───────┘
                  │ persist + route       │ push
                  ▼                       │
           ┌──────────────┐   lookup ┌──────────────┐
           │ Message svc  │─────────▶│ Session       │  user_id → gateway
           │              │          │ registry      │
           └──────┬───────┘          └──────────────┘
                  ▼
        ┌──────────────────┐   offline?   ┌──────────────┐
        │ Message store +  │─────────────▶│ Inbox queue  │ per-user, ordered
        │ per-conv sequence│              │ (undelivered)│
        └──────────────────┘              └──────────────┘
   Presence svc ── online/last-seen (soft state, TTL heartbeats)

Components¶

Gateway nodes. Hold ~100k WebSocket connections each; terminate the client protocol; persist outgoing messages and push incoming ones.

Session registry. Maps each online user_id to the gateway node holding its socket, so messages route to the right node. Soft state, updated on connect/disconnect.

Message service. Assigns server_msg_id/sequence, persists the message, and decides deliver-now (recipient online) vs enqueue (offline).

Message store + per-conversation sequence. Durable messages and the monotonic sequence per conversation that defines order.

Inbox queue. Per-user ordered queue of undelivered messages, drained on reconnect.

Presence service. Online/last-seen from heartbeats; best-effort, TTL'd.

Primary path — Alice → offline Bob → reconnect¶

Alice's client sends SEND {to: Bob, conv_id, client_msg_id, text} over her socket.
Alice's gateway hands it to the message service, which dedups by client_msg_id, assigns the next server_msg_id/sequence for the conversation, and persists it.
The server ACKs Alice (status sent) — she now knows it's durable.
Message service looks up Bob in the session registry. Bob is offline → no session → enqueue into Bob's inbox queue.
Ten minutes later Bob reconnects; his gateway registers his session and sends his last-seen sequence.
The server streams all messages after that sequence, in order, including Alice's. Bob's client stores it and returns a delivery receipt.
The receipt routes back to Alice's gateway (via her session) and her UI flips to delivered; when Bob opens the chat, a read receipt flips it to read.

Storage choices¶

Message store: durable, sharded by conversation. Messages are small and append-only per conversation; sharding by conv_id keeps a conversation's sequence on one shard for cheap ordering.
Inbox queue: durable per-user queue (a partitioned log or a queue table), so offline messages survive server restarts.
Session registry: in-memory, replicated (e.g. Redis) — soft state rebuilt on reconnect if lost.
Presence: in-memory with TTL — cheap, best-effort, not durable.

Scaling¶

Connections scale by adding gateway nodes (~100k each; 500M peak → ~5,000 nodes). The session registry scales as a sharded key-value store keyed by user. Message throughput (millions/second) scales by sharding the message store and inbox queues by conversation/user. Group messages fan out to N member inboxes; the member cap (256) bounds worst-case fan-out per message. Cross-node routing is a direct node-to-node forward using the session registry, avoiding a central broker on the hot path.

Operational signals¶

The healthy signal is end-to-end delivery latency p99 for online→online under 500 ms. The first metric to degrade under trouble is inbox-queue depth — a spike means recipients aren't draining (mass disconnect, or a gateway region down). The misleading metric is message send success: sends can succeed (persisted + acked) while delivery stalls, so track delivery separately from send. The graph an operator opens first during an incident is session-registry churn: a storm of reconnects (e.g. a mobile network blip) reshuffles millions of sessions and drives routing load — expected briefly, dangerous if sustained.

Failure modes and resilience¶

Lost ack after persist. Alice's client retransmits with the same client_msg_id; the server dedups, so no duplicate — at-least-once + dedup in action.
Recipient gateway dies mid-delivery. The message is still in the store/inbox (persist-first), so on Bob's reconnect it redelivers; nothing is lost.
Session registry stale (points to a dead node). Delivery falls back to enqueue; Bob's next reconnect drains it. Registry entries are TTL'd and rebuilt on connect.
Out-of-order arrival. The client renders by per-conversation sequence, not arrival order, so a late message slots into place.
Duplicate delivery. Client dedups by server_msg_id, so a redelivered message isn't shown twice.
Group fan-out partial failure. Per-member enqueue is independent; a member whose enqueue failed simply gets it on retry, without blocking others.

Where this shows up in production¶

WhatsApp — a small fleet of Erlang gateway servers each holding millions of connections; persist-first delivery with per-user offline queues.
Facebook Messenger / MQTT — long-lived MQTT connections for low-overhead mobile push of messages and presence.
Signal — at-least-once delivery with client-side dedup and per-conversation ordering, under end-to-end encryption.
Redis — session registry (user → gateway) and presence with TTL heartbeats.
Kafka / durable logs — per-user inbox queues as partitioned, ordered, durable logs.
Snowflake-style sequences — monotonic per-conversation message ids for ordering.
Apple/Google push (APNs/FCM) — waking an offline app so it reconnects and drains its inbox.