01. Live Streaming Platform — High-Level Design¶
~15 min read · Part 2 of 4 (Overview → HLD → LLD → Q&A)
Architecture¶
broadcaster ─RTMP/SRT─▶ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Ingest server │──▶│ Live transcode│──▶│ Segment │
└──────────────┘ │ (real-time) │ │ packager │
└──────────────┘ └──────┬───────┘
▼ publish seg N
┌──────────────────────┐
│ CDN / edge (warmed │◀── viewers
│ as segments produced)│ (5M)
└──────────┬───────────┘
▼ retain
┌──────────────┐
│ VOD store │ (replay later)
└──────────────┘
chat: viewers ⇅ ┌──────────────┐ pub/sub ┌──────────────┐
│ Chat gateway │◀─────────▶│ Room pub/sub │ per-channel fan-out
│ (sharded) │ │ backbone │
└──────────────┘ └──────────────┘
Components¶
Ingest server. Terminates the broadcaster's RTMP/SRT push; one active ingest per stream (with hot standby).
Live transcoder. Encodes the incoming stream into ABR renditions in real time, segment-by-segment. Latency-critical.
Segment packager. Emits short segments and maintains the rolling live manifest.
CDN / edge. Distributes live segments to viewers; warmed segment-by-segment as they're produced. Absorbs the 20 Tbps fan-out.
VOD store. Retains live segments so the stream can be replayed after it ends (reuses the OTT path).
Chat gateway (sharded). Holds viewers' chat WebSocket connections, sharded across many nodes per channel.
Room pub/sub backbone. Broadcasts each chat message to all gateway nodes serving that channel.
Primary path — video (glass to glass)¶
- Broadcaster pushes RTMP/SRT to an ingest server.
- The live transcoder produces ABR renditions in real time; the packager emits segment N (~2s) and appends it to the rolling manifest.
- Segment N is published to edges as it's produced — the edge is warmed just ahead of demand, not before the stream.
- Viewers' players fetch the rolling manifest and pull the newest segments from the nearest edge, using ABR to match their bandwidth.
- End-to-end latency = capture + encode + segment duration + edge propagation + player buffer — a few seconds, dominated by segment size and buffering.
Primary path — chat under a clutch moment¶
- Each viewer connects to a chat gateway shard for the channel (many shards per big channel).
- A sent message goes to the gateway, which publishes it to the room pub/sub backbone.
- The backbone delivers the message to every gateway node serving that channel; each node pushes it to its connected viewers.
- Under extreme load (50k msgs/s), per-user rate limits (slow mode) and message sampling/batching keep the delivered stream legible and the fan-out bounded — the system delivers a readable subset, not all 50k/s to all 5M.
Storage choices¶
- Live segments: short-lived blob + edge cache, rolling window; only the recent segments matter for live.
- VOD retention: durable blob storage for replay after the stream ends.
- Chat: mostly transient (in-flight via pub/sub); optionally a short buffer for late-joiners and a durable log for moderation/replay.
- Channel/stream metadata: cached store (title, viewer count, live status).
Scaling¶
Video fan-out scales on the CDN exactly like VOD — the 20 Tbps rides thousands of edges — but with live warming per segment rather than pre-positioning. Ingest is trivial (one source per stream). Transcoding scales per active stream (each stream needs its own real-time encoder). Chat scales by sharding connections across gateway nodes and using pub/sub for room broadcast; the hard limit isn't connections but the N×M delivery product, capped by rate-limiting and sampling. Concurrent-viewer count is itself a hot metric aggregated approximately (see failure modes).
Operational signals¶
The healthy signal is glass-to-glass latency holding in the low-seconds band and rebuffer ratio near zero. The first metric to degrade is transcoder/ingest processing lag — if the encoder can't keep real-time, the live edge falls behind and latency grows unbounded, which is worse than a VOD hiccup because there's no catching up. The misleading metric is average viewer bitrate (masks a struggling network subset). The graph an operator watches during the final is segment publish cadence: segments must appear every ~2s like a metronome; a stutter there means the real-time pipeline is losing the race and latency is about to balloon.
Failure modes and resilience¶
- Encoder can't keep up (falls behind real-time). Latency grows and never recovers within the stream. Mitigation: provision transcoders with headroom, drop to fewer renditions under pressure, and shed the highest resolution before falling behind.
- Ingest server failure. The stream drops. Mitigation: hot-standby ingest and broadcaster-side reconnect with a short buffer to bridge the gap.
- Cold edge for a just-produced segment. Inherent to live. Mitigation: push segments to edges proactively as produced; request coalescing so one origin fetch per edge serves the herd.
- Chat fan-out overload. 50k msgs/s × millions of viewers. Mitigation: per-user rate limits, sampling/batching of the displayed scroll, and room sharding; deliver a legible subset.
- Viewer-count hot counter. Millions joining at once. Mitigation: approximate count via sampling/aggregation, not an exact per-join increment.
- Thundering join at stream start. Mitigation: stagger manifest fetches and rely on edge fan-out; the join is a read, so the CDN absorbs it.
Where this shows up in production¶
- Twitch — low-latency HLS with ~2s segments, sharded IRC-derived chat scaling to hundreds of thousands per channel, and slow-mode rate limiting on huge channels.
- YouTube Live — real-time transcode + DASH/HLS with a DVR window and post-stream VOD.
- WebRTC / LL-HLS — the sub-second interactive path versus chunked HLS's few-second latency (the LLD's fork).
- RTMP / SRT — broadcaster ingest protocols feeding the transcode pipeline.
- Redis pub/sub — per-channel chat message broadcast to gateway shards.
- Akamai / Cloudflare live — edge distribution of live segments warmed as produced.
- Cloudflare Stream / AWS IVS — managed live-ingest-to-edge pipelines packaging this whole flow.