Skip to content

01. Video Streaming / OTT — High-Level Design

~16 min read · Part 2 of 4 (Overview → HLD → LLD → Q&A)

Architecture

   master ─▶ ┌──────────────┐   ┌──────────────┐   ┌──────────────┐
             │ Ingest svc   │──▶│ Segmenter    │──▶│ Transcode    │ (parallel
             └──────────────┘   │ (split)      │   │ worker fleet │  per chunk)
                                └──────────────┘   └──────┬───────┘
                                                          ▼
                                              ┌──────────────────────┐
                                              │ Blob store (segments  │
                                              │ + manifests), tiered  │
                                              └──────────┬───────────┘
                                                         │ pre-warm popular titles
                                                         ▼
   viewer ── manifest ─▶ ┌──────────────┐        ┌──────────────┐
          ── segments ─▶ │ Playback API │        │  CDN / edge  │◀── segment fills
                         │ + metadata   │        │ (ISP caches) │      from origin
                         └──────┬───────┘        └──────────────┘
                                ▼
                         ┌──────────────┐
                         │ Catalog + user│  progress, entitlements
                         │ state store   │
                         └──────────────┘

Components

Ingest service. Accepts masters, validates, kicks off the transcode job.

Segmenter. Splits the master into short segments so encoding parallelizes per chunk.

Transcode worker fleet. Encodes each segment into each rendition (resolution × bitrate × codec); horizontally scaled, idempotent per segment.

Blob store (tiered). Immutable segments + manifests. Hot titles on fast storage, long-tail on cold/cheaper tiers.

CDN / edge. Serves segments to viewers; the only tier that touches the 100 Tbps. Popular titles pre-warmed before big releases.

Playback API + metadata. Serves manifests, entitlements (is this viewer allowed?), and DRM license hooks. Stateless.

Catalog + user-state store. Titles, thumbnails, and per-user resume positions/watch history.

Primary path — ingest (offline)

  1. POST /ingest with the master; ingest service creates a transcode job.
  2. The segmenter splits the master into N short segments.
  3. The worker fleet encodes each of the N segments into every rendition in parallel — a 1-hour title's encode is bounded by the slowest chunk, not the full runtime.
  4. Segments and the ABR manifest (listing renditions and segment URLs) are written to blob storage, immutable.
  5. For an anticipated hit, the title is pre-warmed to edge caches ahead of release.

Primary path — playback (online)

  1. The player requests the manifest from the playback API (after an entitlement check).
  2. The manifest lists available renditions (e.g. 240p…4K) and the segment URLs for each.
  3. The player estimates bandwidth, picks a starting rendition, and fetches segment 0 from the nearest edge.
  4. As playback proceeds, the player measures throughput and buffer, and selects each next segment's rendition — dropping quality when bandwidth dips, raising it when it recovers (ABR).
  5. Edges serve segments from cache; only a cache miss triggers an origin fill. For the pre-warmed 8pm release, misses are rare and origin stays quiet.
  6. Playback position is periodically POSTed to user-state for resume.

Storage choices

  • Segments + manifests: immutable blob storage, tiered. Write-once, read-through-CDN; hot/cold tiering matches the long-tail access pattern (a few titles are watched constantly, most rarely).
  • Catalog/metadata: cached relational or document store, read-heavy, eventually consistent.
  • User state (resume, history): key-value by user, small writes, tolerant of seconds of lag.
  • Masters: cold, durable storage — needed only to re-transcode, so cheap and slow is fine.

Scaling

Delivery scales entirely on the CDN: the 100 Tbps peak is served from thousands of edge locations (and ISP-embedded appliances), so aggregate egress scales with edge footprint, not origin. Origin scales trivially because it serves only first-fills. Transcoding scales on the worker fleet and chunk parallelism — more titles or faster turnaround just means more workers. The playback API is stateless and scales horizontally. Pre-warming converts a thundering-herd release into cache hits: instead of 10M viewers racing to fill cold edges, the content is already there.

Operational signals

The healthy signal is rebuffer ratio near zero and startup latency p95 under ~2s — these are what viewers feel. The first metric to degrade during a big release is origin egress / cache-miss rate: a spike means edges are cold (pre-warm failed or a new title wasn't positioned), and origin is being asked to do the CDN's job. The misleading metric is average bitrate served: it can look high while a subset of viewers on poor networks are silently rebuffering — segment by network/region. The graph an operator opens first is cache-hit ratio at the edge per title: for the 8pm release it should be ~99%+; anything lower means the herd is hitting origin.

Failure modes and resilience

  • Cold edges on release (pre-warm missed). Origin egress spikes and startup latency climbs. Mitigation: verify pre-warm completion before release; rate-limit origin fills with request coalescing so one fill serves many viewers per edge.
  • Bandwidth collapse mid-stream. ABR drops the viewer to a lower rendition rather than buffering; the client's buffer absorbs transient dips.
  • Transcode worker failure. A chunk fails to encode; idempotent retry re-encodes just that chunk, not the whole title.
  • A rendition missing from the manifest. The player falls back to the nearest available rendition rather than failing playback.
  • Region/edge outage. CDN reroutes viewers to the next-nearest edge; playback continues at slightly higher latency.
  • Popular long-tail title on cold storage. A sudden spike on an archived title. Mitigation: promote it to a hot tier on access, accept a slower first few streams.

Where this shows up in production

  • Netflix Open Connect — ISP-embedded CDN appliances pre-positioned with popular titles, exactly the pre-warm-the-edge model.
  • YouTube — chunked parallel transcoding of uploads into many renditions and codecs (VP9/AV1/H.264).
  • HLS (Apple) / MPEG-DASH — the manifest + segment ABR protocols the player drives.
  • AWS S3 + CloudFront / Akamai — immutable segment storage fronted by global edge delivery.
  • AWS Elemental / FFmpeg fleets — the transcoding worker layer.
  • Widevine / FairPlay / PlayReady — per-title DRM wrapping of segments.
  • S3 storage classes / Glacier — hot/cold tiering for hot titles vs archived long-tail masters.