Skip to content

00. Design Twitter/X (Home Timeline)

~20 min read · Level: advanced · Part 1 of 4 (Overview → HLD → LLD → Q&A)

Problem

Twitter's home timeline shows a stream of recent tweets from accounts you follow, updated in near real time. It shares the fan-out DNA of a photo feed but adds three twists that change the design: tweets are tiny and high-velocity, retweets create secondary fan-out (a retweet of a popular tweet re-injects it into another whole follower set), and users expect live updates without refreshing.

Thread one scenario: a breaking-news tweet gets retweeted 1,000,000 times in 20 minutes. Each retweet is itself a fan-out event, so the naive design suffers a fan-out storm — one tweet's reach multiplies through a million retweeters' follower graphs. Containing that cascade is the heart of this study.

Functional requirements

  • Post a tweet (≤280 chars, optional media reference).
  • Home timeline: recent tweets from followed accounts, near real time.
  • Retweet and quote-tweet.
  • Follow/unfollow.
  • Like and reply (reply is a tweet with a parent reference).

De-scoped: search/trends (its own indexing problem), DMs (see chat study), and ranking/ML timeline (assume reverse-chronological with a light recency boost). Media handling mirrors the Instagram study and isn't repeated here.

Non-functional requirements

The dominant constraint is write-path fan-out amplification under retweets, on top of a read-heavy timeline.

  • Latency: timeline load < 200 ms; a new tweet visible to followers within seconds.
  • Availability: reading the timeline must stay up; posting can briefly queue.
  • Consistency: eventual — slightly stale or out-of-order-by-seconds timelines are fine.
  • Durability: tweets are permanent public record.

Scale estimation

Assume 300M daily actives reading the timeline ~15×/day → 4.5B reads/day ≈ 52,000/second average, ~180k/s peak. Tweets: ~500M/day → ~5,800 writes/second average. Read:write ≈ 9:1 — read-optimized, but writes are far higher velocity than a photo app.

The retweet cascade is the sharp edge. Our breaking tweet's 1M retweets in 20 minutes is ~830 retweets/second, and each retweet is a fan-out into that retweeter's followers. If retweeters average 500 followers, that's 830 × 500 ≈ 415,000 timeline insertions/second from one original tweet — a fan-out storm an order of magnitude above baseline write load, concentrated in minutes.

Storage: tweets are tiny (500M/day × ~300 B ≈ 150 GB/day); the timeline index (per-user id lists) and the follow graph dominate operationally, not bytes.

API sketch

POST /api/v1/tweets {text, media_id?, parent_id?}     → { tweet_id }
POST /api/v1/tweets/{id}/retweet                        → { retweet_id }
GET  /api/v1/timeline?cursor=…                          → paginated tweet refs
GET  /api/v1/tweets/{id}                                → hydrated tweet
WS   /api/v1/timeline/stream                            → live new-tweet pushes

Solutioning

Start from the same hybrid as any large feed: push tweet ids into follower timelines for normal accounts, pull for high-fan-out accounts at read time. That handles the celebrity-author case. Twitter's extra problem is the retweet, and the key insight is to represent it correctly: a retweet is not a copy of the tweet; it is a small pointer that fans out the original's id into the retweeter's followers. Storing retweets as references (not duplicated tweet bodies) means the 1M retweets are 1M tiny rows pointing at one tweet, and hydration resolves the pointer to the single original — so the timeline stores ids, and the tweet body exists once.

That still leaves the fan-out storm: 415k insertions/second from one tweet. The reframing: the storm is not a throughput problem to brute-force; it is an amplification problem to dampen. Apply the same celebrity rule to the original tweet — if a tweet is being retweeted at high velocity, stop pushing it and let it be pulled. A viral tweet becomes a "hot object" served from a dedicated cache and merged at read time, exactly like a celebrity's own posts. Retweets by normal users of a non-viral tweet fan out cheaply; retweets of a viral tweet don't fan out at all — the tweet is already going to appear via the read-time hot-object merge.

The last piece is live delivery. Rather than clients polling, maintain WebSocket connections and push new-tweet notifications to connected followers via a fan-out-to-connections layer. Offline users catch up via the normal timeline read. The tradeoff is holding millions of persistent connections (a connection-management tier), justified because polling 180k/s would be far more wasteful. The files below detail timeline assembly, the retweet model, and the live layer.