01. Ticketing System — High-Level Design¶

~15 min read · Part 2 of 4 (Overview → HLD → LLD → Q&A)

This file turns the three rails from the overview — queue, reservation, payment — into concrete boxes and the flows between them. Read the architecture top to bottom, follow a fan through a hold and a checkout, then look at what breaks when 100,000 of them arrive at once.

Architecture¶

                           ┌──────────────┐
          client ────────▶ │  Edge / CDN  │  (static seat-map assets, JS)
                           └──────┬───────┘
                                  │
                                  ▼
                           ┌──────────────┐
                           │ API Gateway   │  (auth, rate-limit per IP/account)
                           └──────┬───────┘
                                  │
                 ┌────────────────┼─────────────────┐
                 ▼                ▼                 ▼
         ┌──────────────┐  ┌──────────────┐  ┌──────────────┐
         │ Waiting-Room │  │  Reservation  │  │   Booking /   │
         │   service    │  │    service    │  │   Checkout    │
         │ (queue+admit)│  │ (holds, CAS)  │  │   service     │
         └──────┬───────┘  └──────┬───────┘  └──────┬───────┘
                │                 │                 │
                ▼                 ▼                 ▼
        ┌──────────────┐  ┌──────────────┐  ┌──────────────┐
        │ Queue store   │  │  Seat        │  │  Booking DB   │
        │ (Redis sorted │  │  Inventory   │  │  (RDBMS,      │
        │  set + token) │  │  (RDBMS row  │  │  bookings +   │
        │               │  │  per seat +  │  │  payments)    │
        │               │  │  Redis hold  │  │               │
        │               │  │  leases TTL) │  │               │
        └──────────────┘  └──────┬───────┘  └──────┬───────┘
                                 │                 │
                                 ▼                 ▼
                          ┌──────────────┐  ┌──────────────┐
                          │  Seat-state  │  │   Payment     │
                          │  push (SSE / │  │   gateway     │
                          │  WebSocket)  │  │  (external)   │
                          └──────────────┘  └──────────────┘
                                 │
                                 ▼
                          ┌──────────────┐
                          │  Event bus    │──▶ analytics, notifications,
                          │  (Kafka)      │    ticket issuance, expiry reaper
                          └──────────────┘

Read it as three vertical lanes fed by one gateway. A fan lands on cached static assets from the edge, then every stateful call passes the API gateway, which authenticates and rate-limits before anything expensive happens. The left lane is the waiting room: it owns the queue store and decides who is admitted. Only an admitted fan reaches the middle lane, the reservation service, which is the guardian of the seat inventory and the only component allowed to change a seat's state. The right lane is checkout, which turns a valid hold into a paid booking by calling the external payment gateway and writing the booking DB. Seat-state changes are pushed live to browsers so every shopper sees seats gray out in near real time, and an event bus carries the slow, off-path work — ticket PDFs, emails, analytics, and the reaper that releases expired holds.

Components¶

API Gateway. Authenticates the fan, applies per-account and per-IP rate limits, and — critically for an on-sale — is the first place a bot storm is thinned before it can reach the queue. It also routes: queue calls to the waiting room, seat and hold calls to reservation, checkout to booking.

Waiting-Room service. Accepts every join instantly (a cheap write to the queue store), assigns an arrival-ordered position, and admits fans into the active shopping pool at a controlled rate. It issues a short-lived session token on admission that the reservation service trusts. Its whole reason to exist is backpressure and fairness: it converts the 10,000/s join burst into a stream the seat engine can survive, and it decides that stream in roughly arrival order.

Reservation service. The heart of the system and the only writer to seat inventory. It performs the atomic compare-and-swap that flips a seat available → held, records a hold with a TTL, and later promotes or releases it. Every double-sell defense lives here. It is deliberately narrow — it does one contended thing correctly — and it is sharded by event so one hot on-sale cannot starve another.

Booking / Checkout service. Takes a valid hold, calls the payment gateway, and on success writes a durable booking and flips the seats held → sold. It is where money and inventory meet, so it is idempotent by construction: a retried checkout with the same idempotency key produces the same booking, never a second charge or a second seat.

Queue store. A Redis sorted set keyed by event, scored by arrival timestamp, plus a token→position lookup. Sorted sets give O(log n) insert and instant rank queries, so a fan can be told "you are 41,204" cheaply, and admission is just popping the lowest scores.

Seat Inventory. The system of record for seat state: one row per seat in a relational store (for the durable, strongly-consistent sold truth) fronted by Redis for the hot, high-churn hold leases with native TTL. The relational row is the arbiter of ownership; Redis makes the fast-expiring hold layer cheap.

Booking DB. A relational store holding confirmed bookings and payment records, with the seat-uniqueness constraint enforced here as the final backstop: a unique index on (event_id, seat_id) in the sold-bookings table means the database itself refuses a second sale of one seat even if every layer above failed.

Seat-state push. An SSE or WebSocket fan-out that streams seat-state deltas to admitted shoppers so the map updates live without polling. This is what turns 5,000 reads/second of polling into ~40 pushed deltas/second.

Event bus + reaper. Kafka (or similar) carries asynchronous work off the hot path: issuing tickets, sending confirmations, updating analytics, and running the hold reaper that releases leases whose TTL has passed so the seats return to the pool.

Primary write path (hold then buy)¶

The fan joins the queue: POST /queue writes their token into the sorted set at the current timestamp and returns a position. This call is cheap and always succeeds — it never touches inventory.
When the waiting room admits them, it issues a session token with its own short expiry and the fan's browser opens the seat map, receiving live state via the push channel.
The fan selects seats and calls POST /holds. The reservation service runs an atomic CAS per seat: available → held guarded by the seat's version. Seats that flip are returned as a hold with an 8-minute TTL; any seat that lost its CAS comes back in conflict_seats, and the fan picks again.
On a successful hold, the reservation service writes the hold lease (Redis, TTL 8 min) and emits a seat-state delta so every other shopper sees those seats gray out immediately.
The fan calls POST /holds/{id}/checkout with a payment token and an idempotency key. Checkout verifies the hold is still live, charges via the payment gateway, and on success writes the booking and flips the seats held → sold — the unique index on (event_id, seat_id) guaranteeing the sale is singular even under a retry.
Tickets, email, and analytics are emitted to the event bus; the fan gets 201 with their tickets. If checkout is slow, only that fan waits — the seats are already off the market via their hold.

Primary read path (browse the seat map)¶

Static assets (the venue layout, seat SVG, client JS) come from the CDN and never hit origin.
On admission, the client fetches the current seat map once from the reservation service (GET /seats), a single read of one event's compact inventory.
From then on, the client holds an SSE/WebSocket connection and receives only deltas — seat A1 went held, seat B4 went sold — so the map stays live without repeated full reads. This replaces per-client polling with a shared broadcast: the reservation service publishes a delta once per state change, and the push tier fans it out to all admitted connections.
A fan who is still in the queue (not yet admitted) reads nothing from inventory at all; they only poll their queue position, which is served from the queue store. This is the key property that keeps 90,000 waiting fans from generating any load on the contended seat engine.

Storage choices¶

Seat ownership (the sold truth): relational store, one row per seat. The invariant "sold at most once" wants a hard, transactional uniqueness constraint, and a relational unique index on (event_id, seat_id) provides exactly that as an unforgeable backstop. The access pattern is a point update by seat key inside a transaction — no joins on the hot path — so a well-indexed relational row per seat, sharded by event, fits.
Hold leases: Redis with native TTL. Holds are high-churn (created, expired, re-created thousands of times during a sale) and inherently time-boxed. Redis SET key val EX 480 NX gives atomic acquire-if-absent plus automatic expiry, which is precisely a lease. Putting the fast-expiring layer in Redis keeps that churn off the durable store while the durable row stays the arbiter of the final sold state.
Queue: Redis sorted set. Arrival-ordered admission is a ranked set; sorted sets give cheap position lookups and range pops. It is not durable-critical — losing the exact queue on a crash is recoverable by re-queuing, far less costly than losing a booking.
Bookings and payments: relational store. Money demands ACID, foreign keys, and auditability. Bookings and payment records live in a relational DB with the seat-uniqueness backstop and transactional coupling between "charged" and "seats sold."
Analytics and tickets: off the event bus. Append-heavy, aggregation-queried work goes to a separate pipeline so it never contends with the reservation hot path.

Scaling¶

Sharding by event. Each event's inventory is small (~1.5 MB for 10,000 seats) but hot for a few minutes. Sharding by event_id means one on-sale's contention is isolated to one shard; a stampede for Concert A cannot degrade the seat engine for Movie B. To scale across many simultaneous on-sales, add shards and spread events across them. A single mega-event that outgrows one shard is partitioned by seat section (block 100, block 200), since sections never contend with each other.

Absorbing the burst. The waiting room is the load shock absorber. The 10,000 join-requests/second in the opening ten seconds hit only the queue store — a cheap sorted-set insert — and the reservation service sees the admission-controlled stream instead, capped so active shoppers stay near 1.5× remaining seats. That converts a 10,000/s inventory storm into roughly a 2,000/s opening burst of hold attempts and a ~100/s steady rate thereafter, numbers a single sharded reservation instance handles comfortably.

Hot-seat contention. The aggregate rate is fine; the danger is one seat. Front-row A1 may draw several thousand CAS attempts in a second, all serialized on one row. This is handled not by spreading load (you cannot spread a single row) but by making each attempt cheap and non-blocking: the CAS is a single conditional update that either wins or fails in microseconds, and losers get an instant 409 rather than queuing behind a lock. Optimistic concurrency turns a would-be lock convoy into a flurry of fast, independent yes/no answers.

Read scaling. Trivial once you push instead of poll. The full seat-map read happens once per admitted session; everything after is a shared delta broadcast. Scaling the push tier is adding fan-out nodes, independent of the write path.

Operational signals¶

The healthy signal is the hold-success-to-attempt ratio holding steady and the seat-inventory counter draining monotonically toward zero — seats going available → held → sold and never backward except through honest TTL expiry. The first metric to degrade under trouble is checkout latency, usually because the external payment gateway is slowing down; when it climbs, holds sit longer, conversion drops, and the sale drains slower even though inventory is fine. The misleading metric is the raw join/request rate at the gateway — it spikes to 10,000/s at 10:00:00 and looks alarming, but that is the queue doing its job absorbing the burst; watching it panics you about a number that is supposed to be huge. The graph an experienced operator opens first is hold-attempt rate against the reservation service versus its CAS-conflict rate: a healthy sale shows a modest attempt rate with conflicts concentrated on a few hot seats, while a conflict rate spiking across all seats means the waiting room is admitting too fast and manufacturing contention — the signal to tighten admission.

Failure modes and resilience¶

The on-sale burst itself (the threaded scenario). 100,000 fans, 10,000/s of joins, all in ten seconds. Without the waiting room this stampedes the seat engine and hot seats simultaneously; the store browns out and CAS retries pile up exactly when load is worst. The waiting room is the resilience mechanism: it accepts the burst into a cheap queue and meters admission so the reservation service never sees more than it can serve. When inventory hits zero, remaining queued fans are admitted directly to a sold-out response — the queue drains as a clean tail, not a crash.
Client abandons a hold (crash, closed tab). The seats would be stranded if holds were locks. Because a hold is a TTL lease, the reaper releases it at expiry (8 minutes) and the seats return to the pool automatically. No human intervention, no stuck inventory.
Payment gateway slow or down. Checkout latency rises and conversions stall, but no seat is oversold — unpaid holds simply expire and recycle. Mitigation: a payment timeout shorter than the hold TTL so a hung payment releases the seat rather than letting it sit dead, plus a circuit breaker that fails checkout fast and keeps the hold intact for a retry.
Reservation service or shard crash mid-sale. Redis hold leases may be lost, but the durable sold rows survive, so nothing sold is forgotten; lost holds simply expire into availability, which is the safe direction. On restart the seat map is rebuilt from the durable rows (sold) plus surviving leases (held), and unknown seats default to available — over-release is recoverable, over-sell is not.
Double-submit / network retry on checkout. A fan double-clicks buy, or a timeout triggers a client retry. The idempotency key collapses both into one booking, and the unique index on (event_id, seat_id) refuses a second sale even if two checkouts race. Two failures must both breach for a double-sell, and the DB constraint is the last line that cannot be bypassed.
Queue store loss. The sorted set is not durable-critical. If it is lost, fans re-queue; positions reset, which is unfair but not incorrect — no seat is affected. This is the one place we deliberately chose recoverable-but-lossy over durable, because the queue is a fairness mechanism, not a system of record.

Where this shows up in production¶

Ticketmaster — its "Smart Queue" waiting room admits fans into the purchase flow in arrival order precisely to convert an on-sale stampede into a metered, fair stream, the same role the waiting-room service plays here.
BookMyShow — holds selected seats for a fixed countdown during checkout and releases them on timeout, the canonical hold-then-pay TTL lease.
Amazon / Shopify flash sales — decrement inventory with atomic conditional writes so a limited-stock item is never oversold under a thundering-herd checkout, the same CAS-guards-the-unit pattern as a seat.
Airline seat maps (Amadeus, Sabre) — reserve a seat with a short hold while the fare is confirmed, then commit, showing the same reserve→confirm split under a hard no-double-book rule.
Redis SET NX EX — the primitive behind the hold lease: atomic acquire-if-absent with automatic expiry, so an abandoned hold self-heals without a reaper race.
Shopify checkout idempotency keys — a retried purchase produces one order, not two charges, exactly the idempotent-checkout guarantee that keeps a double-click from double-selling.
Kafka in order pipelines — decouples the fast commit from slow downstream work (ticket issuance, email, analytics) so a slow consumer never delays the sale, mirroring the event bus here.
Cloudflare Waiting Room — an edge-level virtual queue that holds excess visitors before they reach origin during a spike, the productized form of the backpressure the waiting room provides.