01. Food Delivery Service — High-Level Design¶

~16 min read · Part 2 of 4 (Overview → HLD → LLD → Q&A)

This file turns the two-rail narrative from the overview into concrete boxes and the flows between them. Read the architecture top to bottom, then follow an order through placement and dispatch, then follow a courier's location out to a customer's map, then look at what fails during the rush.

Architecture¶

   customer app        courier app        restaurant app
        │                   │                    │
        │ orders/track      │ GPS pings          │ accept / prep-time
        ▼                   ▼                    ▼
   ┌───────────────────────────────────────────────────┐
   │              API Gateway / Load Balancer            │
   └───┬───────────────┬───────────────┬────────────┬───┘
       │               │               │            │
       ▼               ▼               ▼            ▼
 ┌───────────┐  ┌──────────────┐  ┌──────────┐  ┌──────────────┐
 │  Order    │  │  Location    │  │Restaurant│  │  Tracking    │
 │  Service  │  │  Ingestion   │  │ Service  │  │  Service     │
 │ (state    │  │  Service     │  │(accept,  │  │ (WS fanout   │
 │  machine) │  │              │  │ prep ETA)│  │  to clients) │
 └────┬──────┘  └──────┬───────┘  └────┬─────┘  └──────▲───────┘
      │                │               │              │
      │          ┌─────▼──────┐        │              │
      │          │ Geospatial │        │              │
      │          │ Index      │◀───────┼──────────────┤ (courier positions)
      │          │ (Redis GEO │        │              │
      │          │  / H3 cells)        │              │
      │          └─────┬──────┘        │              │
      │                │               │              │
      ▼                ▼               ▼              │
 ┌──────────┐   ┌──────────────────────────┐         │
 │ Orders   │   │   Dispatch Service        │─────────┘
 │ DB       │◀──│  (windowed batch matcher) │  assignments
 │(sharded  │   │      ▲          │         │
 │ by city) │   │      │          ▼         │
 └────┬─────┘   │  ┌────────┐ ┌────────────┐│
      │         │  │  ETA   │ │ Travel-time ││
      │         │  │ Service│ │ matrix cache││
      │         │  └────────┘ └────────────┘│
      │         └──────────────┬────────────┘
      ▼                        ▼
 ┌──────────────────────────────────────────┐
 │  Event Bus (Kafka) → analytics / ML store │
 └──────────────────────────────────────────┘

Reading it top to bottom: three client types hit one gateway, which fans requests to the service that owns each concern. The Order Service runs the durable order lifecycle against the Orders DB. The Location Ingestion Service absorbs the GPS firehose and writes courier positions into the in-memory Geospatial Index, which is the shared "where is everyone" map that both dispatch and tracking read. The Dispatch Service is the heart: on a timer it pulls dispatchable orders, queries the geospatial index for nearby couriers, asks the ETA Service (backed by the travel-time matrix cache) to score candidate trips, solves a batch assignment, and commits courier→order assignments back to the Order Service. The Tracking Service holds the customer websockets and pushes each order's courier position and current ETA. Everything of lasting interest is mirrored to the Event Bus for analytics and for training the ETA and dispatch models. The load-bearing idea is the split: order state on a transactional store, live world on an in-memory index, joined only at the dispatch tick.

Components¶

API Gateway / Load Balancer. Terminates TLS, authenticates the three client types, and routes to services. It separates the high-frequency, low-value GPS ping traffic (courier app) from the transactional order traffic so a ping flood cannot starve order placement.

Order Service. Owns the order lifecycle as an explicit state machine: CREATED → CONFIRMED → ASSIGNED → PICKED_UP → DELIVERED (plus CANCELLED/REJECTED). Every transition is a durable write with a guard, so an order cannot skip from CREATED straight to PICKED_UP. It is the source of truth for "what is happening to this order," and it emits lifecycle events to the bus.

Restaurant Service. Handles the merchant side: pushing new orders to the restaurant's tablet, recording accept/reject, and — the part dispatch depends on — capturing a prep-time estimate that predicts when the food will be ready. Dispatch times courier arrival to that estimate, so a bad prep estimate is a courier idling at a counter or food sitting cold.

Location Ingestion Service. Absorbs ~3,500 pings/second at peak, validates them, and updates the geospatial index and the tracking fanout. It is deliberately thin and stateless so it can be scaled horizontally against the ping rate, and it drops or downsamples pings under overload rather than backing up.

Geospatial Index. An in-memory structure (Redis GEO, or an H3/quadtree cell map) answering one hot query: "which couriers are within radius R of this point, and what is each one's state?" Both dispatch (to find candidates) and supply-monitoring read it. It holds soft state — a lost index rebuilds from the next round of pings within seconds.

Dispatch Service. The windowed batch matcher. It does not react to each order individually; it collects a window of dispatchable orders, builds a candidate set of couriers per order from the geospatial index, scores trips (including batched trips) via the ETA service, solves an assignment that minimizes total food-in-transit time and courier miles, and commits. Details in the LLD.

ETA Service + travel-time matrix cache. Produces the estimated delivery time for an order and scores candidate trips for dispatch. It reduces the expensive per-order routing computation to lookups against a cell-to-cell travel-time matrix refreshed each minute with live traffic, plus the kitchen's prep estimate.

Tracking Service. Holds ~33,000 open websockets at peak and pushes each subscribed customer their courier's position and refreshed ETA every few seconds. It reads courier positions from the same geospatial index the ingestion service fills, so tracking and dispatch always see one consistent picture of the fleet.

Event Bus + analytics/ML store. A durable log (Kafka) decouples the operational path from analytics and model training. Order transitions, assignments, and downsampled location traces flow here; the dispatch and ETA models are trained offline on this history.

Primary write path (place an order and dispatch it)¶

POST /orders reaches the Order Service through the gateway. It writes an order row in state CREATED (guarded by the client's idempotency_key so a retry does not create a second order) and returns an initial ETA from the ETA service.
The Restaurant Service pushes the order to the merchant tablet. The restaurant accepts and enters a prep-time estimate; the order moves to CONFIRMED with a predicted ready_at.
The order becomes dispatchable when its ready_at is close enough that a courier dispatched now would arrive as the food is ready. It joins the dispatch pool for its geographic cell.
On its next tick (every ~15 s), the Dispatch Service pulls the pool, queries the Geospatial Index for candidate couriers near each restaurant, scores single and batched trips with the ETA Service, and solves the assignment. It commits {courier_id → [order_ids]} back to the Order Service, which transitions the affected orders to ASSIGNED — atomically claiming each courier so no other tick can take them.
The courier app receives the offer, accepts, drives to the restaurant, and picks up (PICKED_UP); the app's GPS pings now drive the customer's live map. On drop-off the courier marks DELIVERED. Each transition is a durable write and a bus event.

Primary read path (live tracking)¶

The courier app posts a GPS ping every ~4 s to the Location Ingestion Service, which updates the courier's position in the Geospatial Index and forwards it to the Tracking Service.
The customer app opens WS /orders/{id}/track. The Tracking Service subscribes that socket to the order's assigned courier.
Every ~5 s the Tracking Service pushes the customer the courier's latest position plus a refreshed ETA — the ETA service recomputes from the courier's current cell to the drop-off cell using the travel-time matrix, so the number moves as traffic and progress change.
Because tracking reads the same geospatial index that dispatch reads, the customer's dot and the dispatcher's view of that courier never disagree. If a ping is dropped under load, the customer sees the last known position for a few seconds rather than a gap — stale-but-moving beats frozen-then-jumping.

Storage choices¶

Orders: sharded relational store, partitioned by city. The order lifecycle is transactional (state transitions with guards, payment coupling) and queried by order_id, by courier_id, and by "active orders in this city." A relational store gives the transactions and secondary indexes; sharding by city keeps each shard's working set to one metro and matches the fact that dispatch is inherently local — no cross-city query exists.
Courier positions: in-memory geospatial index (Redis GEO / H3). Chosen for sub-millisecond radius queries and constant overwrite by the ping stream. It is not durable and does not need to be — a courier's position a minute ago is worthless, and the index self-heals from the next pings. Retention is a short trail (last few minutes) for tracking smoothing.
Travel-time matrix: in-memory cache, refreshed per minute. Cell-to-cell travel times under live traffic. Small (thousands of cells → a bounded matrix), read constantly by ETA scoring, rewritten every minute — a cache, not a system of record.
Analytics / ML training: append-optimized store behind the bus. Order histories and downsampled location traces are append-heavy and queried by aggregation for model training and ops dashboards. A columnar/time-series store fits, and keeping it off the operational path protects dispatch and tracking latency.

Scaling¶

Location ingestion and tracking. Both scale with courier and order count, and both scale horizontally: partition couriers and orders by geographic cell, and run ingestion and tracking nodes per cell-group. At 3,500 pings/s and 6,600 pushes/s in one city these are small; the load that grows is connections, so the tracking tier is sized by concurrent websockets (~33,000 here), not by bandwidth. Add cities and each is an independent shard.

Dispatch. The optimizer's cost is combinatorial in the number of orders × candidate couriers considered together, so the key scaling move is geographic partitioning of the optimization: solve each H3 region independently rather than the whole city at once. A citywide solve of 210 orders/tick against thousands of couriers is intractable and pointless — a courier in the north cannot serve a drop-off in the south. Partitioned, each cell's solve is a small matrix (tens of orders × tens of couriers) that finishes in milliseconds, and cells run in parallel. Cell boundaries overlap slightly so a courier near an edge can serve either side.

Number movement. Moving from single to batched dispatch is the concrete lever: at 50,000 orders/hour, single dispatch needs ~20,000 couriers on the road; batching at 1.4 orders/trip needs ~14,000 — a 30% supply cut, at the cost of ~5–6 minutes of extra food-in-transit time on each batched order. Tightening the food-hot budget from +8 to +4 minutes protects meal quality but shrinks the batchable fraction, pushing the fleet requirement back toward 17,000. That dial — batch aggressiveness versus meal quality — is the one an ops team actually turns during a rush.

Operational signals¶

The healthy signal is time-to-assign — the gap between an order becoming dispatchable and a courier accepting it — which should sit at a few seconds and barely move as the rush builds; a rush that keeps time-to-assign flat is the fleet absorbing demand as designed. The first metric to degrade under trouble is unassigned-order age: when the oldest order waiting in the dispatch pool climbs from seconds toward minutes, the fleet is running short in some cell and food is already getting cold before anyone picks it up. The misleading metric is average ETA accuracy — it stays comfortable because the many easy single-order deliveries are dead-on, while it hides the batched-and-delayed tail where the actual damage is; watch p90 ETA error, not the mean. The graph an experienced operator opens first during a rush is the supply–demand ratio per cell — available couriers versus unassigned orders in each H3 region — because that single map tells you where the fleet is short before customers start seeing broken ETAs, and it is the input to every mitigation (surge, incentives, throttling) you might reach for.

Failure modes and resilience¶

Dispatch service degraded during the rush. If the batch optimizer falls behind or crashes at 7pm, orders pile up unassigned and every meal in that window gets colder by the second — the threaded scenario's nightmare. Mitigation: a fallback to greedy single dispatch — drop batching and the global solve, and assign each dispatchable order to its nearest free courier immediately. This serves worse (needs the full ~20,000-courier fleet, no batching savings) but keeps orders moving; batching resumes when the optimizer recovers. The degrade is graceful and explicit, not a stall.
Geospatial index outage. Dispatch and tracking lose the fleet's positions. Mitigation: the index is replicated, and because it self-heals from the ping stream it rebuilds within a few seconds of pings resuming; during the gap, dispatch falls back to each courier's last-known position with a widened radius, accepting slightly worse matches over no matches.
Location ingestion overload. At peak the ping firehose can exceed a node's capacity. Mitigation: backpressure by downsampling — instruct courier apps to ping every 8 s instead of 4 s under load, halving the rate at the cost of coarser tracking, rather than letting the queue back up and go minutes stale.
Restaurant never accepts / bad prep estimate. An unaccepted order times out and is re-routed or cancelled with a refund; a chronically optimistic prep estimate is corrected by learning per-restaurant prep-time models so dispatch stops sending couriers to idle at the counter.
Supply shortage (demand > couriers). No matching fixes an empty fleet. Mitigation, per the overview: lengthen promised ETAs honestly, raise courier incentives to pull supply onto the road, and throttle new-order promises in the worst-hit cells — degrade the promise, not the pipeline.
Tracking fanout overload. If the websocket tier saturates, degrade specific clients to polling (GET /orders/{id} every ~10 s) instead of pushing, trading map smoothness for surviving the connection load.

Where this shows up in production¶

DoorDash — runs a windowed batch assignment optimizer ("DeepRed") that re-solves courier-to-order matching every few seconds rather than greedily on each order, exactly the batch-versus-single tradeoff at fleet scale.
Uber / Uber Eats (H3) — the open-source H3 hexagonal grid is the geospatial partitioning that makes "couriers near this restaurant" a bounded cell query and lets the dispatch optimization shard by region.
Swiggy / Zomato — per-restaurant prep-time prediction feeding dispatch timing, so couriers arrive as food is ready rather than idling or letting it cool.
Instacart — batching multiple shopper orders onto one trip, the same incremental-delay-versus-supply-savings bet made for groceries.
Google Maps / OSRM / Valhalla — the routing engines behind the travel-time matrix; the platform calls them to refresh the cell-to-cell matrix, not per order.
Redis GEO — the in-memory radius query for nearby couriers, overwritten continuously by the ping stream, used as a self-healing soft-state index.
Kafka — the lifecycle event backbone decoupling operational dispatch/tracking from analytics and offline model training.
Lyft / Uber ETA models — learned arrival-time prediction that blends live traffic with historical patterns, the ML behind an ETA that stays honest as conditions change.