Scope the requirements
This overlaps the news feed deliberately — interviewers expect the fan-out core stated quickly and confidently, and then they push on what Twitter adds: time-ordered IDs without coordination, search over immutable tweets, trending topics, and like counts on a tweet the whole planet is tapping.
Functional — what it must do
- Post tweets — 280 characters, optional media.
- Follow users; a home timeline of followees' tweets, reverse-chronological.
- Like and retweet, with visible counts.
- Search tweets by keyword; trending topics as a stretch goal I'll plan for.
- A user timeline (one profile's own tweets) — distinct from the home timeline, and I say so early.
- Explicitly cut: DMs, notifications, ads, ML ranking.
Non-functional — the numbers that shape the design
- Home timeline p99 under 500 ms; a tweet is visible to followers within ~5 seconds — Twitter's real production target.
- Availability over consistency — eventual is fine everywhere, and like counts may be approximate.
- Brutally read-skewed: real Twitter runs roughly 300k timeline reads/s against ~6k tweet writes/s — a 50:1 skew the architecture must reflect.
- Scale target: 200M DAU, 100M tweets/day.
- Tweets are immutable once posted — an assumption I'll exploit for search and caching.
I'll state the constraint that shapes everything up front: this system is 50 parts read to one part write, and tweets never change after creation. Every choice — precomputed timelines, index-once search, aggressive caching — falls out of those two facts.
Back-of-envelope estimates
The Grokking-canonical numbers, and worth having cold — the read/write skew justifies the architecture, and the media line shows where the bytes actually are.
| Quantity | Estimate | How you got there |
|---|---|---|
| Tweet writes | ~1k/s | 100M tweets/day ÷ ~86k s ≈ 1,150/s; peaks 5–10×. |
| Timeline reads | ~300k/s | 200M DAU visiting timelines ≈ 28B tweet views/day. The 50:1 skew, visible in the arithmetic. |
| Text storage | ~30 GB/day | 100M × ~300 B ≈ 30 GB/day, ~11 TB/yr. Tweet text is a rounding error. |
| Media storage | ~24 TB/day | 1 in 5 tweets a photo (200 KB) + 1 in 10 a video (2 MB). Media dwarfs text 800:1 — it goes to S3 + CDN, never the database. |
| Fan-out volume | ~20B inserts/day | 100M tweets × ~200 avg followers ≈ 200k timeline-cache writes/s, absorbed async. |
| Timeline cache entry | ~800 IDs/user | Real Twitter keeps a Redis list of ~800 tweet IDs per active user's home timeline. |
Three hundred thousand reads a second with a 500 ms budget means the home timeline must be a precomputed list in RAM — Redis lists of about 800 tweet IDs per active user, which is exactly what Twitter actually runs.
Sketch the API
Five endpoints. The quiet decision that pays off later: the pagination cursor is the tweet ID itself, which only works because of how I mint IDs.
- The cursor is a Snowflake ID: because IDs embed their creation time, "tweets older than cursor" is a pure ID comparison — no timestamp column, no offset, stable under the firehose of inserts.
- Like/retweet are idempotent per (userId, tweetId) — a unique key on the engagement row means double-taps and retries can't double-count.
Data model
Tweets are immutable and append-only, engagement is counters, timelines are cache. Three very different write patterns, so they get three different stores.
Tweets go to Cassandra or DynamoDB — 6k immutable writes/s with point reads by ID is the textbook partitioned-KV workload, and there are no joins to lose. It's worth saying that real Twitter historically ran sharded MySQL and it worked; at 10M DAU I'd do the same. I'd switch to the partitioned store when operating the shards myself stops being fun — around this scale.
High-level design
One write enters, and Kafka fans it to three consumers with different jobs: timeline fan-out, the search indexer, and the trends pipeline. The read side never sees any of that machinery — it reads finished results out of caches and indexes.
- Tweet: client POSTs → tweet service mints a Snowflake ID, inserts into Cassandra, drops the tweet on the Kafka firehose, returns. Sub-100 ms for the author.
- Fan-out workers push the ID into each follower's Redis timeline list (trim at ~800), skipping flagged celebrity accounts — the ~5 s visibility target is the queue's SLO.
- In parallel, the indexer consumes the same event and writes the tweet into Elasticsearch exactly once — immutability means no update path — while the trends pipeline bumps sliding-window term counts.
- Home timeline read: timeline service fetches the ID list from Redis, hydrates via a tweet cache backed by Cassandra, merges recent tweets from followed celebrities pulled live, attaches counts from the counter cache.
- Search read: query fans in across all Elasticsearch shards, results merge and rank — the mirror image of the timeline's fan-out.
Deep dives — where the interview is won
Snowflake IDs: time-ordered, collision-free, no coordinator
Auto-increment dies the moment tweets live on more than one database node — either the nodes collide or they serialise on a shared sequence, and a central ticket server becomes a single point of failure at 6k writes/s. Hashing gives uniqueness but random IDs, which destroys the thing a timeline needs most: chronological order you can sort and paginate by. Snowflake solves both in 64 bits: 41 bits of millisecond timestamp, 10 bits of machine ID, 12 bits of per-machine sequence — 4,096 IDs per millisecond per machine, minted entirely locally.
The timestamp in the high bits is the trick: sorting by ID is sorting by time, so cursor pagination, timeline merges, and even time-range scans on the partition key all fall out for free, with zero coordination between machines. The operational fine print is clock skew — machines sync via NTP, and a generator that sees its clock jump backwards must refuse to mint until time catches up, or it risks duplicate IDs.
Hybrid fan-out, with Twitter's actual numbers attached
The strategy is the news-feed answer — push for normal accounts, pull-and-merge for celebrities — so I state it fast and spend my time on what production taught Twitter. Home timelines live in Redis as lists of ~800 tweet IDs; fan-out runs at roughly 20B inserts/day (~200k/s), and delivery within ~5 seconds is the pipeline's SLO. Accounts above a follower threshold skip fan-out and are merged at read time. That's the whole core, and it's exactly what Twitter shipped.
The number interviewers push on is RAM. Naively, 800 IDs × 8 B × every registered user is memory you don't want to buy — so only active users hold a timeline in Redis. A dormant user's list is simply absent; on login, the timeline service rebuilds it once by pulling followees' recent tweets, then fan-out resumes for them. Most registered users are dormant at any moment, so this halves the cache several times over.
Search and trends ride the firehose, not the database
Search over tweets cannot be a LIKE '%term%' scan — that's a full table walk per query. It needs an inverted index: term → list of tweet IDs, which is Elasticsearch's whole job. The gift is immutability: a tweet is indexed exactly once, by an async consumer on the Kafka firehose, and never updated. No reindex-on-edit path, no cache invalidation on documents — seconds of indexing lag is the entire consistency story. Note the symmetry worth saying aloud: timelines fan out on write so reads are cheap; search fans in on read — every query hits all index shards and merges — because you can't know at write time which queries will want a tweet.
Trends are a streaming problem, not a query: find heavy-hitter terms over a sliding window of the firehose. Exact counting of every term is pointless — a count-min sketch plus a top-k heap per window gives approximate counts in fixed memory, and nobody audits whether a hashtag had 1.02M or 1.05M mentions. Windowed counts land in a small trends cache that the read path serves in one hit.
Counters that survive a viral tweet
A tweet going viral means millions of likes an hour landing on one key — as a database row that's a lock convoy, as a single Redis key it's better but still one node's ceiling. The base pattern: likes hit Redis INCR, a consumer flushes deltas to durable storage periodically, and the per-user engagement row (for idempotency and un-like) is written async. Reads come from the cache, display values round to "1.2M", and nobody waits on durable writes.
For the truly extreme key, shard the counter: split it into N sub-keys (likes:{id}:0..15), increment one at random, sum on read — write throughput scales by N for the price of a slightly heavier read, which is itself cached. This is the general hot-key recipe: spread writes, aggregate lazily.
Follow-ups you should expect
What's the difference between the home timeline and the user timeline in your design?
A tweet gets deleted after being retweeted and fanned out into millions of timelines. How do you remove it?
How does this go multi-region?
Why not just use database auto-increment for tweet IDs?
Your Redis timeline cluster loses a node. What does the user see?
Where candidates lose the room
- Conflating the home timeline with the user timeline — they have opposite cost profiles, and the celebrity fix depends on knowing the difference.
- Rebuilding the entire news-feed fan-out discussion from scratch and running out of time before search, trends and counters — the parts this question is actually about.
- Proposing auto-increment or UUIDv4 for tweet IDs — one doesn't shard, the other doesn't sort; the Snowflake discussion is where the marks are.
- Treating search as a SQL LIKE query instead of an async-built inverted index — it signals you've never thought about read amplification.
- Designing as if reads and writes were comparable when the skew is 50:1 — every architectural choice should visibly favour the read path.