What it is
Scalability is a system's ability to absorb growing load — traffic, data, users — by adding resources. There are only two directions: vertical (scale up: a bigger machine) and horizontal (scale out: more machines sharing the load). Every architecture decision in an interview is downstream of which one you picked for which tier, and when.
Interviewers rarely ask about scalability directly; they score it through the non-functional requirements step. The signal they want is quantitative restraint: capacity math that says when one box stops being enough, not a reflexive sprawl of microservices for 10k users. "Don't distribute until the math forces you to" is the sentence that separates senior candidates from buzzword candidates.
I'll start on a single beefy box and scale the stateless app tier horizontally behind a load balancer once we pass a few thousand RPS — the database is the tier where I'll stay vertical as long as the numbers let me.
The picture
The variants and when to pick each
Vertical first — and defend it
Scaling up means a bigger machine: more CPU, more RAM, faster disks. No code changes, no distributed-systems complexity, no consistency problems. Cloud ceilings are higher than people think — you can rent 96–128 vCPUs and up to 24 TB of RAM before you hit the wall — so for the database tier especially, vertical first is usually the right call.
The costs are real though: the price curve goes exponential near the top, upgrades often mean downtime, and one machine is one failure domain. Vertical scaling buys time; it doesn't buy availability.
Horizontal scaling requires statelessness
Scaling out means commodity machines behind a load balancer — near-unlimited headroom, fault tolerance by redundancy, and cost that grows linearly. But it only works if any server can serve any request, which means the app tier must be stateless: sessions, carts, and per-user state move out to Redis or a database.
This is the prerequisite interviewers probe. If I draw ten app servers and keep sessions in server memory, sticky sessions become load-bearing and one server death logs users out — say the state moved to Redis before anyone asks.
The three axes: clone, split by function, split by data
The AKF scale cube gives horizontal scaling three distinct axes. X: clone identical servers (the load-balanced app tier, read replicas). Y: split by function — separate services for upload, feed, and search so each scales on its own curve. Z: split by data — sharding, so each node owns a slice of the keyspace.
In a 40-minute interview these appear in exactly that order: clone the app tier early, split out the one service with a genuinely different load profile (video transcoding, notification fan-out), and shard only when write throughput or storage on one database node forces it.
Know what breaks first
A single modern app server handles roughly 1k–10k RPS of real work; a single Postgres or MySQL box does about 5k–10k writes/s comfortably. So for almost every read-heavy consumer design, the database is the first bottleneck — which is why the standard escalation is cache, then read replicas, then shard, in that order of increasing pain.
When the interviewer says "now 10× the traffic", walk the tiers explicitly: app tier scales by adding boxes, reads scale with cache and replicas, writes are the hard part and the reason sharding exists. Naming the order shows I've thought about cost, not only capacity.
Numbers worth memorising
| One app server (real work) | ~1k–10k RPS |
| One Postgres/MySQL node, writes | ~5k–10k writes/s comfortable |
| 1M requests/day | ≈ 12 QPS average — peak 2–5× that |
| Scale-out trigger | ~70% sustained CPU on the tier |
| Vertical ceiling (cloud) | ~128 vCPUs / 24 TB RAM — high, but real |
Where the interviewer pushes
The classic push is "why not just buy a bigger box?" — and the trap is treating it as an insult. The strong answer agrees: vertical first is often correct, especially for the database, because it defers distributed-systems complexity entirely. Then name the two reasons it ends: the cost curve goes exponential near the ceiling, and one machine is one failure domain — so availability, not throughput, is usually what forces the first scale-out.
The second push is "your app servers are stateless — where did the sessions go?" This is a check that I didn't say "stateless" as a slogan. The answer: session data lives in Redis (or a signed token on the client), so any server serves any request, deploys are rolling, and a dead server costs capacity but no user state. If I need sticky sessions for WebSockets, that's a deliberate exception and I say what breaks on reconnect.