System Design Primer: A Tour of Scalability Fundamentals

The System Design Primer is an open-source study guide with around 300k GitHub stars that collects the standard vocabulary of large-scale system design into one place: a long README, a set of Anki decks, and worked solutions to the usual interview questions (design a URL shortener, a Twitter timeline, a web crawler). I went through the repository to see how well it holds up as a reference rather than just as interview prep, and this is my condensed version of it, section by section, with a few notes where I think the material simplifies too hard or has simply aged. The primer's own framing is that everything is a trade-off, and that thread is worth keeping in view throughout.

How the primer frames the problem

It opens with two distinctions that the rest of the document leans on.

Performance versus scalability. A service has a performance problem if it is slow for a single user, and a scalability problem if it is fast for one user but slow under load. The primer's working definition of scalable is that adding resources produces a proportional increase in capacity. If doubling the servers doesn't roughly double the throughput, something in the design isn't scaling.

Latency versus throughput. Latency is the time to service one request; throughput is the number of requests served per unit time. The stated goal is maximal throughput with acceptable latency. The two trade off against each other (batching and queuing buy throughput at the cost of latency), and naming which one a design is optimising for keeps that trade explicit.

Consistency, availability, and the CAP trade-off

The primer states CAP (Consistency, Availability, Partition Tolerance) in the familiar form: a distributed system can provide at most two of three properties — consistency (every read sees the most recent write), availability (every request gets a non-error response), and partition tolerance (the system keeps working despite dropped messages between nodes). Since network partitions are a fact of life, the real choice is between CP (refuse requests during a partition to preserve consistency) and AP (keep answering, and accept that some answers are stale).

That framing is a useful starting point, but it is the version that Eric Brewer — who introduced CAP — walked back in his 2012 retrospective CAP Twelve Years Later. His objections: "two of three" is misleading because partitions are rare, so there is no reason to forfeit C or A when the system isn't partitioned; and all three properties are matters of degree, not binary switches.

The more practical model is PACELC: if partitioned (P), trade availability against consistency (A/C); else (E), in normal operation, trade latency against consistency (L/C). The "else" half is the part CAP omits, and it is the one that governs a system's behaviour almost all of the time. The primer would be stronger if it led with PACELC and treated CAP as the historical predecessor.

Availability patterns

Failover. Active-passive keeps a hot standby that takes over when heartbeats from the active node stop; only one node serves traffic at a time. Active-active has both nodes serving, which also spreads load but requires the rest of the system to cope with both being live.

Replication. Master-slave sends writes to one primary and replicates to read-only replicas. Losing the primary means read-only operation until one is promoted. Master-master accepts writes on more than one node, which removes that single write bottleneck but forces you to deal with write conflicts and, usually, to give up some ACID guarantees or latency to coordinate. This is expanded in the database section below.

Availability in numbers. The primer quantifies the marketing shorthand: 99.9% ("three nines") is about 8h 46m of downtime a year, or 43m a month; 99.99% ("four nines") is about 52m a year, or 4m 23s a month. But it's worth noting that availability figures for components in sequence multiply (two 99.9% services in a request path give 99.8%), whereas components in parallel improve the total, and a published SLA number is a contractual commitment, not a measurement of what you'll actually get.

The request path: DNS, CDN, load balancer, reverse proxy

DNS

DNS resolves a hostname to an IP through a hierarchy of record types (NS, A, CNAME, MX). Managed DNS providers layer routing policies on top: weighted round-robin, latency-based, and geolocation-based responses. The costs the primer names are the added lookup latency, the operational complexity, and DNS being a favourite DDoS target. DNS-based global traffic routing is exactly what Azure Traffic Manager does, and its six routing methods map almost one-to-one onto the policies listed here.

CDN

A CDN serves static assets (and increasingly some dynamic content) from edge locations near the user. Push CDNs have you upload content when it changes — good for small or rarely-changing sites with minimal traffic to the origin. Pull CDNs fetch on the first request and cache with a TTL, which is less for you to manage, at the cost of a slow first hit and the risk of serving stale content until the TTL expires. Cache-busting via versioned URLs is the standard workaround for the staleness problem.

Load balancer

A load balancer spreads requests across a pool, stops routing to unhealthy members, and removes any single server as a point of failure. The primer splits it by OSI layer: Layer 4 balancing routes on IP and port only, which is cheap. Layer 7 balancing terminates the connection and can route on URL path, headers, or cookies, at higher cost. Routing algorithms range from round-robin to least-connections to session-hashed. Two caveats it flags: the balancer itself becomes a bottleneck and a failure point unless it is also made redundant, and horizontal scaling behind it only works if the app servers are stateless — session state has to move to a shared store. Azure's L4/L7 split is the same idea: I compared Azure Load Balancer and the L7 Application Gateway separately, and the trade-offs line up with the primer's.

Reverse proxy

The primer draws a line between a load balancer and a reverse proxy that is worth keeping: a load balancer distributes across many servers and is pointless with one, whereas a reverse proxy is useful even in front of a single backend: it centralises TLS termination, compression, caching, static-file serving, and request filtering behind one public endpoint. In practice the same product (nginx, HAProxy, Application Gateway) often does both jobs.

The application layer

Separating the web tier from the application tier lets each scale and be configured independently. From there the primer describes microservices — small, independently deployable services, each owning one capability (Pinterest split into user profiles, followers, feeds, search, photo upload) — and the service discovery they need — Consul, etcd, or ZooKeeper holding a registry of service names, addresses, ports, and health, often with a key-value store for shared config. This trades code complexity for operational and deployment complexity, and that trade is not free.

Databases

Relational databases

SQL databases give you ACID transactions — atomicity, consistency, isolation, durability. The primer's scaling techniques, roughly in the order you'd reach for them:

Master-slave and master-master replication: read replicas first, multi-primary only when you have to, because conflict resolution and the ACID/latency cost are real.

Federation: splits the database by function — separate databases for users, products, forums. That reduces the read and write volume and replication lag on each. It doesn't help when one table is the problem, and cross-function joins now cross a network.

Sharding distributes rows of one table across databases by a key (user ID, geography). It removes the single-writer bottleneck that federation leaves in place, at the cost of application logic to route queries, uneven "hot" shards that need rebalancing, and joins that now span shards.

Denormalisation copies data into multiple tables so reads avoid expensive joins — trading write cost and the risk of inconsistent copies for read speed. Useful once read/write ratios are lopsided.

SQL tuning: pick tight column types, index the columns used in WHERE/JOIN/ORDER BY (but not everything, because indexes cost write time and memory), denormalise to kill joins on hot paths, and partition hot rows so they stay in memory.

NoSQL

NoSQL stores drop joins and usually full ACID in favour of BASE — basically available, soft state, eventual consistency. Four families:

SQL or NoSQL

The primer's decision guide: SQL when data is structured and relational, transactions and strict schemas matter, and you want the mature tooling. NoSQL when the schema is loose or fast-moving, the data volume is very large, write throughput dominates, or the data is transient. It also notes that plenty of systems run both, each for the workload it suits.

Caching

Caching can happen at the client, the CDN, the web server (a reverse-proxy cache), the database, or the application layer (Redis, Memcached). The primer's real contribution here is naming the four update strategies and their failure modes:

The framing to take away is that cache invalidation is a genuinely hard problem and each of these strategies picks a different thing to get wrong.

Asynchronism

Move expensive work off the request path so the user isn't waiting on it. Message queues (RabbitMQ, Amazon SQS, Redis) hold jobs that workers pull and process, with the user getting an immediate acknowledgement. Task queues (Celery) handle scheduled or compute-heavy jobs. Back pressure is the safety valve: when the queue hits a size limit, start rejecting new work — HTTP 503 with a Retry-After — instead of letting the queue exhaust memory and take everything down. The trade-off named: async is wrong for anything that genuinely needs a synchronous answer, and it adds a moving part. This is the same ground as Azure's messaging and event services, where competing consumers plus queue-depth autoscaling is the concrete version of "workers pull from a queue".

Communication

TCP is connection-oriented and reliable — ordered delivery, retransmission, flow and congestion control — at the cost of handshake latency and per-connection memory (connection pooling mitigates the latter). UDP is connectionless and unreliable but lower overhead, and it supports broadcast/multicast — the right choice for VoIP, streaming, and games where a late packet is worthless anyway.

RPC (gRPC, Thrift) makes a remote call look local. It is efficient and good for internal service-to-service traffic, but it couples client to server and every operation is a new bespoke method. REST is resource-oriented over HTTP verbs, stateless, cacheable, and loosely coupled — the right default for public APIs — but it can be chatty when a client needs a deep object graph, and it is awkward for actions that don't map onto CRUD. The HTTP verb table (GET is safe and idempotent, POST is neither, PUT and DELETE are idempotent but not safe) is the reference worth memorising.

What I'd take from it

As a map of the vocabulary, it is hard to beat: one place to see how DNS, CDN, load balancing, replication, sharding, caching, and queuing fit together, with the trade-offs attached. The four-step interview method it teaches — clarify the use cases and constraints, sketch the high-level design, drill into the core components, then find and address the bottlenecks — is a sound way to approach an open-ended design question, in an interview or on the job.

It is a primer, and it is honest about being one, the depth is meant to come from the linked papers and engineering blogs it points to.

Sources

• donnemartin/system-design-primer — README
• Eric Brewer — CAP Twelve Years Later: How the "Rules" Have Changed (InfoQ)
• Daniel Abadi — Consistency Tradeoffs in Modern Distributed Database System Design (PACELC)