1.1 Scalability — System Design Mastery

Predict

Your web app is getting 10x more traffic than last month. You have one server running everything. What are your options for handling the load? Write down as many strategies as you can think of before reading on.

The two axes of scaling

Every scaling decision starts with the same question: do I make this machine bigger, or do I add more machines?

Vertical scaling (scale up)

Add more resources to a single machine — more CPU cores, more RAM, faster SSDs.

Advantages: Simple. No code changes. No distributed systems headaches.

Limits:

Hardware has a ceiling. The largest cloud instances top out around 448 vCPUs and 24TB RAM.
Cost scales super-linearly. A machine with 2x the CPU often costs 3-4x the price.
Single point of failure. One machine means one failure domain.

Horizontal scaling (scale out)

Add more machines and distribute the work across them.

Advantages: No theoretical ceiling. Better fault tolerance. Often more cost-effective.

Challenges: Your application must be designed for it. State management, data consistency, and network communication all become harder.

	Vertical	Horizontal
Complexity	Low	High
Cost curve	Exponential	Linear
Ceiling	Hardware limits	Effectively unlimited
Fault tolerance	Single point of failure	Can survive node failures
Code changes	None	Significant

The key to horizontal scaling: statelessness

A stateless server stores no session data locally. Every request contains everything the server needs to process it (or can fetch it from a shared store). Any server in the pool can handle any request.

A stateful server remembers things between requests — sessions, caches, file uploads in progress. If that server goes down, the state is lost. If you need to add servers, you need "sticky sessions" to route users back to the same server.

Stateful:   User A → always → Server 1 (has session)
            User B → always → Server 2 (has session)

Stateless:  User A → any server → reads session from Redis
            User B → any server → reads session from Redis

The rule: Move state out of your application servers and into dedicated state stores (databases, caches, object storage). Your application servers become interchangeable workers.

Shared-nothing architecture

The gold standard for horizontal scaling. Each node is independent — it shares no memory, no disk, no state with other nodes. Nodes communicate only through the network.

Why it wins: Adding capacity means adding nodes. Removing a node does not affect others. You can scale linearly with load.

Real-world example: Amazon's web tier runs thousands of stateless application servers behind load balancers. Session state lives in DynamoDB. File uploads go to S3. The servers themselves are disposable — any one can be terminated and replaced in seconds.

Check Your Understanding

You are running a web application on a single 8-core server with 32GB RAM. Traffic has doubled. Your options are: (A) upgrade to a 16-core, 64GB server, or (B) add a second identical 8-core server behind a load balancer. What factors should drive your decision?

Scalability Practice

Problem 1 of 3(intermediate)

A social media startup stores user sessions in memory on each application server and uses sticky sessions (cookie-based routing) to ensure users return to the same server. They want to scale from 2 servers to 20. What is the primary problem with their current approach?

Explain Back

Explain to a junior developer why making servers stateless is the single most important thing they can do to prepare an application for horizontal scaling. What specific things need to move out of the server, and where do they go?

Key Takeaway

Vertical scaling has a ceiling; horizontal scaling does not. The key to horizontal scaling is statelessness — move sessions, caches, and file storage out of your application servers into shared stores. Shared-nothing architectures scale linearly because nodes are independent and interchangeable.

Connect

Scalability tells you how to handle more load. But how do you measure whether your system is fast enough? Module 1.2 covers the two dimensions of performance: latency (how fast) and throughput (how many).