System Design: WhatsApp Architecture Deep Dive

When Facebook acquired WhatsApp in 2014 for $19 billion, the company had 55 employees — including just 35 engineers — serving 450 million monthly active users. By 2017, that same lean team was handling 900 million users sending 65 billion messages per day. Today, WhatsApp serves over 2 billion users. It is perhaps the most dramatic example of engineering efficiency in the history of software.

The secret was a deliberate set of technology choices, starting with one that most companies would never consider: building the entire backend in Erlang.

Why Erlang Changed Everything

WhatsApp’s founders, Jan Koum and Brian Acton, came from Yahoo, where they had seen the pain of scaling C++ and Java systems. When they started WhatsApp in 2009, they chose Erlang — a language originally built by Ericsson in the 1980s for telephone switches.

This choice seemed eccentric. Erlang is not popular, does not have a large talent pool, and looks nothing like mainstream languages. But it has one property that made it perfect for messaging: the BEAM virtual machine handles concurrency better than almost anything else.

The BEAM Advantage

Think of most language runtimes like a highway. Each car (thread) is large, takes up a lane, and switching between lanes is expensive. Now imagine a system where instead of cars, you have bicycles — millions of them — each tiny, each moving independently, and lane-switching is nearly free. That is BEAM.

Key properties of BEAM/Erlang for messaging:

• Lightweight processes — an Erlang process uses only about 300 bytes of memory initially. A single server can run millions of processes simultaneously. Each connected user gets their own process.
• Preemptive scheduling — BEAM automatically time-slices between processes. One slow operation cannot starve others. This is critical when one user sends a large video while millions of others send text.
• Fault isolation — processes are fully isolated. If one crashes (bad message, out-of-memory), only that user’s connection drops. Every other user is unaffected.
• Hot code reloading — Erlang can swap running code without restarting the server. WhatsApp could deploy updates without disconnecting a single user.

%% Simplified WhatsApp connection handler
-module(ws_connection).
-export([start/1, loop/1]).

start(Socket) ->
    %% Each connected user spawns a lightweight process
    %% ~300 bytes initial memory, millions can run concurrently
    Pid = spawn(?MODULE, loop, [Socket]),
    gen_tcp:controlling_process(Socket, Pid),
    Pid.

loop(Socket) ->
    receive
        {tcp, Socket, Data} ->
            Message = decode_message(Data),
            route_message(Message),
            loop(Socket);
        {deliver, Message} ->
            %% Another process is delivering a message to this user
            gen_tcp:send(Socket, encode_message(Message)),
            loop(Socket);
        {tcp_closed, Socket} ->
            %% User disconnected, process terminates
            %% ~300 bytes freed, no impact on other users
            ok
    end.

The Numbers

By 2012, WhatsApp achieved a benchmark that stunned the industry: a single server handling 2 million concurrent connections. By 2015, they pushed this to over 2.5 million connections per server using custom-tuned FreeBSD kernels.

For comparison, a typical Java server with Netty handles around 100,000-500,000 concurrent connections. A Node.js server tops out around 50,000-100,000. WhatsApp’s Erlang servers handled 5-25x more connections per machine.

With roughly 550 servers at peak, WhatsApp handled nearly a billion users. The server-to-user ratio was approximately 1:1,800,000. That efficiency is why 35 engineers could run the entire backend.

Message Delivery: Store-and-Forward

WhatsApp uses a store-and-forward model for message delivery. The message does not go directly from sender to receiver. Instead, it passes through WhatsApp’s servers, which hold it until the recipient comes online.

The Delivery Flow

Message Delivery:

1. Sender's phone encrypts message
   -> Sends to WhatsApp server via persistent connection

2. Server receives encrypted blob
   -> Stores in Mnesia (Erlang's built-in database)
   -> Checks: Is recipient online?

3a. Recipient IS online:
    -> Forward immediately via recipient's connection
    -> Recipient sends delivery ACK
    -> Server deletes stored message
    -> Server sends delivery receipt to sender (double gray check)
    -> Recipient sends read receipt (double blue check)

3b. Recipient is OFFLINE:
    -> Message stays in Mnesia
    -> When recipient reconnects, server pushes all pending messages
    -> Normal ACK flow continues

Messages are deleted from servers once delivered.
No message history is stored server-side.

This design has a profound privacy implication: once a message is delivered, WhatsApp’s servers do not retain it. There is no server-side message history to subpoena, hack, or leak. The servers are a relay, not a repository.

Delivery Receipts

The checkmark system is a state machine:

State	Visual	Meaning
Sent	Single gray check	Server received the message
Delivered	Double gray checks	Recipient’s device received the message
Read	Double blue checks	Recipient opened the chat

Each state transition is an explicit acknowledgment message flowing back through the same connection infrastructure.

Mnesia: The In-Process Database

WhatsApp uses Mnesia, Erlang’s built-in distributed database, for transient message storage. Mnesia is unusual — it runs inside the BEAM VM itself, so there is no network hop to query it. Data is replicated across nodes for durability.

For a message that is delivered within seconds (which is the common case), Mnesia’s in-memory speed means the store-and-forward overhead is negligible. The message hits disk only if the recipient is offline for an extended period.

End-to-End Encryption

In 2016, WhatsApp rolled out end-to-end encryption for all messages, calls, photos, and videos. They adopted the Signal Protocol, developed by Open Whisper Systems (now Signal Foundation).

How It Works

The core idea: each user’s device generates a pair of cryptographic keys — a public key (shared with WhatsApp’s servers) and a private key (never leaves the device). Messages are encrypted with the recipient’s public key and can only be decrypted with their private key.

End-to-End Encryption Flow:

1. Key Exchange (one-time setup):
   Alice's phone generates:
     - Identity Key Pair (long-term)
     - Signed Pre-Key (medium-term)
     - One-Time Pre-Keys (single use)
   Public parts uploaded to WhatsApp server.

2. Alice sends message to Bob:
   a. Alice's phone fetches Bob's public keys from server
   b. Performs X3DH key agreement -> shared secret
   c. Encrypts message with shared secret (AES-256)
   d. Sends encrypted blob to server

3. Server forwards encrypted blob to Bob:
   - Server CANNOT decrypt (no private key)
   - Bob's phone decrypts with its private key

4. Forward Secrecy:
   - Keys ratchet forward after each message
   - Compromising one key does NOT decrypt past messages

WhatsApp’s servers see only encrypted blobs. They know that Alice sent a message to Bob and when, but they cannot read the content. This is not a policy choice — it is a mathematical guarantee.

Group Encryption

Group messaging adds complexity. Rather than encrypting once for the group, the sender encrypts the message separately for each group member using their individual keys. For a group of 100 people, the sender’s device performs 100 encryption operations.

This is why WhatsApp originally capped groups at 256 members and why large groups can be slower on low-powered devices. The computational cost scales linearly with group size.

Group Messaging at Scale

WhatsApp groups (now supporting up to 1024 members) require a different architecture than one-to-one messaging.

Group Message Fan-Out

Group Message Flow:

1. Sender sends message + group ID to server

2. Server looks up group membership
   -> Retrieves list of 1024 member IDs (max)

3. Fan-out: Server creates delivery task for each member
   -> Each task is an independent Erlang process
   -> Tasks execute concurrently

4. For each member:
   -> Check if online -> deliver immediately
   -> If offline -> store for later delivery
   -> Collect delivery receipts

5. Sender receives aggregate delivery status

The fan-out is where Erlang’s concurrency model shines. Spawning 1024 lightweight processes to deliver a group message in parallel is trivial for BEAM. Each process independently handles its recipient — checking online status, delivering or storing, and collecting acknowledgments.

Group Metadata

Group state (members, admins, name, icon) is stored separately from messages. When a member joins or leaves, a “group state update” message is broadcast to all members. Each device maintains its own local copy of the group state.

This is an eventually consistent model. If two admins simultaneously add different members, there is a brief window where different devices have different member lists. The server is the source of truth, and devices sync on their next connection.

Media Storage and Delivery

Text messages are small — a few hundred bytes. Media (photos, videos, documents) can be megabytes or gigabytes. WhatsApp handles them differently.

Media Upload Pipeline

Media Handling:

1. Sender's device:
   a. Compress media (images: JPEG, videos: H.264)
   b. Generate AES-256 encryption key
   c. Encrypt media with that key
   d. Upload encrypted blob to media storage (not Mnesia)

2. Media storage returns a URL

3. Sender's device:
   a. Creates a message containing:
      - Media URL
      - Encryption key
      - Thumbnail (small, sent inline)
   b. Encrypts this message with recipient's key
   c. Sends via normal message channel

4. Recipient's device:
   a. Decrypts message -> gets URL + media encryption key
   b. Downloads encrypted blob from media storage
   c. Decrypts with the media-specific key
   d. Displays media

Server stores encrypted blob but not the decryption key.
The key travels through the E2E encrypted message channel.

This is elegant. The media server stores encrypted files it cannot decrypt. The decryption key travels through the end-to-end encrypted messaging channel. Even if someone compromises the media storage, they get only encrypted blobs.

Media is stored for 30 days on WhatsApp’s servers. After the recipient downloads it, the server-side copy is eventually deleted.

Connection Management

WhatsApp maintains persistent TCP connections with every active device. At peak, this means managing over a billion concurrent connections.

Connection Architecture

Each server runs a connection handler process per user. The servers sit behind load balancers that use consistent hashing to route a user to the same server. This matters because when a message arrives for a user, the routing layer must know which server holds that user’s connection.

Connection Routing:

  User Device
      |
   TLS Termination (Edge)
      |
   Load Balancer (Consistent Hash on User ID)
      |
   Erlang Server Cluster
      |-- Server A: handles users 0-999999
      |-- Server B: handles users 1000000-1999999
      |-- ...
      |-- Server N: handles users (N-1)*M to N*M

When Server A receives a message for a user on Server C:
  Server A -> internal routing -> Server C -> user's process

Handling Disconnections

Mobile devices constantly disconnect and reconnect — entering tunnels, switching from WiFi to cellular, going to sleep. WhatsApp’s system handles this gracefully:

1. When a connection drops, the server keeps the user’s process alive for a grace period
2. Incoming messages accumulate in the process mailbox (Erlang’s built-in message queue)
3. When the device reconnects, it may land on a different server
4. The routing layer updates, and queued messages drain to the new connection

The Scaling Story: 50 Engineers, 900M Users

WhatsApp’s engineering culture was deliberately minimal. No managers, no dedicated QA team, no daily standups. Engineers owned their services end-to-end. The team’s philosophy was captured by Jan Koum’s phrase: “No ads, no games, no gimmicks.”

Several technical decisions enabled this extreme efficiency:

• Erlang/BEAM — 10-50x fewer servers than equivalent Java/Python systems meant 10-50x fewer things to monitor and maintain
• FreeBSD — chosen over Linux for its superior network stack. WhatsApp engineers contributed patches upstream to improve connection handling
• Single responsibility — the product did one thing (messaging) and did it well. No stories, no reels, no marketplace (those came later under Meta)
• No custom infrastructure — unlike Netflix or Uber, WhatsApp did not build custom databases or frameworks. They used Erlang’s built-in tools (Mnesia, OTP) and standard protocols
• No A/B testing framework — features shipped to everyone at once. This eliminated an entire layer of infrastructure complexity

Technology Stack Summary

Language:      Erlang (backend), C/C++ (native mobile clients)
Runtime:       BEAM VM (custom-tuned)
OS:            FreeBSD (custom kernel patches)
Database:      Mnesia (transient messages), custom media storage
Encryption:    Signal Protocol (E2E), TLS 1.3 (transport)
Protocol:      Custom binary protocol over TCP
Load Balancing: Consistent hashing on user ID
Monitoring:    Minimal — Erlang/OTP's built-in supervision trees

Key Takeaways

WhatsApp’s architecture demonstrates that simpler is often better:

• Choose the right runtime for the problem. Erlang was not popular, but its concurrency model was purpose-built for exactly this workload. The 10-50x efficiency advantage over mainstream languages justified the smaller talent pool.
• Delete features, do not add infrastructure. WhatsApp served a billion users without a microservices architecture, without Kubernetes, without a custom service mesh. They did not need those things because the product was simple.
• Privacy as an architecture constraint. End-to-end encryption and store-and-forward-then-delete are not just policies — they are architectural decisions that reduce storage costs and legal liability.
• Small teams can build enormous systems when they choose technologies that minimize operational overhead. Fifty engineers managing 550 servers is manageable. Fifty engineers managing 10,000 servers is not.
• Do not scale what you do not build. Every feature you add is a feature you must scale. WhatsApp scaled messaging to 2 billion users by not building anything else.

The next time you send a WhatsApp message and see the double blue checkmarks appear in under a second, remember that behind that instant feedback is an Erlang process running on a FreeBSD server that is simultaneously managing 2 million other conversations — and the server does not even know what your message says.