Everyday storyA post office that also phones you
Imagine a post office that keeps a register of which counter each person is standing at. When a letter arrives for you, the clerk first files a copy in a locked cabinet, so it can't be lost whatever happens next. Then she checks the register and walks the letter straight to your counter. You sign for it, and the sender gets a slip saying it reached you.
If you've gone home, she doesn't wait at an empty counter. She phones you to say a letter is waiting, and the next time you walk in, she hands over everything that arrived while you were away, in the order it came. If you have a family group, she makes one copy for each member and walks each to the right counter, or rings the ones who are out.
That's a chat app almost exactly. The open counters are live connections to phones. The register is "presence". The locked cabinet is the message store. The phone call is a push notification. And the order of letters is kept by numbering them as they're filed, not by the date the sender wrote on the envelope.
In a real productWhatsApp and other chat apps
You met chat from the user's side in WhatsApp's ticks: one tick when the server has it, two when the phone has it, blue when it's read. Every chat app, whether a messenger, a team chat at work, or the chat with your delivery rider inside a food app, is solving the same problem: get a few words from one phone to another in about a second, never lose them, and keep them in order.
This lesson is the interview version: designing the whole system for crores of people sending thousands of crores of messages a day. It's one of the most common system design questions, because it touches almost everything: long-lived connections, routing, storage at huge scale, offline delivery, groups, ordering and failure. We'll walk through it the way a strong candidate would in a 45-minute interview.
The ideaHow it works
Step 1: what it must do
Start by agreeing what you're building. The , the things the app does:
- one-to-one messages and group messages (say up to 1,000 members);
- delivery to people who are offline, as soon as they come back;
- ticks for sent, delivered and read;
- photos, videos and voice notes;
- online status and "last seen";
- the same chat on a phone and a laptop.
The , how well it must do them:
- messages arrive in about a second when both people are online;
- a message, once the sender sees one tick, is never lost;
- everyone in a chat sees messages in the same order;
- it keeps working when individual servers fail;
- messages are private, ideally .
Say what you'll leave out: voice and video calls, payments, stories and channels. That keeps the interview on the core.
Step 2: a quick estimate
A sum tells you which parts are hard.
- Messages. Say 50 crore daily users sending 40 messages each: 2,000 crore messages a day. A day has about 86,400 seconds, so that's about 2.3 lakh messages a second on average, perhaps 7 lakh at evening peaks and far more at midnight on New Year's Eve.
- Text storage. A text message with its details is about 200 bytes. 2,000 crore × 200 bytes is about 4 TB a day, or about 1.5 PB a year before making copies for safety.
- Media. If 1 message in 20 is a photo of about 200 KB after compression, that's 100 crore photos, about 200 TB a day. Media is fifty times bigger than text.
- Connections. Say 20 crore people are online at a busy moment, each with one open line. If one server can hold about 5 lakh mostly idle connections, that's 400 servers, so plan for 600 with spare room.
- Heartbeats. Each open phone sends a tiny "still here" signal every 30 to 60 seconds. 20 crore phones every 45 seconds is about 45 lakh heartbeats a second, more than the messages themselves.
What this tells you: the message rate is large but manageable once spread out; the real challenges are holding crores of connections, storing petabytes, and keeping media off the message path.
Step 3: the architecture
- Connections. Each phone keeps a (or similar ) open to one chat server, through a . The server can push a message down that line the instant it arrives, instead of the phone asking "anything new?" every second.
- Presence. A fast shared keeps : which chat server each online user is connected to. Entries expire after a short unless refreshed by , so a phone that vanishes stops being "online" within a minute.
- Message store. Every message is saved here before it's delivered. It's built for a huge, steady stream of small writes.
- Push service. For people who aren't connected, it sends a through Google's or Apple's service, using the phone's .
- Media storage. Photos and videos go to , served through a . The message carries only a link and a small preview.
Step 4: following one message
- 1
Asha's phoneChat server A
Message m-77 to Ravi, over her open connection
- 2
Chat server AMessage store
Save m-77 first (one tick for Asha)
- 3
Chat server APresence
Which server is Ravi connected to?
- 4
PresenceChat server A
Chat server B
- 5
Chat server AChat server B
Deliver m-77 to Ravi
- 6
Chat server BRavi's phone
m-77 (Ravi's phone replies: got it, two ticks)
- Asha's phone sends message m-77 over its open connection to chat server A. The phone gave it a unique ID when she pressed send.
- Server A saves it to the message store and gives it the next in this chat. Only now does it reply "got it", and Asha sees one tick. From this moment the message cannot be lost.
- Server A asks presence where Ravi is. Answer: chat server B.
- Server A passes the message to B, which sends it down Ravi's connection.
- Ravi's phone replies with an . The server records it and tells Asha: two ticks. When Ravi opens the chat, his phone sends a , and the ticks turn blue.
Save first, deliver second: that order is what makes "one tick" a real promise.
Step 5: deep dives
Order and no duplicates. Phone clocks disagree by seconds or minutes, so they can't decide order. The server assigns each message the next number in its conversation (1,203, 1,204, …) when it saves it, and every phone shows messages by that number. Networks also flap: Asha's phone may send m-77, lose the reply, and send it again. Because the ID was made on her phone, the server sees m-77 twice and stores it once, a form of . Delivery to Ravi is , and his phone also ignores any ID it already has.
Offline and many devices. Each device remembers the highest sequence number it has for each chat. When it reconnects, it says "I have chat c-17 up to 1,203" and the server sends everything after. This is , and it makes offline delivery the same code path as normal delivery. A phone and a laptop are just two devices, each with its own position, so each catches up on its own.
Groups. Sending to a group is : one message, many deliveries. The message is saved once in the group's conversation, then a job on a queue delivers it to each member: down the live connection for those online, by push for those who aren't.
A 200-member group means 200 deliveries for every message. A busy 1,000-member group sending 10 messages a minute means 10,000 deliveries a minute, which is why fan-out runs on a with workers rather than inside the sender's request. With end-to-end encryption, the sender's phone locks the message once with a that every member's devices hold, so the server never needs to encrypt 1,000 copies.
Storage. The message store is write-heavy and enormous, so it's split into . The is the conversation ID: all of a chat's messages live together, so opening a chat reads one shard, and new messages from crores of chats spread evenly across all shards.
Splitting by time instead would send every new message in the world to the "today" shard, a perfect . Recent months stay on fast disks; older months can move to cheaper .
Presence at scale. 45 lakh heartbeats a second can't each be a database write. Presence lives in memory, split across many machines, and is allowed to be slightly wrong. If "last seen" is 20 seconds out of date, nobody is harmed. Messages need strong promises; presence and typing indicators don't.
Step 6: trade-offs
- Open connections versus polling. Open lines cost server memory and a little phone battery, but they're the only way to get second-level delivery for crores of people. Polling every few seconds would multiply requests many times and still feel slow.
- Keep history or not. Some apps keep every message on their servers forever, so a new laptop sees years of history. Others, especially end-to-end encrypted ones, keep a message only until every device has it, then delete it, relying on phone backups. The first costs petabytes of storage and holds a valuable target; the second saves storage but makes multi-device history harder.
- Encryption versus server features. If servers can't read messages, they can't search them, filter spam by content, or make previews. Those features move onto phones, or work only from .
- Fan-out for huge groups. Pushing every message to every member works up to a few thousand. For broadcast channels with lakhs of followers, flip it: store the message once and let each phone fetch new posts when the channel is opened.
The big idea: keep a live line to every online phone, save every message with a per-chat number before delivering it, and let anyone who wasn't there catch up from the last number they have.
For experienced engineers
Keep connection servers stateless apart from their sockets, with routing in a presence store (TTL'd entries refreshed by heartbeats, or a connection registry updated on connect and disconnect); inter-server delivery goes through a pub-sub or direct RPC keyed by the target server. Use per-conversation ordering with server-assigned sequence numbers (a single writer per conversation, or a conditional increment in the store), at-least-once delivery with client-generated message IDs for dedup, and a per-device cursor for sync. Wide-column stores (Cassandra or HBase-like) with partition key (conversation_id, time_bucket) and clustering by sequence suit the append-heavy pattern; the bucket bounds partition size for very active chats. Keep the inbox-delivery state (per-device pending queue) separate from long-term history so pruning delivered messages is cheap. Connection servers need graceful draining on deploy and jittered client reconnect with exponential backoff to avoid thundering herds. For E2EE, multi-device means per-device keys and sender-side fan-out of message keys (Sesame-style) or sender keys for groups, with rekeying on membership change; server-side abuse detection then relies on metadata, rate limits and user reports. Large groups switch from push fan-out to pull on open, and very hot conversations may need a dedicated sequencer.
Step by stepAsha sends Ravi a photo while he's in a tunnel
It's 7:40 pm. Asha in Pune sends her brother Ravi a photo of their mother's birthday cake. Ravi is on a Mumbai local, and the train has just entered a tunnel. Let's follow the message from her tap to his screen.
-
7:40:00 — The photo goes first. Asha's phone shrinks the 2 MB camera photo to about 200 KB and uploads it straight to media storage, which returns a link. The chat servers never carry the photo itself; with encryption, the phone also encrypts it first and the key travels inside the message.
-
7:40:01 — The message is small. Her phone creates message m-901: the link, a tiny blurred preview, the caption "Amma's cake!", and an ID it generated itself. Total size: about 2 KB. It goes over her open connection to chat server A in about 50 ms.
- 1
Asha's phoneMedia storage
Upload the 2 MB photo, squeezed to about 200 KB
- 2
Asha's phoneChat server
Message m-901: photo link + caption, ID made on her phone
- 3
Chat serverMessage store
Save as number 1,204 in the chat (one tick)
- 4
Chat serverPush service
Ravi is offline: send a silent wake-up
- 5
Push serviceRavi's phone
Waits until the train leaves the tunnel
- 6
Ravi's phoneChat server
Reconnect: I have everything up to number 1,203
- 7
Chat serverRavi's phone
Here is 1,204 (Asha gets two ticks)
- 8
Ravi's phoneMedia storage
Fetch the photo when he opens the chat
-
7:40:01.05 — Saved and numbered. Server A saves m-901 into chat c-17's shard and gives it number 1,204. It replies "got it", and Asha sees one tick. Even if every chat server crashed now, the message is safe.
-
7:40:01.06 — Where is Ravi? Presence has no entry for him: his phone's last heartbeat was 70 seconds ago, longer than the 60-second TTL, so his entry has expired. He's treated as offline.
-
7:40:01.07 — A push is sent. The server asks the push service to send a wake-up notification to Ravi's phone. Google's or Apple's push system holds it, because his phone can't be reached either.
-
7:42:30 — Out of the tunnel. Ravi's phone gets signal. The push arrives and shows "Asha sent a photo". At the same time, the chat app reconnects, landing on chat server D this time, and registers in presence.
-
7:42:31 — Catch up by number. Ravi's phone says: "for chat c-17, I have everything up to 1,203". Server D reads 1,204 from the shard and sends it. The phone checks the ID is new, stores the message and acknowledges it. Asha's phone gets two ticks, 2.5 minutes after she sent it.
-
7:43:10 — Ravi opens the chat. His phone downloads the 200 KB photo from the nearest CDN copy, which takes about a second on 4G, and sends a read receipt. Asha's ticks turn blue.
-
What it cost. One 2 KB message write, one 200 KB photo stored once, one push, one sync request. Had Ravi been online, steps 4 to 7 would have collapsed into one direct delivery taking well under a second. Offline delivery is not a special system, just the normal sync, run a little later.
What happens when…Situations
Real situations this idea runs into, and how well-built systems handle each one.
What happens when a chat server crashes with 5 lakh people on it
A chat server fails at 9 pm. 5 lakh phones lose their connection in the same second. Any message they were sending without a reply is in doubt, and all 5 lakh phones try to reconnect at once. If they all retry immediately, and again every second, the remaining servers and the load balancer are swamped by a wave of connection requests, and a single failure grows into a wider outage.
Phones reconnect after a short random wait (), growing with each failed try (), so the 5 lakh arrive over a minute rather than a second. The load balancer spreads them across the others, which always run with spare room. Any message without a "got it" is resent with the same ID, so nothing is lost and nothing doubles. On planned restarts, servers use , asking phones to move over in batches.
What happens at midnight on New Year's Eve
At 12:00 am, crores of people send "Happy New Year" to every group they're in, all within a minute or two. Message traffic can jump to ten times a normal evening peak. Group fan-out multiplies it again. The message store, the fan-out workers and the push service are all hit at once.
Teams plan for the known spike: extra servers are added in advance, and fan-out queues absorb the surge so deliveries slow from one second to perhaps ten, rather than failing. Less important work is cut back first: typing indicators, "last seen" updates and read receipts can lag or be dropped, a form of . Saving the messages themselves always comes first.
What happens when someone posts in a 1,000-member group
A housing society group has 1,000 members, and the secretary posts "Water supply off tomorrow 10 am to 2 pm". That one message becomes 1,000 deliveries and perhaps 700 push notifications. If the society's group is busy on a Sunday morning, every message multiplies the work by a thousand, and a few such groups on one server can slow everyone else on it.
Save once, fan out in the background. A queue job delivers to members in batches, grouped by the chat server they're connected to, so one server-to-server hop carries messages for many members. Push notifications for a very chatty group are collapsed into "12 new messages" rather than sent one by one. Groups past a size limit become broadcast-style, where phones fetch new posts instead of being pushed each one.
What happens when one person uses a phone and a laptop
Ravi reads a message on his laptop at work. His phone, in his bag, still shows it as unread, and when he picks it up later it buzzes for messages he's already seen. Worse, if the server only tracked one connection per user, messages might go to the laptop and never reach the phone at all.
Treat each device as its own recipient, with its own connection, its own position in each chat, and, with encryption, its own key. A message is delivered to every device of every member. Reads sync too: when the laptop sends a read receipt, the server passes it to Ravi's other devices, which clear the unread badge without buzzing.
What happens when a spammer messages 10,000 strangers
A new account starts sending the same "earn ₹5,000 a day from home" message to thousands of numbers it found online. With end-to-end encryption, the servers can't read the message to judge it, and every delivery costs the system the same as a real one.
Use what the server can see: a new account messaging thousands of people who have never saved its number, in a short time, is suspicious whatever the words say. cap new conversations per hour for young accounts, a scores behaviour and metadata, and "report" buttons let recipients send the offending message itself for review. Repeat offenders' numbers are blocked.
Watch outCommon mistakes
Polling for new messages
A first design has each phone ask "anything new?" every 2 seconds. With 20 crore phones online, that's 10 crore requests a second, nearly all answered "no", while messages still feel up to 2 seconds late. Phone batteries drain from constant radio wake-ups, and the server bill is many times what it needs to be.
Keep one open connection per device and push messages down it the moment they arrive. Use a light heartbeat every 30 to 60 seconds to keep the line alive, and fall back to push notifications when the app is closed.
Delivering before saving
To shave milliseconds, a design forwards the message to the receiver first and saves it afterwards. If the server crashes in between, a message the sender saw tick is gone forever, and if the receiver was offline, there's nothing stored to deliver later.
Save first, number it, then reply "got it" and deliver. The extra few milliseconds buy the most important promise in the app: one tick means the message cannot be lost.
Ordering by phone clocks
A design sorts messages by the time on the sender's phone. One phone's clock is three minutes fast, so its messages appear above replies that were actually sent earlier, and the conversation reads like nonsense. People in the same group see different orders.
Order by a sequence number the server assigns per conversation when it saves each message. Show the phone's time as a label if you like, but never sort by it.
Treating presence like a message
A design writes every heartbeat and every "typing…" signal to the main message database with the same care as a message. That's 45 lakh writes a second of data that's out of date within a minute, and it slows down the writes that actually matter.
Keep presence and typing indicators in memory with short expiry times, and let them be best-effort. If one is lost, the next heartbeat fixes it a few seconds later.
Interview angleIn an interview
"Design WhatsApp" or "design a chat system" is one of the most common system design questions. Interviewers use it to see whether you can handle long-lived connections, routing between servers, ordering, offline delivery and huge storage, and whether you drive the conversation in a sensible order instead of jumping to a database name.
A strong answer covers:
- requirements, both what it does and how well (latency, never lose, same order), and what's out of scope;
- a quick estimate: messages a second, storage a day, connections online, and the heartbeat load;
- the core design: persistent connections, presence, message store, push, media storage;
- the send path, with save-before-deliver and what each tick means;
- ordering with per-chat sequence numbers and dedup with client-made IDs;
- offline and multi-device sync by "last number I have";
- groups by fan-out on a queue, and a different approach for huge channels;
- storage sharded by conversation, and the history-versus-privacy choice.
Follow-up: "How does chat server A know where to send Ravi's message?" It looks Ravi up in presence, which maps each online device to its chat server, and forwards the message to that server; if there's no entry, Ravi is offline and gets a push.
Follow-up: "What if Ravi's phone receives the same message twice?" Every message carries an ID made on the sender's phone, and Ravi's phone ignores IDs it already has. Delivery is at least once; showing is effectively once.
Follow-up: "How would you support a group of 5 lakh followers?" Stop pushing to everyone. Store each post once in the channel, notify lightly or not at all, and have phones fetch new posts when the channel is opened.
Try it in the simulator
Goal: Switch a chat server off and keep messages flowing with retries.
Two chat servers sit behind a load balancer. Run it, and switch one off with Chaos: phones connected to it lose their messages until they reconnect elsewhere. Give the users 2 retries with a 100 ms backoff and try again: most messages now get through on the second attempt, the way a phone quietly reconnects and resends.
Quick checkCheck your understanding
Words in this lessonThe jargon, in plain English
Every technical word used above. In the lesson, tap any word with an ⓘ to see its meaning right there.
- Acknowledgement
- A short "got it" reply that confirms a message arrived. Without one, the sender can't be sure, so it keeps trying.
- At least once
- A delivery promise: every message arrives, but some may arrive twice, so receivers must handle repeats.
- Back-of-the-envelope estimate
- A quick, rounded calculation, like 1 crore requests a day ≈ 100 a second, used to decide which parts of a design matter. Close enough beats exact.
- CDN
- Content Delivery Network: many caches spread around the world that serve files from a location near each user.
- Cold storage
- A cheaper kind of storage for files that are rarely opened. Keeping files there costs less, but fetching them is slower or costs a little each time.
- Deduplication
- Spotting and dropping repeats, so the same message or action only counts once.
- Device token
- An address for one app on one phone, given out by Google or Apple, that a server uses to send that phone notifications.
- Draining
- Taking a server out of service gently: it stops getting new requests, finishes the ones already in progress, and only then shuts down.
- End-to-end encryption
- Scrambling a message on the sender's phone so only the receiver's phone can unscramble it. Servers in between can pass it on but can't read it.
- Exponential backoff
- Waiting longer after each failed try (say 1 s, then 2 s, then 4 s) so a struggling server gets time to recover.
- Fan-out
- Sending one thing to many places, like telling every follower about a new post or every viewer about a new comment.
- Functional requirement
- Something the system must do, such as "people can send a message" or "opening a short link redirects you".
- Graceful degradation
- When a system is overloaded, it switches off the nice-to-have parts and keeps the essential ones working, instead of failing completely.
- Group key
- A shared key that every member of a group chat receives, each copy locked for that member, so a message can be locked once and read by the whole group.
- Heartbeat
- A small "I'm alive" message a machine sends every second or so. When heartbeats stop, others assume it has failed.
- Hot key
- One item that everyone wants at once, like a celebrity's post. All the traffic lands on the one place it's stored and can overload it.
- Jitter
- A small random extra wait added to retries, so many clients that failed together don't all retry at the same instant.
- Key-value store
- A simple, very fast database that saves each value under a name, its key, and finds it again by that name, like a locker numbered by product ID.
- Load balancer
- A component that receives requests and spreads them across several servers.
- Message queue
- A buffer that holds messages until a consumer is ready to process them, so producers don't have to wait.
- Metadata
- Small facts about a bigger thing: a photo's owner, caption, size and where it's stored, but not the photo itself.
- Non-functional requirement
- How well the system must do its job: how fast, for how many people, how reliably, and how fresh the data must be.
- Object storage
- A cheap, practically unlimited store for big files like photos and videos. You save and fetch each file by a key, like a cloakroom ticket.
- Persistent connection
- A line between your phone and the server that stays open, so either side can send something the moment it happens instead of asking again and again.
- Presence
- Knowing whether a user is online right now, and which server their phone is connected to. It is kept in a fast shared store and refreshed by regular heartbeats, so it can be slightly out of date.
- Push notification
- A message the app's server sends to your phone even when the app is closed, like "Your order is on the way".
- Rate limiting
- Capping how many requests a client may make in a period, rejecting the excess.
- Read receipt
- A small message sent back when you open a chat, telling the sender you've seen their message (the blue ticks).
- Sequence number
- A number that goes up by one with every update, so an app can put updates in order and ignore old ones that arrive late.
- Shard key
- The piece of data (often the user ID) used to decide which shard a record lives on.
- Sharding
- Splitting one large dataset across several databases, each holding a part.
- Spam filter
- Software on the mail server that checks every incoming email and decides whether it goes to your inbox, your Spam folder, or nowhere at all.
- Store and forward
- Holding a message safely until the receiver is reachable, then delivering it. How chat apps deliver messages sent while your phone was off.
- TTL
- Time to live: how long a cached copy is trusted before it expires and is fetched again.
- WebSocket
- A long-lived, two-way connection between client and server, used for live updates like chat or delivery tracking.
Enjoyed this? Keep going for free
This lesson is a free preview. Sign up free to save your progress and earn XP, and to unlock all 232 lessons, the simulator challenges and mock interviews.
Sign up free