Design: a chat app

The full lesson, with a worked example: Design: a chat app.

Requirements

Functional

• One-to-one and group messages, groups up to about 1,000 members.
• Offline delivery: messages reach people who were offline as soon as they reconnect.
• Ticks: sent (saved on the server), delivered (on the phone), read.
• Media: photos, videos, voice notes.
• Presence: online status and last seen.
• Many devices: the same chats on a phone and a laptop.

Non-functional

• About a second from send to screen when both are online.
• Never lose a message once the sender sees one tick.
• Same order for everyone in a chat.
• Survive server failures without losing or doubling messages.
• Private, ideally end-to-end encrypted.

Out of scope (say so): voice and video calls, payments, stories, broadcast channels (mentioned at the end).

Estimates

• Messages: 50 crore daily users × 40 messages = 2,000 crore a day ÷ about 86,400 seconds ≈ 2.3 lakh a second; evening peaks about 7 lakh; New Year's midnight several times that.
• Text storage: about 200 bytes a message × 2,000 crore ≈ 4 TB a day, about 1.5 PB a year before copies.
• Media: 1 in 20 messages is a photo of about 200 KB: 100 crore photos ≈ 200 TB a day, fifty times the text.
• Connections: 20 crore online at a busy moment; about 5 lakh mostly idle connections per server → 400 servers, plan about 600 with spare room.
• Heartbeats: 20 crore phones every 45 seconds ≈ 45 lakh a second, more than the messages.

What this says: message volume is manageable once spread out. The hard parts are holding crores of connections, storing petabytes, and keeping media and presence off the message path.

API

Most traffic flows over one WebSocket per device, as small typed frames:

• Client to server: send (client message ID, chat ID, body or media link), ack (chat ID, sequence number), read (chat ID, up to sequence number), typing, sync (for each chat, the highest number the device has).
• Server to client: saved (client message ID → sequence number: one tick), message, delivered and read receipts, presence updates.

Plain HTTP for everything else:

• POST /v1/media → an upload link to object storage; the phone uploads directly and puts the returned link in the message.
• GET /v1/chats/:chatId/messages?before=:seq&limit=50 → older history when you scroll up.
• POST /v1/groups, POST /v1/groups/:id/members → create groups, add or remove members.
• POST /v1/devices → register a device and its push device token.

Data model

• messages, sharded by chat_id: chat_id, seq (per-chat sequence number), msg_id (made on the sender's phone, for dedup), sender_id, body or encrypted blob, media_url, sent_at (display only). Sorted by seq within a chat.
• chats: chat_id, type (direct or group), members, last_seq.
• device_cursors: device_id, chat_id, delivered_seq, read_seq. One row per device per chat, so each device catches up on its own.
• presence (in memory, not a database): user_id and device_id → chat server, with a TTL of about 60 seconds refreshed by heartbeats.
• devices: device_id, user_id, push token, public key for encryption.

High-level design

• Chat servers hold the open connections, through a load balancer. Apart from the sockets, they keep no state.
• Presence maps each online device to its chat server, in a fast in-memory store.
• The message store saves every message before delivery; built for a huge stream of small appends.
• The fan-out queue and workers turn one group message into many deliveries.
• The push service wakes offline phones through the phone makers' push systems.
• Media storage holds photos and videos, served through a CDN; messages carry only a link and a tiny preview.

The send path:

1. 1

   Asha's phoneChat server A

   Send m-77 (ID made on her phone) over the open line

2. 2

   Chat server AMessage store

   Save, give it number 1,204 in chat c-17

3. 3

   Chat server AAsha's phone

   Saved: one tick

4. 4

   Chat server APresence

   Where is Ravi connected?

5. 5

   PresenceChat server A

   Chat server B (or nothing: send a push)

6. 6

   Chat server AChat server B

   Deliver 1,204 to Ravi

7. 7

   Chat server BRavi's phone

   Message 1,204 down his open line

8. 8

   Ravi's phoneChat server B

   Got it: Asha sees two ticks

Save first, deliver second: that order is what makes one tick a real promise.

Deep dives

Ordering and duplicates

• Order by the per-chat number assigned when the server saves the message, never by phone clocks, which disagree by minutes.
• One writer per chat (or a conditional increment in the store) hands out numbers, so two messages can't get the same one.
• Duplicates: a phone that loses the saved reply resends with the same msg_id; the server stores it once. This is deduplication.
• Delivery is at least once; the receiving phone ignores IDs it already has, so showing is effectively once.

Offline delivery and many devices

• Each device remembers the highest number it has per chat. On reconnect it sends sync: "chat c-17 up to 1,203", and the server sends 1,204 onwards.
• This is store and forward, and it means offline delivery is the normal path run later, not a separate system.
• A phone and a laptop are two recipients, each with its own cursor, connection and key. Read receipts sync across a user's devices so the phone stops buzzing for messages read on the laptop.

Groups and pub-sub

• One message is saved once in the group's chat, then one job goes on a queue; workers deliver to each member's devices. This is fan-out.
• Workers batch members by chat server, so one server-to-server hop carries messages for many members. Delivery between chat servers can run over pub-sub topics keyed by server.
• Numbers: a 200-member group means 200 deliveries a message; a busy 1,000-member group at 10 messages a minute is 10,000 deliveries a minute.
• With encryption, the sender encrypts once with a group key every member's devices hold, and the key changes when members leave.
• Push notifications for chatty groups collapse into "12 new messages".

Storage

• Shard key: conversation ID. Opening a chat reads one shard; writes from crores of chats spread evenly.
• Not by time: a "today" shard would take every new message in the world, a perfect hot spot.
• Very active chats split further into time buckets so no single partition grows forever.
• Recent months on fast disks; older months to cold storage.

Presence at scale

• 45 lakh heartbeats a second can't each be a database write. Presence lives in memory, split across machines, and may be slightly wrong.
• A missed heartbeat lets the entry expire after about 60 seconds, so a phone in a tunnel becomes "offline" and gets a push instead.
• Typing indicators and last seen are best-effort and are the first things dropped under load.

Server failure and reconnects

• A chat server crash drops about 5 lakh connections at once. Phones reconnect with exponential backoff and jitter, spreading the wave over a minute.
• Unacknowledged messages are resent with the same ID: nothing lost, nothing doubled.
• Planned restarts use graceful draining, moving phones over in batches.

Trade-offs

• Open connections vs polling. Connections cost server memory and a little battery; polling every 2 seconds from 20 crore phones is 10 crore requests a second, nearly all "nothing new", and still feels late.
• Keep history vs delete after delivery. Keeping everything gives instant history on a new laptop but costs petabytes and holds a valuable target. Deleting once every device has a message saves storage but leans on phone backups.
• Encryption vs server features. If servers can't read messages, search, content spam filters and previews move to phones or work only from metadata.
• Push vs pull for big groups. Pushing to every member works up to a few thousand. Broadcast channels with lakhs of followers store each post once and let phones fetch on open.
• Strong vs loose promises. Messages get strong ones (saved, ordered, never lost); presence and typing get loose ones, which is what makes the scale affordable.

Interview talking points

• Agree requirements, including "never lose after one tick" and "same order", and what is out of scope.
• Estimate messages a second, storage a day, connections and the heartbeat load.
• Draw persistent connections, presence, message store, push and media storage.
• Walk the send path: save, number, one tick, look up presence, deliver, two ticks.
• Explain per-chat sequence numbers and client-made IDs.
• Show offline and multi-device sync as "send me everything after N".
• Fan out groups through a queue; switch to pull for huge channels.
• Shard by conversation and explain why not by time.

For experienced engineers

Keep per-device pending-delivery state separate from long-term history, so pruning delivered messages is cheap. Wide-column stores with a partition key of chat plus time bucket and rows ordered by sequence number suit the append-heavy pattern.

Follow-up questions

How does chat server A know where Ravi is? It looks him up in presence, which maps each online device to its chat server, and forwards the message there. No entry means offline: send a push and wait for his sync.

What if Ravi's phone receives a message twice? Every message carries an ID made on the sender's phone, and his phone ignores IDs it already has. Delivery is at least once; showing is once.

What happens at midnight on New Year's Eve? Add servers in advance, let fan-out queues absorb the surge so delivery slows from one second to perhaps ten, and shed typing, last seen and read receipts before anything else. This is graceful degradation; saving messages always comes first.

How would you support a channel with 5 lakh followers? Stop pushing to everyone. Store each post once, notify lightly or not at all, and have phones fetch new posts when the channel is opened.

How do you fight spam if servers can't read messages? Use behaviour and metadata: a new account messaging thousands of strangers is suspicious whatever it says. Rate limit new conversations for young accounts and let recipients report the message itself.

Why not send the photo through the chat servers? At about 200 TB a day it would swamp them. The phone uploads straight to object storage and sends a 2 KB message with the link; the receiver downloads from a nearby CDN copy.