HLD · 8 min read

Design A Chat System

System design for a real-time chat application.

On this page
  1. Problem statement
  2. Requirements and scope
  3. Functional requirements
  4. Non-functional requirements
  5. Capacity estimation
  6. Protocol choice: WebSockets
  7. Core design: store-and-forward
  8. Write path (sender online)
  9. Cross-server message routing
  10. Online presence
  11. Database choices
  12. Group chat
  13. Media handling
  14. Message search
  15. Auth and rate limiting
  16. Bottlenecks and trade-offs

Problem statement

Design a real-time chat system like WhatsApp, WeChat, or Messenger. Users should be able to send and receive messages instantly, individually or in groups, with delivery receipts and online presence indicators.

Requirements and scope

Functional requirements

  • One-on-one and group messaging (up to 1024 members per group)
  • Real-time message delivery
  • Delivery receipts: sent (one tick), delivered (two ticks), read (blue ticks)
  • Online presence and last seen
  • Media support (images, video at a high level)
  • Message search

Non-functional requirements

  • 10M DAU
  • Highly available and low latency
  • Eventual consistency acceptable for delivery receipts
  • Messages must never be lost

Out of scope: End-to-end encryption, calls, stories.

Capacity estimation

MetricValue
DAU10M
Messages per user per day50
Total messages per day500M
Messages per second~5,800 msg/sec (500M ÷ 86,400)
Avg message size (text)~100 bytes
Write throughput~580 KB/sec
Effective operations/sec~12,000 (store + deliver)
Concurrent WebSocket connections~2M (20% of DAU active)
Chat server instances needed~40 (at 50K connections each)

Protocol choice: WebSockets

HTTP is request-response. The client asks, the server answers. Chat requires the server to push messages to the client the moment they arrive. The options:

  • Short polling: Client asks every N seconds. Even counting only the ~2M users online at once, polling every 5 seconds is 400,000 requests/second just for polling, and almost all of them come back empty. Unacceptable.
  • Long polling: Client holds connection until a message arrives. Better but still half-duplex, expensive at scale.
  • WebSockets: Full-duplex, persistent, bi-directional. Server pushes instantly. This is what WhatsApp, Messenger, and Slack all use.

Each chat server maintains ~50,000 concurrent WebSocket connections. With ~2M concurrent users, roughly 40 chat server instances sit behind the load balancer.

Core design: store-and-forward

A message is always persisted before delivery is attempted, first to Kafka, which is replicated and durable, and then to Cassandra. This is non-negotiable. If the recipient's device dies mid-delivery, the message must survive. The flow is always: write first, deliver second. Never skip persistence optimistically.

Write path (sender online)

  1. Sender sends a message over their WebSocket connection to Chat Server A
  2. Chat Server A generates a unique Snowflake ID (time-sortable, distributed) for the message
  3. Chat Server A publishes the message to Kafka, keyed by conversationId so all messages of one chat stay in order on the same partition
  4. Chat Server A immediately returns a one-tick acknowledgement to the sender, meaning "we have it"
  5. Delivery workers consume from Kafka and write the message durably to Cassandra
  6. Check recipient's online status in the Presence store (Redis)

If recipient is online:

  1. Worker publishes a delivery event to the recipient's Redis Pub/Sub channel
  2. Chat Server B, which is subscribed to that channel because the recipient is connected to it, pushes the message to the recipient over WebSocket
  3. Recipient device sends an ACK back through WebSocket
  4. Chat Server A notifies the sender and two ticks appear

If recipient is offline:

  1. Message is already safely in Cassandra
  2. Worker triggers the Notification service to send a push notification to the recipient's device
  3. When recipient comes back online, their client pulls missed messages from Cassandra
  4. ACK flows back through WebSocket, two ticks update

Read receipts (blue ticks):

  1. Triggered when the recipient opens the specific conversation
  2. Client sends a read event over WebSocket
  3. Chat Server updates the message status in Cassandra
  4. Sender gets notified and blue ticks appear
mermaid
sequenceDiagram
    participant S as Sender
    participant A as Chat Server A
    participant K as Kafka
    participant W as Delivery worker
    participant DB as Cassandra
    participant B as Chat Server B
    participant R as Recipient
    S->>A: message
    A->>K: publish (Snowflake ID)
    A-->>S: ✓ sent
    K->>W: consume
    W->>DB: write message
    W->>B: Redis Pub/Sub (recipient's channel)
    B->>R: push over WebSocket
    R-->>B: ACK
    B-->>A: delivered
    A-->>S: ✓✓ delivered

Cross-server message routing

With 40 chat server instances, the sender and recipient are rarely connected to the same server. Chat servers use Redis Pub/Sub to solve this. Each server subscribes to channels for its connected users. When a delivery worker needs to push to a recipient on Server B, it publishes to that user's Redis channel, and Server B picks it up and delivers over WebSocket.

Redis Pub/Sub is fire-and-forget: if nobody is subscribed at that moment, for example because the recipient is reconnecting to a different server, the event is simply dropped. That's fine here, because the message is already in Cassandra, and the client syncs anything it missed when it reconnects. Pub/Sub is only the fast path, never the source of truth.

Online presence

Each client sends a heartbeat ping to its chat server every 5 seconds. The server updates a Redis key presence:{userId} with the current timestamp and a TTL of 15 seconds. If 3 consecutive heartbeats are missed, the TTL expires and the user is marked offline automatically, no explicit logout is needed.

"Last seen" is the timestamp of the last heartbeat recorded. Since the presence key disappears when its TTL runs out, last seen has to be kept somewhere that doesn't expire, for example a separate lastSeen:{userId} key that the server updates on each heartbeat (or when the WebSocket closes) without a TTL.

redis
SET presence:42 1717230000 EX 15     # on every heartbeat
SET lastSeen:42 1717230000           # no TTL, survives going offline

Database choices

StoreTechnologyWhy
Message storeCassandraPartitioned by conversationId, sorted by timestamp, so fetching a page of a chat is one sequential read. Best-in-class write throughput. Discord famously stored its messages in Cassandra
Presence storeRedis (KV)Sub-millisecond reads, TTL-based expiry for automatic offline detection
Cross-server routingRedis Pub/SubLightweight, fast, already in the stack
Message searchElasticsearchFull-text search across messages, CDC pipeline syncs from Cassandra asynchronously

Why Cassandra for messages specifically: The core access pattern is "fetch the messages in conversation X, ordered by time, starting from a given message." Cassandra partitions data by conversationId, and all messages of a chat live on the same node. Within a partition, rows are sorted by message ID (a Snowflake ID, so effectively by time), making range reads extremely fast. It also handles the ~5,800 writes/second comfortably.

sql
CREATE TABLE messages (
  conversation_id bigint,
  message_id      bigint,      -- Snowflake ID, sorts by time
  sender_id       bigint,
  body            text,
  PRIMARY KEY ((conversation_id), message_id)
) WITH CLUSTERING ORDER BY (message_id DESC);

One thing to watch: a very old, very busy group chat can grow into a huge single partition, which is a hot partition problem. The usual fix is to add a time bucket to the partition key, for example (conversation_id, month), so each partition stays a manageable size.

Group chat

For groups up to 1024 members, fan-out happens at the delivery worker level. When a message arrives in a Kafka partition for a group conversation:

  1. Worker fetches the full member list from a Group service (backed by a SQL DB)
  2. For each member, worker checks their presence status in Redis
  3. Online members get delivery via Redis Pub/Sub to WebSocket
  4. Offline members get a push notification
  5. The message is written once to Cassandra under the group conversationId, not duplicated per member

This approach works well up to the 1024 member cap. Beyond that, every message turns into thousands of deliveries at once, and a hybrid approach (fan-out on read for very large groups, the same idea as the celebrity problem in a news feed) would be needed.

Media handling

Text messages flow through the WebSocket -> Kafka -> Cassandra pipeline. Media (images, video) is handled separately to keep the message pipeline lean:

  1. Client uploads the media file directly to S3 via a pre-signed URL
  2. S3 stores the raw asset and acts as the origin for CloudFront (CDN), which pulls and caches it at the edge the first time it's requested
  3. The chat message contains only the media URL and metadata, not the binary
  4. Recipients fetch media directly from CloudFront, not through the chat servers

This keeps the WebSocket servers free of large payloads and lets CDN edge nodes serve media at scale globally.

A CDC (change-data-capture) pipeline tails Cassandra writes and syncs new messages to Elasticsearch asynchronously. Users searching for messages hit a dedicated Search service backed by Elasticsearch, completely decoupled from the real-time delivery path. This avoids putting full-text search load on Cassandra, which is not designed for it.

Auth and rate limiting

Authentication is handled at the load balancer / API gateway layer using JWT tokens. The WebSocket upgrade handshake includes a token in a header or the query string, and the gateway validates it before the connection is established. (A header is safer where the client allows it, since query strings tend to end up in access logs. A common middle ground is a short-lived, single-use connection ticket in the query string.) Rate limiting on message sends is enforced per user to prevent abuse and protect Kafka from ingest spikes.

Bottlenecks and trade-offs

  • Delivery latency vs durability: Persisting before delivery adds a small latency. This is the right trade-off: a slightly slower delivery is always better than a lost message.
  • Cross-server routing complexity: Redis Pub/Sub adds a hop for cross-server delivery. At 40 servers this is manageable, but as the fleet grows, a more sophisticated service mesh or consistent hashing scheme (routing users to a deterministic server) becomes worthwhile.
  • Group fan-out ceiling: The 1024 member cap is an architectural constraint. Beyond it, per-member delivery worker fan-out becomes expensive. Very large group broadcasts would benefit from a pub/sub tree model rather than a flat fan-out.
  • Presence accuracy: Heartbeat-based presence has a ~15-second inaccuracy window. A user who loses connectivity appears online for up to 15 seconds. This is a known and accepted trade-off in every major chat system.