StreamingStaff Level

Real-Time Chat & Messaging Architecture

Q: How do you handle group chats with 500 members?

Instead of fanout-on-write to 500 separate user inboxes, use a single conversation stream where participants fetch updates based on their read cursor.

Scale a real-time messaging pipeline for 500M daily active users with message delivery receipts.

Practice in ClawPad Desktop ← All System Design Problems

Target Scale

Engineering Scale & Performance SLAs

Target production parameters expected in a senior or staff interview round.

Peak Throughput

1,200,000 Messages/sec Peak

Sustained peak request volume during high-traffic events.

Active Users

500 Million DAU

Daily active users generating read and write operations.

Storage Ingestion

50 TB / day

Projected data ingestion and replication storage capacity.

Latency Budget

Message latency < 100ms

Strict end-to-end percentile latency SLA constraint.

Stage 01

Functional & Non-Functional Requirements

Establish clear problem boundaries before proposing architectural components.

Functional Scope

Core System Capabilities

1-on-1 private messaging and group messaging (up to 500 participants).
Online/offline status and typing indicators.
Message status tracking (Sent, Delivered, Read).

Non-Functional Scope

Reliability & Latency SLAs

End-to-end message delivery under 100ms.
Zero message loss with persistent storage.
Support millions of concurrent persistent WebSocket connections.

Stage 02

Capacity Estimation Math

Step-by-step arithmetic conversions for QPS, storage, and bandwidth.

Dimension	Calculation Formula	Estimated Result
Message Volume	500M DAU × 100 messages/day = 50B messages/day	~580,000 avg msg/sec (~1.2M Peak msgs/sec)
Daily Message Storage	50B messages × 1 KB average metadata and text payload	~50 Terabytes / day
Concurrent WebSocket Connections	500M DAU with 20% concurrent peak = 100M simultaneous sockets	100M open TCP/TLS connections

Stage 03

Multi-Tier Architecture & Component Topology

How requests navigate ingress gateways, application logic, caching, and persistence.

Connection Gateway Tier

WebSocket Gateway Cluster · TCP Epoll / Netty Connection Handlers

Maintains persistent bi-directional TLS WebSocket connections with active client apps.

Presence & Routing Tier

Presence Service · Redis Pub/Sub State Hash

Tracks which server host ID holds the open socket for each active user UUID.

Message Pipeline Tier

Kafka Message Bus · ScyllaDB / Cassandra Message Store

Buffers messages, coordinates push notifications for offline users, and writes to disk.

Stage 04

Database Schemas & Partitioning Strategy

Entity models, indexing, and primary key partitioning.

Table: messages

PK: (conversation_id, message_id)

conversation_id (UUID)
message_id (TIMEUUID)
sender_id (UUID)
content (TEXT)
status (TINYINT)

Clustered by message_id descending for fast historical pagination.

Stage 05

Critical Architectural Trade-Offs

How to defend engineering compromises when challenged by interviewers.

Decision Point

WebSocket vs Long Polling / SSE

Option A: WebSockets (Bi-directional, low frame overhead)

Option B: Server-Sent Events / HTTP Long Polling

Rationale: WebSockets provide lowest latency bi-directional messaging with single TLS handshake.

Decision Point

Cassandra vs Relational DB for Messages

Option A: ScyllaDB / Cassandra (LSM-tree optimized for write throughput)

Option B: PostgreSQL with Sharding

Rationale: Cassandra provides horizontal write scalability and natural partitioning by conversation_id.

Technical FAQ

Frequently Asked Questions: Real-Time Chat & Messaging Architecture

Key interview questions and conceptual defenses.

How do you handle group chats with 500 members?