50,000 Messages/sec Peak
Sustained peak request volume during high-traffic events.
StreamingStaff Level
Slack & Discord Real-Time Guild Messaging
Manage massive shared channels with live fanout to 100,000+ online members, message edit history, and reaction counters.
Target Scale
Engineering Scale & Performance SLAs
Target production parameters expected in a senior or staff interview round.
50 Million DAU
Daily active users generating read and write operations.
15 TB/day message logs
Projected data ingestion and replication storage capacity.
Channel fanout p99 < 80ms
Strict end-to-end percentile latency SLA constraint.
Stage 01
Functional & Non-Functional Requirements
Establish clear problem boundaries before proposing architectural components.
Core System Capabilities
- Send and receive rich messages in channels, threads, and direct messages.
- Live fanout of messages, typing indicators, and emoji reactions to all online channel members.
- Channel message history pagination and full-text search.
- Member presence tracking (online, idle, offline) across 100,000-member communities.
Reliability & Latency SLAs
- Message delivery p99 latency under 80ms worldwide.
- Zero message loss (durable append-only storage).
- Connection resilience: seamless session reconnection without message duplication.
Stage 02
Capacity Estimation Math
Step-by-step arithmetic conversions for QPS, storage, and bandwidth.
| Dimension | Calculation Formula | Estimated Result |
|---|---|---|
| Peak Message Ingestion QPS | 1 Billion messages/day / 86,400s * 3x peak factor | 34,722 peak messages/sec |
| Real-Time WebSocket Fanout Volume | Average active channel has 200 members -> 35k msgs * 200 members | 7,000,000 WebSocket frames delivered/sec |
| Daily Message Storage Growth | 1B messages * 600 bytes (content + metadata + formatting) | 600 GB/day compressed message storage |
Stage 03
Multi-Tier Architecture & Component Topology
How requests navigate ingress gateways, application logic, caching, and persistence.
WebSocket Gateway Fleet (Elixir/Go) · BGP Anycast Routing · Connection State Ring
Maintain millions of persistent TLS WebSocket connections, heartbeat health checks, and packet compression (Zlib).
Distributed Pub/Sub (Redis / NATS) · Guild Session Router · Fanout Workers
Route outbound messages to the specific gateway servers hosting connected channel members.
Message Service · Distributed NoSQL (ScyllaDB / Cassandra) · Snowflake ID Mint
Assign time-ordered 64-bit IDs and write immutable message records partitioned by channel_id.
Presence Service · Reaction Coalescing Worker · Redis In-Memory Hashes
Debounce rapid emoji reaction clicks and maintain aggregate counts without overloading primary storage.
Stage 04
Database Schemas & Partitioning Strategy
Entity models, indexing, and primary key partitioning.
PK: (channel_id, bucket, message_id)
- channel_id (BIGINT)
- bucket (INT)
- message_id (BIGINT)
- author_id (BIGINT)
- content (TEXT)
- created_at (TIMESTAMP)
Partitioned by channel_id and 10-day time bucket to avoid massive unbounded partitions in ScyllaDB.
PK: (channel_id, user_id)
- channel_id (BIGINT)
- user_id (BIGINT)
- role (VARCHAR(32))
- joined_at (TIMESTAMP)
Indexed on user_id to resolve all subscribed channels on initial login sync.
PK: (message_id, emoji_code, user_id)
- message_id (BIGINT)
- emoji_code (VARCHAR(32))
- user_id (BIGINT)
- created_at (TIMESTAMP)
Clustered for fast reaction tally counts and user emoji toggling.
Stage 05
Critical Architectural Trade-Offs
How to defend engineering compromises when challenged by interviewers.
Storage Engine: Relational PostgreSQL vs Distributed ScyllaDB
Option A: PostgreSQL with Sharding
Option B: Distributed Wide-Column ScyllaDB (C++ Cassandra rewrite)
Rationale: Discord famously migrated from MongoDB to Cassandra and ultimately ScyllaDB because trillions of chat messages require linear write scalability and predictable sub-millisecond p99 latencies without GC pauses.
Large Guild Fanout: Broadcast to All vs Lazy Presence Sync
Option A: Broadcast presence to every member in a 500k-member guild
Option B: Lazy Presence (only send presence updates for visible chat viewport)
Rationale: Broadcasting presence in a 500k-member guild creates quadratic O(N^2) traffic storms. Lazy sync sends presence only for members currently visible in the active channel window.
Technical FAQ
Frequently Asked Questions: Slack & Discord Real-Time Guild Messaging
Key interview questions and conceptual defenses.
How does Discord paginate channel message history efficiently?
Clients query messages with `BEFORE {message_id}` using time-sorted Snowflake IDs, executing high-speed range queries across clustered partition keys without offset pagination.
How do you prevent duplicate messages when a mobile client reconnects on flaky LTE?
Clients attach a client-generated UUID nonce to each message. The gateway checks a 5-minute Redis idempotency cache before inserting.
Simulate this architecture
Download ClawPad