2,000,000 QPS
Sustained peak request volume during high-traffic events.
StorageSenior Level
Distributed In-Memory Cache with Consistent Hashing
Build a fault-tolerant, low-latency in-memory cache supporting LRU eviction and Raft replication.
Target Scale
Engineering Scale & Performance SLAs
Target production parameters expected in a senior or staff interview round.
N/A (Infrastructure Tier)
Daily active users generating read and write operations.
500 GB RAM cluster
Projected data ingestion and replication storage capacity.
Read < 1ms
Strict end-to-end percentile latency SLA constraint.
Stage 01
Functional & Non-Functional Requirements
Establish clear problem boundaries before proposing architectural components.
Core System Capabilities
- Support O(1) Get(key) and Set(key, value, ttl) operations.
- Automatic LRU / LFU memory eviction when memory threshold is reached.
- Automatic node discovery and rebalancing.
Reliability & Latency SLAs
- Sub-millisecond read/write latency.
- High availability with master-replica failover via Raft consensus.
- Even key distribution with zero hot-shard clustering.
Stage 02
Capacity Estimation Math
Step-by-step arithmetic conversions for QPS, storage, and bandwidth.
| Dimension | Calculation Formula | Estimated Result |
|---|---|---|
| Throughput Capacity | 2,000,000 QPS across 50 cache nodes | ~40,000 QPS per node (Easily within single-thread memory bounds) |
| Data Sharding Space | 500 GB cache capacity ÷ 50 nodes | 10 GB memory per shard node with 2x replicas |
Stage 03
Multi-Tier Architecture & Component Topology
How requests navigate ingress gateways, application logic, caching, and persistence.
Consistent Hash Ring Router · Virtual Node Ring (256 vnodes/node)
Hashes keys to determine the target cache shard without querying a centralized coordinator.
Concurrent Hash Map · Doubly Linked List LRU Queue
Executes O(1) lookups and evictions in memory with non-blocking read locks.
Gossip Protocol / Zookeeper · Raft State Machine
Monitors node liveness and triggers replica promotion within 500ms of node failure.
Stage 04
Database Schemas & Partitioning Strategy
Entity models, indexing, and primary key partitioning.
PK: node_id
- node_id (UUID)
- ip_address (VARCHAR)
- vnode_tokens (ARRAY<INT>)
- status (ENUM)
Synchronized across client SDKs via gossip broadcasts.
Stage 05
Critical Architectural Trade-Offs
How to defend engineering compromises when challenged by interviewers.
Client-Side Routing vs Proxy Routing
Option A: Client-Side Consistent Hashing (Zero extra network hop)
Option B: Twemproxy / Envoy Proxy (Centralized management, adds 1ms hop)
Rationale: Client-side routing chosen for sub-millisecond execution in high-throughput internal microservices.
LRU vs LFU Eviction
Option A: LRU (Evicts least recently used, handles recency spikes)
Option B: LFU (Evicts least frequently used, retains historical anchors)
Rationale: LRU with 2-Q segmented queue handles both burst scans and steady-state read workloads.
Technical FAQ
Frequently Asked Questions: Distributed In-Memory Cache with Consistent Hashing
Key interview questions and conceptual defenses.
How do virtual nodes prevent hot spotting in consistent hashing?
By assigning 256 virtual tokens on the hash circle per physical machine, keys distribute uniformly across physical nodes, minimizing variance from 30% to under 2%.
Simulate this architecture
Download ClawPad