500,000 QPS Peak
Sustained peak request volume during high-traffic events.
High-ThroughputSenior Level
Distributed Rate Limiter & Token Bucket
Protect multi-region microservices from traffic spikes, brute-force attacks, and noisy neighbors.
Target Scale
Engineering Scale & Performance SLAs
Target production parameters expected in a senior or staff interview round.
50 Million DAU
Daily active users generating read and write operations.
15 GB memory cache
Projected data ingestion and replication storage capacity.
p99 < 2ms
Strict end-to-end percentile latency SLA constraint.
Stage 01
Functional & Non-Functional Requirements
Establish clear problem boundaries before proposing architectural components.
Core System Capabilities
- Limit client requests based on API key, user ID, or client IP address.
- Return HTTP 429 Too Many Requests with Retry-After headers when exceeded.
- Support tiered rate limits across Free, Pro, and Enterprise client tiers.
Reliability & Latency SLAs
- Ultra-low latency overhead (< 2ms overhead per API request).
- High availability with fail-open fallback during cache partition.
- Distributed synchronization with minimal cross-region locking.
Stage 02
Capacity Estimation Math
Step-by-step arithmetic conversions for QPS, storage, and bandwidth.
| Dimension | Calculation Formula | Estimated Result |
|---|---|---|
| Peak Request Volume | 50M DAU × 100 requests/day ÷ 86,400s × 8x peak factor | ~462,000 QPS (Round up to 500k QPS) |
| Memory Footprint | 50M active users × 64 bytes (Key + Counter + Timestamp) | ~3.2 GB RAM (Triple replication = ~9.6 GB) |
| Network Bandwidth | 500k QPS × 128 bytes sync payload | ~64 MB/s ingress sync |
Stage 03
Multi-Tier Architecture & Component Topology
How requests navigate ingress gateways, application logic, caching, and persistence.
Envoy Proxy Sidecar · Geo-DNS / Cloudflare WAF
Inspects incoming request headers, queries local in-memory token bucket, routes or rejects.
Redis Cluster / Memcached · Sliding Window Counter
Stores authoritative sliding window counters with Lua scripts for atomic increments.
Kafka Event Stream · Prometheus / Grafana
Asynchronously logs rate-limited drop events for security and abuse detection.
Stage 04
Database Schemas & Partitioning Strategy
Entity models, indexing, and primary key partitioning.
PK: tier_id
- tier_id (VARCHAR)
- max_requests (INT)
- window_seconds (INT)
- burst_capacity (INT)
Cached in memory across Envoy edge instances on startup.
PK: client_key
- client_key (VARCHAR)
- current_bucket (BIGINT)
- last_refilled_at (TIMESTAMP)
Managed as atomic Redis hash keys with TTL expiration.
Stage 05
Critical Architectural Trade-Offs
How to defend engineering compromises when challenged by interviewers.
Token Bucket vs Sliding Window Log
Option A: Token Bucket (Memory efficient, fixed capacity)
Option B: Sliding Window Log (Exact precision, high memory per request)
Rationale: Selected Token Bucket with local sliding sub-windows to guarantee sub-millisecond lookups at 500k QPS.
Fail-Open vs Fail-Closed on Redis Partition
Option A: Fail-Open (Allow traffic, preserve user experience)
Option B: Fail-Closed (Block traffic, strictly protect backend)
Rationale: Fail-open with local in-memory rate limiting sidecars to avoid cascading global outages.
Technical FAQ
Frequently Asked Questions: Distributed Rate Limiter & Token Bucket
Key interview questions and conceptual defenses.
Where should the rate limiter sit in the request pipeline?
For lowest latency, deploy rate limiters as sidecar filters within the API gateway (Envoy) rather than an isolated standalone service that requires an additional network hop.
How do you handle multi-region rate synchronization?
Use local in-memory token buckets at each regional edge that periodically batch-sync delta increments to the regional Redis cluster, avoiding cross-continent locking.
Simulate this architecture
Download ClawPad