Rate Limiting

How it works

Rate limiting caps how many requests a client can make in a window, protecting services from abuse and overload. Token bucket, leaky bucket, and sliding-window counters each balance burst tolerance against strictness.

Mental models

• Token bucket refills at a fixed rate and allows controlled bursts.
• Leaky bucket queues requests and drains at a constant rate, smoothing output.
• Fixed window is cheap but lets a boundary burst hit up to 2× the limit.
• Sliding window counts the trailing interval — fairer, but tracks more state.
• Limits live at the edge (API gateway) keyed by IP, user, or API key.

Common pitfalls

• Per-instance counters undercount across a fleet — distributed rate limiting needs shared state (Redis) or the real limit is (per-instance limit × instance count).
• Rejecting with a bare error invites retry storms; return 429 with a Retry-After header so well-behaved clients back off.
• Too strict and you throttle legitimate bursty usage; too loose and one bad actor can still degrade the service for everyone else.