Rate Limiting
Rate limiting protects services by capping the number of requests allowed in a time window: excess requests are rejected or queued. It prevents traffic spikes from overwhelming a system, and also powers anti-abuse and API metering. In high-availability design it is the core trade of sacrificing some requests to keep the whole service up.
How it works
Three algorithms dominate: fixed-window counters are simplest but can admit double traffic at window boundaries; sliding windows and token buckets smooth that out, and token buckets additionally allow controlled bursts, which is why most gateways default to them. Limiting can sit at the CDN/WAF edge, in the reverse proxy (Nginx's limit_req), or inside the application (connection pools, semaphores). Two practical pitfalls: first, status code semantics — 429 Too Many Requests is the correct answer, yet Nginx's limit_req returns 503 by default, polluting failure monitoring; set limit_req_status 429 explicitly (our 503 troubleshooting article analyzes this). Second, pair rejections with the Retry-After header and client-side exponential backoff, otherwise rejected clients retry immediately and amplify the load.
Examples
- 1.An API gateway caps each API key at N requests per minute, returning 429 beyond that
- 2.Login endpoints are rate limited per IP to blunt credential stuffing
- 3.Nginx throttles download paths so a few users cannot saturate the bandwidth