Building Resilient APIs

An API that works perfectly at 100 requests per second but falls over at 10,000 isn't a production API — it's a prototype. Resilience is what separates the two.

Rate limiting with graduated responses. We use a token bucket algorithm with three tiers: at 60% capacity, we add a warning header. At 80%, we start slowing responses by 50ms. At 100%, we return 429 with a Retry-After header. Graduated pressure lets clients adapt without hard cutoffs.

Idempotency for all mutations. Every POST and PATCH endpoint accepts an Idempotency-Key header. The server deduplicates within a 24-hour window. This makes client retries safe — the network can drop a response, the client retries, and the server returns the original result instead of creating a duplicate.

Graceful degradation. When a downstream dependency fails, the API should degrade, not crash. If the recommendation service is down, return an empty list instead of a 500. If the search index is unavailable, fall back to a database query. Every dependency gets a timeout and a fallback.

Structured error responses. Every error returns a consistent shape: `{ error: string, code: string, details?: unknown }`. The code field maps to a documented enum so clients can handle errors programmatically. No raw stack traces, no HTML error pages, no inconsistent shapes between endpoints.

We test resilience with a chaos proxy that sits between our API and every dependency. It can inject latency, drop requests, or return garbage responses. Every endpoint gets exercised against each failure mode before it ships.