How to design retries with exponential backoff, jitter, retry budgets, cancellation, idempotency and tests for JavaScript, TypeScript and Python services.
Decide whether the operation is retryable before calculating a delay
The first question is not how long to sleep. It is whether the operation can be repeated without creating another side effect. Reads are often safe. Writes may require an idempotency key, a conditional update or a transaction-specific recovery path.
Retry transient failures such as selected timeouts, connection resets, rate limits and explicit service-unavailable responses. Do not automatically retry authentication failures, malformed input, permission errors or deterministic business-rule failures.
When the server returns Retry-After, respect it within the client’s overall deadline. The server has more information about its recovery window than a generic client formula.
Backoff limits request pressure; jitter prevents synchronization
Exponential backoff grows the delay after each failed attempt. A cap prevents the delay from becoming impractical. Jitter randomizes the selected delay so thousands of clients do not wake at the same boundary and hit the recovering service together.
Full jitter chooses a random value between zero and the capped exponential delay. Equal or decorrelated jitter can reduce variance for other workloads. The important property is that clients do not share one deterministic schedule.
ceiling = min(max_delay, base_delay * 2 ** attempt)
delay = random_between(0, ceiling)
sleep(delay)Use a retry budget, not only a maximum attempt count
Attempt limits are easy to understand but incomplete. Three attempts can take milliseconds for one service and thirty seconds for another. Track both maximum attempts and maximum elapsed time, then stop when either boundary is reached.
Propagate cancellation from the caller. A user navigation, deployment shutdown or expired request should interrupt sleep and in-flight work. Background retries that outlive their owner consume capacity and make incidents harder to diagnose.
- Maximum attempts.
- Maximum total elapsed time.
- Per-attempt timeout.
- Backoff cap.
- Caller cancellation signal.
- Operation idempotency state.
- Allowed error and status classes.
Log the reason and budget without leaking credentials
A retry metric should distinguish first-attempt success from eventual success. Record the operation name, attempt number, error class, selected delay and final outcome. Redact authorization headers, tokens and request bodies.
High retry volume can be an early incident signal. Track retry rate, exhausted budgets and added latency by dependency. If most requests succeed only after retries, the service is not healthy simply because the user eventually received a response.
Test time deterministically
Inject the clock, random source and sleep function so the test suite does not wait in real time. Verify the range of jittered delays, cap behavior, retry eligibility and cancellation. A deterministic random sequence makes edge cases reproducible.
For side-effecting operations, test the idempotency behavior at the integration boundary. The retry utility can schedule another attempt; only the operation and its storage contract can prove that a repeated request is harmless.
- Non-retryable errors fail immediately.
- Transient errors stop after the attempt or elapsed-time budget.
- Jitter remains inside the expected range.
- Delay never exceeds the configured cap.
- Cancellation interrupts waiting and work.
- Retry-After is respected within the caller deadline.
- Idempotency prevents repeated side effects.
FAQ
Frequently asked questions
What is jitter in a retry strategy?
Jitter randomizes retry delays so clients do not synchronize and overwhelm a recovering dependency at the same time.
Which errors should be retried?
Retry only errors classified as transient and only when the operation is safe to repeat. Authentication, validation and deterministic business errors should normally fail immediately.
How many retry attempts should I use?
There is no universal number. Combine an attempt limit with a maximum elapsed-time budget and the caller’s deadline.
Does exponential backoff make a write safe to retry?
No. Safety comes from the operation’s idempotency or transaction design. Backoff only changes request timing.