Retry Pattern in Microservices: Exponential Backoff, Jitter and Retry Budgets

Retry is additional traffic

During a dependency outage, retry can help or it can become the outage amplifier.

If 10,000 requests fail and each one retries three times, the unhealthy dependency receives up to tens of thousands of additional attempts.

My default attitude is:

retry less, classify carefully, and stay inside the original deadline.

Retryable vs non-retryable

Usually retryable:

connection reset
temporary DNS/network failure
HTTP 502/503
HTTP 429 when Retry-After is present
short-lived timeout for an idempotent operation

Usually not retryable:

400 validation failure
401/403 authentication/authorization
business rejection
unsupported request
duplicate business operation without idempotency

Exponential backoff + jitter

attempt 1 -> failure
wait 100ms + random jitter

attempt 2 -> failure
wait 200ms + random jitter

attempt 3 -> stop/fallback

NestJS/RxJS:

return firstValueFrom(
  this.http.get(url, {
    timeout: 1200,
  }).pipe(
    retry({
      count: 2,
      delay: (error, retryCount) => {
        if (!isTransient(error)) {
          throw error;
        }

        const base =
          100 * 2 ** (retryCount - 1);

        const jitter =
          Math.floor(Math.random() * 75);

        return timer(base + jitter);
      },
    }),
  ),
);

Why jitter matters

Without jitter, 1,000 pods can fail at the same time and all retry exactly 100 ms later.

That creates synchronized traffic spikes.

Randomization spreads the load.

Deadline-aware retry

Suppose the API SLA is 2.5 seconds.

Bad design:

attempt 1 timeout 2s
retry
attempt 2 timeout 2s
retry
attempt 3 timeout 2s

The user can wait six seconds or more.

I compute the remaining deadline:

async function callWithDeadline(
  deadlineAt: number,
) {
  for (let attempt = 1; attempt <= 2; attempt++) {
    const remaining =
      deadlineAt - Date.now();

    if (remaining <= 100) {
      throw new DeadlineExceededError();
    }

    try {
      return await callDependency(
        Math.min(1200, remaining - 50),
      );
    } catch (error) {
      if (
        attempt === 2 ||
        !isTransient(error)
      ) throw error;

      await sleep(
        Math.min(
          150 + Math.random() * 75,
          remaining / 3,
        ),
      );
    }
  }
}

Retry-After

For HTTP 429/503:

const retryAfter =
  response.headers['retry-after'];

if (retryAfter) {
  const delayMs = parseRetryAfter(retryAfter);
  await sleep(
    Math.min(delayMs, remainingBudget),
  );
}

Respecting server backoff signals is better than blindly hammering it.

Retry ownership

I define where the retry happens.

Example:

Client           no automatic retry for booking create
Gateway          no retry
Booking service  one controlled provider retry
Provider SDK     retries disabled

That keeps the total attempt count understandable.

Payment and idempotency

Never retry payment capture safely without a provider/merchant idempotency reference.

A timeout means:

result unknown

not:

operation failed

Reconcile first if the provider cannot guarantee idempotent capture.

Retry budget

At platform level I monitor:

retry attempts / original requests

If normal traffic is 10,000 RPS and retries add 4,000 RPS during an outage, the retry layer itself is a major source of load.

I set alerts on retry ratio.

Production checklist

• Retry only transient errors.
• Require idempotency for side effects.
• Keep attempts low.
• Use exponential backoff.
• Add jitter.
• Respect Retry-After.
• Stay inside caller deadline.
• Disable duplicate retry layers.
• Monitor retry ratio.
• Pair persistent failure with circuit breaker.

FAQ

How many retries?

Often one or two. I start small and measure.

Why not retry every 500?

Some 500 responses represent deterministic application bugs. Classification needs domain/dependency knowledge.

Should users retry?

For reads, often yes. For side-effecting commands, the API should provide idempotency so a client retry is safe.

Related architecture guides

• Circuit Breaker with Resilience4j

• Bulkhead Pattern

  Architect's final take

  Retry is not a reliability checkbox. I treat it as controlled extra load and make its maximum cost visible.