Kafka Backpressure in Production NestJS Systems

Kafka can deliver work faster than a consumer, database or downstream API can safely absorb. Backpressure is the architecture discipline that keeps that mismatch from becoming an outage.

Measure the real bottleneck first

Track consumer lag, processing latency, database connection saturation, heap usage and downstream error rates. Scaling consumers without measuring dependencies can simply move the bottleneck into MySQL, MongoDB or an external API.

Bound concurrency deliberately

Use a finite worker pool or controlled batch size. Increase parallelism only while the full dependency chain remains within healthy latency and error thresholds. A consumer that accepts unlimited in-flight work is effectively turning memory into an uncontrolled queue.

Make retries safe

Retries need idempotency keys, bounded retry counts and a clear dead-letter strategy. For state changes, keep enough audit history to determine what was attempted, what succeeded and what can be replayed safely.

Protect dependencies and recovery paths

Circuit breakers, timeouts and load shedding prevent one failing dependency from consuming all worker capacity. Recovery should be testable: restart consumers, replay events, verify idempotency and confirm that lag returns to normal without duplicate business actions.