Skip to content
C Codeloom
Backend

Message Broker Comparison: Kafka, RabbitMQ, SQS, NATS

Compare popular message brokers across delivery guarantees, ordering, throughput, and operations to pick the right tool for the job.

·3 min read · By Codeloom
Intermediate 10 min read

What you'll learn

  • ✓How log-based and queue-based brokers differ
  • ✓Where each broker shines
  • ✓Delivery guarantees and ordering
  • ✓Operational cost realities
  • ✓When to pick managed services

Prerequisites

  • •Familiar with HTTP and databases

What and Why

Message brokers move data between services so producers and consumers do not have to be online at the same time. They handle buffering, retries, and fan-out. But brokers are not interchangeable. The wrong choice can cost months of rework.

This post compares four common ones: Apache Kafka, RabbitMQ, AWS SQS, and NATS.

Mental Model

Brokers fall into two families:

  • Log brokers (Kafka, Kinesis): an immutable, replayable, partitioned log. Consumers track their own position.
  • Queue brokers (RabbitMQ, SQS, NATS JetStream): items are delivered, acknowledged, and removed.

Logs are great for streams of events you might want to reprocess. Queues are great for tasks you process once and discard.

Architecture

Log broker (Kafka)            Queue broker (RabbitMQ/SQS)
Producer -> Partition         Producer -> Queue
            |                              |
     offset 0,1,2,3...               item1, item2 (FIFO)
            |                              |
     Consumer A (offset=2)           Consumer (ack/nack)
     Consumer B (offset=0)
Log vs queue model

Kafka excels at high throughput, retention, and replay. It is heavier to operate and assumes long-lived consumers.

RabbitMQ shines at flexible routing (exchanges, bindings) and per-message control. Throughput is good but well below Kafka.

SQS is a managed queue with simple semantics. Throughput is high; ordering is per-message-group; cost scales with usage.

NATS is lightweight pub/sub for microservices. JetStream adds persistence. It is fast and simple but with a smaller ecosystem.

Trade-offs

Exactly-once delivery is hard everywhere. Kafka offers transactional writes, but exactly-once still requires idempotent consumers in practice. Treat all brokers as at-least-once and design consumers accordingly.

Ordering is global only in single-partition Kafka topics and FIFO queues. Most systems accept per-key ordering and benefit from the parallelism.

Operations differ dramatically. Self-hosted Kafka requires real expertise. Managed SQS is one API call. Choose accordingly to your team capacity.

Practical Tips

If you need event sourcing, audit, or replay, pick Kafka or a similar log.

If you need flexible routing and per-message control, RabbitMQ is hard to beat.

If you want minimal ops and AWS is already your platform, SQS plus SNS covers most needs.

If you need low-latency in-cluster messaging for microservices, NATS is the lightest option.

Always pair the broker with a dead-letter queue or topic. Without it, poison messages will keep failing forever or, worse, get silently dropped.

Use schemas (Avro, Protobuf, JSON Schema). Brokers do not care about payload shape; downstream consumers do. A registry saves you when producers change.

Wrap-up

There is no best broker, only the best broker for your workload. Pick by access pattern (queue vs log), operational appetite (managed vs self-hosted), and ecosystem fit. The decision is sticky, so spend a day prototyping with realistic data before you commit. Once your data flows through a broker, it tends to stay there for years.

Related articles