Choose Apache Kafka when you need an event streaming platform that stores streams of events durably and lets many consumers read and replay them, for data pipelines and event-driven systems at scale. Choose RabbitMQ when you need a flexible message broker that routes messages between services, speaks open protocols such as AMQP 1.0 and MQTT 5.0 and, by its own account, is easy to deploy in the cloud, on premises or on your own machine. The line has blurred, though: RabbitMQ now has Streams, an append-only log you can read repeatedly, much like Kafka's model.
This choice shapes how your services talk to each other for years, so it's worth getting right. I read the Apache Kafka introduction and use cases pages and RabbitMQ's home page and its docs on streams, queues and protocols.
Kafka and RabbitMQ compared
| Apache Kafka | RabbitMQ | |
|---|---|---|
| Type of tool | Event streaming platform | Messaging and streaming broker |
| Core model | Topics of events kept for a set time | Queues, plus Streams for append-only logs |
| After a message is read | Not deleted; consumers can read again | Queues hand messages on; Streams keep them until expiry |
| Protocols | Kafka's own TCP protocol | AMQP 1.0, MQTT 5.0 and others |
| Data integration | Kafka Connect and Kafka Streams | Routing, filtering, federation |
| Commercial backing | Managed services from various vendors | Broadcom's enterprise offering with 24/7 support |
| Licence | Open source Apache project | Free and open source |
RabbitMQ now does logs too
The classic advice says Kafka keeps messages and RabbitMQ deletes them once they're consumed. RabbitMQ's own docs complicate that. RabbitMQ Streams is a persistent, replicated data structure that models an append-only log of messages that can be read repeatedly until they expire, which the docs call non-destructive consumer semantics. It also supports super streams, which are partitioned streams.
Kafka's introduction makes the same point about its own design: events in a topic can be read as often as needed and aren't deleted after consumption; you set how long Kafka keeps them with a per-topic setting. So both can replay data now. The real difference is that Kafka is built around streams from the ground up, while RabbitMQ adds streams to a broker built around routing messages.
Where Kafka fits
Kafka describes three capabilities in one platform: publishing and subscribing to streams of events, storing them durably for as long as you want, and processing them as they happen or afterwards. It runs as a cluster of servers that can span data centres or cloud regions, with brokers forming the storage layer and Kafka Connect moving data in and out of systems such as relational databases. You can run it yourself or use a fully managed service from one of several vendors.
A lighter messaging system is compared in our NATS vs Kafka comparison.
Where RabbitMQ fits
RabbitMQ calls itself a powerful, enterprise grade open source messaging and streaming broker used by millions worldwide. Its strength is flexibility: you can combine routing, filtering, streaming and federation to decide how messages travel from publishers to consumers, acknowledge deliveries and replicate messages across a cluster. Version 4.4.0 is the current release on its site, and Broadcom offers a commercial edition with round-the-clock support.
Managed streaming on AWS is covered in our Kafka vs Kinesis guide.
Choose with these questions
- Do many teams need to read the same events, now and later? Kafka's topics suit that.
- Do you mostly pass jobs or commands between services? RabbitMQ's queues and routing fit.
- Do you have devices speaking MQTT? RabbitMQ supports MQTT 5.0.
- Are you building data pipelines into databases and warehouses? Kafka Connect is made for it.
- Is your team small with no streaming specialist? RabbitMQ says it's easy to deploy, even on a laptop.
If you're designing the services around it, my explainer on monoliths versus microservices sets the scene, and this guide to monitoring data in your business covers what to watch once messages flow.
Want AI doing the heavy lifting in your marketing?
I build the systems that handle the boring 80 percent, so you get your week back. Done properly, with the human kept in.
My advice on Kafka or RabbitMQ
For a small or growing business connecting a handful of services, I'd start with RabbitMQ: it's flexible, speaks standard protocols and now offers streams if you need replay later. Move to Kafka when event data becomes a product in itself, with several teams or systems reading the same streams and pipelines feeding analytics. If you go with Kafka and don't have a specialist, a managed service will save a lot of late nights.
Do you make messaging or data streaming tools? Tool makers can ask to be considered for a featured place in comparisons like this one.
Frequently asked questions
Is Kafka a message queue?
Kafka describes itself as an event streaming platform. Events in a topic aren't deleted after being read, and retention is set per topic.
Does RabbitMQ support streaming?
Yes. RabbitMQ Streams is a persistent, replicated append-only log that consumers can read repeatedly until messages expire.
Which protocols does RabbitMQ support?
RabbitMQ supports several open standard protocols, including AMQP 1.0 and MQTT 5.0.
Are Kafka and RabbitMQ free?
Yes. Both are open source. Broadcom sells a commercial RabbitMQ offering, and various vendors offer managed Kafka services.
Can Kafka run in the cloud?
Yes. Kafka's docs say it runs on bare metal, virtual machines and containers, on premises or in the cloud, self-managed or as a managed service.
