- You might not need to change your clients
- Grouped by the job they do
- The seven, A to Z
- Where the messages end up on disk
- Stream or queue: name your workload
- Who runs it at 3am?
- When the platform you have is still the right one
- Open source, foundation-run or vendor-backed
- What to measure during a trial
- Testing a move without losing messages
- Frequently asked questions
Look further if you'd like a classic message broker with queues, a lightweight messaging system that ships as one binary, or a fully managed cloud stream with no cluster to run. If you'd rather keep your existing clients, some platforms speak the same protocol, and teams already settled on one cloud usually start with that provider's managed stream.
Apache Kafka calls itself an open-source distributed event streaming platform for data pipelines, streaming analytics and integration. If you're a data or platform engineer looking past it, the reason is usually practical: running the cluster takes more effort than the team wants, the workload is really a queue rather than a stream, or the company has settled on one cloud and wants to use what's there.
You might not need to change your clients
Leaving the platform doesn't always mean leaving its protocol. Redpanda's docs say producers and consumers talk to it using the Apache Kafka API, and Microsoft says Azure Event Hubs has built-in Kafka compatibility, so existing Kafka workloads can run there without code changes.
That changes the size of the job. If your producers and consumers are spread across dozens of services, rewriting them all is a huge project. Moving to something that speaks the same API can turn it into an infrastructure change instead. Test that claim with your own clients and settings before you plan around it, but it's worth checking first.
The two differ in one way that matters. Microsoft describes Event Hubs as fully managed, which suits teams who want to keep their clients and stop running servers, as long as they're happy on that cloud.
If you're moving because the model itself is wrong for you, say you need classic queues, that compatibility won't help. Then the brokers further down are the better place to look.
Grouped by the job they do
| Option | Group |
|---|---|
| Amazon Kinesis Data Streams | Managed cloud stream |
| Apache ActiveMQ Artemis | Classic broker |
| Apache Pulsar | Multi-tenant messaging |
| Azure Event Hubs | Speaks the Kafka API, managed |
| NATS | Lightweight |
| RabbitMQ | Classic broker |
| Redpanda | Speaks the Kafka API |
Keeping your clients points you to the two rows that speak the Kafka API. Wanting queues points you to the brokers. If you'd like no cluster at all, Kinesis and Event Hubs are the managed rows. Nobody needs to evaluate all seven; pick the rows that match your reason and trial those.
The seven, A to Z
Amazon Kinesis Data Streams
Takes in and processes data records in real time.
Apache ActiveMQ Artemis
Multi-protocol messaging from the Apache project.
Apache Pulsar
Multi-tenant server-to-server messaging, first built at Yahoo.
Azure Event Hubs
Microsoft's fully managed real-time streaming platform.
NATS
A single binary for pub/sub, request/reply and JetStream.
RabbitMQ
An open source messaging and streaming broker.
Redpanda
Producers and consumers use the Kafka API with it.
Where the messages end up on disk
Storage decides what happens after a restart, so it's worth reading before any trial. Pulsar's docs say it offers several subscription types and uses Apache BookKeeper for persistent message storage, which means a second project to learn alongside it. NATS keeps its core small and adds persistent streaming through JetStream when you need it.
For the brokers and the managed streams, read each vendor's own storage and retention pages rather than assuming they behave like the log you have now. Then ask one plain question: if a node dies mid-afternoon, what does a consumer see when it comes back?
Want AI doing the heavy lifting in your marketing?
I build the systems that handle the boring 80 percent, so you get your week back. Done properly, with the human kept in.
Stream or queue: name your workload
A lot of switching starts with a mismatch. A stream keeps an ordered log that many readers can replay. A queue hands each message to a worker and moves on. If your team mostly uses topics as job queues, a broker built around queues may feel simpler. If you rely on replaying history or many independent readers, you'll want something that treats the log as the main thing.
Write down how your top three topics are used today. That one page of notes will tell you more than any feature chart. We've also compared Kafka against RabbitMQ and how NATS stacks up against Kafka if one of those pairings is on your shortlist.
Who runs it at 3am?
The quieter question behind most moves is operational. A self-run cluster needs someone who understands storage, upgrades and recovery. A managed stream moves that work to a cloud provider, in exchange for tying you to that provider's way of doing things.
If your team is small, be honest about who answers the pager. A lightweight system that ships as one file may be easier to run than you expect. A managed service may be easier still, if you're already on that cloud. A large, multi-tenant setup only makes sense if several teams need to share it.
When the platform you have is still the right one
If your cluster is stable, someone on the team understands it well and your workloads really are streams that get replayed and read by many consumers, there's no reason to move just because a newer name looks simpler. Upgrades and tuning can be tedious, but so is migrating every producer and consumer.
The strong signals for change are specific: topics that are really work queues, a company-wide move to one cloud, or a team that can't staff the operational side any more. Write down which of those applies before you start a trial.
Open source, foundation-run or vendor-backed
These projects are governed in different ways. Pulsar now sits under the Apache Software Foundation, and Artemis is presented as an Apache project. RabbitMQ says it's free and open source software, with Broadcom offering a separate commercial edition. Others are offered by a single company or cloud provider. None of that makes a tool better or worse, but it shapes who decides the roadmap and where you go for support.
What to measure during a trial
- End-to-end delay from producer to consumer under your normal load.
- What happens to in-flight messages when a node restarts.
- How long it takes to add a new topic or queue and grant access to it.
- How readable the logs and metrics are for someone on call.
Testing a move without losing messages
- Pick one non-critical topic with steady traffic.
- Mirror its messages to the candidate system while the old one stays in charge.
- Point one consumer at the new system and compare what it receives.
- Restart a node on purpose and check nothing is lost or duplicated beyond what you expect.
- Measure how long a new teammate takes to set up a producer and consumer.
- Move a second topic only when the first has run cleanly for a few weeks.
Want your tool on this list? Featured spots and reviews here are paid partnerships. Here's how to partner with me.
Frequently asked questions
Can I keep my existing Kafka clients?
Possibly. Run a staging copy of your real traffic through Redpanda or Event Hubs before you rely on it.
Who first built Pulsar?
Yahoo did. Its docs say the project now sits under the Apache Software Foundation.
What does multi-tenant mean for Pulsar?
Several teams can share one deployment.
Can NATS keep messages after delivery?
Yes, JetStream adds persistent streaming.
How long should a trial run?
Long enough to cover a restart and a normal week of traffic; a few weeks on one topic is a fair start.
