Choose Amazon Kinesis Data Streams if you're on AWS and want serverless streaming with no clusters to run: on-demand mode scales capacity for you and data is replicated across three Availability Zones. Choose Apache Kafka if you want an open source platform you can run anywhere, keep events for as long as you like and connect to other systems with Kafka Connect. If you like Kafka but want AWS to run it, Amazon also offers Amazon MSK, a fully managed Kafka service.
Streaming choices get political fast, so I stuck to the documentation: Kafka's introduction and uses pages, and Amazon's Kinesis Data Streams product page, developer guide and FAQs, plus the Amazon MSK page.
Kafka and Kinesis compared
| Apache Kafka | Amazon Kinesis Data Streams | |
|---|---|---|
| Type | Open source event streaming platform | Serverless streaming data service on AWS |
| Where it runs | Bare metal, virtual machines or containers, on premises or in any cloud | AWS |
| Who runs it | You, or a managed service from a vendor | AWS |
| Keeping events | A per-topic setting, as long as you want | 24 hours by default, up to 365 days |
| Scaling unit | Partitions spread across brokers | Shards, or on-demand mode |
| Ecosystem | Kafka Connect and Kafka Streams | Firehose, Managed Service for Apache Flink and the Kinesis Client Library |
| Free option (as of October 2026) | Yes, open source | Not mentioned on the pages I read; you pay for what you use |
Retention is the dividing line people overlook
Kafka's introduction says it stores streams of events durably and reliably for as long as you want, with retention set per topic, and that its performance is effectively constant with respect to data size, so storing data for a long time is perfectly fine. That makes Kafka a system of record for events, not just a pipe, and it's why teams use it to replay history when they build something new.
Kinesis works differently. Its FAQs say the default retention period is 24 hours, extended retention lets you reprocess data for up to seven days, and long-term retention goes up to 365 days for things like back testing, backfills and auditing. Both longer options are optional features with additional charges, so long retention isn't the default; it's a setting you'll need to plan for.
How Kinesis capacity works
Amazon describes Kinesis Data Streams as serverless, so there's nothing for you to patch or manage. By default it scales capacity automatically in on-demand mode, and you can choose provisioned mode if you'd rather manage throughput yourself. Capacity is counted in shards: each shard supports 1 MB per second and 1,000 records per second for writes and 2 MB per second for reads, and producers or consumers that go over those limits get throttled. Data is replicated synchronously across three Availability Zones, so you don't have to design that yourself. Compute has the same serverless question, which I look at in Lambda against Fargate.
Message brokers are another angle, compared in our Kafka vs RabbitMQ comparison.
Why teams choose Kafka
Under the hood it runs as a cluster of servers, some acting as brokers for storage and others running Kafka Connect to import and export data, talking to clients over a TCP protocol. Topics are partitioned across brokers so clients can read and write in parallel, and Kafka offers guarantees such as exactly-once processing. You can self-manage it or use fully managed services from a variety of vendors, which is how Amazon MSK fits in for AWS users who want Kafka itself.
Storing events and processing them are different jobs, as our Kafka vs Flink guide explains.
Five checks before you commit
- Are you all-in on AWS and short of people to run clusters? Kinesis removes that work.
- Do you need to keep events for months or years and replay them freely? Kafka's per-topic retention suits that.
- Do you need to run on premises or across several clouds? Kafka runs anywhere.
- Do you want Kafka but not the operations? Look at Amazon MSK or another managed Kafka service.
- Is your traffic spiky? Kinesis on-demand mode scales capacity automatically.
Streaming usually feeds other systems, so my notes on data integration challenges are worth reading first, along with this list of data analytics tools for what happens downstream.
My advice on Kafka or Kinesis
If your whole product runs on AWS and you want a stream working this week without running brokers, Kinesis Data Streams is the pragmatic choice. If events are central to your business, you'll want to replay them for a long time or you can't commit to one cloud, choose Kafka and decide separately whether to run it yourself or use a managed service such as Amazon MSK. Don't pick on throughput claims alone; retention and who runs the system matter more.
Want AI doing the heavy lifting in your marketing?
I build the systems that handle the boring 80 percent, so you get your week back. Done properly, with the human kept in.
Do you make data tools? Tool makers can ask to be considered for a featured place in comparisons like this one.
Related: writing for us on streaming.
Frequently asked questions
Is Kinesis the same as Kafka?
No. Kinesis Data Streams is Amazon's own serverless streaming service. Amazon sells managed Kafka separately as Amazon MSK.
How long does Kinesis keep data?
Kinesis Data Streams keeps data for 24 hours by default. Extended retention goes up to seven days and long-term retention up to 365 days, both as optional features.
Is Apache Kafka free?
Yes. Apache Kafka is open source, and you can self-manage it or use a managed service from a vendor.
What is a Kinesis shard?
A shard is the base throughput unit of a Kinesis data stream. Amazon says each shard supports 1 MB per second and 1,000 records per second for writes and 2 MB per second for reads.
How long can Kafka keep events?
Kafka's introduction says you can store streams of events for as long as you want, using a per-topic retention setting.
