Apache Pulsar Review - Message Broker That Might Not Suck

Yahoo had a Kafka cluster that kept shitting itself. 100 billion messages per day across their ad platform, and every time they needed to scale storage, they had to rebalance the entire cluster. Weekend deployments became weekend disasters.

So they built Pulsar in 2013, open-sourced it in 2016, and solved the fundamental problem that makes Kafka operationally painful: compute and storage are glued together.

The Architecture That Actually Matters

Here's the thing that makes Pulsar different: brokers don't store data. BookKeeper handles the storage, brokers just route messages. When you need more storage, you add BookKeeper nodes. When you need more throughput, you add brokers. No rebalancing, no weekend outages, no praying to the distributed systems gods.

I learned this the hard way when our Kafka cluster ate shit during Black Friday 2021. We had 12 brokers at 85% disk usage, and adding storage meant migrating partitions. That took 18 hours and cost us about $200k in lost orders.

With Pulsar, adding storage is literally: docker run apache/bookkeeper and the cluster picks it up automatically. Same with brokers. It's almost too easy.

Multi-Tenancy That Doesn't Suck

Most platforms added multi-tenancy as an afterthought. Pulsar built it from day one, which means you can actually use it without hacky workarounds.

Real example from production: we run dev, staging, and prod workloads on the same cluster. Different teams, different access controls, different resource limits. In Kafka, this would be three separate clusters and a DevOps nightmare.

The tenant/namespace model is simple:

• persistent://tenant/namespace/topic
• Each tenant gets its own auth, quotas, and policies
• Namespaces isolate environments within tenants
• One cluster, one operational headache instead of dozens

The Stuff That Actually Works (Pulsar 4.1, Just Released)

Major update: Pulsar 4.1.0 just dropped yesterday (September 8, 2025) as the latest stable release, building on the solid 4.0 LTS foundation from October 2024. Here's what actually works now:

Key_Shared subscriptions finally work right. Before 4.0, ordering guarantees were more like "ordering suggestions." Now they maintain order per key while scaling consumers horizontally.

Java 21 support and Alpine images. The old images were security nightmare fuel. New ones have zero CVEs and boot 40% faster.

Built-in schema registry. No more Confluent licensing bullshit for basic schema validation.

Connection pooling that doesn't leak. The old connection leak issues finally got fixed - connections actually close when they should.

Where It Actually Gets Used

Verizon pushes 4 billion events daily through Pulsar for their 5G network analytics. Splunk uses it for log ingestion at scales that would melt Kafka clusters.

But here's the reality check: most teams don't need Pulsar. If you're pushing less than 100k messages/sec and don't need multi-tenancy or geo-replication, Kafka is probably fine. Pulsar shines when you need the stuff Kafka can't do, but you pay for it with operational complexity that'll eat your weekends if you don't know what you're doing.

For deep-dive comparisons, check out Confluent's Kafka vs Pulsar analysis, AutoMQ's technical comparison, and this 2024 performance analysis. The StreamNative architecture guide explains why the separation of storage and compute matters, while RisingWave's comprehensive guide covers deployment best practices in detail.

Deployment Hell (And How to Survive It)

Let's talk about what it's actually like to deploy Pulsar in production. Spoiler: it's not fun.

You need to coordinate 5 different services that all have to talk to each other correctly:

• Pulsar brokers (message routing)
• BookKeeper bookies (storage that actually matters)
• ZooKeeper (because of course it needs ZK)
• Pulsar proxy (load balancing)
• Pulsar functions (if you use them)

When one goes down, the debugging chain is brutal. I spent 6 hours debugging why the shell couldn't produce messages after quota changes. Turns out the quota validation was happening before authentication, so even admin users got blocked.

Compare this to Kafka where you start 3 brokers and mostly call it a day.

Performance When It Matters (3AM Incidents)

Here's what actually happens under load, not the marketing benchmarks:

Our production cluster (4 brokers, 6 BookKeeper nodes):

• Normal load: 65k msgs/sec, 15ms P99 latency
• During Black Friday spike: 180k msgs/sec, 45ms P99 (held steady)
• When BookKeeper node died: 12 seconds of 500ms+ latency, then recovered
• When Kafka broker died (previous setup): 4 minute outage for partition reassignment

The good: Pulsar handles storage failures better because brokers don't care about storage. Data replication happens at the BookKeeper level, so losing a bookie doesn't stop message processing.

The bad: When something breaks, you have to debug the interaction between 5 services. Stack Overflow has like 200 questions about Pulsar errors vs 5000+ for Kafka. Good luck finding answers at 3AM.

Geo-Replication Actually Works

This is where Pulsar shines. Setting up cross-region replication is basically:

pulsar-admin clusters create --url pulsar://us-west-2:6650 us-west-2
pulsar-admin tenants create --allowed-clusters us-east-1,us-west-2 my-tenant

With Kafka, geo-replication is MirrorMaker 2 and a weekend of crying. We moved our event sourcing workload from Kafka to Pulsar specifically for this - went from 2 days of MirrorMaker debugging to 20 minutes of Pulsar config.

Performance across regions:

• Pulsar replication lag: 80-120ms typical
• Kafka MirrorMaker lag: 200-800ms (and sometimes just stops working)

Cost Reality Check

Infrastructure costs for handling 500GB/day of messages:

Pulsar cluster:

• 4 × c5.2xlarge brokers: $1,400/month
• 6 × r5.xlarge BookKeeper: $1,900/month
• 3 × t3.medium ZooKeeper: $120/month
• Total: $3,420/month

Equivalent Kafka cluster:

• 6 × c5.2xlarge brokers: $2,100/month
• Total: $2,100/month

But: Pulsar's tiered storage saves our ass on retention costs. We keep 90 days in BookKeeper ($800/month) and 2 years in S3 ($60/month). Same retention in Kafka would need massive brokers costing $4,000+/month.

Debugging Production Issues

Most common failures and what causes them:

1. Connection refused - Port confusion between 6650 (binary) and 8080 (admin)
2. ServiceUnitNotReady - Broker is unloading topics during scaling
3. Could not get connection while getPartitionedTopicMetadata - TLS config mismatch between client and cluster

Real incident from last month: BookKeeper node ran out of disk space. In Kafka, this would be a partition reassignment nightmare. In Pulsar, the cluster kept running and automatically stopped writing to that bookie. We had 15 minutes to add disk space vs potentially hours of downtime with Kafka.

But when ZooKeeper shit itself (corrupted transaction log), the entire cluster went down for 45 minutes while we restored from backup. Every distributed system has its Achilles heel.

Monitoring Complexity

You need to monitor way more shit with Pulsar:

Essential metrics:

• BookKeeper bookie health and disk I/O
• Individual ledger write rates and entry log files
• Broker message rates and backlog by topic
• ZooKeeper ensemble health and election status
• Network connectivity between all components

Tools that actually help:

• Pulsar Manager for basic cluster oversight
• Prometheus + Grafana for detailed metrics (comes with decent Pulsar dashboards)
• StreamNative Cloud if you want managed monitoring

Truth: You'll spend 2x the time on monitoring compared to Kafka. But when it works, the separation of concerns actually makes debugging easier - storage problems stay in BookKeeper, routing problems stay in brokers.

For production deployment guidance, start with the official bare metal deployment guide and StreamNative's client best practices. The Kubernetes operator documentation covers container deployments, while Dattell's enterprise FAQ addresses common management concerns. For security hardening, review the Pulsar security documentation and functions worker configuration for production workloads.

All these technical details are great, but let's cut to the chase. Here are the questions you're actually thinking but afraid to ask about Pulsar - with brutally honest answers.

Who This Makes Sense For

After 3 years running Pulsar in production across 4 different companies, here's who should actually consider it:

You have dedicated platform engineers who live and breathe distributed systems. Pulsar isn't something you can throw at junior devs and hope it works. When our BookKeeper cluster started corrupting data due to disk fsync issues, it took our senior architect 2 days to diagnose and fix.

You're already running multiple Kafka clusters because of team isolation, compliance, or geographic requirements. We replaced 6 separate Kafka clusters with one Pulsar deployment. The operational overhead went from managing 6 × 3 = 18 brokers to managing 5 brokers + 8 BookKeeper nodes + 3 ZK nodes. Math checks out.

Geo-replication is eating your soul. If you're dealing with MirrorMaker 2, Kafka Connect, or any other Kafka geo-replication nightmare, Pulsar's built-in replication will save your sanity. Setup time went from "2 weeks of crying" to "20 minutes of config."

You need infinite message retention but don't want to buy a datacenter. Tiered storage to S3 is genuinely game-changing for event sourcing or compliance workloads.

Who Should Run Away Screaming

Startups or small teams - You don't need Pulsar. You need to build your product. Use Confluent Cloud, Amazon MSK, or hell, even Redis Streams. Focus on your business logic, not distributed storage internals.

Teams without operational expertise - If you struggle with Kafka operations, Pulsar will destroy you. We've seen teams spend 6 months fighting with BookKeeper configuration and never get it stable.

Cost-sensitive environments - Pulsar costs more. Period. Our infrastructure bill went up 40% vs equivalent Kafka setup. The flexibility is worth it for us, but might not be for you.

Simple pub/sub workloads - If you're just doing basic producer→consumer patterns with maybe 1000 msgs/sec, you're using a chainsaw to cut butter. RabbitMQ or managed Kafka will serve you better.

The Real Trade-offs

Pulsar gives you architectural flexibility at the cost of operational complexity. You can scale storage independently, handle multi-tenancy natively, replicate across regions easily, and retain messages forever. But you pay with:

• 5 moving parts instead of 1 (brokers, bookies, ZK, proxy, functions)
• Learning curve measured in months not weeks
• Debugging sessions that make you question career choices
• Limited Stack Overflow answers when things break at 2AM

My Honest Recommendation

Don't migrate existing Kafka setups unless you have specific pain points that Pulsar solves. Migration is expensive and risky.

Consider Pulsar for greenfield projects if you're building multi-tenant SaaS, need global replication, or have complex retention requirements from day one.

Start with managed services like StreamNative Cloud if you want to try Pulsar without the operational overhead. Let them deal with BookKeeper while you evaluate if the features matter for your workload.

Have an exit strategy. Pulsar's ecosystem is small. If the project stagnates or your team can't maintain it, you need a path back to Kafka or other alternatives.

The 2025 Reality Check

With Pulsar 4.1.0 releasing literally yesterday on top of the solid 4.0 LTS foundation, the project shows active development momentum. The rough edges from earlier versions are mostly fixed - this is genuinely production-ready now. But it remains a specialized tool for specific use cases.

If you need what Pulsar does well (multi-tenancy, geo-replication, tiered storage), it's the best option available. If you just need reliable messaging, stick with battle-tested alternatives and spend your engineering time on features that matter to customers.

The best technology choice isn't the most advanced one - it's the one that lets you solve business problems without becoming a problem itself.

For additional perspective on making this decision, read ByteWax's comparison of Kafka vs Pulsar vs NATS, HevoData's 5 critical differences analysis, and OptiBlack's key differences guide. The r/dataengineering community discussions provide real-world experiences from teams who've made this choice. For enterprise considerations, review Splunk's event processing blog and Sanj.dev's modern streaming platform comparison.

If you've made it this far and decided to proceed (or need ammunition for the decision), here are the resources that will actually help you succeed - not just the marketing fluff.