StreamNative Wants the Stream to Outlive the Broker

A data stream is a rather odd place to hide a business. It has no shop window, no recognizable sound, and, if the engineers are doing their jobs, very little drama. A payment clears. A threat signal arrives. A visitor clicks a button and a sales tool notices before lunch. Between those moments sits a system passing messages from one machine to another. StreamNative sells the service of keeping that system dependable, scalable, and less expensive to run.

The founders knew the machinery before there was a company. At Yahoo, Guo and Merli worked on the distributed messaging system that became Apache Pulsar. Guo later led messaging infrastructure work at Twitter; both helped found Streamlio before it was acquired by Splunk. StreamNative was their next attempt to make that experience usable by teams that had applications to build and no desire to babysit a cluster.

A good broker is a terrible landlord

The original offer was easy to grasp: let the people who helped create Pulsar operate it. StreamNative Cloud handles provisioning, monitoring, upgrades, and the unpleasant arithmetic of growing traffic. A developer can use it to move events between services, feed real-time analytics, hold messages for later replay, or connect data from sensors and applications. Pulsar remains part of the offer, while Kafka-compatible endpoints invite a much larger installed base. Managed Flink, developed with Ververica, adds a way to process streams alongside the messaging layer.

Its customers reveal what this work is for. Q6 Cyber uses Pulsar as a transport layer for threat intelligence as it works with more than 85 billion collected records. It began alongside Google Pub/Sub, tested high-throughput flows, then expanded. Safari AI uses StreamNative to move structured output from computer-vision systems into business metrics. Unify uses it to react to prospect behavior in seconds. These are very different companies, but all lose something when a message waits, vanishes, or costs too much to keep.

The customer numbers deserve their proper owners. Q6's record count describes Q6's collection, not StreamNative's customer base. Safari AI's saving is Safari AI's reported comparison with its previous setup, not a universal discount. Both stories are useful precisely because they name the situation: unpredictable ingestion, structured machine-learning data, retention, and an engineering team that would rather work on its product.

The bill that changed the architecture

Traditional Kafka deployments keep durable records on broker disks and replicate them across machines and availability zones. That architecture is familiar and often appropriate, especially when latency is unforgiving. It also brings costs: duplicated storage, network traffic, and capacity reserved for the next spike. Data destined for analytics is frequently copied again into a lakehouse. StreamNative's response was Ursa, introduced in 2024 and generally available on AWS bring-your-own-cloud deployments in March 2025.

Ursa uses stateless, leaderless serving and cloud object storage for its cost-optimized path. The company describes a stream being written to an object-store log, then materialized into open formats such as Iceberg or Delta for analytical use. Kafka and Pulsar clients can keep speaking familiar protocols. The important change is under the floorboards: the durable bytes are less tightly bound to the broker that accepted them.

There is a price for cheaper storage, and here it is time. StreamNative's 2024 public-preview description put one object-storage write path at roughly 500 milliseconds, up from roughly 200 milliseconds for the comparison path. Those figures describe its preview architecture, not every Ursa configuration. The choice is therefore practical: a team counting clicks, telemetry, or historical events may prefer lower infrastructure costs; a system that needs tighter acknowledgement latency may need the latency-optimized profile instead. The interesting engineering work is making that choice explicit.

“A protocol tells you how to talk to a system. It says nothing about the bytes after they land.”StreamNative, on opening Lakestream in 2026

Pulsar people enter the Kafka room

One can imagine the temptation to defend the original technology forever. StreamNative chose a bigger addressable problem. Kafka is the language many data teams already speak; asking them to rewrite producers and consumers would make every sale a migration project. Kafka compatibility gives them an easier first step. Universal Linking offers a route for copying data between existing streaming systems, while Ursa for Kafka adds a diskless topic option inside a Kafka distribution. In the same cluster, other topics can continue using ordinary disk-backed Kafka storage.

That last detail is surprisingly humane. Infrastructure changes tend to be sold as dramatic replacements, but a topic-by-topic experiment is easier to reverse and easier to measure. A team can compare latency, cloud spend, and failure behavior on one slice of traffic before moving the rest. Q6 Cyber's earlier parallel rollout, though with Pulsar and Pub/Sub, shows the same cautious instinct. The old system need not disappear on day one.

Opening the basement door

In September 2026, StreamNative made its most consequential move: it released the Lakestream specification, Ursa 1.0 storage engine, and Ursa for Kafka under the Apache 2.0 license. The specification describes how a stream lives in object storage, how offsets and cursors work, and how closed segments can become tables. Its ambition is interoperability at the storage layer, so the records need not be understood only by the software that first wrote them. That is a proposal to the industry, not an accomplished standard.

This is also a business decision. StreamNative still sells a managed platform: operations, security, support, networking, and the convenience of having someone else get paged at night. Its public pricing lists serverless starting at $73 a month, dedicated at $505, and bring-your-own-cloud at $365. Usage, configuration, and the underlying cloud bill change the total. The company is betting that open storage will make its paid service more attractive, even if another vendor eventually builds on the same specification.

The technical claim has earned notice: the Ursa paper won Best Industry Paper at VLDB 2025. An award does not settle a cloud bill, but it suggests that the design has attracted scrutiny beyond a product launch. The more decisive test comes next. Will independent tools implement Lakestream? Will customers see enough saving after accounting for storage requests, networking, operational changes, and latency? Will open tables remove a copy of data in their actual pipeline? Those answers will vary by workload.

For a team considering StreamNative, the copyable lesson is unromantic: measure before replacing. Put one representative topic on the platform. Record producer acknowledgement time, consumer lag, retention needs, cross-zone traffic, and the cost of the analytics copy. Try a cost-optimized path only where its latency fits. Then decide whether managed Pulsar, Kafka compatibility, or open lakehouse access solves the problem you actually have. StreamNative's grand idea may be that the stream should outlive the broker. The smaller, immediately useful idea is to stop treating every stream as if it had the same job.