Jay
Kreps

A literature major's revenge on the database

The Confluent office in Mountain View sits a freeway exit away from where most enterprise software gets built, which is fitting. Jay Kreps spent the last decade arguing the database should be turned inside out - that the log, not the table, is the real thing - and the world quietly came around. In December 2025, IBM agreed to buy his company for $11 billion in cash. He still types like an engineer, blogs in 12,000-word increments, and answers email at jay@confluent.io.

The story of Apache Kafka is the story of an idea that sounded boring at every dinner party for ten years and turned out to be load-bearing for the modern internet. Banks use it for fraud detection. Ride-share apps use it to know where every car is. Streaming services use it to remember what you almost watched. Generative AI shops use it to feed models the fresh data without which a chatbot is a museum piece. The plumbing is Kafka. The plumber, more often than not, is Kreps.

He did not arrive at this position through the usual founder pipeline. He left high school after a single year because he had decided traditional schooling was inefficient, then convinced his parents to let him teach himself the rest. He has admitted the freedom was probably wasted on a teenager. He filled the gaps later at UC Santa Cruz, picking up a bachelor's in 2002 and a master's in 2005, both in computer science. He also took, by his own account, a lot of literature classes - a detail that would become surprisingly load-bearing.

The LinkedIn years

Before Kafka there was NexTag, where a young Kreps spent a year as a senior data-mining engineer working on merchant profitability. The job was an apprenticeship in the dull, expensive problem of moving a lot of data from one place to another reliably. He arrived at LinkedIn in June 2007 and stayed seven years, ultimately as Principal Staff Engineer leading the relevance and data systems work. The professional network was growing fast and breaking in the way fast-growing networks do: too many systems, too many integration scripts, every team reinventing the same pipeline at a slightly different angle.

The fix Kreps and his collaborators, Neha Narkhede and Jun Rao, eventually settled on was deceptively small. Instead of point-to-point integrations between every database, search index, cache and analytics system, every event would be appended to one ordered, replicated log. Subscribers would read from it at their own pace. The log, not the table, would be the source of truth. They called the project Kafka because writers write and writers were on Jay's mind, and because, as he later admitted, the name sounded cool for an open-source project. It is still one of the all-time great pieces of software branding.

The blog post that started a company

In December 2013 Kreps published an essay on the LinkedIn engineering blog called "The Log: What every software engineer should know about real-time data's unifying abstraction." It ran roughly twelve thousand words. It read like a paper and was passed around like one. By late 2014 the essay had become a slim O'Reilly book, "I Heart Logs," and a company. Confluent was incorporated in September 2014 with Kreps, Narkhede and Rao on the founding cap table. The pitch was straightforward enough to fit on a napkin: Apache Kafka was already running inside everything that mattered. Confluent would help companies actually operate it.

For the first few years the business was the kind that engineers like and analysts find slightly confusing - support contracts, schema registries, connectors, an enterprise edition. Then came Confluent Cloud, a managed service that took the operational pain out of running a streaming platform and turned Kafka into something a Visa-toting developer could spin up in fifteen minutes. The managed product is now the bulk of the business and is the reason the company has the size it does.

The arc, when you map it, is unusually clean. NexTag taught him how badly data movement is usually done. LinkedIn gave him a place big enough that he had to fix it. The fix, generalized, became Kafka. Kafka became Confluent. Confluent became, in late 2025, IBM's idea of where the AI era's plumbing should live.