Edo Liberty and the trouble with knowing everything

Before Edo Liberty left Amazon Web Services, he and his wife opened a spreadsheet. They had three young children, including one-year-old twins. He had an established research job. Starting a company would replace a salary with an experiment. The immediate question was whether they could keep paying the mortgage.

They worked through the numbers. They could. His wife encouraged him to go ahead. Liberty later recounted the conversation without pretending that the downside was mysterious: years of hard work, depleted savings, perhaps failure, followed by another job search. Pinecone began with a technical conviction and a household calculation. Even an ambitious idea has to get past the kitchen table.

He founded the company in 2019, several years before ChatGPT made AI infrastructure a subject for people who had previously enjoyed discussing almost anything else. The opportunity he saw concerned vector search: the machinery that helps software find related information inside enormous collections. He had encountered its usefulness at large technology companies. He wanted other engineers to be able to use it, too.

An inconveniently large picture

Liberty’s route into computing began with some reluctance. At Tel Aviv University, he wanted to become a physicist. Programming seemed like a necessary accompaniment, something he ought to learn rather than something he expected to enjoy. Algorithms changed the attraction. Computer science became a way to think about mathematics, abstraction and complexity.

He earned a bachelor’s degree in physics and computer science, then moved to the United States in 2004 for doctoral study at Yale. A research problem there supplied a particularly direct introduction to limited resources. He was working with hyperspectral images of about 1.5 gigabytes each. His desktop had 512 megabytes of memory. The picture was larger than the machine’s working space.

A computer confronted with that mismatch does not become more cooperative because the question is interesting. You need another method. Liberty’s interest settled around the theory and algorithms of large datasets, followed by a postdoctoral fellowship in applied mathematics at Yale. The constraint would remain familiar even as the machines became larger: an abundance of information, a limited budget for handling it.

The useful art of keeping less

Some of Liberty’s research asks how much of a dataset can be discarded while preserving the information needed for a particular calculation. The qualifier matters. A useful summary has to keep something measurable intact. Making a file smaller is easy if you are indifferent to what disappears.

His 2013 paper, Simple and Deterministic Matrix Sketching, introduced Frequent Directions and won a best paper award at KDD. It describes a way of retaining a compact approximation of the structure in a stream of vectors. In ordinary terms, a much smaller mathematical object can preserve useful relationships in a much larger one.

Another line of work, with Zohar Karnin and Kevin Lang, produced the KLL algorithm for estimating quantiles in streaming data. Quantiles help answer questions such as where a measurement falls within a distribution. A system watching a continuing flow of observations cannot always keep every observation and start again from the beginning.

These problems have a certain economy of temperament. They reward asking what is sufficient, what can be proved and what can run within an actual machine’s limits. Liberty’s research record includes best paper awards at SODA in 2011 and PODS in 2021 as well. The prizes recognize individual results; the recurring concern is how to make large amounts of data manageable.

From a research lab to somebody’s working day

The work took him through environments where those limits arrived with customers attached. He joined Yahoo’s research center in Israel in 2009 after an invitation from Yoelle Maarek, and later headed Yahoo’s New York research lab. At AWS, he became director of research and head of Amazon AI Labs.

His engineering record ranges across machine-learning platforms, streaming data and the algorithms and distributed systems behind Amazon SageMaker. Those are collective projects, reflected in papers and patents with long lists of collaborators. They put research inside services that other people depend on, where an elegant method must also survive implementation.

There is a change of audience involved. A mathematical argument can establish that an approach works under specified assumptions. A platform has to make the approach usable by an engineer whose immediate problem is somewhere else. That second obligation would become central to Pinecone: reduce the specialist work required before somebody can build an application.

A database before the dinner-party conversation

At Yahoo and AWS, Liberty had seen AI models and vector search improve applications such as recommendations and spam detection. Such systems required specialist engineering. He assumed an accessible, packaged version already existed for teams without a technology giant’s resources. Discovering that gap helped prompt Pinecone.

The idea takes a little unpacking. A model can represent a piece of text or an image as a vector: a list of numbers. Similarity between those representations gives software a way to retrieve related material. It can look for meaning and relevance rather than depend entirely on matching the same words.

Pinecone offered that capability as a managed database, publicly introducing its vector database in 2021. Engineers could store and search vectors without building the entire underlying service themselves. The product decision was as significant as the mathematical one. A technique available inside a large research organization could become a service another team could adopt.

Language models brought another use for the same machinery. An application could retrieve relevant passages from its own documents, then give those passages to a model to help it compose an answer. Liberty described this process through a school assignment: find several books on a subject, read the relevant material, then write a summary. The sequence is recognizable even when the library has become a database.

That connection gave Pinecone a place in retrieval-augmented generation, usually shortened to RAG. A model’s general capabilities could be combined with information selected for a particular question. Liberty’s 2023 TEDAI talk addressed the distinction between what AI can think through and what information it can access, presenting vector databases as part of a route toward long-term memory.

In April 2023, Pinecone announced a $100 million Series B led by Andreessen Horowitz, at a $750 million valuation. ICONIQ Growth, Menlo Ventures and Wing Venture Capital participated. Those figures belong to the company’s financing history. They also mark how far the business had traveled from its pre-ChatGPT beginnings, when explaining the need came before benefiting from the demand.

A little Shackleton on the reading list

For a person occupied with modern infrastructure, Liberty has found leadership material in a distinctly inconvenient journey. He has reread Endurance, the account of Ernest Shackleton’s expedition, and bought copies for members of his team. He has spoken about its resourcefulness, hope and lessons in leadership.

The attraction is understandable without turning a startup into an Antarctic expedition. A founder makes decisions with incomplete knowledge and other people depending on the outcome. Plans need revising. An excellent initial idea supplies remarkably little instruction about the next unpleasant surprise. Shackleton is at least company on the bookshelf.

Teaching offers another connection between Liberty’s research and its audience. His courses have included data-mining algorithms at Tel Aviv University, long-term memory in AI at Princeton, and algorithms in large language models at Columbia. Explaining a method asks a researcher to expose its assumptions. Students, like engineers, tend to notice the step a confident explanation has skipped.

“My current focus is on long-term memory for AI.”

Edo Liberty

Returning the keys, keeping the question

In September 2025, Liberty handed Pinecone’s CEO role to Ash Ashutosh and became chief scientist. He had led the business for six years. The new role would give him more time with models, prototypes and code. After years of running an organization around research, he was making more room for the research itself.

His announcement also contained a smaller ambition. His seven-year-old daughter had wished him a fun year at Pinecone with nice, helpful colleagues. Liberty repeated the wish appreciatively. It was an unusually economical description of a desirable workplace, and required no slide deck.

A further transition followed. On August 26, 2026, Ashutosh announced that Liberty was stepping back from day-to-day responsibilities while remaining a shareholder and board member. That is his current relationship with the company he founded. The CEO handover and the move away from daily operations were separate changes, a year apart.

The public research record continued to grow. In September 2026, Liberty, Ashwin Padaki and Amir Ingber coauthored the announcement of VQ-bench, an open-source framework for comparing vector quantization methods. Quantization reduces the bits required to store vectors. The project addresses a practical irritation: methods evaluated with different datasets and measures are difficult to compare fairly.

It is a fitting problem to find near the latest end of his story. Before a database can retrieve the useful thing, it has to store it within a budget. Before a family can back an experiment, it has to check the mortgage. Liberty’s career has given those constraints different scales and different names. The work begins when the numbers meet the available room.