He built a faster way to feed machines that never stop eating
Modern GPUs can swallow terabits of data per second. The file formats feeding them were designed for a world of MapReduce and overnight batch jobs, a world where a person, eventually, would look at the answer. Will Manning noticed the mismatch and refused to call it normal. His company, Spiral, exists to close the gap between what accelerators can consume and what storage can deliver. His open file format, Vortex, is how he is closing it.
Spiral launched out of stealth in September 2025 with $22 million across a seed and Series A round, led by Amplify Partners and General Catalyst. The product is an exabyte-scale, object-store-native data system aimed at the workloads that legacy warehouses handle worst: GPU data loading, reinforcement-learning systems, agent platforms, and anything that lives in the awkward range between a 1KB embedding and a 25MB image. Manning likes to describe Spiral's specialty bluntly as "making GPUs go brr."
To understand why a founder would obsess over decompression throughput, it helps to know that this is the second exabyte-scale data system of his career. The first was Palantir Foundry.
The clue to his temperament is in the unglamorous specifics he reaches for when he describes the status quo. Ask most founders about the AI data problem and you get a slide about market size. Ask Manning and you get the five-step ordeal that every machine-learning team performs thousands of times a day: read a Parquet file, explode it to Arrow at ten times the memory, convert that to tensors, cache the result, train, then repeat forever. Each step is a small tax. Stack them across a fleet of accelerators and the tax becomes the dominant cost of doing AI at all. He has counted the steps because he intends to delete them.
Every team that read or wrote bytes
From 2013 to 2022, Manning helped create and scale Palantir Foundry. The phrase he uses for his remit is precise and a little terrifying: he ran "every engineering team that read or wrote bytes." Data systems, databases, compute, formats, scheduling. He also stood up Palantir's European commercial operations along the way, which is a different kind of plumbing entirely.
Before Palantir there was a stranger résumé. Between 2009 and 2011 he was an early AI researcher at Princeton, working on Bayesian nonparametrics and reinforcement learning, years before either phrase was fashionable. Then a turn as a derivatives quant in finance. The pattern across all of it: problems where the data is enormous, the math is unforgiving, and the latency budget is measured in microseconds.
Modern GPUs can consume terabits per second, but legacy storage formats are a huge bottleneck - they effectively require CPUs to sit in the middle, decompressing data before passing it on.
LLVM, but for file formats
Vortex is the part of the story Manning seems most proud of, partly because he gave it away. In August 2025 the format was donated to the Linux Foundation's AI & Data arm, with backing from an unusual trio of supporters: Microsoft, Snowflake, and Palantir. Manning chairs its Technical Steering Committee.
The pitch he uses on stage is "Vortex: LLVM for File Formats." The analogy is exact. LLVM let compiler authors stop reinventing backends and focus on the interesting part, the language. Vortex aims to do the same for storage: stop writing format specifications from scratch, start composing encodings and layouts. The numbers he quotes are roughly 100x to 200x faster random access, 10-20x faster scans, and 5x higher write throughput, with decompression designed to run on the GPU itself via SIMT rather than forcing a CPU to chew the data first.
That's broken. Vortex fixes it.
His pinned GitHub repositories tell the same story without a press release: vortex, fastlanes, and fsst. All compression libraries. All written in Rust. His idea of a side interest is a faster way to pack bytes. The work draws openly on recent research with names like BtrBlocks and FastLanes, and Vortex is built to make those ideas composable rather than locked inside one vendor's binary.
Give away the format, keep the company
There is a strategic puzzle buried in all of this. Vortex is the technical heart of Spiral, and Manning handed it to the Linux Foundation as an open, incubation-stage project that anyone can use. Founders are not usually in the habit of donating their crown jewels. The bet is that a file format only matters if it becomes a standard, and standards do not happen behind a paywall. A format that Microsoft, Snowflake, and Palantir all support is worth more to the industry, and ultimately to Spiral, than a proprietary one nobody trusts to outlive its maker.
It also fits the man. He sponsors six open-source developers and projects out of his own pocket, among them the Zig programming language and its tooling. The instinct to fund the unglamorous infrastructure that other people build on is the same instinct that made him donate Vortex. The company, in his telling, is not the format. The company is everything you wrap around the format to make it safe, governed, and fast at exabyte scale.
Three people who have done this before
Manning did not build Spiral alone. He co-founded it with Rob Kruszewski and Nick Gates, a pair General Catalyst summarized as "three of the sharpest technical minds in data infrastructure." It is a deliberately small, deeply senior group; Spiral runs lean, with a headcount still measured in the high teens. The shared background is the point. These are people who have already lived through the operational reality of data systems at enormous scale, which tends to produce strong opinions about what is broken and a low tolerance for rebuilding the same mistakes.
Manning's own pre-startup detour through Citadel and quantitative finance left a mark on how Spiral thinks about latency and cost. In a trading firm, a few microseconds and a few wasted dollars per query are not academic; they compound into real money. He carries that sensibility into AI infrastructure, where the wasted resource is no longer a trader's time but a rack of accelerators sitting idle, waiting on data that should already be there.
The third age of data systems
Manning frames the whole company around a tidy bit of history. There have been, he argues, three ages of data infrastructure.
Human in, human out
The Postgres era. People type the data, people read the results. Comfortable, slow, and roughly as old as Taylor Swift.
Machine in, human out
Big Data and the Lakehouse. Machines generate floods of data, but a dashboard and a human still sit at the end.
Machine in, machine out
AI systems are the primary consumers now. No human at the end. Throughput, cost, and composability beat dashboards. This is Spiral's bet.
The problem with living in Age III on Age II infrastructure, he points out, is waste. A single H100 can be fed by something like four million 100KiB images per second in theory; in practice these expensive accelerators sit idle around 70% of the time, starved by data pipelines duct-taped together from a half-dozen tools that leak credentials and compromise permissions on the way. Spiral's answer is one API and end-to-end permissioning over object storage, from a tiny embedding to a massive video file.
The awkward middle is where he lives. Traditional warehouses are tuned for tidy rows and columns; blob stores are tuned for enormous files. The data that AI actually trains on sits in between, in the range from roughly one kilobyte to twenty-five megabytes, and almost nothing handles it gracefully. That gap is not a footnote in Spiral's pitch. It is the whole company. Manning keeps returning to it on stage and in writing, from a Carnegie Mellon database seminar in October 2025 to essays on Conway's law and how the shape of an organization quietly dictates the shape of the software it ships.
Quotable
Taylor Swift, Postgres, and I are all roughly the same age.
Making GPUs go brr.
Read Parquet, explode to Arrow at 10x memory, convert to tensors, cache, train, repeat. That's the ordeal Spiral deletes.
We created Vortex to support this next generation of workloads.