llm-inference
(5)
Modal
Modal is a serverless cloud platform that lets developers run compute-intensive AI and data workloads - inference, training, fine-tuning, batch jobs, and sandboxed code - without managing infrastructure. Built from scratch in Rust with a custom container runtime, file system, and GPU memory snapshotting, it delivers sub-second cold starts and elastic autoscaling from zero to thousands of GPUs, billed by the second. Founded in 2021 by Erik Bernhardsson and Akshat Bubna, the New York-based company reached roughly $300M in annualized revenue and raised a $355M Series C in 2026 at a $4.65B valuation.
Enfabrica
Enfabrica is a Silicon Valley chip startup building the networking and memory plumbing that lets tens of thousands of GPUs behave like one giant computer. Its Accelerated Compute Fabric SuperNIC (ACF-S) moves data at 3.2 terabits per second, and its EMFASYS memory fabric lets AI clusters borrow cheap DDR5 DRAM over Ethernet instead of buying more GPUs for capacity. Founded in 2019 by ex-Broadcom and Google networking engineers, the company raised roughly $290M before Nvidia paid over $900M in September 2025 to license its technology and hire CEO Rochan Sankar and much of his team.
Parasail
Parasail is a San Mateo-based AI inference cloud that lets developers run open-source and custom models without buying or committing to their own GPUs. Instead of owning chips, it orchestrates rented GPU capacity across roughly 40 data centers in 15 countries plus its own clusters, automatically routing each workload to the cheapest, fastest hardware. Founded in 2023 by Mike Henry (ex-Groq CPO, Mythic founder) and Tim Harris (Swift Navigation founder), the company processes on the order of 500 billion tokens a day and pitches a token-based economy - pay for output, not hardware.
Quadric
Quadric is a semiconductor IP company that licenses the Chimera family of general-purpose neural processing units (GPNPUs) - a single, fully C++ programmable processor core that runs machine learning inference alongside classic DSP and control code in one unified architecture. Founded in 2016 by three engineers who previously built bitcoin computing hardware, Quadric aims to fix the fragmentation of on-device AI silicon: instead of a fixed-function neural accelerator that breaks whenever a new model appears, its cores scale from 1 to 864 TOPS and can adopt new networks - including large language models up to 30 billion parameters - through a software port rather than a chip respin.
Positron AI
Positron AI designs purpose-built inference hardware for transformer models, aiming to make Nvidia GPUs optional for running large language models at production scale. Its first product, Atlas, ships from US fabs and claims roughly 3x lower latency and 4x better performance-per-watt versus an H100 system.