continuous-batching
(2)
FriendliAI
FriendliAI is a San Francisco-based AI inference cloud built by the researchers who invented continuous batching - the technique now standard across the industry. Its platform serves open-weight and custom generative AI models in production with high throughput, lower GPU costs, and 99.99% reliability, used by customers including LG and Twelve Labs.
Byung-Gon Chun
Byung-Gon Chun is the CEO and Co-founder of FriendliAI, and a professor of Computer Science and Engineering at Seoul National University currently on leave. A systems researcher turned founder, he is best known for inventing continuous batching - the scheduling technique that became the default standard in every major LLM inference engine, from vLLM to TensorRT-LLM. His lab published the foundational ORCA paper at OSDI 2022, and he then turned that academic insight into FriendliAI, an enterprise AI inference platform that raised $26.7M and supports over 550,000 models from Hugging Face. With a career spanning Intel, Yahoo!, Microsoft, and Facebook, Chun brings rare depth across both research and production AI infrastructure.