Inferact, the company founded by vLLM's core maintainers, thanks a16z for profiling its story. The company says inference is still far from solved and is hiring across all teams in San Francisco and Singapore.
The post links vLLM's open-source history to Inferact's funding and hiring, showing how an inference project's maintainers are building a company around it.
Thank you @a16z for telling the Inferact story.
Four years in, inference is still far from solved. Models keep getting bigger, workloads keep getting more varied, and the hardware landscape keeps expanding. Inferact exists to build the future of inference with the vLLM community.
We’re hiring across all teams in SF and Singapore: https://inferact.ai/careers
In 2022, Berkeley PhD student Woosuk Kwon thought he could make Meta's new open-weight model, OPT-175B, run faster in two weeks. Four years and 3,000 contributors later, that project is @vllm_project, one of the largest open-source inference engines, running on ~500,000 GPUs at any given time. He and fellow researchers Simon Mo, Kaichao You, and Roger Wang borrowed "virtual memory" from operating systems, handing out memory only as it's needed so each GPU can juggle requests of wildly different sizes. (The "v" in vLLM is for virtual.) Their first user, Berkeley neighbor @arena, cut its GPU needs in half. Today Google, Amazon, and Meta all run vLLM in production. In January, the core maintainers founded @inferact and raised $150M to give vLLM a full-time team. They're hiring across eng, marketing, and devrel in SF and Singapore: https://a16zjobs.substack.com/p/inferact-raised-one-of-the-largest @simon_mo_ @woosuk_k @KaichaoYou @rogerw0108
Source: Inferact · x.comPublished · added here
