IBM-backed llm-d releases 1.0 with controls for running AI models in-house
AIIBM, Red Hat and Google, the lead maintainers of the open-source llm-d project, release version 1.0 of its Kubernetes-based distributed inference framework. The release adds multi-tenant quality-of-service controls, zero-downtime upgrades and routing that reuses cached context for AI agents.
Source details
SiliconANGLE · AI · Paul Gillin · siliconangle.comPublished · added here
Original title: IBM-backed llm-d framework adds controls for running AI models in-house