4Paradigm and RiseUnion open-source ModelSphere LLM deployment platform
Overview
4Paradigm and RiseUnion have open-sourced ModelSphere under the Apache 2.0 license, a platform that manages compute, model serving and runtime tuning for large language models.
It runs on top of inference engines such as vLLM and SGLang and schedules Huawei Ascend, Iluvatar, Moore Threads and Hygon DCU hardware through an EngineX inference layer.
The companies claim about 285% higher peak throughput than an internal baseline, according to Pandaily's report. The companies have not had that figure independently verified.
Written by AI from the articles below · updated Oct 16, 4:41 AM ET
Check the sources:
Article timeline
The articles in this story. Times are ET.
- PandailyNews4Paradigm and RiseUnion open-source ModelSphere for heterogeneous LLM serving
AI4Paradigm and RiseUnion have open-sourced ModelSphere under Apache 2.0, a platform that manages compute, model serving and runtime tuning above engines like vLLM and SGLang. It schedules Huawei Ascend, Iluvatar, Moore Threads and Hygon DCU resources through an EngineX inference layer. The companies claim about 285% higher peak throughput versus an internal baseline, a figure they have not had independently verified.
Heat trend
Not enough continuous observations to show a trend yet.