Zyphra says its ZOE serving system speeds up GLM-5.3-Flash on AMD MI300X
Article timeline
The articles in this story. Times are ET.
ZyphraOfficialZyphra's ZOE makes GLM-5.3-Flash on MI300X profitable at 64 sessionsAIZyphra says its ZOE serving system makes GLM-5.3-Flash on MI300X profitable, earning $2.95 per hour per GPU after $2.02/hr rent at 64 sessions per node. Stock SGLang earns $0.13 per hour under the same conditions. Running 96 sessions earns more, but p90 time to first token rises from 2.9s to 21.8s, so Zyphra runs 64.
- ZyphraOfficialZyphra's ZOE cuts p90 time to first token 10.7x versus stock SGLang
AIZyphra says ZOE has 10.7x lower p90 time to first token than stock SGLang at 64 concurrent sessions. The post reports ZOE at 2.9 s and stock SGLang at 31.4 s.
- ZyphraOfficialZyphra adds GLM-5.3-Flash to Zyphra Inference on AMD MI300X
AIZyphra says GLM-5.3-Flash from Z.ai is now served on Zyphra Inference using its Zyphra Optimization Engine (ZOE) on AMD Instinct MI300X. Zyphra claims ZOE makes p90 generation 3.1x faster per user and p90 time to first token 10.7x lower than stock SGLang.
- ZyphraOfficialZyphra's ZOE runs 3.1x faster per user than stock SGLang on AgentX
AIZyphra says its ZOE generates 3.1x faster per user than stock SGLang on one 8xMI300X node in the SemiAnalysis AgentX replay of multi-turn coding agent sessions. At 64 concurrent sessions, ZOE reaches 52.8 tokens/s per user at p90 generation speed, versus 16.9 tokens/s for stock SGLang.
