ZOE has 10.7x lower p90 time to first token (TTFT) than stock SGLang at 64 concurrent sessions:
ZOE = 2.9 s
Stock SGLang = 31.4 s
Zyphra's ZOE cuts p90 time to first token 10.7x versus stock SGLang
AIZyphra says ZOE has 10.7x lower p90 time to first token than stock SGLang at 64 concurrent sessions. The post reports ZOE at 2.9 s and stock SGLang at 31.4 s.
Source details
Zyphra · x.comPublished · added here
Original title: ZOE has 10.7x lower p90 time to first token (TTFT) than stock SGLang at 64 concurrent sessions: