From chips to clusters: simulating LLM serving at scale.
Zhenhua Zhu from Tsinghua Shenzhen International Graduate School on an open-source platform for finding bottlenecks across compute, memory, interconnects and serving software.
Open-source platform simulates LLM serving bottlenecks from chips to clusters
AIZhenhua Zhu of Tsinghua Shenzhen International Graduate School presents an open-source platform for finding bottlenecks in LLM serving. It models compute, memory, interconnects and serving software, from chips to clusters.
Source details
LMSYS Org · x.comPublished · added here
Original title: From chips to clusters: simulating LLM serving at scale.