Baseten cuts delta weight sync to 15 seconds with 0.8-second pause
Overview
Baseten says it rebuilt its delta weight sync path so each GPU's existing weights serve as the parent, removing a full CPU copy of the checkpoint.
On GLM-5.3, Baseten reports a sync that took 36 seconds with a 6-second pause now takes 15 seconds with a 0.8-second pause.
Baseten also says the host memory freed by this change is used for KV cache offloading, which raises Kimi K3 cache hit rates from 2% to 92% at concurrency 48. These figures come from Baseten's own blog post and have not been independently verified.
Written by AI from the articles below · updated Oct 30, 3:51 PM ET
Check the sources:
Article timeline
The articles in this story. Times are ET.
- Baseten BlogOfficialPickBaseten cuts delta weight sync to 15 seconds with a 0.8-second pause
AIBaseten says it rebuilt its delta weight sync path so each GPU's existing weights serve as the parent, removing a full CPU copy of the checkpoint. On GLM-5.3, a sync that took 36 seconds with a 6-second pause now takes 15 seconds with a 0.8-second pause. The freed host memory is used for KV cache offloading, which Baseten says raises Kimi K3 cache hit rates from 2% to 92% at concurrency 48.
Heat Trend
Not enough continuous observations to show a trend yet.