BAAI proposes time-division multiplexing to serve SLOs on fewer GPUs

BAAI says a single stream can meet target SLOs on fewer GPUs when extreme minimum latency is not required.

Its approach decouples pipeline stages from physical GPUs, so one GPU runs multiple stages in different time slices. It also merges tasks from different chunks on the same Transformer block into one batch, which BAAI says amortizes weight loading and kernel launch overhead. The claim comes from a single post by BAAI's official account on X, and no measured results are included in the report.