PhysisServe pipelines denoising steps and Transformer blocks across GPUs

AISummary

BAAI says PhysisServe, a system for autoregressive denoising inference, splits the model into stages that each correspond to one Transformer block at one denoising step.

It pipelines both dimensions at once, so a single stream can use as many GPUs as the number of denoising steps times the number of blocks.

In steady state, output latency depends only on the slowest stage, and adding steps or depth does not reduce output frequency.

Post on XView on X

BAAI

@BAAIBeijing

Part of a thread · earlier post

Existing autoregressive denoising inference schemes use coarse pipeline granularity, bottlenecking parallelism and output frequency. PhysisServe solves this across three system layers, starting with single-stream acceleration:

Stage-wise Pipelining

• Splits the model into fine-grainedstages—each corresponding to a single Transformer block at a specific denoising step—as the basic unit of compute allocation and scheduling.

• Pipelines both denoising steps and Transformer blocks simultaneously, scaling the number of GPUs usable by a single stream to the product of the two dimensions.

• In steady state, output latency is bounded only by the slowest stage; adding denoising steps or model depth increases pipeline depth and GPU count without degrading output frequency.

Source: BAAI · x.comPublished Oct 15, 9:41 AM ET · added here 9:52 AM ET

PhysisServe pipelines denoising steps and Transformer blocks across GPUs

Topics