vLLM says its training forward pass calls its inference kernels directly, aligning FP8 GEMM, sparse-attention Top-K, FP8 KV-cache reads, MoE communication and CUDA Graph execution. The aim is for the same token to get the same result across validated batch sizes, phases and parallel layouts. The backward pass uses separate checks.
Part of a thread · earlier post
2/ The training forward calls vLLM's inference kernels directly.
We align FP8 GEMM, sparse-attention Top-K, FP8 KV-cache reads, MoE communication and CUDA Graph execution so the same token gets the same result across the validated batch sizes, phases and parallel layouts.
Shared numerics in the forward pass, with independent checks for the backward pass.
Source: vLLM · x.comPublished · added here
