vLLM reports GB200 forward-pass gains from Inferact kernels in microbenchmarks

1. 11:26 AM
   vLLM@vllm_projectOfficial

   Inferact's kernel work cuts GB200 forward times in microbenchmarks

   AIInferact's fused all-reduce plus norm, packed QKV, and tanh softcap kernels cut GB200 forward microbenchmark times, per vLLM. TP4 batch 1 fell from 4.23 ms to 3.89 ms, and TP8 batch 128 from 8.15 ms to 6.47 ms. The benchmarks used dummy weights and measured forward passes, not end-to-end serving.

   
2. 9:23 AM
   vLLM@vllm_projectOfficial

   vLLM shares GB200 forward microbenchmark gains from Inferact's kernel work

   AIvLLM says Inferact's fused all-reduce plus norm, packed QKV paths, and tanh softcap kernel cut GB200 forward time from 4.23 to 3.89 ms at TP4 batch 1, and from 8.15 to 6.47 ms at TP8 batch 128. The figures come from forward microbenchmarks with dummy weights, not end-to-end serving results.

   

The trend compares only continuously observed participants, so its scope may be smaller than the current heat total. Hover or tap for hourly heat; use arrow keys with a keyboard.