SGLang and NVIDIA speed up Kimi K3 inference on early-access Vera Rubin hardware

AIRadixArk says Miles runs reinforcement learning end to end on NVIDIA Vera Rubin, using SGLang rollouts, Megatron training and one container image. Agentic RL runs 64 concurrent sandboxes on the Vera CPU next to the GPUs. The linked SGLang post reports that Kimi K3 inference gained up to 20% faster FP8 MLA at batch 1 with 128K context, and a 5.9% end-to-end speedup from MoE tail fusion.

AISGLang optimized attention, MoE, and speculative verification kernels for Kimi K3 NVFP4 on early-access NVIDIA Vera Rubin hardware. Reported gains include up to 20% faster FP8 MLA at batch 1 and 128K context, 20% faster KDA verification with bitwise-identical output, and a 5.9% end-to-end speedup from MoE tail fusion. Miles, from RadixArk, uses SGLang for rollouts in end-to-end RL training on Rubin, including agentic RL with 64 concurrent sandboxes on the Vera CPU.

AIMiles runs the full reinforcement learning loop on Nvidia Rubin, using SGLang for rollout, Megatron for training, and one container image. On a single 4-GPU tray, Qwen3-30B-A3B's GSM8K reward rises from about 45% to about 95% over 50 rollouts, matching the GB300 curve. The post also reports DeepSeek-V4-Flash end-to-end rollout and training, and Qwen3.5-35B-A3B agentic RL with 64 concurrent mini-SWE-agent sandboxes on SWE-bench Verified, where reward holds near 0.6 and median response length falls about 30%.

AISGLang says restructured MLA decode kernels on Rubin, which fit a deeper pipeline in 327 KiB of shared memory, deliver a 16% speedup at batch 16 with 128K context and bit-identical output. The post also reports 20% faster full FP8 MLA at batch 1 and 20% faster KDA verify kernels after keeping weights in registers and reducing synchronization. It additionally covers fusing MoE finalization, the shared expert, 8-GPU all-reduce, and RMSNorm into one collective kernel.

AISGLang says it worked with NVIDIA to optimize attention, MoE, and speculative verification kernels for Kimi K3 inference on early-access Rubin hardware. It reports up to 20% faster FP8 MLA at batch 1 with 128K context, 20% faster KDA verification with bitwise-identical output, and a 5.9% end-to-end speedup from MoE tail fusion that removes 276 kernel launches per decode step. The post also says SGLang powers Miles' end-to-end RL training on Rubin, including agentic RL with 64 concurrent sandboxes on the Vera CPU.