ALERT🚨🚨🚨: AMD's @AnushElangovan and his team have achieved up to 2x better performance per dollar than even B200 on Kimi K3 Agentic Inference! This is due to various software optimization efforts, including tuning AITER kernels and MonoKernel decode.
Along with MI355X having 2304GB of HBM per server, this allows it to avoid crossing slow 8x400Gbit/s scale-out networking between servers, which B200 needs to do due to having less than 1.5TB of HBM.
Great work to Anush, Peng Sun, Ramine, Emad, and the vLLM team for this amazing work!