3 ways to boost MoE inference on NVIDIA DGX Spark
AINVIDIA's technical blog says batch-size-1 profiling of Inclusion AI's Ling-3.0-flash on NVIDIA DGX Spark showed decoding was memory-bound, with reading model weights driving execution time. The post identifies the mixture-of-experts computation and the output-layer projection as the two operations that stood out.
Source details
NVIDIA Technical Blog · Yang Liu · developer.nvidia.comPublished · added here