Great work from @RedHat_AI bringing FP8 + DSpark to NVIDIA Nemotron 3.5 Super VL with LLM Compressor, Speculators, and vLLM!
Their benchmarks on 4× H100 show how the optimizations complement each other: FP8 improves throughput under load, while DSpark boosts per-user streaming speed. Together, they outperform either optimization alone across HumanEval, math reasoning, and RAG workloads.
Both checkpoints are open and ready to explore on Hopper.
Red Hat AI releases FP8 and DSpark checkpoints for Nemotron 3.5 Super VL
AISummary
Red Hat AI says FP8 quantization and DSpark speculative decoding, run on 4× H100 with vLLM, together outperform either optimization alone on HumanEval, math reasoning, and RAG workloads. FP8 improves throughput under load, and DSpark raises per-user streaming speed. Both checkpoints for NVIDIA Nemotron 3.5 Super VL, a 120B multimodal MoE, are open on Hugging Face.
Post on XView on X
vLLMVerified on X
@vllm_project
Two optimized checkpoints for NVIDIA Nemotron 3.5 Super VL, a 120B multimodal MoE for long-running agents. Quantization and speculative decoding stack for close to 2x on Hopper, accuracy intact. FP8 (LLM Compressor): https://huggingface.co/RedHatAI/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-FP8 DSpark (Speculators): https://huggingface.co/RedHatAI/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B.dsparkView quoted post on X
Source: vLLM · x.comPublished · added here