Skip to content
View original post on X: SGLangOfficial· 25/100AI score25/100

SGLang publishes a cookbook for StepFun's Step-5-Preview model

AISummary

SGLang (@sgl_project) shares a cookbook for running StepFun's Step-5-Preview model. The guide covers a B300 setup with tensor parallelism of 4, FP8 quantization, TensorRT-LLM MoE, and MTP speculative decoding, with HiCache off and bf16 KV cache.

Post on XView on X
SGLangVerified on X
@sgl_project

Part of a thread · earlier post

@StepFun_ai Cookbook：https://docs.sglang.io/cookbook/autoregressive/StepFun/Step-5-Preview#hw=b300&parallel=tp4&quant=fp8&moe=trtllm&hicache=off&kvDtype=bf16&spec=mtp

Source: SGLang · x.comPublished

Topics

Open-source ecosystemDeployment & engineeringTutorials & how-to