StepFun releases open weights for Step 5 Preview and publishes its architecture blog
Original titleStepFun Releases Open Weights for Step 5 Preview, Publishes Technical Blog on Its Architecture
StepFun released the open weights for its Step 5 Preview model, a sparse mixture-of-experts model with 600B total parameters, 27B active per token, and a 1 million token context window.
The company published a technical blog describing a 92-layer design and Sparse GQA attention that indexes in blocks of 8 tokens.
The weights use the StepFun Model License 1.0, which StepFun describes as Apache-2.0-style with additional attribution and commercial-license conditions.
The article details the 92-layer design, Sparse GQA attention, and training choices behind a long-context agent model, which helps readers compare architectures for million-token workloads.
Source: Pandaily · pandaily.comPublished · added here