vLLM describes Beam's MoE, attention and speculative drafting design

vLLM (@vllm_project) publishes a description of Beam's architecture.

The model routes each token to 8 of 256 experts across 52 layers. Its attention alternates between a 512-token sliding window and global NoPE attention. A separate 5-layer drafter supplies speculative tokens for the main model to verify. The report is a single post from vLLM's X account, and the post does not include benchmark results.