Skip to content
View original post on X: PaddlePaddleOfficial· 34/100AI score34/100

FlashMask library speeds LLM long-context training up to 1.55x over FlashAttention-4

AISummary

PaddlePaddle announces FlashMask, an attention library for LLM training that supports complex attention masks and runs up to 1.55x faster than FlashAttention-4. It adds context parallelism with load balancing and computation–communication overlap, with PaddlePaddle and PyTorch support via CuTe DSL on SM90 and SM100 GPUs. The team says FlashMask-3 has been accepted at NeurIPS 2026.

Post on XView on X
PaddlePaddleVerified on X
@PaddlePaddle

FlashMask: Faster, More Flexible Attention for Long-Context Training
We’re excited to share FlashMask, a high-performance Attention library for LLM training!
🔸 Flexible support for complex attention masks
🔸 Up to 1.55× faster than FlashAttention-4
🔸 Efficient context parallelism with load balancing and computation–communication overlap
🔸 PaddlePaddle & PyTorch support via CuTe DSL
🔸 SM90 & SM100 support
🔸 Large head dimensions: 192/128, 256/256, 512/512, 576/512

FlashMask-3 is accepted at NeurIPS 2026!
🌐 GitHub: https://github.com/PaddlePaddle/FlashMask
💻 Paper: https://arxiv.org/pdf/2610.17901
🤗 Project: https://enigmatisms.github.io/flashmask-3

Source: PaddlePaddle · x.comPublished · added here

Topics

Open-source ecosystemDeployment & engineeringData & trainingProduct updates