Skip to content
View original post on X: ZyphraOfficial
Published
27/100

Zyphra's ZYFA cuts FP8 attention numerical error 62–78% versus AITER

AIZyphra reports that its ZYFA method reduces numerical error by 62–78% compared with AITER's FP8 kernel at head dimension 128 on Mistral-7B and Qwen3-8B at 4K tokens. ZYFA rescales blocks before rounding so small attention weights are not incorrectly rounded to zero in FP8, and it normalizes using the rounded values.
ZyphraVerified on XOfficial
@ZyphraAI
View on X

Part of a thread · earlier post

Small attention weights can be incorrectly rounded to zero in FP8. ZYFA rescales blocks before rounding and normalizes using the rounded values.

At head dimension 128, this yields 62–78% less numerical error than AITER’s FP8 kernel on Mistral-7B and Qwen3-8B at 4K tokens.

Source details

Zyphra · x.comPublished · added here

Original title: Small attention weights can be incorrectly rounded to zero in FP8. ZYFA rescales blocks before rounding and normalizes using the rounded ...