Small attention weights can be incorrectly rounded to zero in FP8. ZYFA rescales blocks before rounding and normalizes using the rounded values.
At head dimension 128, this yields 62–78% less numerical error than AITER’s FP8 kernel on Mistral-7B and Qwen3-8B at 4K tokens.
