Skip to content
View original post on X: ZyphraOfficial
Published
38/100

Zyphra's ZYFA cuts softmax work per score from 14 to 6 cycles

AIZyphra says its ZYFA "lean P" replaces softmax's exponential with a multiply-add and an integer conversion, cutting vector work per score from 14 to 6 cycles. The post notes that FP8 makes matrix math 2x faster than BF16 but leaves softmax unchanged, making softmax the bottleneck in long-context prefill.
ZyphraVerified on XOfficial
@ZyphraAI
View on X

Part of a thread · earlier post

FP8 makes matrix math 2x faster than BF16 but does not speed up softmax, so softmax becomes the bottleneck in long-context prefill.

ZYFA's "lean P" replaces softmax's exponential with a multiply-add and an integer conversion, cutting vector work per score from 14 to 6 cycles.

Source details

Zyphra · x.comPublished · added here

Original title: FP8 makes matrix math 2x faster than BF16 but does not speed up softmax, so softmax becomes the bottleneck in long-context prefill.

Topics

Deployment & engineeringTutorials & how-to