# Zyphra finds most FP8 error comes from 8-bit input conversion

> Original title: Across the three models we studied, most FP8 error comes from converting the inputs to 8 bits.

- Source: Zyphra
- Published: 2026-11-06T02:36:49.000Z
- Bad Influence: https://badinfluence.news/items/fp3qbekw6t0avjscj7nniwvmv
- Original: https://x.com/ZyphraAI/status/2118533029669027954

## Summary

Zyphra says that across three models it studied, most FP8 error comes from converting inputs to 8 bits. On Mistral-7B, rotating queries and keys to spread out large values roughly halves input error, with almost no extra time when fused with quantization.

## Post

Across the three models we studied, most FP8 error comes from converting the inputs to 8 bits.

On Mistral-7B, rotating queries and keys to spread out large values roughly halves input error, with almost no extra time when fused with quantization.

![](https://badinfluence.news/api/img-proxy?u=https%3A%2F%2Fpbs.twimg.com%2Fmedia%2FHWaKdohaYAAJMqy.jpg&mode=thumb&exp=1794182400&sig=94ea51b53978fb76)