Skip to content
View original post on X: Daniel Han
Published
34/100

Daniel Han says V3.1 quants beat GSQ and V3 on efficiency

AIDaniel Han says the UD quants for V3.1 beat GSQ and V3 quants on the Pareto efficiency front. UD-IQ3_XXS recovers 99.9% of average accuracy and UD-IQ2_XS recovers 99.4%, both smaller than IQ3_S. The largest gap is on the smallest quant, where UD-IQ1_S recovers 97.9% versus 90.5% for GSQ.
Daniel HanVerified on X
@danielhanchen
View on X

Yes our main goal was to make quants better on the Pareto efficiency front! So vs GSQ & V3 quants, V3.1 is much better!

Avg recovery of UD-IQ3_XXS is 99.9% & UD-IQ2_XS at 99.4% whilst being smaller than IQ3_S - the diffs are largest on small bits - UD-IQ1_S 97.9% vs GSQ 90.5%

orenoa@orenoa7
UD-IQ3_XXSを動かしてみたけど、結局サイズが既存IQ3_Sと変わらず速度向上はdecodeで+0.7〜2.3%程度 ただしベンチマークスコアはUD-IQ3_XXS側がTerminalBench2.1はじめ9/12の指標で上回っている UD系列で面白いのはUD-IQ2_XSだと考えていて、サイズは75.8GBでIQ3_Sより7.8GB減量している こちらはBF16比精度がIQ3_Sに対し+0.4pt改善していて、各種ベンチマークスコアも既存IQ3_Sに近い IQ3_Sがサイズの問題で使えなかった人はUD-IQ2_XSに乗り換える意義が大きと思う
View quoted post on X
Source details

Daniel Han · x.comPublished · added here

Original title: Yes our main goal was to make quants better on the Pareto efficiency front! So vs GSQ & V3 quants, V3.1 is much better!

Topics

Deployment & engineeringProduct updates