we got the inference goats 🐐
I will shortly teach you how quantization works, modern ideas (double quantization, calibration) and llama.cpp quants (k-quants, i-quants)
super excited and starstruck 🥵
Merve Noyan teaches quantization and llama.cpp k-quants in Hugging Face livestream
AIMerve Noyan says she will shortly teach how quantization works, covering double quantization, calibration, and llama.cpp k-quants and i-quants. A Hugging Face livestream next Tuesday will give a primer on quantization with llama.cpp and vLLM, featuring Merve Noyan, Maxime Labonne, Aditya Tadimeti, Bartowski, and Mgoin.
@mervenoyann
Join us in this livestream next Tuesday, learn how to shrink models to fit in your laptop and make them go brrr 💨 We will have @mervenoyann @maximelabonne @TadimetiAditya @bartowski1182 and @mgoin_ giving a primer and know-hows on quantization with llama.cpp and vLLM ⚡️