llama serve -hf ggml-org/Muse-Spark-Open-816B-A42B-GGUF:Q2_K (on two mac studios).
Well done @finkd & @johnternus!
Hugging Face's Clément Delangue shares llama serve command for Muse-Spark-Open GGUF
AISummary
Clément Delangue posts a llama serve command that runs ggml-org/Muse-Spark-Open-816B-A42B-GGUF at Q2_K quantization on two Mac Studios. He credits @finkd and @johnternus for the setup.
Post on XView on X
Source: clem 🤗 · x.comPublished · added here