llama.cpp v0.7.0, 128GB Mac/Spark:
llama serve -hf ggml-org/Qwen3.8-Flash-Next-GGUF:IQ4_NL --spec-type draft-mtp --lazy-mode on
llama.cpp v0.7.0 runs Qwen3.8-Flash-Next on 128GB Macs and Spark
AISummary
Georgi Gerganov shares a command to serve the Qwen3.8-Flash-Next GGUF model with llama.cpp v0.7.0 on 128GB Mac and Spark systems. The example uses the IQ4_NL quantization, draft-mtp speculative decoding, and lazy mode.
Post on XView on X
Georgi GerganovVerified on X
@ggerganov
Source: Georgi Gerganov · x.comPublished · added here