We worked on a llama.cpp impl for DS4.1-Flash + made Unsloth dynamic GGUFs for it!
The engram / PLE can be swapped out from MXFP8 to Q6_K to Q5_K and IQ4_NL precision - we found Q6_K approx = MXFP8 and IQ4_NL loses a tiny bit of accuracy.
The engram can stream fully from disk!
DeepSeek V4.1-Flash-1017 can now run locally! 🐳 We made 1-bit retain ~88.2% accuracy while shrinking size from 508GB to 250GB (−52%). Run on a 128GB+ RAM device. V4.1 Flash 1017 is the strongest open model to date Guide: https://unsloth.ai/docs/models/deepseek-v4.1-flash GGUF: https://huggingface.co/unsloth/DeepSeek-V4.1-Flash-1017-GGUF
Daniel Han · x.comPublished
Original title: We worked on a llama.cpp impl for DS4.1-Flash + made Unsloth dynamic GGUFs for it!
