OpenBMB's MiniCPM-V 4.7 runs a 35B multimodal model on iPhone with about 3GB of memory
Original title (Chinese)
iPhone断网跑35B多模态！国产开源新模型，只占3G内存
AISummary
OpenBMB and ModelBest open-sourced MiniCPM-V 4.7, which includes a 35B-A3B MoE version and a 1B version. The 35B-A3B model activates 3B parameters per inference and runs offline on an iPhone 17 Pro Max at over 13 tokens per second using about 3GB of memory. The article says its OmniDocBench v1.6 score of 94.24 leads four other models by more than 12 points.
AIWhy it matters
The article explains how the model's MoE design and expert prefetching allow a 35B model to run on about 3GB of memory, which matters for on-device deployment.
Source: QbitAI · qbitai.comPublished · added here