OpenBMB adds Lookahead Expert Scheduling to MiniCPM-V 4.7 for on-device MoE inference

OpenBMB says its Lookahead Expert Scheduling (LES) in MiniCPM-V 4.7 speeds up running a 35B mixture-of-experts model on memory-limited devices, with expert weights stored on SSD. LES predicts which experts the next layer may need, computes with experts already loaded, and loads the rest in parallel, so generation does not stall while waiting for every expert to load, according to OpenBMB's X posts. OpenBMB says LES requires no additional training and that the original token-level router still selects the experts actually used. Fine-grained layer-wise mixed-precision quantization and contiguous expert weight layouts further reduce storage and transfer overhead, according to the first post.