Love seeing these real-world comparisons! 💙 Thanks for including MiniCPM5 in your tests. Exciting to see how far small models have come! 🚀
OpenBMB thanks PaulGug for testing MiniCPM5 in real-world comparisons
AIOpenBMB replies to PaulGug's personal-agent tests, which compared MiniCPM5 with Spark X2.6 1.7B and LFM3 at 2B and under. PaulGug calls MiniCPM5 the balanced model of the three, while Spark X2.6 leads on context recall at 8K and 16K.
@OpenBMB
Small models are going through a real growth spurt right now. On my personal-agent tests (PA Core, reasoning on, RTX 3080), three models at 2B and under now land within 21 points of each other, and rubbing shoulders with older 4B/9B weights. Newest on the bench: Spark X2.6 1.7B from @SparkLLM - PA total: 172.3/300 - Tool calls: 30/30 in every run - Context recall: 9.3 at 8K, 8.7 at 16K, best of the three! - Schedule and draft (PA2): 98 and 100 in two runs, one 54 pulled the mean to 84 - Daily organization (PA1): 35.7, the weak spot Under 2B, three different strengths. @liquidai LFM3 plans, @SparkLLM Spark reads, @OpenBMB MiniCPM5 is the balanced one. Leave thinking on. With it off, Spark is a much weaker model on these tasks.
Source details
OpenBMB · x.comPublished
Original title: Love seeing these real-world comparisons! 💙 Thanks for including MiniCPM5 in your tests. Exciting to see how far small models have come! 🚀