AA-Omniscience v1.1 largely preserves the rank order from v1, but models score ~2 points lower on Net Score on average on the refreshed set. AA-Omniscience v1.1 has a 0.98 Spearman rank correlation with v1, and the median decrease in Net Score is 0.6 points. Some models see much larger declines, particularly MiniMax M3.1 Flash, down 22 points (16 to -6), and MiMo V2.6 Pro, down 13 points (8 to -5)
Artificial Analysis refreshes AA-Omniscience to v1.1 with lower model scores
AIArtificial Analysis says AA-Omniscience v1.1 keeps the v1 rank order, with a 0.98 Spearman correlation, and models score about 2 points lower on Net Score on average. The median Net Score drop is 0.6 points, but MiniMax M3.1 Flash falls 22 points, from 16 to -6, and MiMo V2.6 Pro falls 13 points, from 8 to -5.
Source details
Artificial Analysis · x.comPublished
Original title: AA-Omniscience v1.1 largely preserves the rank order from v1, but models score ~2 points lower on Net Score on average on the refreshed s...