ARC Prize reports power-curve link between ARC-AGI-1 and ARC-AGI-2 model scores

1. 1:05 PM
   Mike Knoop@mikeknoopX

   Mike Knoop says the science of AI evals is still nascent

   AIMike Knoop says the science of evals is quite nascent given how important they are. The post cites ARC Prize data showing a power-curve fit of ARC-AGI-2 scores from ARC-AGI-1 scores (v2 = v1^4.53) explaining about 98% of the variation. Models tied on ARC-AGI-1 can still differ by over 20 percentage points on ARC-AGI-2.

2. 12:05 PM
   ARC Prize@arcprizeOfficial

   ARC Prize says ARC-AGI-1, -2, and -3 show a progression in model capability

   AIARC Prize says the relationships across ARC-AGI-1, ARC-AGI-2, and ARC-AGI-3 point to a natural progression. It says future benchmarks will help track progress and separate increasingly capable models. Model scores are listed on its leaderboard at

3. 12:05 PM
   ARC Prize@arcprizeOfficial

   ARC-AGI score curves link v1 and v2 results across models

   AIARC Prize says ARC-AGI-2 scores follow a power curve, v2 = v1^4.53, that explains about 98% of the variation in scores. Models tied on ARC-AGI-1 can still differ by over 20 percentage points on ARC-AGI-2.

   

The trend compares only continuously observed participants, so its scope may be smaller than the current heat total. Hover or tap for hourly heat; use arrow keys with a keyboard.