The science of evals is quite nascent given how important they are
Mike Knoop says the science of AI evals is still nascent
AIMike Knoop says the science of evals is quite nascent given how important they are. The post cites ARC Prize data showing a power-curve fit of ARC-AGI-2 scores from ARC-AGI-1 scores (v2 = v1^4.53) explaining about 98% of the variation.
Mike KnoopVerified on X
@mikeknoop
Plotting ARC-AGI model scores reveals a strong relationship across all three benchmarks. For example, the power curve v2 = v1^4.53 explains about 98% of the variation in ARC-AGI-2 scores. Models tied on ARC-AGI-1 can still differ by over 20 percentage points on ARC-AGI-2.
Source details
Mike Knoop · x.comPublished
Original title: The science of evals is quite nascent given how important they are