ARC Prize reports DeepSeek V4.1 Flash scores 3.3% on ARC-AGI-3 for under $1k

ARC Prize reports that DeepSeek V4.1 Flash scored 3.3% on ARC-AGI-3 at a total evaluation cost of $878 using the provider adapter harness.

Under the standard harness, the model scored 0.9% at a cost of $1.2k, according to the same post. ARC Prize says it is the first model it has evaluated on ARC-AGI-3 to score above 3% for under $1k in total cost.

On the earlier ARC-AGI benchmarks, ARC Prize reports DeepSeek V4.1 Flash scored 94.5% on ARC-AGI-1 at $0.07 per task and 72.9% on ARC-AGI-2 at $0.13 per task. Those scores are 5.5 and 11.5 points above V4 Flash's best results, but ARC Prize says they cost about 250% more per task. ARC Prize has published the full results, along with its benchmarking code and testing policy, on its leaderboard.