Joel Niklaus releases Harness Leaderboard comparing nine coding-agent harnesses

Joel Niklaus releases the Harness Leaderboard, which compares nine coding-agent harnesses running open models across 28,849 trials and $4,746 of inference, according to a post by Teknium on X.

Averaged over three models, mini-swe-agent solves 51.4% of tasks and Pi solves 49.4%. Pi costs $0.29 per solved task, about 45% less than mini-swe-agent's $0.53. The post says the best harness depends on the model, and opencode costs $3.35 per solved task, 11.7 times Pi's cost.