# Joel Niklaus releases Harness Leaderboard comparing nine coding-agent harnesses

> Original title: Hermes supremacy tweet #9138

- Source: Teknium
- Published: 2026-11-06T07:20:55.000Z
- Bad Influence: https://badinfluence.news/items/rd21whq0n48z7lofc02jj31fn
- Original: https://x.com/Teknium/status/2118604525003251800

## Summary

Joel Niklaus releases the Harness Leaderboard, comparing nine coding-agent harnesses on open models across 28,849 trials and $4,746 of inference. Averaged over three models, mini-swe-agent solves 51.4% of tasks, while Pi solves 49.4% at $0.29 per solved task, about 45% less than mini-swe-agent's $0.53. The best harness depends on the model, and opencode costs $3.35 per solved task, 11.7 times Pi's cost.

## Post

Hermes supremacy tweet #9138

![](https://badinfluence.news/api/img-proxy?u=https%3A%2F%2Fpbs.twimg.com%2Fmedia%2FHWbLlHZbkAAJ6io.png&mode=thumb&exp=1794182400&sig=27f30a231098fe9e)

## Quoted post @joelniklaus

> I burned a shitload of compute so you don't have to. Releasing the Harness Leaderboard.
> 
> Which harness should you run your open model in? I ran 28,849 trials to find out.
> 
> The leaderboard compares nine coding-agent harnesses on open models. In each comparison only the harness changes: the model, provider and reasoning effort stay fixed, and every task gets three attempts.
> 
> The trials ran GLM-5.3-Flash, DeepSeek-V4.1-Flash and Flint on Terminal-Bench 2.1 and DeepSWE, for $4,746 of inference.
> 
> Averaged over the three models, mini-swe-agent (@KLieret), a bash-only agent, solves 51.4% of tasks and @openclaw 27.3%. Four others are statistically tied with mini-swe-agent.
> 
> One of them is Pi (@pidotdev), at 49.4% for $0.29 per solved task, about 45% less than mini-swe-agent's $0.53.
> 
> The best harness depends on the model: mini-swe-agent on GLM-5.3-Flash, Hermes Agent (@NousResearch) on DeepSeek-V4.1-Flash and Kimi Code (@Kimi_Moonshot) on Flint.
> 
> @opencode costs $3.35 per solved task, 11.7 times as much as Pi.
> 
> See Harness Hub's Harness Lens for what the nine harnesses do differently. It documents the prompts, tools and compaction of 34 coding-agent harnesses and links every claim to its source.
> 
> Leaderboard: https://huggingface.co/spaces/joelniklaus/harness-leaderboard
> Harness Lens: https://huggingface.co/spaces/harness-hub/harness-lens

https://x.com/joelniklaus/status/2118373808692437403