Skip to content
View original post on X: elvis
Published
40/100

Qwen3-14B's Spider 2.0-SQLite score rises from 22.2% to 54.8% with harness training

AIHarnessSQL trains Qwen3-14B inside the same execution harness used at deployment, raising its Spider 2.0-SQLite score from 22.2% to 54.8%. The method uses isolated executable databases with hidden answer checks, keeps only verified trajectories for SFT, then applies execution-reward RL. Qwen3-8B rises from 15.5% to 45.2%, and both models transfer to BIRD-Interact and LiveSQLBench.
elvisVerified on X
@omarsar0
View on X

Pay close attention to custom harnesses.

This is a super interesting paper showing the impact of training inside the harness.

Qwen3-14B goes from 22.2% to 54.8% on Spider 2.0-SQLite when it is trained in the same execution harness it uses at deployment.

Text-to-SQL models are usually trained to write one static query. Deployed database agents inspect schemas, run probe queries and revise, and the harness for that only appears at inference time.

HarnessSQL builds isolated executable databases with hidden answer checks. Teachers run inside the target harness; we keep only verified trajectories for SFT, then apply execution-reward RL.

In addition, Qwen3-8B rises from 15.5% to 45.2%, and both models transfer to BIRD-Interact and LiveSQLBench.

Paper: https://arxiv.org/abs/2610.12274

Chat with Paper: https://academy.dair.ai/papers/harnesssql-harness-native-training-for-sql-agents-in-realistic-database-environm-2610.12274

Source details

elvis · x.comPublished · added here

Original title: Pay close attention to custom harnesses.