LLMs with scaffolds have lagged on text-to-SQL, a task that relies on human judgment. By folding expert judgment into every part of RLVR on Tinker, @maxYuxuanZhu and @ddkang (UIUC and Bridgwater) trained the first text-to-SQL model to beat the human mark.
https://thinkingmachines.ai/news/putting-task-expertise-into-rl
UIUC and Bridgewater train first text-to-SQL model to beat human experts
AISummary
Researchers Yuxuan Zhu and Daniel Kang, from UIUC and Bridgewater, trained the first text-to-SQL model to surpass the human benchmark by folding expert judgment into every part of RLVR on Tinker. The post says LLMs with scaffolds had lagged on this task, which relies heavily on human judgment.
Post on XView on X
@tinkerapi
Source: Tinker · x.comPublished · added here