NYU and Amazon paper: a few persistently useful skills match larger distilled skill banks

NYU and Amazon researchers publish a paper finding that distilling only a few skills matches or beats distilling a skill bank up to 11 times larger.

The method, SGUID, keeps only skills that help early in training and still help late. The paper reports that with 6 such skills, 3 of 4 models matched or beat the full bank of 30 to 71 skills on math contest tests.

A second round adding 3 new skills raised Qwen3-8B from 64.3% to 66.3%, according to the paper's report as relayed by Rohan Paul on X.