Apple paper finds app and database diversity matters more than task count for tool-use agents

Apple researchers report that for training tool-use agents, more diverse apps and database states improve generalization more than more training tasks.

At a fixed budget of 32,768 tasks, adding apps and database versions raised a 4B model's score on AppWorld's challenge split from 38.4% to 45.0%, according to a post by Rohan Paul on X summarizing the paper. The researchers' data-mix scaling law predicted larger held-out runs with 0.0285 error, compared with 0.1706 for a law that uses task count alone.