Plan-then-retrieve tool discovery helps multi-tool LLM agent queries but hurts single-tool ones
Original titlePlan or Retrieve First? A Scale Study of Tool Discovery Architectures
AISummary
Apple researchers compare two ways LLM agents find APIs in large registries: retrieving tools from the user query versus first generating Abstract Tool Stubs for retrieval.
On ToolBench's 46,984 endpoints, the plan-then-retrieve approach beats retrieve-then-plan on multi-tool queries but loses on single-tool ones.
The extra discovery call roughly doubles inference cost, which is why the authors suggest routing queries by complexity.
Source: Apple Machine Learning Research · machinelearning.apple.comPublished · added here