Skip to content
View original post on X: elvis
Published
62/100

IBM's JevAct adds a runtime decision gate to tool-using LLM agents

AIIBM researchers introduce JevAct, a runtime check that sits between an LLM agent and its tools and verifies selected tool calls before they run. Failed calls return to the agent with feedback for revision, and the paper reports that Opus 4.6's three-run success on τ²-bench Telecom rose from 39.5% to 80.7%.
elvisVerified on X
@omarsar0
View on X

Banger paper from IBM on decision gates for agents.

They add a runtime gate to tool-using agents and keep the models and tools the same.

JevAct is a check that sits between the agent and its tools. An LLM first turns the domain policy into code checks and yes/no questions. Before a selected tool call runs, the code checks the facts in the tool records, and a Jev-like decision model answers the questions about the conversation. If the call fails a check, it goes back to the agent with feedback so the agent can revise it.

On τ²-bench Telecom, Opus 4.6's success across all three runs rose from 39.5% to 80.7%.

On Telecom, all eight agents improved by 16.9 points on average in three-run success. All nine improved on BFCL. Airline results were smaller and mixed.

They show that the gains come from the harness.

https://arxiv.org/abs/2610.38620

Chat with Paper: https://academy.dair.ai/papers/think-before-you-act-decision-gate-for-llm-agents-2610.38620

Source details

elvis · x.comPublished · added here

Original title: Banger paper from IBM on decision gates for agents.