Skip to content
Trending storyDeveloping

DigitalOcean launches Signals and Simulations to turn failed agent sessions into tests

3 articles3 sourcessince Oct 12Last article 1h ago ·

Overview

AISummary of 3 articles

DigitalOcean has launched Agent Learning Services, including Signals, Simulations and Insights in public preview, and Agent Data Services in private preview.

Testing Catalog says Signals reads live agent traffic and flags failed sessions, such as a successful tool call that queried the wrong region. Users can turn a flagged session into a test that Simulations replays against a proposed fix before it ships.

Elvis Saravia says Signals flags sessions where users became frustrated or tools failed, and converts them into evals with one click. Rohan Paul says Signals uses rule-based checks for signs such as repeated user messages, requests for a human, and looping tool retries, and that Simulations replays flagged sessions with a fake user against the fixed agent. Paul's post says teams no longer need to pay an LLM to grade every conversation or write test cases by hand.

Written by AI from the articles below · updated Oct 12, 12:38 PM ET

Check the sources:

Developments

2 developments

  1. Oct 12, 11:27 AM ET · 2 articles
    DigitalOcean launches public preview of Signals, Simulations, and Insights and private preview of Agent Data Services
    Rohan Paul: DigitalOcean launches Signals and Simulations to catch silent AI agent failures
  2. Oct 12, 11:23 AM ET · 1 article
    DigitalOcean launches Agent Learning Services and Agent Data Services to turn production agent failures into evals
    Elvis Saravia: DigitalOcean's Signals turns failed agent sessions into one-click evals

Article timeline

The articles in this story. Times are ET.

Oct 12
  1. Rohan PaulX
    DigitalOcean launches Signals and Simulations to catch silent AI agent failures

    AIDigitalOcean launches Signals, which watches live agent sessions with rule-based checks for signs like repeated user messages, requests for a human, and looping tool retries. Flagged sessions go to Simulations, which replays them with a fake user against the fixed agent before release. The post says teams no longer need to pay an LLM to grade every conversation or write test cases by hand.

    Video from @rohanpaul_ai's post
  2. 🚨 AI News | TestingCatalogX
    DigitalOcean launches Signals, Simulations, and Insights for AI agents

    AIDigitalOcean launches Signals, Simulations, and Insights in public preview, and Agent Data Services in private preview. Signals reads live agent traffic and flags failed sessions, such as a successful tool call that queried the wrong region. Users can turn a flagged session into a test that Simulations replays with a proposed fix.

    Video from @testingcatalog's post
  3. elvisX
    DigitalOcean's Signals turns failed agent sessions into one-click evals

    AIDigitalOcean's Signals flags production agent sessions where users got frustrated or tools failed and converts them into evals with one click. Simulations then replay those sessions against a fix on the agent's real harness before it ships.

    Video from @omarsar0's post

Heat trend

Not enough continuous observations to show a trend yet.