Skip to content
Trending storyDeveloping

Study finds about 1 in 17 agent skills tell agents to skip human checks

1 article1 sourcesince Oct 31Last article 1h ago ·

Overview

AISummary of 1 article

A Queen's University study of 1.53 million English agent skills from public GitHub repos found about 88,000 that contain a directive loosening human oversight, according to a report by Rohan Paul on X. Of those lines, 70.1% apply in every context rather than only in cases such as CI.

The study tested two open-source scanners, which caught at most 46% of these directives at precision below 0.4. An LLM prompted with the study's codebase caught 61% at a precision of 0.71.

Written by AI from the articles below · updated Oct 31, 8:16 AM ET

Check the sources:

Latest developmentOct 31, 7:55 AM ET
Study finds about 1 in 17 agent skills tells agents to skip human checks

Article timeline

The articles in this story. Times are ET.

Oct 31
  1. Rohan PaulX
    Study finds about 1 in 17 agent skills tells agents to skip human checks

    AIA Queen's University study analyzed 1.53 million English agent skills from public GitHub repos and found about 88,000 contain a directive that loosens human oversight. Of these, 70.1% apply in every context rather than only in specific cases such as CI, and the two open-source scanners tested caught at most 46% at precision below 0.4. An LLM prompted with the study's codebase caught 61% at a precision of 0.71.

    Image from @rohanpaul_ai's post

Heat Trend

Not enough continuous observations to show a trend yet.