OpenAI's textGrain watermark detection fails after word edits and on short text
Overview
OpenAI's tests show its textGrain watermark detection falls sharply when text is edited or short.
In an evaluation of 400-token passages, replacing 10 percent of words with synonyms cut detection from 92 percent to 66 percent, and replacing up to 25 percent dropped it to 17 percent, according to OpenAI. The report also finds detection often fails on texts under 150 words.
The EU AI Act's Article 50 requires machine-readable marking of AI-generated outputs, but IEEE Spectrum reports its criteria lack measurable thresholds, so compliance with textGrain is uncertain. Independent researchers have not yet evaluated textGrain, and OpenAI says watermarking alone is not enough.
Written by AI from the articles below · updated Nov 5, 10:09 AM ET
Check the sources:
Article timeline
The articles in this story. Times are ET.
- IEEE Spectrum · AINewsOpenAI's textGrain watermark may struggle to meet EU AI text rules
AITests on OpenAI's textGrain watermark show detection often fails after editing or when text is under 150 words. The EU AI Act's Article 50 requires machine-readable marking of AI-generated outputs, but its criteria lack measurable thresholds, so compliance is uncertain. Independent researchers have not yet evaluated textGrain, and OpenAI says watermarking alone is not enough.
Heat Trend
Not enough continuous observations to show a trend yet.