Skip to content
View original post on X: Rohan PaulX· 50/100AI score50/100

Pangram misses 79.8% of Meta Muse-Glimmer-rewritten scientific abstracts

AISummary

Rohan Paul reports that the AI-text detector Pangram missed 79.8% of scientific abstracts rewritten by Meta's Muse-Glimmer while flagging just 1 of 5,000 human abstracts. He cites a Tokyo Metropolitan University paper finding that Pangram caught 93.5% of GPT-5 rewrites, so its miss rate depended mostly on which LLM did the rewriting.

Post on XView on X
Rohan PaulVerified on X
@rohanpaul_ai

Pangram, the AI-text detector, missed 79.8% of scientific abstracts rewritten by Meta's Muse-Glimmer, while flagging just 1 of 5,000 human abstracts.

In a new paper from Tokyo Metropolitan University, reseaerchers find the share of AI-rewritten abstracts that Pangram misses depends strongly on the LLM version

Shows that it caught 93.5% of GPT-5 rewrites but missed 79.8% from another new model. Its miss rate depended mostly on which model did the rewriting.

Source: Rohan Paul · x.comPublished · added here

Topics

Meta / LlamaSafety & alignmentPapers & researchEvals & benchmarks