By changing the prose but keeping the arguments and labels, IdeaLens detects where a person or AI came up with the idea, not just who wrote them down. Trained on Tinker, and the code+data+model are open sourced as well!
IdeaLens detects whether ideas originated from humans or AI
AISummary
IdeaLens is a detector that identifies whether the ideas in a text came from a human or an AI, rather than judging the prose alone. On mixed-provenance benchmarks, it reached 81.3% average idea-detection accuracy, versus 25.4% for ProseLens and 25.9% for Pangram 4. The model, code, and data are open-sourced, and it was trained on Tinker.
Post on XView on X
@tinkerapi
Traditional AI detectors are accurate when ideas and prose come from the same source. However, on mixed provenance benchmarks where they differ, they follow the prose and not the ideas in the writing. IdeaLens correctly detects who came up with the ideas and not who wrote the prose on these mixed benchmarks, with an average idea detection accuracy of 81.3% vs ProseLens's 25.4% and Pangram 4's 25.9%. This suggests that ideas are a strong differentiator of human and AI writing.
Source: Tinker · x.comPublished · added here