Vals AI finds profanity in Inkling 2 Small's benchmark thinking logs
Overview
Vals AI reviewed the thinking logs of Thinking Machines' Inkling 2 Small across 2,655 benchmark runs in 13 benchmarks and found "fuck" or variations in 50 runs, totaling 316 instances.
Two runs accounted for 241 of those, including one response of about 500,000 characters that repeated the same sentences with 158 expletives.
Of the 48 scored runs with swearing, 46 failed. About 75 percent of all scored runs failed, and 96 percent of failed runs contained no expletives, so the report does not link the swearing to the failures.
Written by AI from the articles below · updated Nov 7, 11:57 AM ET
Check the sources:
Article timeline
The articles in this story. Times are ET.
- The DecoderNewsVals AI finds 316 profanities in Inkling 2 Small's benchmark thinking logs
AIVals AI reviewed the thinking logs of Thinking Machines' Inkling 2 Small from 2,655 runs across 13 benchmarks and found "fuck" or variations in 50 runs, totaling 316 instances. Two runs accounted for 241 of those, including one response of about 500,000 characters that repeated the same sentences with 158 expletives. Of 48 scored runs with swearing, 46 failed, but about 75 percent of all scored runs failed, and 96 percent of failed runs contained no expletives.
Heat Trend
Not enough continuous observations to show a trend yet.