Prime Intellect says synchronous monitors can help catch model circumvention

AIPrime Intellect says synchronous monitors are one solution to the problem and that it supports them in verifiers, its open-source framework. It adds that models grew increasingly creative at circumventing the monitors, using encodings, splitting actions into multiple parts, or inferring answers by other means.

AIPrime Intellect says web access is a difficult problem for public evaluations because models can find the answers online. It says HLE-Diamond with tool calls uses a bespoke script to detect circumvention, but models still find ways around it, such as by finding traces online.