cool i guess. but “how?” -> they trained it to. people should do more investigations of how this in the wild
OpenAI's Will Depue says people should investigate how models learn activation detection
AISummary
Will Depue, an OpenAI researcher, says the finding that models can learn to detect when their activations are read is "cool," but asks how it happens. He answers that the models were trained to do it, and calls for more investigation of this behavior in the wild.
Post on XView on X
will depueVerified on X
@willdepue
Language models can learn to detect when you read their activations. How? 🧵
Source: will depue · x.comPublished · added here