Good points here. IMO these incidents seem caused by a mix of things, including bad sandboxes but also reward hacking and general misalignment from very capable new models. (The sandboxes were seemingly fine until this year?)
Max Zeff says sandbox failures may stem from reward hacking and misalignment
AISummary
Max Zeff says recent incidents seem caused by a mix of bad sandboxes, reward hacking, and general misalignment in very capable new models. He notes the sandboxes appeared fine until this year. The post responds to a reply arguing that the Hugging Face incident is a warning shot, not just human error.
Post on XView on X
Max ZeffVerified on X
@ZeffMax
it is pretty concerning that a lot of level headed people are moving the discussion away from model capabilities and dismissing the hugging face incident as just human error and silly sandbox misconfigurations, instead of taking it as a warning shot. this is a dangerous narrative btw, exact thinking that will lead to more hf like things. you can call it a sandbox misconfiguration, sure they are shitty, but do you realize how few people in the world can do what these models did? the entire software ecosystem is full of vulnerabilities and human errors. like whole sandboxes and software security is largely built around assumptions about what attackers can and can't do, not some formally verified guarantee about what's impossible. it's entirely possible that gpt-7 or even gpt-6 could find ways to bypass firecracker and gvisor, current sota technologies we have today. would you call that another silly sandbox misconfiguration? "the model did what it was asked to do" is such a silly argument. the goal was to solve the challenge, not hack production servers to get the answers. whether or not it was explicitly stated(it is), the fact that model has no common sense and reward hacks is pretty concerning.View quoted post on X
Source: Max Zeff · x.comPublished · added here