New language added on page 29 of Anthropic’s S-1/A this morning says that internal estimates indicate they may imminently meet the RSI danger threshold of 'dramatically accelerated progress attributable to AI' - as much progress in one year as two years of the 2018–2024 baseline!
Anthropic says it may imminently meet its AI R&D danger threshold
AIAndrew Curran says a new passage on page 29 of Anthropic's S-1/A says internal estimates for future models indicate the company may imminently meet its Responsible Scaling Policy definition of dramatically accelerated progress attributable to AI. The filing says no current model has been designated as exceeding that threshold, and that Anthropic is considering updates to its policy to require human review of certain R&D decisions.
AIWhy it matters.
The filing language gives the company's own estimate of its approach to a dangerous AI R&D threshold, which matters for judging how it plans to govern future model development.
Andrew CurranVerified on X
@AndrewCurran_
Anthropic updated its S1 this morning. Notable additions re automated AI R&D and human oversight of it: "[...] our internal estimates and our expectations regarding future models, rather than our current models, indicate that we may imminently meet our Responsible Scaling Policy’s definition of 'dramatically accelerated progress attributable to AI.' We have not designated any of our current models as exceeding this threshold, and we continue to assess the underlying data. We are also evaluating the implications of meeting the AI research and development capability threshold and will release further reports in the future as we set forth in our Responsible Scaling Policy." "We are also considering additional measures, including updates to our Responsible Scaling Policy, to require human review and approval of certain decisions in our research and development processes as our models perform an increasing share of that work." "We believe our investments in safety techniques will enable us to manage these risks and responsibly realize substantially more leverage from Claude than we do today in developing AI models. However, we do not anticipate being able to safely use Claude to fully autonomously design and build successive generations of frontier models without meaningful human supervision or understanding, if and when that becomes technically possible. Accordingly, we are considering updates to our Responsible Scaling Policy to keep humans in control of frontier model development at Anthropic." https://www.sec.gov/Archives/edgar/data/2130357/000162828026068676/anthropicpbc-sx1ax1.htm
Source details
Andrew Curran · x.comPublished · added here
Original title: New language added on page 29 of Anthropic’s S-1/A this morning says that internal estimates indicate they may imminently meet the RSI da...