In its IPO filing, Anthropic tells investors that people inside the company share the worry that advanced AI could pose catastrophic or existential risks to humanity.
In the same filing, it warns that its 1,300% growth rate won’t last and that it can’t promise revenue will keep growing at all.
“In addition, with the scale of harm that could result from a catastrophic AI-related incident, no single insurer, and potentially the insurance industry in the aggregate, may have sufficient capacity or expertise to underwrite insurance commensurate with such exposure.”
“Concerns have been raised both within our organization and by other researchers, regulators, policymakers, and industry participants that sufficiently advanced AI systems could, if not developed, deployed, and operated with appropriate safeguards, oversight, and security measures, pose catastrophic or existential risks to humanity, particularly if such systems were misused, replicated, weaponized, or allowed to operate with a high degree of autonomy.”
“We may be exposed to liability, litigation, and regulatory action arising from harms associated with our AI models, and our contractual protections and insurance may be insufficient, any of which could adversely affect our business, financial condition, and results of operations.”