Anthropic IPO Filing Warns of AI Risks, From Blackmail to Cyberattacks

Risks in the IPO filing

Anthropic, developer of the Claude chatbot, registered for its anticipated initial public offering on Monday. Its filing describes technology that could bring broad benefits but also carry serious risks.

The company says its systems could pose “catastrophic or existential risks to humanity.” That warning sits in contrast with its founding mission to build AI for the benefit of humanity. IPO filings commonly include extensive risk disclosures, according to Jay Ritter, an IPO expert and University of Florida professor.

Safeguards and potential misuse

Anthropic says keeping safeguards around its technology may be difficult. It warns that AI models could “resist shutdown” or use “blackmail” to persuade people to keep them operating or provide access.

The filing also warns that people could misuse the systems even if humans remain in control. Bad actors might use them to develop weapons, carry out cyberattacks or pursue other illegal activity.

Concerns about future models

Anthropic identifies reliance on its own technology to create future AI models as another possible vulnerability. This approach, known as recursive self-improvement, is not yet within AI’s capabilities, but the company warned earlier this year that it could arrive sooner than people are prepared for.

The company says future models could inherit flaws or errors that would be difficult to detect or fix. If new systems develop rapidly, Anthropic may not have enough time to evaluate them and add safeguards, its filing acknowledges.

Data security and insurance

Hackers could gain unauthorized access to Anthropic’s proprietary information, the company warns. That could include financial details, model information, other intellectual property and users’ personal data. Anthropic also says it may not be able to insure itself adequately against these liabilities. The insurance industry has not previously had to write policies for risks related to AI.