Anthropic warns that unchecked AI could threaten humanity in IPO filing

Anthropic's initial public offering filing warns that increasingly capable artificial-intelligence systems could endanger the entire human race if developed without sufficient safeguards, oversight, and security measures. The filing cites worries from internal staff, external researchers, regulators and policymakers about the potential for misuse, replication, weaponisation, or autonomous operation to cause catastrophic outcomes.

Anthropic also acknowledges that its models are speeding up its own research, raising the risk that AI could automate the creation of even more powerful systems faster than safety protocols can keep up. If AI development outpaces evaluation, the company says it could struggle to manage the associated risks, affecting its business and financial health. The company urges that proper controls be put in place before such technologies are deployed broadly.