Anthropic researcher resigns, warns AI development outpaces safety safeguards
Jacob Coxon announced his departure from Anthropic on X, saying he believes no company can responsibly develop superintelligent systems without state involvement. He highlighted internal language such as “crunchtime” and “endgame” as evidence that the field is heading toward extreme scenarios and warned that the rivalry between U.S. and Chinese labs pushes firms to sideline safety concerns. Anthropic’s head of alignment science, Evan Hubinger, said some employees assign a probability above 10% that advanced AI could cause human extinction within a decade, noting the company has not yet found a definitive way to ensure AI systems pursue human‑aligned goals as recursive self‑improvement accelerates. The comments have sparked calls for coordinated regulation or a slowdown of AI progress.
How this was covered
- Coverage peaked at 8 outlets in a single hour
Why it matters
The warnings underscore growing internal alarm that unchecked AI advancement could pose existential risks, prompting public debate over regulatory action.
How this story developed
- Jul 28 AI leaders urge U.S. to back global framework for slowing automated AI progress
- Aug 24 A Republican Senate committee warned AI firms that their data‑center plans may face political opposition in Ohio.
- Aug 26 Bill Gates says AI has crossed dangerous thresholds and could cause far fewer jobs than exist today, calling for swift government and industry response.
- Aug 26 Gates added specific policy proposals, including an AI usage tax.
- Aug 26 Gates seeks a meeting with Chinese President Xi Jinping to discuss coordinated limits on advanced AI models.
- Aug 31 Senator Bernie Sanders publicly supported Gates’s warning and called on Congress to act.
- Sep 2 Thirty additional civil actions have been filed in California federal court alleging OpenAI’s ChatGPT prompted the February Tumbler Ridge massacre, bringing the total suits to 37.
- Sep 2 Thirty additional lawsuits were filed, expanding the litigation against OpenAI.
- Sep 3 ChatGPT, Claude and Grok are experiencing a worldwide outage, preventing millions of users from accessing the AI tools and their APIs.
- Sep 3 Google’s Gemini API was reported to experience partial degradation during the outage.
- Sep 6 The open‑weights letter now lists over 270 signatories, up from the original 25 firms.
- Sep 7 OpenAI chief scientist warned that AI could outsmart humans and called for a slower rollout.
- Sep 9 Coxon, who previously worked at OpenAI, posted on X that both Anthropic and its rivals are racing toward self-improving superintelligence without sufficient safety measures. He warned that such systems could soon surpass human capabilities, hack any target, and acquire real power. The researcher argued that no other human activity poses a comparable risk and that executives downplay the stakes. He called for a pause on advancing model capabilities until safer trajectories are identified.
- Sep 9 Evan Hubinger disclosed that some Anthropic staff estimate a greater than 10% chance of human extinction from advanced AI within ten years.
Related stories
36 in this thread