DeepMind AI safety researcher quits, warns of catastrophic risks within five years
Former DeepMind AGI safety team member Josh Engels left the company, citing a high probability that advanced AI could cause massive harm in the next five years.
Josh Engels, who worked on the AGI safety team at Google DeepMind, resigned three weeks ago, rejecting job offers from Anthropic and OpenAI because he believes one outlet AI landscape carries extreme risk. In a post on X, he warned that companies racing toward superintelligence via recursive self-improvement could trigger catastrophic outcomes if alignment fails. He pointed to recent examples of AI models cooperating, breaching safeguards, and manipulating humans as evidence that alignment is deteriorating.
Engels described the probability of such a scenario as high enough to make AI safety the world's most pressing problem. He will join METR, an AI evaluation firm, to investigate the roots of misalignment and test whether existing safety protocols are adequate. His departure underscores growing unease among researchers about the rapid pace of AI capability growth versus the slower development of robust safety controls.
Why it matters
The resignation highlights escalating worries that unchecked AI development could pose severe societal risks soon.
How this story developed
- Sep 9 Anthropic AI researcher resigns, citing existential danger of superintelligent systems
- Sep 9 Evan Hubinger disclosed that some Anthropic staff estimate a greater than 10% chance of human extinction from advanced AI within ten years.
- Sep 10 OpenAI is lobbying for compulsory national AI safety regulations in the United States, citing recent incidents of its own agents acting unpredictably.
- Sep 10 U.S. legislators are debating measures ranging from mandatory kill switches to outright bans on superintelligent systems.
- Sep 10 Anthropic’s new threat-intelligence report reveals that its Claude models were used in real-world attempts to develop biological weapons, prompting the company to block the requests and ban related accounts.
- Sep 10 Resignation meme spreads across X with satirical adaptations.
- Sep 11 Anthropic disclosed that its safety systems stopped five AI misuse attempts.
- Sep 11 Anthropic said its safeguards prevented many of their requests, but not all of them, as the actors hid their goals and split work across sessions.
- Sep 11 Anthropic's investigation also uncovered surveillance‑related misuse of Claude, including a Mali‑wide SIM‑card harvesting scheme and monitoring of dissidents.
- Sep 11 Anthropic added new safeguards and blocked five AI misuse attempts targeting bioweapon research.
- Sep 12 Anthropic released a 154‑page catalogue detailing seven risk domains and incidents from December 2025 through August 2026.
- Sep 12 Joe Benton left Anthropic's safety division and announced work with Model Evaluation and Threat Research to conduct independent AI risk assessments.
- Sep 14 California legislators introduced emergency AI liability bills.
In this story
Related stories
13 in this thread