Anthropic AI researcher resigns, citing existential danger of superintelligent systems

Coxon, who previously worked at OpenAI, posted on X that both Anthropic and its rivals are racing toward self-improving superintelligence without sufficient safety measures. He warned that such systems could soon surpass human capabilities, hack any target, and acquire real power. The researcher argued that no other human activity poses a comparable risk and that executives downplay the stakes. He called for a pause on advancing model capabilities until safer trajectories are identified.

Jacob Coxon, an AI model-training expert who left Anthropic after a stint at OpenAI, announced his resignation on X, expressing alarm over the industry’s push toward self-improving superintelligence. He contended that Anthropic and competing firms are racing to build systems that could eventually outstrip human control, potentially endangering humanity. Coxon warned that these future systems might be capable of hacking any target, revolutionizing fields overnight, and amassing real resources.

He claimed that the danger is unique among human endeavors and that many executives mask the severity in public statements. According to him, Anthropic understands the risk, yet OpenAI continues to pursue the technology to stay ahead. He suggested that only extraordinary confidence in alternative paths could justify accelerating AI development, and he advocated for measures such as a temporary halt on capability improvements.