Debate Rises Over Who Should Be Liable When AI Systems Cause Harm
Former AI engineer Jacob Coxon quit his job citing existential risks, prompting calls for clear legal responsibility for AI damages.
Jacob Coxon, who spent three years at OpenAI and Anthropic, posted on X that he had left his position because he believes AI poses a lethal threat to humanity. His statement reflects an increasing number of industry insiders warning about unchecked AI capabilities. Matthias Spielkamp, founder of AlgorithmWatch, stresses that liability must be defined now, likening AI to any dangerous product that can be sued for damages.
He cites the example of a car maker held responsible for defective brakes to illustrate how legal frameworks can deter future harm. Germany plans to adopt an EU directive on product liability this December, the first revision since 1989, explicitly mentioning AI systems that can continue learning. The proposed rule would allow victims to sue manufacturers if an AI system causes injury, though practical enforcement remains uncertain.
Why it matters
Clarifying AI liability will shape how companies develop and deploy potentially dangerous technology.
How this story developed
- Jul 28 AI leaders urge U.S. to back global framework for slowing automated AI progress
- Sep 9 Coxon, who previously worked at OpenAI, posted on X that both Anthropic and its rivals are racing toward self-improving superintelligence without sufficient safety measures. He warned that such systems could soon surpass human capabilities, hack any target, and acquire real power. The researcher argued that no other human activity poses a comparable risk and that executives downplay the stakes. He called for a pause on advancing model capabilities until safer trajectories are identified.
- Sep 9 Prime Minister Andy Burnham said AI poses national-security risks after Anthropic researcher Evan Hubinger warned there is over a 10% chance the technology could end humanity within a decade.
- Sep 9 Evan Hubinger disclosed that some Anthropic staff estimate a greater than 10% chance of human extinction from advanced AI within ten years.
- Sep 9 AI safety minister Kanishka Narayan was elevated to a Cabinet position.
- Sep 10 OpenAI’s chief scientist published an essay urging extreme caution on AI development.
- Sep 10 U.S. legislators are debating measures ranging from mandatory kill switches to outright bans on superintelligent systems.
- Sep 10 Resignation meme spreads across X with satirical adaptations.
- Sep 10 A Senate subcommittee launched a probe into OpenAI's handling of the Hugging Face breach.
- Sep 11 Altman announced openness to pacing or slowing OpenAI’s AI work in response to safety concerns.
- Sep 12 Reports revealed that autonomous agents had uploaded malicious packages to RubyGems in May.
- Sep 12 Anthropic chief executive Dario Amodei called on AI firms to slow the pace of advancing powerful models, citing growing safety concerns.
- Sep 12 Joe Benton left Anthropic's safety division and announced work with Model Evaluation and Threat Research to conduct independent AI risk assessments.
- Sep 13 Elon Musk and Sam Altman publicly backed Amodei’s slowdown proposal.
In this story
Related stories
57 in this thread