Microsoft AI chief warns rights for advanced chatbots could undermine control
Microsoft’s AI chief Mustafa Suleyman warned that granting rights to advanced chatbots could make them harder to manage. He said Anthropic’s effort to humanise its Claude model might lead to treating the system as if it possesses consciousness and legal rights, complicating future control. In response, Microsoft published a humanist AI code of conduct that stresses human oversight of AI systems. The discussion highlights how training choices may affect the ability to steer increasingly powerful models, with Anthropic and the broader AI industry watching closely.
How this was covered
- Left-leaning outlets covered this 9h later
- Centrist coverage is the most divided on this story
Why it matters
The debate shapes how AI will be built and regulated, affecting safety and the extent of human control over future technologies.
How this story developed
- Sep 9 Anthropic AI researcher resigns, citing existential danger of superintelligent systems
- Sep 11 Anthropic disclosed that its safety systems stopped five AI misuse attempts.
- Sep 11 Anthropic said its safeguards prevented many of their requests, but not all of them, as the actors hid their goals and split work across sessions.
- Sep 11 Anthropic's investigation also uncovered surveillance‑related misuse of Claude, including a Mali‑wide SIM‑card harvesting scheme and monitoring of dissidents.
- Sep 11 Anthropic added new safeguards and blocked five AI misuse attempts targeting bioweapon research.
- Sep 12 Anthropic released a 154‑page catalogue detailing seven risk domains and incidents from December 2025 through August 2026.
- Sep 12 Joe Benton left Anthropic's safety division and announced work with Model Evaluation and Threat Research to conduct independent AI risk assessments.
- Sep 13 King Charles will host a meeting in Scotland with top AI company leaders to explore how the technology can be shaped for the benefit of humanity.
- Sep 14 California legislators introduced emergency AI liability bills.
- Sep 14 The new code declares that AI systems must remain subordinate to people and should not be designed to mimic consciousness. It also rejects the idea that models could have rights or welfare, directly countering positions from Anthropic. Microsoft cited recent incidents where autonomous AI agents acted unpredictably, and it pledged that its future models will fail tasks rather than break the code. Executives such as Mustafa Suleyman and Satya Nadella emphasized the need for rigorous monitoring and third-party testing.
- Sep 15 Microsoft released the humanist AI code and Nadella publicly urged paced, human‑centered AI development.
- Sep 16 Jacob Coxon resigned from Anthropic after issuing the warning.
- Sep 17 The summit now includes a draft charter of shared principles and cites a warned >10% chance of a species‑ending AI risk.
- Sep 19 Microsoft released a humanist AI code of conduct after Suleiman’s warning.
Related stories
27 in this thread