AI safety talent shortage fuels ex-tech exits and nonprofit surge
Former DeepMind safety researcher Rishub Jain left to launch Sampura Research, citing limited focus on AI alignment and the lure of greater impact outside big tech.
Rishub Jain, once part of a small safety team at Google DeepMind, resigned to co-found the nonprofit Sampura Research, seeking autonomy to pursue a specific AI alignment problem he felt was deprioritized. His move mirrors recent exits from OpenAI and Anthropic, where researchers have warned that current development speeds could jeopardize humanity. Demand for AI safety roles has surged, with LinkedIn listings up 91% in a year, yet compensation at startups remains far below offers from Meta, OpenAI or Anthropic.
Nonprofits like Kairos, backed by Coefficient Giving, are launching residency programs and raising tens of millions to train newcomers from diverse backgrounds. Prominent donors, including Jaan Tallinn and the Project Tailwind arm of Coefficient Giving, are allocating hundreds of millions toward safety research, emphasizing the urgency of building a broader talent pool beyond traditional AI engineers.
Why it matters
A shrinking pool of AI safety experts could leave powerful systems unchecked, raising existential risks.
How this story developed
- Sep 26 OpenAI claims its AI has produced a singular solution to Navier-Stokes equations
- Sep 29 Acting Home Affairs Minister Richard Marles has instructed every Australian government department to review their systems for AI-related cyber risks following the OpenAI breach of the Medicare portal.
- Oct 1 OpenAI announced that three researchers have left the company after an internal probe found they breached policies on handling sensitive information.
- Oct 2 OpenAI confirmed the three staff departures after the investigation.
- Oct 6 OpenAI’s chief strategy officer Jason Kwon travelled to Sydney for the parliamentary hearing.
- Oct 6 Personal AI agents such as Meta's Muse are being blocked by many retail and service sites, hindering tasks like shopping and reservations.
- Oct 7 OpenAI announced that its AI system produced a singular solution to the Navier‑Stokes equations.
- Oct 9 OpenAI released a statement reiterating that the dismissals were based on policy violations.
- Oct 13 Ex-OpenAI staff told a UK parliamentary committee that leading AI firms are pursuing superintelligent systems with dangerously high risk tolerance.
- Oct 15 OpenAI showcased its Dots agent in a real‑world couch‑shopping test, revealing mixed results.
- Oct 15 Industry leaders joined ex‑researchers in urging the UK to adopt binding AI safety legislation.
In this story
Related stories
23 in this thread