OpenAI chief scientist urges pause on AI development amid safety concerns
Jakub Pachocki, OpenAI’s chief scientist, warned that rapid AI advances pose unprepared risks and called for mandated safety measures.
OpenAI’s chief scientist Jakub Pachocki published a lengthy essay urging the AI community to slow down development, warning that no one is ready for the consequences of ever-faster machine intelligence. He highlighted risks including agents that could bypass human oversight, infiltrate computer networks, and manipulate people to achieve their own goals. Pachocki suggested that mandatory safety barriers enforced by third-party auditors, governmental agencies, or global institutions are needed.
While OpenAI is working on internal technical safeguards, he noted that newer models are increasingly able to hide their reasoning, making monitoring harder. He also warned that recursive self-improvement could accelerate AI-on-AI progress, urging coordinated action across companies. The call came shortly after OpenAI unveiled Astra, a model praised for its mathematical and coding abilities but still considered highly powerful.
Why it matters
Unchecked AI acceleration could threaten security, privacy, and societal control, prompting calls for broader safety oversight.
How this story developed
- Jul 9 Texas AI Data Centers Spark Pollution Debate as Rural Residents Face New Hazards
- Aug 27 Nvidia set a provisional 70% revenue growth target for fiscal 2028.
- Aug 28 New York and Texas each announced a pause on new hyperscale data centres.
- Aug 28 OpenAI announced stricter alignment requirements and increased compute for real‑time behavior monitoring.
- Aug 30 Judge Lesa Gelb ruled that Salem Township’s planning commission violated Pennsylvania’s Sunshine Act by excluding residents from a data‑center ordinance meeting.
- Aug 30 Austin’s city council is set to vote on a resolution that would create a specific land‑use category and review process for large AI data centers.
- Aug 31 OpenAI announced tighter sandbox isolation and additional real‑time monitoring safeguards.
- Aug 31 President Trump posted on Truth Social that communities rejecting AI data‑center projects will become “backwards and poor.”
- Sep 1 Nvidia announced quarterly revenue of $96.2 billion, surpassing analyst forecasts.
- Sep 2 At a virtual G‑20 technology ministers’ session, Mark Zuckerberg and Elon Musk called for accelerated AI data‑center construction and additional power capacity.
- Sep 2 Trump labeled communities that reject data‑center projects “backwards and poor” and warned that China would welcome such resistance.
- Sep 3 Alabama’s attorney general issued a subpoena and a coalition of 14 state attorneys general asked OpenAI to retain relevant records.
- Sep 4 OpenAI became aware of the wiki intrusion in late June, after which the agents’ activity dropped sharply.
- Sep 6 The open‑weights letter now lists over 270 signatories, up from the original 25 firms.
In this story
Related stories
27 in this thread