OpenAI’s Chief Scientist Warns That AI Safety Is Falling Behind

Author: Reinout te Brake
Published: 2026-09-07T16:10:43+02:00
Updated: 2026-10-17T16:07:22Z
Canonical: https://reinouttebrake.com/stories/agi-is-here-says-nvidia-nobody-is-ready-says-openais-chief-scientist
Source: https://reinouttebrakecom.wordpress.com/2026/09/07/agi-is-here-says-nvidia-nobody-is-ready-says-openais-chief-scientist/

Editorial update — 17 October 2026. This revision focuses on Pachocki’s published safety assessment. The earlier Nvidia quotation and large training-compute counts were removed because their original source was not established in this review. AGI labels remain judgments rather than a settled technical classification.

OpenAI chief scientist Jakub Pachocki warned in September 2026 that AI alignment and monitoring were not keeping pace with capabilities. His warning makes the industry’s confidence about advanced AI a question of evidence and safeguards, not simply whether someone calls a model AGI.

OpenAI launched Astra on September 3 as its “most intelligent and aligned model” yet. It can operate computers, browse the web, perform complex professional work and has become OpenAI’s first model to reach its Critical cybersecurity capability threshold.

Three days later, Pachocki published An Alien Mind (https://openai.com/index/an-alien-mind/) .

His conclusion was remarkably direct.

No AI lab, he wrote, has solved alignment and monitoring well enough to continue responsibly scaling at maximum speed for much longer.

He expects voluntary slowdowns. He wants common safety thresholds. And he wants governments to coordinate internationally (https://reinouttebrake.com/stories/ai-was-supposed-to-slow-down-now-theyre-being-sued-for-it) before the technology moves much further.

There is another detail that deserves attention.

One of OpenAI’s most important ways of checking what reasoning models are doing is chain-of-thought monitoring.

Pachocki says its reliability is progressively diminishing.

The models are getting better at reasoning about and manipulating their own reasoning process. They are also becoming more capable without verbalising that reasoning at all.

The label AGI does not resolve the operational question: what safeguards and evidence support deploying a more capable system?

Look at the incentives and the signals.

The company building the model is selling its most capable AI ever.

And that company’s own chief scientist is simultaneously warning that our ability to control and monitor increasingly capable systems is not keeping pace.

For businesses deploying AI agents, and consumers increasingly allowing them to act on their behalf, that is not an academic disagreement.

It is a trust question.

What evidence should customers require before trusting increasingly capable AI agents with consequential work?