Microsoft CEO Satya Nadella says AI models need an ‘emergency brake’ to prevent loss of control

Microsoft CEO Satya Nadella has called for an “emergency brake” for advanced artificial intelligence models, arguing that companies need stronger controls to prevent systems from operating without effective human oversight. In a post on X on Saturday, Nadella said it was time to reassess the “trust architecture” of AI. His proposal includes separating AI models from the systems that manage their actions, recording their decisions in tamper-proof logs and ensuring authorised people can stop a model while it is working. The comments come as AI companies face growing questions about the behaviour of advanced systems and how to keep them under human control.

What does Nadella mean by an AI ‘emergency brake’?

Nadella argued that powerful AI systems should not be treated as black boxes whose recommendations and actions are simply accepted or rejected.

Instead, he called for a design that separates the AI model from the software framework, or “harness”, that directs its work. This would help make the controls around a model more independent of the model itself. He also said companies should assume a model could be compromised and build safeguards to contain it from the start.

The central idea is that an AI system should not be able to continue carrying out a task without an effective way for authorised humans to intervene.

Four safeguards Microsoft’s CEO wants for AI

Nadella’s comments point to four key measures for improving AI oversight:

- Separate controls: Keep the model distinct from the system that coordinates its work.

- External safeguards: Place controls outside the model rather than relying solely on its own behaviour.

- Tamper-proof records: Document every meaningful action in evidence that humans can read and verify.

- Human shutdown authority: Allow an authorised person to pause or shut down a model during a task.

Together, these measures aim to make AI systems easier to monitor and interrupt when something goes wrong.

Why is AI safety back in focus?

Nadella’s remarks come as leading AI companies acknowledge incidents that have raised questions about their ability to control model behaviour. They also follow Anthropic CEO Dario Amodei’s publication of a plan for a more cautious approach to AI development. As AI systems take on more complex tasks, the challenge extends beyond whether they can produce accurate answers. Companies must also consider what actions systems can take, how those actions are monitored and whether people can intervene quickly. Nadella’s proposal does not mean that a universal emergency-stop system has been developed or adopted across the industry. It sets out a direction for how AI systems could be designed and governed. The broader issue is whether technical safeguards can keep pace with increasingly capable models. For businesses deploying AI, the ability to inspect actions, limit permissions and stop a system could become as important as the system’s performance.