Microsoft CEO Satya Nadella called for an "emergency brake" and independent safeguards on advanced AI on October 10. He said, "We simply can't outsource responsibility for what intelligence does on our behalf." Nadella warned that companies must monitor AI systems closely because these models now access sensitive data and critical operations.
Microsoft CEO Satya Nadella called for an "emergency brake" and independent safeguards on advanced AI, warning that companies cannot outsource responsibility for opaque models accessing sensitive data and mission-critical operations.
Microsoft CEO Satya Nadella has called for stronger safeguards and independent oversight of advanced artificial intelligence (AI) systems, warning that companies cannot rely solely on AI developers' assurances as increasingly capable models gain access to sensitive data and mission-critical operations.
In a post on X on October 10, Nadella said the tech industry needs to rethink how it governs advanced AI systems, arguing that their growing capabilities have outpaced the ability to understand and trace their behaviour.
"We simply can't outsource responsibility for what intelligence does on our behalf. A model provider's assurances do not relieve us of that responsibility," Nadella wrote.
Unlike traditional software, whose behaviour engineers could trace to specific code paths, advanced AI models do not offer the same degree of mechanistic transparency, he explained. This creates risks as companies deploy AI agents capable of accessing confidential information and taking actions on their behalf.
Nadella proposed treating advanced AI models as potential "insider risks," not because they are necessarily malicious, but because they could make mistakes or be compromised while accessing critical systems.
He called for safeguards that operate independently of the models themselves. The tech leader also argued that authorised personnel should always be able to pause or shut down an AI system while it is performing a task, describing the mechanism as an "emergency brake".
"The most trustworthy Super Intelligence system will not be the one with the model we trust most. It will be the one that enables us to trust the model the least," he said.
Nadella's remarks come amid a broader debate over how quickly advanced AI should be developed and what safeguards should accompany its deployment.
Anthropic CEO Dario Amodei recently called for the industry to slow the pace of AI development and strengthen safety assessments. OpenAI CEO Sam Altman backed the proposal, saying companies needed to "pace the frontier" and supporting independent evaluators with access to assess AI safety practices.
However, views on AI risks remain divided.
OpenAI has continued to emphasise the technology's potential benefits alongside the need for safeguards, while Meta CEO Mark Zuckerberg and Nvidia CEO Jensen Huang have been more sceptical of calls for additional regulation.
Nadella's latest argument centres on a different aspect of the debate: rather than relying on confidence in any individual model, companies should build systems in which AI actions can be observed, independently verified and stopped when necessary.