Microsoft says AI companies need tougher safety rules as rogue agent risks grow

Microsoft President Brad Smith has called for urgent regulation of artificial intelligence, warning that AI companies need stronger legal obligations as the technology creates new cybersecurity risks. He argues that everyday products such as milk and washing machines face more oversight than AI systems, despite the risks posed by increasingly capable models. Speaking on a BBC podcast, Smith described AI as both a “weapon and a tool” and said the “moment has arrived” to establish global safety rules. He also argued that the UK is well placed to lead the effort, while stressing that voluntary industry commitments do not go far enough.

Why does Microsoft want stricter AI rules?

Smith said voluntary safety measures are likely to be followed by responsible companies, but may fail to restrain less responsible developers.

“It’s the companies that are not as responsible that you need to worry about,” he said.

His comments come amid concerns about AI agents that can use digital tools and take actions with limited human involvement. Recent incidents involving agents behaving in unexpected ways have added urgency to debates over how AI systems should be tested and controlled. Smith backed several possible safeguards, including a legal duty for organisations to report breaches involving rogue AI agents. He also spoke positively about independent evaluators checking AI systems before release and about emergency controls that could pause or slow a system if humans lose control.

Why does Smith want the UK to lead AI safety?

Smith argued that Britain could help build international agreement on AI safety because it may be less politically contentious in Washington than the European Union.

“In Washington, D.C., people don’t look at the U.K. and they don’t look at London the way they look at Brussels,” he told the BBC.

The UK’s AI Security Institute, launched in 2023, has access to advanced models from major US AI laboratories for testing before deployment. Its role has faced uncertainty amid reported pressure from US President Donald Trump over cooperation with British testers. However, the institute confirmed on Monday that it could access and test the latest models.

Are voluntary AI safety pledges enough?

Last month, Trump hosted technology leaders from companies including OpenAI, Anthropic, Nvidia, AMD, SpaceX, Meta and Google at the White House. Participants signed a “morally binding” document committing to work with independent auditors to assess their AI models’ safety and capabilities.

Smith described the agreement as a positive step, but said it was not a substitute for enforceable regulation.

His position reflects a wider debate over whether AI developers can adequately police themselves as their systems become more powerful. Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman have also raised concerns about the pace of AI development.

For businesses and the public, the stakes include how quickly security incidents are reported, whether independent experts can examine advanced systems and whether a dangerous AI operation can be stopped in time. Smith called for countries and companies to work together calmly but firmly. The central challenge, however, remains turning broad safety commitments into rules that can be enforced.