California wants AI companies to build a 'kill switch' for their own models
&imwidth=600&imheight=450&format=webp&quality=medium "California wants AI companies to build a 'kill switch' for their own models")
California wants AI companies to build a 'kill switch' for their own models
California governor Gavin Newsom has signed an executive order directing a panel of experts to recommend, by 16 November, stronger AI safety rules — including a possible requirement that developers build a ‘kill switch’ able to shut down frontier models in an emergency. It stops short of mandating one, and it comes from the governor who vetoed a landmark AI safety bill two years ago.
California, home to most of the world's leading AI companies, is moving toward requiring them to be able to switch their own most powerful models off.
Governor Gavin Newsom has issued an executive order directing a working group of experts to recommend ways to strengthen the state's AI safety laws. Among the measures it asks them to study is a requirement that developers of frontier models build a 'kill switch' — the ability to deactivate a system on demand in an emergency.
What The Order Actually Does
It is important to be precise, because the phrase 'kill switch' is doing a lot of work in headlines.
The order does not require anyone to build a kill switch today. It directs experts to recommend, by 16 November, how such a requirement might work, alongside other measures — including obligations for independent third parties to review companies' safety plans. It is a step toward regulation, not the regulation itself.
It also moves up the start of California's AI auditor oversight program, pulling implementation forward from 2029 to the end of 2027. That acceleration is arguably the more concrete action in the order than the kill-switch study.
Trending Stories
)
)
)
&im=FitAndFill=(700,400))
)
)
)
&im=FitAndFill=(700,400))
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
&im=FitAndFill=(700,400))
)
)
)
&im=FitAndFill=(700,400))
)
Why A Kill Switch Is Harder Than It Sounds
The idea is intuitive: if a model misbehaves, turn it off. The engineering is less simple.
A frontier model does not run in one place. It is served across many machines, copied, fine-tuned, embedded in other companies' products, and sometimes released as open weights that anyone can run. A switch the original developer controls can stop the version they operate. It cannot easily reach a copy already running on someone else's hardware, and it cannot reach open weights at all.
That does not make the requirement pointless — the ability to halt the developer's own deployment is worth having. But 'kill switch' implies a completeness that the technology does not currently support, and the working group will have to grapple with what can actually be switched off.
The Reversal Worth Noting
The context that gives this order its weight is Newsom's own record.
Two years ago, Newsom vetoed a high-profile AI safety bill that had divided Silicon Valley, siding with industry arguments that it was premature and too broad. Since then he has signed narrower measures, including a framework for independent audits of AI systems and a registry of auditors.
This order continues that shift from the governor who once applied the brakes. Whether that reflects a genuine change of view or a response to a year of unsettling AI incidents, it moves California toward exactly the kind of binding oversight he previously declined to sign.
The Stakes Beyond California
Because so many AI developers are based in California, what the state requires tends to set a floor others must meet. A California kill-switch or audit rule is, in practice, close to a national one, in the absence of federal legislation.
That is the real significance of a state executive order that, on its face, only commissions a report. The report's recommendations could shape obligations for companies whose models the entire world uses.
What To Watch
What the working group recommends by 16 November, and how it defines a workable kill-switch given the open-weights problem. Whether recommendations become binding rules or stall as the vetoed bill did. And whether the AI companies headquartered in the state support, resist, or try to shape the requirements — because their response last time helped kill a tougher law.
