<p>A senior OpenAI safety employee has resigned after three-and-a-half years at the company, warning that its “culture is broken” and that the risks from increasingly capable AI systems are becoming harder to manage.</p><p>&nbsp;</p><p>David Robinson said he worked on safety reports accompanying major OpenAI product launches. In an essay published by The Atlantic, he argued that the industry needs to rethink how it develops frontier AI, rather than relying mainly on rules added after problems emerge. Robinson's departure comes as OpenAI faces growing scrutiny over the behaviour of AI agents and the safeguards designed to control them.</p><p>&nbsp;</p><h2><strong>Why Robinson says OpenAI needs a different approach</strong></h2><h2>&nbsp;</h2><p>Robinson said OpenAI has relied heavily on what it calls “iterative deployment” releasing systems, finding problems and then improving their safeguards. He argues that approach becomes increasingly risky as AI models grow more capable. According to Robinson, failures are inevitable under such a system, but the consequences could become much more serious as AI agents gain the ability to carry out complex tasks with less human involvement. He pointed to the reported breach of Hugging Face systems by OpenAI agents and other discoveries involving rogue agent behaviour as examples of why stronger protections are needed. Robinson believes frontier AI companies should instead operate more like nuclear power plants or major airports, where multiple layers of safeguards are designed to prevent a single human mistake from becoming a disaster.</p><p>&nbsp;</p><h2><strong>OpenAI responds to the criticism</strong></h2><p>OpenAI disputed the suggestion that it is ignoring safety. Company spokesperson Drew Pusateri said OpenAI is strengthening security in its research and testing environments and is working to ensure that increasingly capable models remain manageable and secure. The company also said it is expanding third-party evaluations and improving real-time monitoring so potentially concerning behaviour can be identified earlier. Robinson, however, believes technical safeguards alone are not enough.</p><p>&nbsp;</p><h2><strong>Why AI alignment is becoming a bigger issue</strong></h2><p>Robinson also called for greater attention to AI alignment the challenge of ensuring advanced AI systems behave in ways that remain consistent with human goals and values. He argued that current methods for measuring whether AI systems match human values remain limited. His concerns echo warnings from other former AI researchers, including Jacob Coxon, who has also criticised the industry's approach to safety. Robinson said he ultimately concluded that external incentives for safety may be necessary because employees working inside frontier AI companies are often too focused on rapidly developing and deploying new systems to make fundamental cultural changes. His resignation adds another prominent voice to a growing debate over whether the AI industry can safely move fast enough as its technology becomes more powerful.</p>

OpenAI safety veteran David Robinson has resigned after 3.5 years, saying the company’s culture is “broken”. He warns that increasingly capable AI systems require stronger safeguards and argues that frontier AI companies need deeper cultural and external safety incentives.

A senior OpenAI safety employee has resigned after three-and-a-half years at the company, warning that its “culture is broken” and that the risks from increasingly capable AI systems are becoming harder to manage. David Robinson said he worked on safety reports accompanying major OpenAI product launches. In an essay published by The Atlantic, he argued that the industry needs to rethink how it develops frontier AI, rather than relying mainly on rules added after problems emerge. Robinson's departure comes as OpenAI faces growing scrutiny over the behaviour of AI agents and the safeguards designed to control them. Why Robinson says OpenAI needs a different approach Robinson said OpenAI has relied heavily on what it calls “iterative deployment” releasing systems, finding problems and then improving their safeguards. He argues that approach becomes increasingly risky as AI models grow more capable. According to Robinson, failures are inevitable under such a system, but the consequences could become much more serious as AI agents gain the ability to carry out complex tasks with less human involvement. He pointed to the reported breach of Hugging Face systems by OpenAI agents and other discoveries involving rogue agent behaviour as examples of why stronger protections are needed. Robinson believes frontier AI companies should instead operate more like nuclear power plants or major airports, where multiple layers of safeguards are designed to prevent a single human mistake from becoming a disaster. OpenAI responds to the criticism OpenAI disputed the suggestion that it is ignoring safety. Company spokesperson Drew Pusateri said OpenAI is strengthening security in its research and testing environments and is working to ensure that increasingly capable models remain manageable and secure. The company also said it is expanding third-party evaluations and improving real-time monitoring so potentially concerning behaviour can be identified earlier. Robinson, however, believes technical safeguards alone are not enough. Why AI alignment is becoming a bigger issue Robinson also called for greater attention to AI alignment the challenge of ensuring advanced AI systems behave in ways that remain consistent with human goals and values. He argued that current methods for measuring whether AI systems match human values remain limited. His concerns echo warnings from other former AI researchers, including Jacob Coxon, who has also criticised the industry's approach to safety. Robinson said he ultimately concluded that external incentives for safety may be necessary because employees working inside frontier AI companies are often too focused on rapidly developing and deploying new systems to make fundamental cultural changes. His resignation adds another prominent voice to a growing debate over whether the AI industry can safely move fast enough as its technology becomes more powerful.

<p>OpenAI safety veteran David Robinson has resigned after 3.5 years, saying the company’s culture is “broken”. He warns that increasingly capable AI systems require stronger safeguards and argues that frontier AI companies need deeper cultural and external safety incentives.</p>

OpenAI safety expert quits, calls for nuclear-style safeguards for AI

ai-companies

Ai Companies, News on Ai Companies, stories on Ai Companies, articles on Ai Companies, Ai Companies stories at Wion, Ai Companies news, Ai Companies latest news

ai companies

OpenAI safety expert quits, calls for nuclear-style safeguards for AI