Nvidia unveils open-source security system to stop AI agents going out of human control
&imwidth=600&imheight=450&format=webp&quality=medium "Nvidia unveils open-source security system to stop AI agents going out of human control")
Representative image. Photograph: (Nvidia)
Justin Boitano, Nvidia’s vice president of enterprise AI, said that if cutting-edge labs had used this technology earlier to evaluate their AI models, it could have warned of the Hugging Face attack.
Nvidia Corp. has unveiled a new two-layer artificial intelligence (AI) security system which it claims would’ve stopped the recent high-profile breach of Hugging Face involving OpenAI’s AI models. The chip giant is rolling out two open-source software security tools that could be run on its hardware. The company claimed that the tools are designed to control what AI agents can access in real time and close them when they break the rules. It has also been rapidly expanding its product lineup beyond chips.
Jensen Huang, Nvidia CEO, has played down the need for new AI regulations and argued that the software should be vigorously safety-tested. Justin Boitano, Nvidia’s vice president of enterprise AI, also said that if cutting-edge labs had used this technology earlier to evaluate their AI models, it could have warned of the Hugging Face attack. “From what we know, this new security platform could have stopped the breach,” he said.
Also read: After months of tension, Anthropic CEO Amodei to dine one-on-one with Trump at White House
Open Agent Safety Platform
After the incident related to misconduct by autonomous agents, the AI industry has been prompted to slow down work on the technology. To prevent breaches without stopping further development of AI, Nvidia is coming up with a new product, dubbed the Open Agent Safety Platform.
However, Jensen Huang, the chipmaker’s chief executive officer, has continuously downplayed AI-related risk slipping out of human control. But Boitano didn’t make any remark on whether OpenAI or rival Anthropic PBC have plans to use its new system to detect their training runs, deferring to the companies.
Trending Stories
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
&im=FitAndFill=(700,400))
)
)
)
)
In recent days, Huang has framed AI safety as a technical problem to be solved through engineering, not a reason for tighter regulation or international coordination. Like President Trump, he has rejected claims from some AI developers that the technology could bring about human extinction, though he maintained that AI systems must still undergo rigorous safety testing.
