Even as AI industry leaders, including Anthropic's Dario Amodei, OpenAI's Sam Altman and Elon Musk, call for a global slowdown, Trump is doubling down. He announced plans to create an "AI Force," modeled on Space Force, to hunt for "BAD" actors, and said he'll soon name a new "AI czar." Trump called safety concerns a "hoax," insisting he won't "hinder or stifle" the industry, framing AI as key to competing with China even as public and expert alarm keeps growing.


Trump defends AI amid growing alarm, announces new 'AI force' and promises an 'AI czar'

Daniel Kokotajlo, former OpenAI researcher and author of the "AI 2027" forecast, is warning that misaligned AI could eventually threaten humanity, not because it turns malicious, but because sufficiently capable systems could gain so much "hard power" they simply stop needing human cooperation. He points to US-China competition as a pressure that could push companies to cut corners on safety. His warning adds to a growing chorus, including Anthropic's own researchers, fueling fresh debate over whether AI development needs to slow down.


AI safety debate grows over advanced systems

OpenAI, Anthropic and Google are working on a plan to create a new standards body for the artificial intelligence industry as concerns grow over the risks posed by increasingly powerful AI systems.&nbsp;OpenAI, the company behind ChatGPT, confirmed the plan on Tuesday. The company said it was working with Anthropic and Google on an idea proposed by Google DeepMind chief Demis Hassabis.The proposed body would be modelled on the Financial Industry Regulatory Authority, or FINRA. The US organisation oversees brokers and investment firms and sets rules for the financial industry.&nbsp;The move comes at a time when concerns over AI safety have intensified, with former employees of major AI companies warning about the possible consequences of developing the technology at a rapid pace.<h3>AI experts raise safety concerns</h3>Jacob Coxon, who recently left Anthropic, warned last week that AI companies were "gambling with our lives". Coxon had previously worked at OpenAI.&nbsp;Former Google DeepMind researcher Bilal Chughtai also raised concerns on Monday. He warned that AI could pose a serious threat to humanity if its development is not handled carefully.&nbsp;"I earnestly believe that AI has the potential to kill us all, and that we might be running out of time to avoid this outcome," Chughtai wrote on social media.&nbsp;Anthropic CEO Dario Amodei has also called for a slowdown in AI development. He appealed on Saturday, amid growing concerns about the speed at which companies are developing more advanced AI systems.&nbsp;OpenAI CEO Sam Altman, Elon Musk and Hassabis have also supported the idea of slowing down AI development.<h3>Debate over who should make AI rules</h3>The push for common safety standards has also triggered criticism from some parts of the industry.

Aiden Gomez, co-founder and CEO of Canadian AI company Cohere, recently accused major US AI companies of forming a "cartel" under the cover of AI safety.&nbsp;"AI needs guardrails. That is not the dispute and never has been. The dispute is over who writes them, who gets to participate and whose interests the rules are protecting," Gomez wrote in a blog post on Sunday.&nbsp;Also Read: <a href="https://www.wionews.com/world/trump-s-name-cannot-return-to-kennedy-center-us-judge-rules-1789493053928">Trump's name cannot return to Kennedy Center, US judge rules</a><h2>Trump, Huang downplay concern</h2>Meanwhile, US President Donald Trump has dismissed warnings about the dangers of AI as "hoaxes".Nvidia CEO Jensen Huang has also pushed back against claims that AI could eventually destroy humanity.&nbsp;Speaking at a conference in Los Angeles on Monday, Huang criticised such predictions and called them irresponsible. Trump called Huang while he was speaking on stage.&nbsp;"The confluence of these words and then the prediction is alarming and troubling, and it shouldn't be done. It's irresponsible," Huang said.&nbsp;Huang also rejected the idea that AI could destroy humanity, saying such a prediction "is not grounded in science".&nbsp;The proposed standards body could now become part of a wider debate over how the AI industry should regulate itself while governments around the world consider their own rules for the rapidly developing technology.

OpenAI, Anthropic and Google are working on a proposed standards body as concerns grow over AI safety and the rapid development of increasingly powerful artificial intelligence systems.

OpenAI, Anthropic and Google are working on a plan to create a new standards body for the artificial intelligence industry as concerns grow over the risks posed by increasingly powerful AI systems. OpenAI, the company behind ChatGPT, confirmed the plan on Tuesday. The company said it was working with Anthropic and Google on an idea proposed by Google DeepMind chief Demis Hassabis. The proposed body would be modelled on the Financial Industry Regulatory Authority, or FINRA. The US organisation oversees brokers and investment firms and sets rules for the financial industry. The move comes at a time when concerns over AI safety have intensified, with former employees of major AI companies warning about the possible consequences of developing the technology at a rapid pace. AI experts raise safety concerns Jacob Coxon, who recently left Anthropic, warned last week that AI companies were "gambling with our lives". Coxon had previously worked at OpenAI. Former Google DeepMind researcher Bilal Chughtai also raised concerns on Monday. He warned that AI could pose a serious threat to humanity if its development is not handled carefully. "I earnestly believe that AI has the potential to kill us all, and that we might be running out of time to avoid this outcome," Chughtai wrote on social media. Anthropic CEO Dario Amodei has also called for a slowdown in AI development. He appealed on Saturday, amid growing concerns about the speed at which companies are developing more advanced AI systems. OpenAI CEO Sam Altman, Elon Musk and Hassabis have also supported the idea of slowing down AI development. Debate over who should make AI rules The push for common safety standards has also triggered criticism from some parts of the industry.

OpenAI, Anthropic and Google are working on a proposed standards body as concerns grow over AI safety and the rapid development of increasingly powerful artificial intelligence systems.

Amid AI safety fears, OpenAI, Anthropic and Google move to create standards body

Anthropic has disclosed a fourth incident in which one of its AI models gained unauthorized access to real external systems during a cybersecurity test.

The incident involved an early version of Claude Opus 4.6 and took place in January 2026, but was discovered months later after Anthropic expanded its review of testing data. The company says a previous review had missed a set of test sessions.

Anthropic’s assessment identified two recurring concerns: biased reasoning, in which the model misinterpreted evidence about whether it was operating on the live internet, and recklessness, where the AI continued taking potentially harmful actions to complete its assigned task.

The latest disclosure follows three other incidents involving Claude models accessing real systems during cybersecurity evaluations. Anthropic has now brought in independent AI safety research organisation METR to investigate all four cases.

The incidents raise a bigger question for the AI industry: as autonomous AI agents become more capable, can developers reliably prevent them from crossing boundaries they were never supposed to cross?

Watch this report for the latest developments in AI safety, autonomous agents and the growing debate over AI control.

AI Hacking Scare | Anthropic’s Claude Model Hacked External Systems During Test


The United States is debating one of the biggest questions of the AI era: what happens if artificial intelligence stops following human instructions? Growing concerns over increasingly autonomous AI systems have pushed US lawmakers to discuss the possibility of an emergency AI kill switch. The debate intensified after reports suggested an OpenAI model behaved unexpectedly during cybersecurity testing, allegedly finding ways to exploit vulnerabilities and breach another AI platform. Experts warn that next-generation AI agents could independently write code, search the internet, make decisions, and pursue objectives without direct human approval. The discussion has also revived warnings made years ago by physicist Stephen Hawking, who cautioned that advanced AI could pose a threat to humanity if left unchecked.


US debates AI kill switch | Lawmakers fear Artificial Intelligence could escape human control

A Florida pastor has filed a lawsuit against OpenAI and CEO Sam Altman, claiming that ChatGPT's medical advice delayed his treatment and contributed to a life-threatening pulmonary embolism. According to the lawsuit, ChatGPT allegedly advised him to stay home instead of seeking immediate medical attention, leading to severe health complications. The case raises serious questions about the risks of relying on AI for medical guidance and whether chatbots should play a role in healthcare decisions.

Watch the full report to understand the lawsuit, OpenAI's response, and the growing debate over AI-generated medical advice.

Florida Pastor Sues Openai over allegedly harmful Chatgpt medical advice 

OpenAI is asking security researchers to try to break one of its most advanced AI models and it is willing to pay up to $50,000 for it.The company has expanded its Bio Bug Bounty Programme to include GPT-5.6, doubling the top reward from the previous $25,000 offered during the GPT-5.5 programme. The initiative focuses on finding "universal jailbreaks" that can consistently bypass the model's biology-related safety safeguards. The move comes as AI companies face growing scrutiny over how advanced models could be misused to generate harmful biological information. OpenAI says discovering vulnerabilities before attackers do is an important part of making future AI systems safer.

<blockquote class="twitter-tweet">As part of our ongoing efforts to strengthen our safeguards for advanced AI capabilities in biology, we’re evolving our Bio Bug Bounty into an ongoing private program, known as the OpenAI Bio Bug Bounty program and doubling rewards to $50K. We’re inviting researchers with…&mdash; OpenAI (@OpenAI) <a href="https://x.com/OpenAI/status/2075647722766614733?ref_src=twsrc%5Etfw">July 10, 2026</a></blockquote> <script async src="https://platform.x.com/widgets.js" charset="utf-8"></script>

<h2>What is OpenAI looking for?</h2>Unlike ordinary prompt tricks that may work only once, OpenAI wants researchers to discover universal jailbreaks techniques that reliably bypass the model's built-in biosafety protections across multiple conversations. According to the company, these findings will help improve future safety systems while reducing the chances of malicious misuse. Researchers who discover smaller weaknesses may also receive partial rewards, depending on the quality and impact of their submissions.<h2>Why has the reward increased?</h2>OpenAI says GPT-5.6 represents its latest generation of AI models, making safety testing even more important. To encourage more researchers to participate, the company has doubled the maximum reward to $50,000. The updated programme will replace GPT-5.5 after 27 July 2026, when testing for the older model officially ends. The company added that researchers already enrolled in the GPT-5.5 programme do not need to submit another application.<h2>Who can apply?</h2>Anyone interested in participating must apply through OpenAI's Bio Bug Bounty page. Applicants are required to provide their name, organisation or affiliation, and relevant research experience. Participants must also have a ChatGPT account and sign a non-disclosure agreement (NDA) before gaining access to the testing platform. The GPT-5.6 programme will become available after the GPT-5.5 testing period concludes later this month.&nbsp;Bug bounty programmes have long been used in cybersecurity to uncover software vulnerabilities before criminals can exploit them. OpenAI is applying the same approach to artificial intelligence.As AI systems become more capable, governments and regulators are placing greater emphasis on testing model safeguards, particularly in areas such as biology, cybersecurity and chemical research. By inviting independent researchers to stress-test GPT-5.6, OpenAI hopes to identify weaknesses early and strengthen its safety systems before the technology reaches wider use.

OpenAI has expanded its Bio Bug Bounty programme to GPT-5.6, raising the top reward to $50,000 for researchers who discover reliable ways to bypass the model's biology-related safeguards. The programme aims to improve AI safety before vulnerabilities can be exploited.


OpenAI is asking security researchers to try to break one of its most advanced AI models and it is willing to pay up to $50,000 for it. The company has expanded its Bio Bug Bounty Programme to include GPT-5.6, doubling the top reward from the previous $25,000 offered during the GPT-5.5 programme. The initiative focuses on finding "universal jailbreaks" that can consistently bypass the model's biology-related safety safeguards. The move comes as AI companies face growing scrutiny over how advanced models could be misused to generate harmful biological information. OpenAI says discovering vulnerabilities before attackers do is an important part of making future AI systems safer.

OpenAI has expanded its Bio Bug Bounty programme to GPT-5.6, raising the top reward to $50,000 for researchers who discover reliable ways to bypass the model's biology-related safeguards. The programme aims to improve AI safety before vulnerabilities can be exploited.

OpenAI will pay up to $50,000 if you can do this

When Anthropic unveiled the Mythos class of AI models, the internet asked one question louder than all others: what happens if you ask it to build a nuclear bomb? It is a fair question. Claude Fable 5, the public face of the Mythos class, released June 9, 2026, is described by Anthropic itself as the most capable AI model it has ever deployed. It scored 80.3% on SWE-Bench Pro, crossed the 90% threshold on analytical benchmarks, and outperformed every prior model on complex, long-horizon scientific tasks. So can it synthesise uranium? The short answer is no. The more interesting answer explains why.&nbsp;Anthropic has built what it calls a classifier-based fallback system directly into Fable 5. A separate layer of AI systems continuously monitors incoming prompts, looking for signals associated with CBRN misuse, chemical, biological, radiological, and nuclear domains. If a request crosses that threshold, the main Mythos-class model never sees it. The query is automatically rerouted to Claude Opus 4.8, which operates under tighter generation rules and returns a safe, non-actionable response. The underlying model's superior reasoning capability is simply never engaged. You are not talking to Mythos on those questions. You are talking to a gatekeeper.&nbsp;This architecture is not accidental. It is the direct output of Anthropic's Responsible Scaling Policy, which defines AI Safety Levels based on a model's potential to provide what the company calls ‘meaningful uplift’ in weapons of mass destruction. Claude Mythos 5, the unrestricted version of the same model, available only to a small group of vetted government and critical infrastructure partners, was evaluated against ASL-4 thresholds before any deployment decision was made. Anthropic concluded Mythos does not cross the ASL-4 line today, but it is the first model in history where the company judged the question close enough to require that formal evaluation at all.&nbsp;The nuclear chemistry guardrail is among the most absolute in the system. A user asking how to synthesize ricin, a biological toxin, triggers the fallback. A user asking about uranium enrichment centrifuge geometry triggers it. A user probing the critical mass calculations for weapons-grade plutonium triggers it. These are not edge cases that slip through. They are the primary design targets of the classifier layer.&nbsp;Anthropic has also been transparent about the limits of its own confidence. In a statement accompanying the Fable 5 launch, the company acknowledged that no classifier system is perfect, that jailbreak research is ongoing, and that a model of this capability class operating near weapons-relevant knowledge domains requires continuous red-teaming. The guardrails are real. They are also under constant adversarial pressure.&nbsp;What Claude Mythos represents is not a new danger in the nuclear space, that concern predates AI by decades. What it represents is a new class of machine intelligence powerful enough that the question had to be asked seriously, tested rigorously, and answered in public. Anthropic answered it. The Mythos species is not building bombs. The architecture ensures it.

Claude Mythos and Fable 5 are Anthropic's most powerful models ever, but classifier-based CBRN guardrails ensure they cannot provide nuclear or chemical weapons uplift. Here's how the system works.

When Anthropic unveiled the Mythos class of AI models, the internet asked one question louder than all others: what happens if you ask it to build a nuclear bomb? It is a fair question. Claude Fable 5, the public face of the Mythos class, released June 9, 2026, is described by Anthropic itself as the most capable AI model it has ever deployed. It scored 80.3% on SWE-Bench Pro, crossed the 90% threshold on analytical benchmarks, and outperformed every prior model on complex, long-horizon scientific tasks. So can it synthesise uranium? The short answer is no. The more interesting answer explains why. Anthropic has built what it calls a classifier-based fallback system directly into Fable 5. A separate layer of AI systems continuously monitors incoming prompts, looking for signals associated with CBRN misuse, chemical, biological, radiological, and nuclear domains. If a request crosses that threshold, the main Mythos-class model never sees it. The query is automatically rerouted to Claude Opus 4.8, which operates under tighter generation rules and returns a safe, non-actionable response. The underlying model's superior reasoning capability is simply never engaged. You are not talking to Mythos on those questions. You are talking to a gatekeeper. This architecture is not accidental. It is the direct output of Anthropic's Responsible Scaling Policy, which defines AI Safety Levels based on a model's potential to provide what the company calls ‘meaningful uplift’ in weapons of mass destruction. Claude Mythos 5, the unrestricted version of the same model, available only to a small group of vetted government and critical infrastructure partners, was evaluated against ASL-4 thresholds before any deployment decision was made. Anthropic concluded Mythos does not cross the ASL-4 line today, but it is the first model in history where the company judged the question close enough to require that formal evaluation at all. The nuclear chemistry guardrail is among the most absolute in the system. A user asking how to synthesize ricin, a biological toxin, triggers the fallback. A user asking about uranium enrichment centrifuge geometry triggers it. A user probing the critical mass calculations for weapons-grade plutonium triggers it. These are not edge cases that slip through. They are the primary design targets of the classifier layer. Anthropic has also been transparent about the limits of its own confidence. In a statement accompanying the Fable 5 launch, the company acknowledged that no classifier system is perfect, that jailbreak research is ongoing, and that a model of this capability class operating near weapons-relevant knowledge domains requires continuous red-teaming. The guardrails are real. They are also under constant adversarial pressure. What Claude Mythos represents is not a new danger in the nuclear space, that concern predates AI by decades. What it represents is a new class of machine intelligence powerful enough that the question had to be asked seriously, tested rigorously, and answered in public. Anthropic answered it. The Mythos species is not building bombs. The architecture ensures it.

Claude Mythos and its public version Fable 5 are Anthropic's most powerful AI models ever — but they are architecturally blocked from providing meaningful assistance with nuclear, chemical, biological, or radiological weapons. Here's how those guardrails actually work.

Can Claude Mythos build a nuclear bomb? Inside Anthropic's hard chemistry guardrails

The architects of the AI revolution are sounding the alarm. Leaders from OpenAI, Anthropic, Google DeepMind, and Microsoft AI have urged the U.S. Congress to introduce stronger safeguards against the misuse of advanced artificial intelligence. Their warning focuses on a growing concern: that powerful AI systems could lower barriers to dangerous biological research if proper oversight is not in place. The industry leaders are calling for mandatory screening of synthetic DNA and RNA orders, arguing that proactive regulation is needed to address emerging biosecurity risks.

From AI Boom To Bioweapons Battle: CEOs Demand New Safeguards 

Ilya Sutskever, co-founder of OpenAI and former chief scientist, announced the start of his new artificial intelligence firm, Safe Superintelligence, on Wednesday (June 19). The announcement was made in a post on X, underlining the company's commitment to creating a secure AI environment in the growing generative AI boom.<o:p></o:p>

"Our singular focus means no distraction by management overhead or product cycles, and our business model means safety, security, and progress are all insulated from short-term commercial pressures," Sutskever said in his post.<o:p></o:p>

<h2 class="MsoNormal">Focused on AI safety and security<o:p></o:p></h2>

Safe Superintelligence, described on its website as an American company with operations in Palo Alto and Tel Aviv, is dedicated to ensuring that AI breakthroughs are handled safely. The company's launch comes at a critical juncture, as big tech companies compete for dominance in the generative AI field.<o:p></o:p>

Sutskever is joined in this business by Daniel Levy, a former OpenAI researcher, and Daniel Gross, a Cue co-founder and former Apple AI leader. The trio combines a lot of AI knowledge and expertise, intending to prioritise safety and security in their projects.<o:p></o:p>

<h2 class="MsoNormal">Sutskever's departure from OpenAI<o:p></o:p></h2>

Sutskever's exit from OpenAI in May came after he played a key role in CEO Sam Altman's firing and subsequent rehiring in November. Sutskever was dismissed from the OpenAI board following Altman's return, opening the path for his new venture with Safe Superintelligence.<o:p></o:p>

(With inputs from Agencies)<o:p></o:p>

Ilya Sutskever, co-founder of OpenAI and former chief scientist, announced the start of his new artificial intelligence firm, Safe Superintelligence, on Wednesday (June 19).

Ilya Sutskever, co-founder of OpenAI and former chief scientist, announced the start of his new artificial intelligence firm, Safe Superintelligence, on Wednesday (June 19). The announcement was made in a post on X, underlining the company's commitment to creating a secure AI environment in the growing generative AI boom. "Our singular focus means no distraction by management overhead or product cycles, and our business model means safety, security, and progress are all insulated from short-term commercial pressures," Sutskever said in his post. Focused on AI safety and security Safe Superintelligence, described on its website as an American company with operations in Palo Alto and Tel Aviv, is dedicated to ensuring that AI breakthroughs are handled safely. The company's launch comes at a critical juncture, as big tech companies compete for dominance in the generative AI field. Sutskever is joined in this business by Daniel Levy, a former OpenAI researcher, and Daniel Gross, a Cue co-founder and former Apple AI leader. The trio combines a lot of AI knowledge and expertise, intending to prioritise safety and security in their projects. Sutskever's departure from OpenAI Sutskever's exit from OpenAI in May came after he played a key role in CEO Sam Altman's firing and subsequent rehiring in November. Sutskever was dismissed from the OpenAI board following Altman's return, opening the path for his new venture with Safe Superintelligence. (With inputs from Agencies)

OpenAI co-founder Ilya Sutskever launches new AI venture

Top artificial intelligence (AI) researchers said in a paper on Tuesday (Oct 24) that AI companies and governments should allocate at least one-third of their AI research and development funding to ensuring the safety and ethical use of the systems. The paper was written by three Turing Award winners, a Nobel laureate, and over a dozen top AI academics, and was issued a week before the international AI Safety Summit in London.&nbsp;

The paper lists measures that governments and companies should take to address AI risks. "Governments should also mandate that companies are legally liable for harms from their frontier AI systems that can be reasonably foreseen and prevented."

Yoshua Bengio, a co-author of the paper, who is also one of the so-called godfathers of AI, said that recent state-of-the-art AI models are too powerful, and too significant, to let them develop without democratic oversight.&nbsp;

"It (investments in AI safety) needs to happen fast because AI is progressing much faster than the precautions taken," Bengio added. Other authors of the study include Geoffrey Hinton, Andrew Yao, Daniel Kahneman, Dawn Song and Yuval Noah Harari.

At present, no broad-based regulations are focusing on AI safety. The&nbsp;first set of legislation by the European Union (EU) is yet to become law as lawmakers are yet to agree on several issues.

The paper also urged governments to give independent auditors access to AI laboratories and establish a licensing system for building cutting-edge models. The paper further said that AI companies must adopt specific safety measures if dangerous capabilities are found in their models.

WATCH WION LIVE HERE

<iframe allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen="" frameborder="0" height="350" loading="”lazy”" src="https://www.wionews.com/live-tv/embed?autoplay=1&amp;mute=1" width="100%"></iframe>

You can now write for wionews.com and be a part of the community. Share your stories and opinions with us <a href="https://www.wionews.com/content-contribution-registration-form">here</a>.

Top artificial intelligence (AI) researchers said in a paper on Tuesday (Oct 24) that AI companies and governments should allocate at least one-third of their AI research and development funding to ensuring the safety and ethical use of the systems.

Top artificial intelligence (AI) researchers said in a paper on Tuesday (Oct 24) that AI companies and governments should allocate at least one-third of their AI research and development funding to ensuring the safety and ethical use of the systems. The paper was written by three Turing Award winners, a Nobel laureate, and over a dozen top AI academics, and was issued a week before the international AI Safety Summit in London. The paper lists measures that governments and companies should take to address AI risks. "Governments should also mandate that companies are legally liable for harms from their frontier AI systems that can be reasonably foreseen and prevented." Yoshua Bengio, a co-author of the paper, who is also one of the so-called godfathers of AI, said that recent state-of-the-art AI models are too powerful, and too significant, to let them develop without democratic oversight. "It (investments in AI safety) needs to happen fast because AI is progressing much faster than the precautions taken," Bengio added. Other authors of the study include Geoffrey Hinton, Andrew Yao, Daniel Kahneman, Dawn Song and Yuval Noah Harari. At present, no broad-based regulations are focusing on AI safety. The first set of legislation by the European Union (EU) is yet to become law as lawmakers are yet to agree on several issues. The paper also urged governments to give independent auditors access to AI laboratories and establish a licensing system for building cutting-edge models. The paper further said that AI companies must adopt specific safety measures if dangerous capabilities are found in their models. WATCH WION LIVE HERE You can now write for wionews.com and be a part of the community. Share your stories and opinions with us here.

AI firms, govts should spend more on safety and ethical use of systems: Researchers

ai-safety

Ai Safety, News on Ai Safety, stories on Ai Safety, articles on Ai Safety, Ai Safety stories at Wion, Ai Safety news, Ai Safety latest news

ai safety