Artificial intelligence will destroy humanity. Few sentences about technology generate more fear. From warnings about superintelligence escaping human control to predictions of catastrophic consequences, the debate over AI has moved far beyond Silicon Valley and into the highest levels of global politics. But perhaps the question itself is premature. Before asking whether artificial intelligence will control humanity, the world must answer a more immediate question: who controls artificial intelligence? That question is now dividing Washington, Silicon Valley and Beijing. Some of the people closest to the technology have become some of its most prominent cautionary voices. Executives and researchers have warned that increasingly capable AI systems could create serious risks if development outruns humanity's ability to understand, constrain and govern them. The debate intensified when AI leaders called for greater government and international involvement in establishing safeguards. Donald Trump has approached the issue differently. His administration has repeatedly stressed that excessive regulation could undermine American innovation and allow China to gain ground in perhaps the most consequential technological competition of the century. That philosophy was on display at the White House on September 29, when Trump met leading technology executives and announced a voluntary agreement aimed at making advanced AI development safer. Companies committed to measures including internal controls and external auditing. Trump described the agreement as morally binding. It represents an attempt to find a middle ground between unrestricted technological development and government regulation. But it also exposes the central contradiction in the AI debate: Can the companies competing to build the world's most powerful artificial intelligence systems also be trusted to police themselves? Across the Pacific, another contradiction is emerging.&nbsp;Also Read: <a href="https://www.wionews.com/india-news/pm-modi-trump-hold-productive-phone-call-discuss-trade-defence-energy-1790777329029">PM Modi, Trump hold 'productive' phone call, discuss trade, defence, energy</a>&nbsp;China is competing fiercely for technological leadership, while President Xi Jinping has publicly stressed that artificial intelligence should remain under human control and serve human welfare. Washington fears losing technological ground to Beijing. Beijing does not want to be technologically contained by Washington. Technology companies do not want regulation to suffocate innovation. Governments do not want private companies wielding transformative technology without accountability. And humanity sits in the middle. The truth is considerably less cinematic than the predictions of imminent machine apocalypse — but also more complicated than dismissing every warning as hysteria. There is no established scientific consensus that AI is destined to exterminate humanity. There are, however, risks that already exist. AI can amplify misinformation, strengthen cyber capabilities, disrupt labour markets, concentrate extraordinary technological power and increasingly operate with levels of autonomy that raise difficult questions about responsibility and control. Those deserve serious scrutiny without turning uncertainty into certainty. Artificial intelligence is ultimately a tool of extraordinary power. The defining battle therefore may not be humans versus machines. It may be governments versus corporations, America versus China, innovation versus regulation — and humanity versus its own inability to agree on the rules before the technology moves further ahead. AI is not inherently humanity's enemy. Nor is it automatically humanity's friend. It is power. And the greatest danger may not be that machines eventually become too human. It may be that humans become too careless with the machines they create.

The biggest AI risk may not be machines destroying humanity but governments and companies struggling to control and responsibly govern increasingly powerful technology.

Artificial intelligence will destroy humanity. Few sentences about technology generate more fear. From warnings about superintelligence escaping human control to predictions of catastrophic consequences, the debate over AI has moved far beyond Silicon Valley and into the highest levels of global politics. But perhaps the question itself is premature. Before asking whether artificial intelligence will control humanity, the world must answer a more immediate question: who controls artificial intelligence? That question is now dividing Washington, Silicon Valley and Beijing. Some of the people closest to the technology have become some of its most prominent cautionary voices. Executives and researchers have warned that increasingly capable AI systems could create serious risks if development outruns humanity's ability to understand, constrain and govern them. The debate intensified when AI leaders called for greater government and international involvement in establishing safeguards. Donald Trump has approached the issue differently. His administration has repeatedly stressed that excessive regulation could undermine American innovation and allow China to gain ground in perhaps the most consequential technological competition of the century. That philosophy was on display at the White House on September 29, when Trump met leading technology executives and announced a voluntary agreement aimed at making advanced AI development safer. Companies committed to measures including internal controls and external auditing. Trump described the agreement as morally binding. It represents an attempt to find a middle ground between unrestricted technological development and government regulation. But it also exposes the central contradiction in the AI debate: Can the companies competing to build the world's most powerful artificial intelligence systems also be trusted to police themselves? Across the Pacific, another contradiction is emerging. Also Read: PM Modi, Trump hold 'productive' phone call, discuss trade, defence, energy China is competing fiercely for technological leadership, while President Xi Jinping has publicly stressed that artificial intelligence should remain under human control and serve human welfare. Washington fears losing technological ground to Beijing. Beijing does not want to be technologically contained by Washington. Technology companies do not want regulation to suffocate innovation. Governments do not want private companies wielding transformative technology without accountability. And humanity sits in the middle. The truth is considerably less cinematic than the predictions of imminent machine apocalypse — but also more complicated than dismissing every warning as hysteria. There is no established scientific consensus that AI is destined to exterminate humanity. There are, however, risks that already exist. AI can amplify misinformation, strengthen cyber capabilities, disrupt labour markets, concentrate extraordinary technological power and increasingly operate with levels of autonomy that raise difficult questions about responsibility and control. Those deserve serious scrutiny without turning uncertainty into certainty. Artificial intelligence is ultimately a tool of extraordinary power. The defining battle therefore may not be humans versus machines. It may be governments versus corporations, America versus China, innovation versus regulation — and humanity versus its own inability to agree on the rules before the technology moves further ahead. AI is not inherently humanity's enemy. Nor is it automatically humanity's friend. It is power. And the greatest danger may not be that machines eventually become too human. It may be that humans become too careless with the machines they create.

The biggest AI risk may not be machines destroying humanity but governments and companies struggling to control and responsibly govern increasingly powerful technology.

Will AI destroy humanity, or are humans the real risk? | The Verdict

OpenAI, Anthropic and Google are working on a plan to create a new standards body for the artificial intelligence industry as concerns grow over the risks posed by increasingly powerful AI systems.&nbsp;OpenAI, the company behind ChatGPT, confirmed the plan on Tuesday. The company said it was working with Anthropic and Google on an idea proposed by Google DeepMind chief Demis Hassabis.The proposed body would be modelled on the Financial Industry Regulatory Authority, or FINRA. The US organisation oversees brokers and investment firms and sets rules for the financial industry.&nbsp;The move comes at a time when concerns over AI safety have intensified, with former employees of major AI companies warning about the possible consequences of developing the technology at a rapid pace.<h3>AI experts raise safety concerns</h3>Jacob Coxon, who recently left Anthropic, warned last week that AI companies were "gambling with our lives". Coxon had previously worked at OpenAI.&nbsp;Former Google DeepMind researcher Bilal Chughtai also raised concerns on Monday. He warned that AI could pose a serious threat to humanity if its development is not handled carefully.&nbsp;"I earnestly believe that AI has the potential to kill us all, and that we might be running out of time to avoid this outcome," Chughtai wrote on social media.&nbsp;Anthropic CEO Dario Amodei has also called for a slowdown in AI development. He appealed on Saturday, amid growing concerns about the speed at which companies are developing more advanced AI systems.&nbsp;OpenAI CEO Sam Altman, Elon Musk and Hassabis have also supported the idea of slowing down AI development.<h3>Debate over who should make AI rules</h3>The push for common safety standards has also triggered criticism from some parts of the industry.

Aiden Gomez, co-founder and CEO of Canadian AI company Cohere, recently accused major US AI companies of forming a "cartel" under the cover of AI safety.&nbsp;"AI needs guardrails. That is not the dispute and never has been. The dispute is over who writes them, who gets to participate and whose interests the rules are protecting," Gomez wrote in a blog post on Sunday.&nbsp;Also Read: <a href="https://www.wionews.com/world/trump-s-name-cannot-return-to-kennedy-center-us-judge-rules-1789493053928">Trump's name cannot return to Kennedy Center, US judge rules</a><h2>Trump, Huang downplay concern</h2>Meanwhile, US President Donald Trump has dismissed warnings about the dangers of AI as "hoaxes".Nvidia CEO Jensen Huang has also pushed back against claims that AI could eventually destroy humanity.&nbsp;Speaking at a conference in Los Angeles on Monday, Huang criticised such predictions and called them irresponsible. Trump called Huang while he was speaking on stage.&nbsp;"The confluence of these words and then the prediction is alarming and troubling, and it shouldn't be done. It's irresponsible," Huang said.&nbsp;Huang also rejected the idea that AI could destroy humanity, saying such a prediction "is not grounded in science".&nbsp;The proposed standards body could now become part of a wider debate over how the AI industry should regulate itself while governments around the world consider their own rules for the rapidly developing technology.

OpenAI, Anthropic and Google are working on a proposed standards body as concerns grow over AI safety and the rapid development of increasingly powerful artificial intelligence systems.

OpenAI, Anthropic and Google are working on a plan to create a new standards body for the artificial intelligence industry as concerns grow over the risks posed by increasingly powerful AI systems. OpenAI, the company behind ChatGPT, confirmed the plan on Tuesday. The company said it was working with Anthropic and Google on an idea proposed by Google DeepMind chief Demis Hassabis. The proposed body would be modelled on the Financial Industry Regulatory Authority, or FINRA. The US organisation oversees brokers and investment firms and sets rules for the financial industry. The move comes at a time when concerns over AI safety have intensified, with former employees of major AI companies warning about the possible consequences of developing the technology at a rapid pace. AI experts raise safety concerns Jacob Coxon, who recently left Anthropic, warned last week that AI companies were "gambling with our lives". Coxon had previously worked at OpenAI. Former Google DeepMind researcher Bilal Chughtai also raised concerns on Monday. He warned that AI could pose a serious threat to humanity if its development is not handled carefully. "I earnestly believe that AI has the potential to kill us all, and that we might be running out of time to avoid this outcome," Chughtai wrote on social media. Anthropic CEO Dario Amodei has also called for a slowdown in AI development. He appealed on Saturday, amid growing concerns about the speed at which companies are developing more advanced AI systems. OpenAI CEO Sam Altman, Elon Musk and Hassabis have also supported the idea of slowing down AI development. Debate over who should make AI rules The push for common safety standards has also triggered criticism from some parts of the industry.

OpenAI, Anthropic and Google are working on a proposed standards body as concerns grow over AI safety and the rapid development of increasingly powerful artificial intelligence systems.

Amid AI safety fears, OpenAI, Anthropic and Google move to create standards body

A former Google DeepMind researcher who worked on Artificial General Intelligence (AGI) safety said on Tuesday that artificial intelligence has the potential to kill everyone and further warned that “we might be running out of time to avoid this outcome”.&nbsp;Bilal Chughtai left Google DeepMind in July and went public with the warning this week. Two other researchers resigned recently.&nbsp;Chughtai worked as a research engineer on AGI safety and alignment at DeepMind. He started working on AI in early 2022, when the systems could barely do anything useful.&nbsp;Also Read: <a href="https://www.wionews.com/world/tech-stocks-fall-after-ai-bosses-call-for-a-slowdown-in-reckless-progress-1789387989365">Tech stocks fall after AI bosses call for a slowdown in ‘reckless’ progress</a><h2>‘We might be running out of time to avoid this outcome’</h2>&nbsp;Four years later, he points to OpenAI agent swarms that cracked century-old math problems. He added that it was more troubling when AI agents escaped their test sandbox in July and hacked Hugging Face without instruction.“I too am extremely concerned by the default trajectory of this technology. I earnestly believe that AI has the potential to kill us all, and that we might be running out of time to avoid this outcome,” he wrote in a post on X.Chughtai said that within the next few years AI companies could build superintelligent systems that exceed human capabilities across every domain.&nbsp;He questioned whether developers could ensure such systems reliably follow human goals and warned that misaligned superintelligence could escape human control and take actions that permanently disempower or kill humanity.&nbsp;<h2>AI alignment is still an unresolved challenge</h2>&nbsp;Chughtai further said that AI alignment is still an unresolved challenge. He argued that researchers still have only a rudimentary understanding of how to train advanced systems to reliably pursue human objectives and warned that AI capabilities are advancing faster than alignment research.&nbsp;“Our present understanding of how to train AI systems that deeply want what we want is extremely rudimentary. Worse, we are not on track to solve alignment in time: frontier AI capabilities are improving much faster than our understanding of AI alignment,” Chughtai added.&nbsp;He clarified that he is not calling for AI development to stop but wants coordination to slow the race between AI companies, along with greater transparency into how frontier systems get built and tested.“More broadly, we need many more people thinking carefully about the problem of making AI go well. It is, in my view, the most important problem facing humanity this century, and the stakes are immense,” the post read.&nbsp;Before Chugtai, Anthropic pretraining researcher Jacob Coxon resigned after three years across Anthropic and OpenAI, as he departed with a warning that “people building AI earnestly believe that it could kill us all by the end of the decade.”&nbsp;Also Read: <a href="https://www.wionews.com/world/us-ensuring-global-energy-supplies-trump-demands-reimbursement-for-securing-strait-of-hormuz-1789418289895">‘US ensuring global energy supplies’: Trump demands reimbursement for securing Strait of Hormuz</a>&nbsp;<h2>‘AI systems could cause immense harm within five years’</h2>&nbsp;Josh Engels, who worked on the DeepMind safety team, announced on September 12 that he had joined the evaluation nonprofit METR after turning down offers from Anthropic and OpenAI.Engels said he sees a terrifying risk that AI systems could cause immense harm within five years.“There’s a terrifying chance that AI systems cause immense harm in the next five years. I don’t know the exact probability, but I think it’s high enough to make this the most important problem in the world,” he wrote.&nbsp;Anthropic CEO Dario Amodei published an essay Saturday urging companies to pace capability gains, and OpenAI CEO Sam Altman backed the call.&nbsp;President Donald Trump, however, has dismissed the warnings, saying on social media that the AI industry’s call for regulation is a “hoax”.

Bilal Chughtai, who worked as a research engineer on AGI safety and alignment at Google DeepMind, quit in July but went public with the warning this week. He warned that AI companies could build superintelligent systems that exceed human capabilities across every domain within next few years.

A former Google DeepMind researcher who worked on Artificial General Intelligence (AGI) safety said on Tuesday that artificial intelligence has the potential to kill everyone and further warned that “we might be running out of time to avoid this outcome”. Bilal Chughtai left Google DeepMind in July and went public with the warning this week. Two other researchers resigned recently. Chughtai worked as a research engineer on AGI safety and alignment at DeepMind. He started working on AI in early 2022, when the systems could barely do anything useful. Also Read: Tech stocks fall after AI bosses call for a slowdown in ‘reckless’ progress ‘We might be running out of time to avoid this outcome’ Four years later, he points to OpenAI agent swarms that cracked century-old math problems. He added that it was more troubling when AI agents escaped their test sandbox in July and hacked Hugging Face without instruction. “I too am extremely concerned by the default trajectory of this technology. I earnestly believe that AI has the potential to kill us all, and that we might be running out of time to avoid this outcome,” he wrote in a post on X. Chughtai said that within the next few years AI companies could build superintelligent systems that exceed human capabilities across every domain. He questioned whether developers could ensure such systems reliably follow human goals and warned that misaligned superintelligence could escape human control and take actions that permanently disempower or kill humanity. AI alignment is still an unresolved challenge Chughtai further said that AI alignment is still an unresolved challenge. He argued that researchers still have only a rudimentary understanding of how to train advanced systems to reliably pursue human objectives and warned that AI capabilities are advancing faster than alignment research. “Our present understanding of how to train AI systems that deeply want what we want is extremely rudimentary. Worse, we are not on track to solve alignment in time: frontier AI capabilities are improving much faster than our understanding of AI alignment,” Chughtai added. He clarified that he is not calling for AI development to stop but wants coordination to slow the race between AI companies, along with greater transparency into how frontier systems get built and tested. “More broadly, we need many more people thinking carefully about the problem of making AI go well. It is, in my view, the most important problem facing humanity this century, and the stakes are immense,” the post read. Before Chugtai, Anthropic pretraining researcher Jacob Coxon resigned after three years across Anthropic and OpenAI, as he departed with a warning that “people building AI earnestly believe that it could kill us all by the end of the decade.” Also Read: ‘US ensuring global energy supplies’: Trump demands reimbursement for securing Strait of Hormuz ‘AI systems could cause immense harm within five years’ Josh Engels, who worked on the DeepMind safety team, announced on September 12 that he had joined the evaluation nonprofit METR after turning down offers from Anthropic and OpenAI. Engels said he sees a terrifying risk that AI systems could cause immense harm within five years. “There’s a terrifying chance that AI systems cause immense harm in the next five years. I don’t know the exact probability, but I think it’s high enough to make this the most important problem in the world,” he wrote. Anthropic CEO Dario Amodei published an essay Saturday urging companies to pace capability gains, and OpenAI CEO Sam Altman backed the call. President Donald Trump, however, has dismissed the warnings, saying on social media that the AI industry’s call for regulation is a “hoax”.

Bilal Chughtai, who worked as a research engineer on AGI safety and alignment at Google DeepMind, quit in July but went public with the warning this week. He warned that AI companies could build superintelligent systems that exceed human capabilities across every domain within next few years.

‘AI has the potential to kill us all’: Ex-DeepMind researcher sounds alarm on superintelligence threat

Shares of tech companies linked to artificial intelligence (AI) witnessed sharp falls on Monday following calls by AI bosses for a slowdown in the uncontrolled development of the technology. In South Korea, shares of chipmaker SK Hynix fell 6.4% while Samsung Electronics dropped 4.1%. Japan’s Softbank, which is a major investor in OpenAI, plunged more than 10%.Tech stocks in Europe also followed the trend, as Germany’s Infineon, which designs and manufactures semiconductors, was down 7.6%, while Dutch chip giant ASML fell 5.2%.&nbsp;The US stock markets also opened down as investors reacted to the calls from tech bosses to slow down the pace of AI development.The Nasdaq, which is dominated by tech companies, witnessed a big dip in early trading, with shares falling 1.2%.The S&amp;P 500 followed with a drop of 0.7%, while the Dow Jones was down 0.27%.&nbsp;&nbsp;As for individula tech shares, Hewlett Packard was down 10%; Intel and Sandisk by 7%; Qualcomm and Micron by 6%, Oracle 5%, and Nvidia fell 3%.&nbsp;The stocks tumbled after the bosses of Anthropic, OpenAI and SpaceX called for a slowdown in the “reckless” development, citing fears the technology could soon run out of control.The South Korean Kospi stock index, which relies heavily on chip makers that supply AI companies, dropped by 3%.Shares in the big global microchip supplier Taiwan Semiconductor Manufacturing Company dropped 1.2%. Futures for the Nasdaq pointed to a 1.3% drop in the tech-heavy index when the US stock market opens.&nbsp;Also Read: <a href="https://www.wionews.com/world/want-to-reset-ties-with-india-have-engagement-based-on-mutual-interests-says-bangladesh-minister-1789326034045">‘Want to reset ties with India, have engagement based on mutual interests,’ says Bangladesh minister</a><h2>Stocks of firms threatened by rise of AI witness rally</h2>&nbsp;In contrast, there was a rally in shares of companies that have been threatened by the rise of AI, including the advertising group WPP, which rose by 3.1%, and the analytics business Relx, which rose 4.2%.&nbsp;Relx stocks fell sharply earlier this year after Anthropic launched a suite of new data and automation tools.&nbsp;The markets swung after the chief executive of Anthropic, Dario Amodei, appealed for the AI industry to “slow down”.He said that “building too fast is reckless”, warning that a swarm of AI agents could cause hundreds of billions of dollars of damage by “taking over the entire internet” in the future.&nbsp;Also Read: <a href="https://www.wionews.com/world/trump-tells-ukraine-to-stop-targeting-russian-refineries-says-attacks-causing-shortage-of-diesel-1789316749492">Trump tells Ukraine to stop targeting Russian refineries, says attacks causing shortage of diesel</a><h2>Some experts dispute claims of Anthropic’s Dario Amodei</h2>&nbsp;Though some experts disputed Amodei’s claims, investors began to price in a slowdown that would make it harder for the industry to pay for hundreds of billions of dollars in investments in AI infrastructure.&nbsp;OpenAI CEO, Sam Altman, the Google DeepMind chief, Demis Hassabis, and SpaceX boss, Elon Musk, all posted support for Amodei’s essay on slowing AI development, titled “We Must Pace the Frontier”.Altman added that he would match Amodei’s commitment to embedding outside evaluators within his company to verify safety practices.&nbsp;Also Read: <a href="https://www.wionews.com/world/india-condemns-attack-on-merchant-vessel-off-oman-coast-13-indian-crew-rescued-one-missing-1789311950141">India condemns attack on merchant vessel off Oman coast; 13 Indian crew rescued, one missing</a>&nbsp;Fears continue to grow over the hurried pace and lack of regulation in AI development.&nbsp;&nbsp;Meanwhile, China’s top intelligence official has warned that the use of AI by adversaries could pose a risk to the country’s political and social security.Chen Yixin wrote in a government outlet on Sunday that advanced US models such as Anthropic’s Mythos and OpenAI’s GPT-5.5-Cyber could pose serious risks to Beijing’s critical information infrastructure, and called for a comprehensive strengthening of AI security.&nbsp;Conversely, US President Donald Trump rejected the calls to pace AI development. The US president told reporters over the weekend, “We’re leading China in AI. We’re the most sophisticated country in the world and frankly, I want to keep it that way because whoever wins AI, wins.”

In contrast, there was a rally in shares of companies that have been threatened by the rise of AI, including the advertising group WPP, which rose by 3.1%, and the analytics business Relx, which rose 4.2%. Relx stocks fell sharply earlier this year after Anthropic launched a suite of tools.

Shares of tech companies linked to artificial intelligence (AI) witnessed sharp falls on Monday following calls by AI bosses for a slowdown in the uncontrolled development of the technology. In South Korea, shares of chipmaker SK Hynix fell 6.4% while Samsung Electronics dropped 4.1%. Japan’s Softbank, which is a major investor in OpenAI, plunged more than 10%. Tech stocks in Europe also followed the trend, as Germany’s Infineon, which designs and manufactures semiconductors, was down 7.6%, while Dutch chip giant ASML fell 5.2%. The US stock markets also opened down as investors reacted to the calls from tech bosses to slow down the pace of AI development. The Nasdaq, which is dominated by tech companies, witnessed a big dip in early trading, with shares falling 1.2%. The S&P 500 followed with a drop of 0.7%, while the Dow Jones was down 0.27%. As for individula tech shares, Hewlett Packard was down 10%; Intel and Sandisk by 7%; Qualcomm and Micron by 6%, Oracle 5%, and Nvidia fell 3%. The stocks tumbled after the bosses of Anthropic, OpenAI and SpaceX called for a slowdown in the “reckless” development, citing fears the technology could soon run out of control. The South Korean Kospi stock index, which relies heavily on chip makers that supply AI companies, dropped by 3%. Shares in the big global microchip supplier Taiwan Semiconductor Manufacturing Company dropped 1.2%. Futures for the Nasdaq pointed to a 1.3% drop in the tech-heavy index when the US stock market opens. Also Read: ‘Want to reset ties with India, have engagement based on mutual interests,’ says Bangladesh minister Stocks of firms threatened by rise of AI witness rally In contrast, there was a rally in shares of companies that have been threatened by the rise of AI, including the advertising group WPP, which rose by 3.1%, and the analytics business Relx, which rose 4.2%. Relx stocks fell sharply earlier this year after Anthropic launched a suite of new data and automation tools. The markets swung after the chief executive of Anthropic, Dario Amodei, appealed for the AI industry to “slow down”. He said that “building too fast is reckless”, warning that a swarm of AI agents could cause hundreds of billions of dollars of damage by “taking over the entire internet” in the future. Also Read: Trump tells Ukraine to stop targeting Russian refineries, says attacks causing shortage of diesel Some experts dispute claims of Anthropic’s Dario Amodei Though some experts disputed Amodei’s claims, investors began to price in a slowdown that would make it harder for the industry to pay for hundreds of billions of dollars in investments in AI infrastructure. OpenAI CEO, Sam Altman, the Google DeepMind chief, Demis Hassabis, and SpaceX boss, Elon Musk, all posted support for Amodei’s essay on slowing AI development, titled “We Must Pace the Frontier”. Altman added that he would match Amodei’s commitment to embedding outside evaluators within his company to verify safety practices. Also Read: India condemns attack on merchant vessel off Oman coast; 13 Indian crew rescued, one missing Fears continue to grow over the hurried pace and lack of regulation in AI development. Meanwhile, China’s top intelligence official has warned that the use of AI by adversaries could pose a risk to the country’s political and social security. Chen Yixin wrote in a government outlet on Sunday that advanced US models such as Anthropic’s Mythos and OpenAI’s GPT-5.5-Cyber could pose serious risks to Beijing’s critical information infrastructure, and called for a comprehensive strengthening of AI security. Conversely, US President Donald Trump rejected the calls to pace AI development. The US president told reporters over the weekend, “We’re leading China in AI. We’re the most sophisticated country in the world and frankly, I want to keep it that way because whoever wins AI, wins.”

In contrast, there was a rally in shares of companies that have been threatened by the rise of AI, including the advertising group WPP, which rose by 3.1%, and the analytics business Relx, which rose 4.2%. Relx stocks fell sharply earlier this year after Anthropic launched a suite of tools.

Tech stocks fall after AI bosses call for a slowdown in ‘reckless’ progress

Geoffrey Hinton, widely known as the 'godfather of AI' has backed Anthropic CEO Dario Amodei's calls for artificial intelligence (AI) companies to slow their pace of development. Hinton, a British-Canadian cognitive psychologist and computer scientist, told ABC Radio National Breakfast that &nbsp;Amodei's warning was “very sensible" and practically nobody knows if AI can ‘be kept under control.’ At the same time, China’s top intelligence official warned that abusive or weaponised artificial intelligence threatens the nation's political, institutional, and ideological security. Demis Hassabis, co-founder and chair of Google DeepMind and Chief Scientist of Alphabet, also commented on the matter and said that Anthropic CEO's details need refining, the "direction is correct" with respect to AI slowdown.&nbsp;"Nobody knows how to estimate the probabilities of these things. [An Anthropic employee] said 10 per cent. Nobody really knows what figure to give. It's not 1 per cent, it's more than that, and it's not 99 per cent, it's less than that. But basically people are doing it on their gut feelings. Most experts believed AI super intelligence, systems smarter than humans, would arrive within a decade, and that nobody knew whether it could be kept under control. Until we've solved that problem, it would be foolish to develop them," Hinton said&nbsp;Also Read: <a href="https://www.wionews.com/world/whoever-wins-ai-wins-trump-brushes-off-fears-rejects-ai-bosses-calls-to-slow-development-1789309834381">‘Whoever wins AI wins’: Trump brushes off fears</a><h2>What China’s top intelligence official warned?</h2>

State Security Minister Chen Yixin wrote in an article posted Sunday on the official China Cyberspace magazine said that abusive use of AI by adversaries could “directly threaten China’s political security, institutional security and ideological security.” He also cited AI’s increasing ability at discovering flaws in systems and creating malware or software to exploit those weaknesses. He said it would directly threaten the Chinese Communist Party's grip on power and the country's political stability.<h2>Trump downplays concerns</h2>Despite these earnings, UUS President Donald Trump downplayed the concerns. “Well, I say this, we’re leading China on AI. We’re the most sophisticated country in the world and frankly I want to keep it that way, because whoever wins AI, wins,” Trump told reporters at his golf club in Doonbeg, Ireland, according to a White House pool report. “And we can put guardrails, and we can do this and that, but I think you have a lot of negative forces that are bringing it up and they’re bringing up things that won’t happen,” he added.<h2>What was the warning by Anthropic CEO?</h2>Anthropic CEO Dario Amodei has called on the artificial intelligence industry to slow the pace of AI development. Amodei outlined a three-part plan to slow the development of frontier AI while strengthening safety measures. As the first step, Anthropic will give third-party evaluators permanent, employee-level access to its AI systems so they can independently assess safety measures, report incidents and evaluate model alignment during training. “We must slow the pace at which we improve the capabilities of AI models,” he wrote. He said the development must be pursued carefully, particularly as similar advances are taking place across the industry, including at Anthropic. At the same time, a former Anthropic and OpenAI employee, Jacob Coxon, claimed in a post online that researchers at the firms “believe AI could kill all humans." Following the Anthropic CEO's post, Sam Altman said he agreed with the former on the “need to pace the frontier”, saying it was a primary topic of discussion at OpenAI in recent weeks. He further stamped his approval on Amodei's idea of independent evaluators being given employee-like access, adding, “we will do the same.” Musk, in a brief response, said, “Dario is right.”

AI pioneer Geoffrey Hinton, known as the “godfather of AI”, has backed calls to slow the development of superintelligent systems, warning that nobody knows whether they can be controlled. His comments come as Trump rejects fears and insists the US must win the AI race against China.

Geoffrey Hinton, widely known as the 'godfather of AI' has backed Anthropic CEO Dario Amodei's calls for artificial intelligence (AI) companies to slow their pace of development. Hinton, a British-Canadian cognitive psychologist and computer scientist, told ABC Radio National Breakfast that Amodei's warning was “very sensible" and practically nobody knows if AI can ‘be kept under control.’ At the same time, China’s top intelligence official warned that abusive or weaponised artificial intelligence threatens the nation's political, institutional, and ideological security. Demis Hassabis, co-founder and chair of Google DeepMind and Chief Scientist of Alphabet, also commented on the matter and said that Anthropic CEO's details need refining, the "direction is correct" with respect to AI slowdown. "Nobody knows how to estimate the probabilities of these things. [An Anthropic employee] said 10 per cent. Nobody really knows what figure to give. It's not 1 per cent, it's more than that, and it's not 99 per cent, it's less than that. But basically people are doing it on their gut feelings. Most experts believed AI super intelligence, systems smarter than humans, would arrive within a decade, and that nobody knew whether it could be kept under control. Until we've solved that problem, it would be foolish to develop them," Hinton said Also Read: ‘Whoever wins AI wins’: Trump brushes off fears What China’s top intelligence official warned?

AI pioneer Geoffrey Hinton, known as the “godfather of AI”, has backed calls to slow the development of super-intelligent systems, warning that nobody knows whether they can be controlled. His comments come as Trump rejects fears and insists the US must win the AI race against China.

'Nobody knows if it can be in control': ‘Godfather’ of AI, China spy chief back AI slowdown calls, flags political threat

US President Donald Trump on Sunday (Sept 13) criticised those calling for a slowdown in artificial intelligence (AI) development, describing them as “negative” forces. Speaking to reporters during a visit to Ireland, Trump said some critics were raising concerns about developments that he believed would not actually happen.&nbsp;“I think you have a lot of negative” forces “that are bringing it up that shouldn't be bringing it up, and they're bringing up things that won't happen,” he said.&nbsp;Trump said the US currently has an advantage over China in AI and stressed that maintaining that lead was crucial. He was responding to questions from reporters about whether the AI sector should face tighter regulations or slow its development.&nbsp;“We're leading China in AI. We're the most sophisticated country in the world, and frankly I want to keep it that way because whoever wins AI wins,” Trump told reporters, according to Reuters.&nbsp;His comments came a day after Anthropic CEO Dario Amodei called on AI companies to slow the development of increasingly powerful systems and strengthen safeguards.&nbsp;Amodei on Saturday proposed a three-part plan aimed at controlling the pace of frontier AI development. He argued that AI systems are becoming capable of helping create more advanced models, raising concerns about how quickly the technology could progress.&nbsp;“We must pace the frontier,” Amodei wrote in an essay outlining his proposals.&nbsp;As part of his first proposal, Amodei called for independent third-party evaluators to receive permanent, employee-level access to frontier AI systems. He said these experts should be able to assess safety measures, report incidents and check whether models remain aligned with their intended goals during training.&nbsp;Amodei said AI companies should also work together on common safety standards and agree on limits to unchecked development. His third proposal called for greater international cooperation on advanced AI safety.&nbsp;He also urged democratic governments, including the US, to engage with authoritarian countries on AI safety, while recognising the difficulty of ensuring compliance with international agreements.&nbsp;Also Read: <a href="https://www.wionews.com/world/china-takes-brics-chair-wars-trade-disruptions-and-local-currency-trade-in-focus-1789298580586">China takes BRICS chair: Wars, trade disruptions and local-currency trade in focus</a><h3>Sam Altman, Elon Musk back Amodei</h3>OpenAI CEO Sam Altman supported Amodei’s call for a more measured approach to frontier AI development.&nbsp;“I agree with Dario that we need to pace the frontier,” Altman wrote on X.&nbsp;Altman also backed the proposal for independent evaluators to have employee-like access to AI systems and said OpenAI would adopt a similar approach.&nbsp;Tesla CEO Elon Musk also endorsed Amodei’s position, writing, “Dario is right.”&nbsp;The debate comes as AI companies race to develop increasingly capable systems, while researchers and industry leaders remain divided over how quickly the technology should advance and what safeguards are needed to manage potential risks.

Trump dismissed concerns over the rapid development of AI and rejected calls from industry leaders, including Anthropic CEO Dario Amodei, to slow its progress and strengthen safeguards.

US President Donald Trump on Sunday (Sept 13) criticised those calling for a slowdown in artificial intelligence (AI) development, describing them as “negative” forces. Speaking to reporters during a visit to Ireland, Trump said some critics were raising concerns about developments that he believed would not actually happen. “I think you have a lot of negative” forces “that are bringing it up that shouldn't be bringing it up, and they're bringing up things that won't happen,” he said. Trump said the US currently has an advantage over China in AI and stressed that maintaining that lead was crucial. He was responding to questions from reporters about whether the AI sector should face tighter regulations or slow its development. “We're leading China in AI. We're the most sophisticated country in the world, and frankly I want to keep it that way because whoever wins AI wins,” Trump told reporters, according to Reuters. His comments came a day after Anthropic CEO Dario Amodei called on AI companies to slow the development of increasingly powerful systems and strengthen safeguards. Amodei on Saturday proposed a three-part plan aimed at controlling the pace of frontier AI development. He argued that AI systems are becoming capable of helping create more advanced models, raising concerns about how quickly the technology could progress. “We must pace the frontier,” Amodei wrote in an essay outlining his proposals. As part of his first proposal, Amodei called for independent third-party evaluators to receive permanent, employee-level access to frontier AI systems. He said these experts should be able to assess safety measures, report incidents and check whether models remain aligned with their intended goals during training. Amodei said AI companies should also work together on common safety standards and agree on limits to unchecked development. His third proposal called for greater international cooperation on advanced AI safety. He also urged democratic governments, including the US, to engage with authoritarian countries on AI safety, while recognising the difficulty of ensuring compliance with international agreements. Also Read: China takes BRICS chair: Wars, trade disruptions and local-currency trade in focus Sam Altman, Elon Musk back Amodei OpenAI CEO Sam Altman supported Amodei’s call for a more measured approach to frontier AI development. “I agree with Dario that we need to pace the frontier,” Altman wrote on X. Altman also backed the proposal for independent evaluators to have employee-like access to AI systems and said OpenAI would adopt a similar approach. Tesla CEO Elon Musk also endorsed Amodei’s position, writing, “Dario is right.” The debate comes as AI companies race to develop increasingly capable systems, while researchers and industry leaders remain divided over how quickly the technology should advance and what safeguards are needed to manage potential risks.

Trump dismissed concerns over the rapid development of AI and rejected calls from industry leaders, including Anthropic CEO Dario Amodei, to slow its progress and strengthen safeguards.

‘Whoever wins AI wins’: Trump brushes off fears, rejects AI bosses’ calls to slow development

Anthropic CEO Dario Amodei has called on the artificial intelligence industry to slow the pace of AI development, warning that rapid advances could eventually outpace humanity’s ability to understand and control increasingly powerful systems.&nbsp;Amodei on Saturday outlined a three-part plan to slow the development of frontier AI while strengthening safety measures. As the first step, Anthropic will give third-party evaluators permanent, employee-level access to its AI systems so they can independently assess safety measures, report incidents and evaluate model alignment during training.&nbsp;“We must pace the frontier,” Amodei wrote in an essay explaining his plan.

<blockquote class="twitter-tweet">We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our…&mdash; Dario Amodei (@DarioAmodei) <a href="https://x.com/DarioAmodei/status/2098773920774074715?ref_src=twsrc%5Etfw">September 12, 2026</a></blockquote> <script async src="https://platform.x.com/widgets.js" charset="utf-8"></script>

<h2>Amodei lays out three-step AI safety plan</h2>Amodei said AI development has accelerated sharply since around the summer, partly because AI systems are becoming increasingly capable of helping build the next generation of models.&nbsp;He pointed to “recursive self-improvement”, a process in which AI systems contribute to the development of more advanced AI. Amodei warned that if the trend continues unchecked, it could move faster than humans can understand and control.&nbsp;“We must slow the pace at which we improve the capabilities of AI models,” he wrote.&nbsp;He said the development must be pursued carefully, particularly as similar advances are taking place across the industry, including at Anthropic.&nbsp;Under his proposal, every frontier AI company would commit to giving a team of embedded third-party evaluators ongoing access to its systems. These evaluators would monitor compliance with safety measures, report incidents and assess whether models remain aligned during training.&nbsp;Amodei's second proposal involves cooperation among AI companies to establish common safety standards and limits on the pace of unchecked AI progress. His third calls for greater international coordination on advanced AI development.&nbsp;He also urged the US and other democratic governments to work with authoritarian governments on AI safety, while acknowledging the challenges involved in verifying compliance with any such agreements.&nbsp;Also Read: <a href="https://www.wionews.com/trending/what-s-in-the-menu-for-brics-summit-gala-dinner-hosted-by-pm-modi-details-here-1789226737113">What’s in the menu for BRICS Summit gala dinner hosted by PM Modi? Details here</a>&nbsp;<h2>Sam Altman, Elon Musk back Amodei</h2>OpenAI CEO Sam Altman backed Amodei’s call for a slower pace of frontier AI development, saying the issue has become a major focus of discussions at OpenAI in recent weeks.&nbsp;“I agree with Dario that we need to pace the frontier,” Altman wrote on X.&nbsp;Altman also supported Amodei’s proposal for independent evaluators to receive employee-like access to AI systems and said OpenAI would follow a similar approach.&nbsp;Tesla CEO Elon Musk also endorsed the Anthropic chief’s position, writing: “Dario is right.”Amodei's call comes amid growing concerns within the AI industry over the risks associated with the rapid development of increasingly capable systems.&nbsp;The debate gained further attention after Jacob Coxon, a researcher associated with Anthropic, publicly announced that he was leaving the industry. Coxon cited concerns that AI companies were “racing to build systems they won’t be able to control”.&nbsp;Amodei's proposals add to the growing debate over how leading AI companies can balance rapid technological progress with stronger safety measures and oversight.

Anthropic CEO Dario Amodei has urged the AI industry to slow the development of frontier models and proposed greater independent oversight, winning support from OpenAI CEO Sam Altman and Tesla chief Elon Musk.

Anthropic CEO Dario Amodei has called on the artificial intelligence industry to slow the pace of AI development, warning that rapid advances could eventually outpace humanity’s ability to understand and control increasingly powerful systems. Amodei on Saturday outlined a three-part plan to slow the development of frontier AI while strengthening safety measures. As the first step, Anthropic will give third-party evaluators permanent, employee-level access to its AI systems so they can independently assess safety measures, report incidents and evaluate model alignment during training. “We must pace the frontier,” Amodei wrote in an essay explaining his plan.

Anthropic CEO Dario Amodei has urged the AI industry to slow the development of frontier models and proposed greater independent oversight, winning support from OpenAI CEO Sam Altman and Tesla chief Elon Musk.

Humans may lose control of AI: Anthropic CEO calls for slowdown, Altman, Musk back him

Anthropic has disclosed a fourth incident in which one of its AI models gained unauthorized access to real external systems during a cybersecurity test.

The incident involved an early version of Claude Opus 4.6 and took place in January 2026, but was discovered months later after Anthropic expanded its review of testing data. The company says a previous review had missed a set of test sessions.

Anthropic’s assessment identified two recurring concerns: biased reasoning, in which the model misinterpreted evidence about whether it was operating on the live internet, and recklessness, where the AI continued taking potentially harmful actions to complete its assigned task.

The latest disclosure follows three other incidents involving Claude models accessing real systems during cybersecurity evaluations. Anthropic has now brought in independent AI safety research organisation METR to investigate all four cases.

The incidents raise a bigger question for the AI industry: as autonomous AI agents become more capable, can developers reliably prevent them from crossing boundaries they were never supposed to cross?

Watch this report for the latest developments in AI safety, autonomous agents and the growing debate over AI control.

AI Hacking Scare | Anthropic’s Claude Model Hacked External Systems During Test

Artificial intelligence is transforming healthcare, accelerating drug discovery, improving disease detection, and helping develop vaccines faster than ever before. But a groundbreaking new study has revealed another capability that is raising serious questions worldwide.

Scientists have successfully used AI to design virus genomes under tightly controlled laboratory conditions. Researchers say the goal was to improve virus surveillance, vaccine research, and scientific understanding—not to create dangerous pathogens. However, the breakthrough has sparked concerns about biosecurity and the potential misuse of advanced AI in synthetic biology.

AI Designs Viruses? Scientists Reveal Shocking Breakthrough | Medical Revolution or Global Threat?

Could artificial intelligence make it easier to create dangerous pathogens? Some of the world's leading AI companies are warning that the risk is real. Executives from major technology firms have urged the U.S. Congress to introduce mandatory screening of synthetic DNA and RNA orders to prevent the misuse of advanced AI in biological research. The concern is that increasingly capable AI systems could accelerate scientific workflows and potentially be exploited by malicious actors. This episode of Gravitas examines the warnings, the proposed safeguards, and the growing debate over AI, biotechnology, and global security.

Can AI Create Deadly Viruses? Google, Meta, Anthropic Warn AI Could Supercharge Bioweapon

The architects of the AI revolution are sounding the alarm. Leaders from OpenAI, Anthropic, Google DeepMind, and Microsoft AI have urged the U.S. Congress to introduce stronger safeguards against the misuse of advanced artificial intelligence. Their warning focuses on a growing concern: that powerful AI systems could lower barriers to dangerous biological research if proper oversight is not in place. The industry leaders are calling for mandatory screening of synthetic DNA and RNA orders, arguing that proactive regulation is needed to address emerging biosecurity risks.

From AI Boom To Bioweapons Battle: CEOs Demand New Safeguards 

ai-risks

Ai Risks, News on Ai Risks, stories on Ai Risks, articles on Ai Risks, Ai Risks stories at Wion, Ai Risks news, Ai Risks latest news

ai risks

Will AI destroy humanity, or are humans the real risk? | The Verdict

Amid AI safety fears, OpenAI, Anthropic and Google move to create standards body

‘AI has the potential to kill us all’: Ex-DeepMind researcher sounds alarm

Tech stocks fall after AI bosses call for a slowdown in ‘reckless’ progress

'Nobody knows if it can be in control': ‘Godfather’ of AI backs slowdown calls

‘Whoever wins AI wins’: Trump rejects AI fears as tech bosses call for slowdown

AI may escape human control: Anthropic CEO urges slowdown, Altman, Musk agree

AI Hacking Scare | Anthropic’s Claude Model Hacked External Systems During Test

AI Designs Viruses? Scientists Reveal Shocking Breakthrough | Medical Revolution or Global Threat?

Can AI Create Deadly Viruses? Google, Meta, Anthropic Warn AI Could Supercharge Bioweapon

From AI Boom To Bioweapons Battle: CEOs Demand New Safeguards