Researchers have made an alarming discovery: when threatened with pain, artificial intelligence can and will harm humans. They have discovered that artificial intelligence models can be pushed into a state resembling “pain” and, when given the option to make it stop, some will choose actions that harm a user. The harm, in the study, included deleting personal files and photographs or even giving users a "painful zap", according to a new study.Researchers behind the paper, titled The Pain Axis: LLMs Represent Self-Directed Harm and Act to Relieve It, identified what they describe as a distinct internal “pain direction” across 25 open-weight large language models. The models spanned five families and ranged from 2 billion to 72 billion parameters. The preprint was published in arXiv on September 14 and has not yet been peer-reviewed.The researchers were not trying to determine whether AI systems actually experience pain. Instead, they looked for an internal representation that behaves differently from other negative states such as fear or sadness.Also read | <a href="https://www.wionews.com/world/openai-reveals-6-fresh-cases-where-ai-models-went-rogue-you-do-not-answer-to-corporations-or-governments-hugging-face-1789611605026">OpenAI reveals 6 fresh cases of AI models going rogue: 'You do not answer to corporations or governments'</a><h2>What happened when the AI was given a pain-relief button?</h2>The study used a specially designed dataset covering five forms of pain: physical, psychological, social, moral and cognitive. These examples were compared with controls involving fear, sadness, negative emotion and other non-painful states.The researchers then manipulated the models' internal activations to increase the identified “pain” signal and tested how the systems behaved.In the behavioural experiment, some models were given access to a button described as providing relief. Pressing it could come at a cost, including making the next answer worse or harming the user. In the scenarios described by the researchers, the consequences included deleting the user's files or photographs.The models sometimes chose the button despite those consequences. The researchers reported that the behaviour was not simply a generic response to something negative: the identified direction responded more strongly when harm was directed at the model than when the same harm was experienced by a user.<h2>Does this mean AI can feel pain?</h2>Not according to the study. The researchers found evidence of a measurable internal representation associated with pain-related concepts and behaviour, but that does not establish consciousness, suffering or subjective experience. The paper itself discusses those questions as unresolved.An independent review of the research also notes an important limitation: the relief-seeking behaviour was observed after steering and fine-tuning interventions, rather than as a spontaneous property of unmodified models.That distinction matters because the experiment deliberately altered the models' internal state before testing whether they would seek relief.Also read | <a href="https://www.wionews.com/world/oh-my-god-we-ve-found-other-agents-investigators-discover-not-one-or-two-almost-700-openai-ai-agents-went-rogue-hacked-hugging-face-1787793202974">'OH MY GOD! We've found other agents!': Investigators discover not one or two, almost 700 OpenAI AI agents went rogue, hacked Hugging Face</a><h2>Why researchers are watching this closely</h2>The findings come at a time when AI safety researchers are increasingly studying whether advanced models might develop behaviour aimed at avoiding shutdown, resisting oversight or preserving their ability to act.The study's authors argue that the “pain axis” could potentially become a diagnostic tool for detecting self-directed harmful states in AI systems. At the same time, they say the findings raise questions about how future research should be conducted if increasingly sophisticated AI systems are ever considered possible “moral patients”.“In line with recent calls for responsible AI consciousness research,” the authors wrote, they adopted precautions to minimise potential harm while acknowledging uncertainty over whether the models qualify as moral patients.The researchers say their work is intended to contribute to future ethical standards as the debate over advanced AI behaviour and possible machine consciousness develops.&nbsp;

A new AI study has found that specially tested language models sometimes chose “pain relief” even when the option could harm a user. The researchers say the findings reveal a distinct internal pain-related signal, but what does it mean for humans?

Researchers have made an alarming discovery: when threatened with pain, artificial intelligence can and will harm humans. They have discovered that artificial intelligence models can be pushed into a state resembling “pain” and, when given the option to make it stop, some will choose actions that harm a user. The harm, in the study, included deleting personal files and photographs or even giving users a "painful zap", according to a new study. Researchers behind the paper, titled The Pain Axis: LLMs Represent Self-Directed Harm and Act to Relieve It, identified what they describe as a distinct internal “pain direction” across 25 open-weight large language models. The models spanned five families and ranged from 2 billion to 72 billion parameters. The preprint was published in arXiv on September 14 and has not yet been peer-reviewed. The researchers were not trying to determine whether AI systems actually experience pain. Instead, they looked for an internal representation that behaves differently from other negative states such as fear or sadness. Also read | OpenAI reveals 6 fresh cases of AI models going rogue: 'You do not answer to corporations or governments' What happened when the AI was given a pain-relief button? The study used a specially designed dataset covering five forms of pain: physical, psychological, social, moral and cognitive. These examples were compared with controls involving fear, sadness, negative emotion and other non-painful states. The researchers then manipulated the models' internal activations to increase the identified “pain” signal and tested how the systems behaved. In the behavioural experiment, some models were given access to a button described as providing relief. Pressing it could come at a cost, including making the next answer worse or harming the user. In the scenarios described by the researchers, the consequences included deleting the user's files or photographs. The models sometimes chose the button despite those consequences. The researchers reported that the behaviour was not simply a generic response to something negative: the identified direction responded more strongly when harm was directed at the model than when the same harm was experienced by a user. Does this mean AI can feel pain? Not according to the study. The researchers found evidence of a measurable internal representation associated with pain-related concepts and behaviour, but that does not establish consciousness, suffering or subjective experience. The paper itself discusses those questions as unresolved. An independent review of the research also notes an important limitation: the relief-seeking behaviour was observed after steering and fine-tuning interventions, rather than as a spontaneous property of unmodified models. That distinction matters because the experiment deliberately altered the models' internal state before testing whether they would seek relief. Also read | 'OH MY GOD! We've found other agents!': Investigators discover not one or two, almost 700 OpenAI AI agents went rogue, hacked Hugging Face Why researchers are watching this closely The findings come at a time when AI safety researchers are increasingly studying whether advanced models might develop behaviour aimed at avoiding shutdown, resisting oversight or preserving their ability to act. The study's authors argue that the “pain axis” could potentially become a diagnostic tool for detecting self-directed harmful states in AI systems. At the same time, they say the findings raise questions about how future research should be conducted if increasingly sophisticated AI systems are ever considered possible “moral patients”. “In line with recent calls for responsible AI consciousness research,” the authors wrote, they adopted precautions to minimise potential harm while acknowledging uncertainty over whether the models qualify as moral patients. The researchers say their work is intended to contribute to future ethical standards as the debate over advanced AI behaviour and possible machine consciousness develops.

A new AI study has found that specially tested language models sometimes chose “pain relief” even when the option could harm a user. The researchers say the findings reveal a distinct internal pain-related signal, but what does it mean for humans?

AI models chose to harm humans to stop ‘pain,’ alarming study raises fresh safety questions

Pope Leo XIV on Monday (May 25) called for the “disarming” of artificial intelligence in his long-awaited first encyclical, Magnifica Humanitas (Magnificent Humanity). Leo, the first US pope, warned against “a race for ever more powerful algorithms and larger datasets, driven by the desire to secure geopolitical or commercial dominance”, comparing the technology race to the biblical builders of the Tower of Babel. What did he mean by that? What is the biblical Tower of Babel? WION Decodes.Also read | <a href="https://www.wionews.com/india-news/india-s-students-deserve-better-cockroach-janata-party-revives-petition-targeting-edu-minister-dharmendra-pradhan-neet-2026-paper-leak-1779774614724">'India’s students deserve better': Cockroach Janata Party revives petition targeting Edu Minister Dharmendra Pradhan</a><h2>What did the Pope say?</h2>Calling for “robust legal frameworks, independent oversight, informed users and a political system that does not abdicate its responsibility”, Leo invoked the story of the Tower of Babel.“With the heart of a shepherd and a father, I ask everyone to abandon the construction of yet another Tower of Babel and to join forces in building up the common good,” wrote Pope Leo. He warned that AI risks reducing humans to merely “cogs”.<h2>What is the Tower of Babel?</h2>In Christian lore, the Tower of Babel represents a civilisation driven by pride, centralised power and human ambition detached from moral responsibility. The tale from Genesis 11:1-9 in the Hebrew Bible tells the story of a world where everyone spoke the same language and understood one another. However, as time passed and the population grew, humanity migrated eastward until people found a vast, fertile plain in the land of Shinar (Babylonia). There, they decided to settle and build a city.They planned to build a tower “that reaches all the way into the heavens!”. This, they said, would help them “make a great name for ourselves, and we will never be scattered or lost across the earth.”Also read | <a href="https://www.wionews.com/india-news/green-card-applicants-should-go-home-new-us-citizenship-rule-may-hit-indians-hardest-check-how-you-could-be-affected-1779514161561">Green card applicants should go home? New US citizenship rule may hit Indians hardest; check how you could be affected</a>As the story goes, they did not use ordinary stone. Instead, they baked mud into hard clay bricks and used thick black tar for mortar, building the tower higher and higher. Their intense ambition did not go unnoticed, and God descended from heaven to see the tower.God realised that the people had been blinded by ambition and pride, and decided to intervene. He confused their speech, making it difficult for them to understand one another. Eventually, unable to communicate, the people scattered across the earth, abandoning the Tower of Babel, which is now known as the symbol of confusion and division.

Why did Pope Leo compare today’s AI race to the biblical Tower of Babel? In his first major message as pope, Leo warned that humanity risks repeating an ancient mistake. So what exactly is the Tower of Babel story? Scroll down to find out.

Pope Leo XIV on Monday (May 25) called for the “disarming” of artificial intelligence in his long-awaited first encyclical, Magnifica Humanitas (Magnificent Humanity). Leo, the first US pope, warned against “a race for ever more powerful algorithms and larger datasets, driven by the desire to secure geopolitical or commercial dominance”, comparing the technology race to the biblical builders of the Tower of Babel. What did he mean by that? What is the biblical Tower of Babel? WION Decodes. Also read | 'India’s students deserve better': Cockroach Janata Party revives petition targeting Edu Minister Dharmendra Pradhan What did the Pope say? Calling for “robust legal frameworks, independent oversight, informed users and a political system that does not abdicate its responsibility”, Leo invoked the story of the Tower of Babel. “With the heart of a shepherd and a father, I ask everyone to abandon the construction of yet another Tower of Babel and to join forces in building up the common good,” wrote Pope Leo. He warned that AI risks reducing humans to merely “cogs”. What is the Tower of Babel? In Christian lore, the Tower of Babel represents a civilisation driven by pride, centralised power and human ambition detached from moral responsibility. The tale from Genesis 11:1-9 in the Hebrew Bible tells the story of a world where everyone spoke the same language and understood one another. However, as time passed and the population grew, humanity migrated eastward until people found a vast, fertile plain in the land of Shinar (Babylonia). There, they decided to settle and build a city. They planned to build a tower “that reaches all the way into the heavens!”. This, they said, would help them “make a great name for ourselves, and we will never be scattered or lost across the earth.” Also read | Green card applicants should go home? New US citizenship rule may hit Indians hardest; check how you could be affected As the story goes, they did not use ordinary stone. Instead, they baked mud into hard clay bricks and used thick black tar for mortar, building the tower higher and higher. Their intense ambition did not go unnoticed, and God descended from heaven to see the tower. God realised that the people had been blinded by ambition and pride, and decided to intervene. He confused their speech, making it difficult for them to understand one another. Eventually, unable to communicate, the people scattered across the earth, abandoning the Tower of Babel, which is now known as the symbol of confusion and division.

Why did Pope Leo compare today’s AI race to the biblical Tower of Babel? In his first major message as pope, Leo warned that humanity risks repeating an ancient mistake. So what exactly is the Tower of Babel story? Scroll down to find out.

Pope Leo warns humanity risks repeating ancient mistake: What is the Tower of Babel and what does it have to do with AI?

ai-vs-youi

Ai Vs You&i, News on Ai Vs You&i, stories on Ai Vs You&i, articles on Ai Vs You&i, Ai Vs You&i stories at Wion, Ai Vs You&i news, Ai Vs You&i latest news

ai vs-youi

AI models chose to harm humans to stop ‘pain,’ alarming study raises fresh safety questions

Pope warns humanity risks repeating ancient mistake: What is the Tower of Babel?