Urgent UN warnings: time to tame killer robots is running out

WASHINGTON – “My main concern is with lethal autonomous weapons systems. Let us call them what they are: killer robots. Machines selecting and engaging their target and taking a life – without human control and judgement. This is morally repugnant.”

So said United Nations Secretary-General António Guterres at a meeting in July that brought together AI leaders from UN member states, international organisations, the private sector, academia, and civil society leaders.

Officially named the first Global Dialogue on Artificial Intelligence, it reflected growing global concern that rapidly advancing AI is outrunning the ability of governments to regulate it. While that applies to all aspects of modern life, it is particularly worrying in warfare.

Also read | India's next terror threat may not cross the border. It may enter through your phone | WION Decodes

Therefore, Guterres argued, the deployment of LAWS (lethal autonomous weapons systems) “is politically unacceptable. And it must be banned by international law.”

International law and international bans have a poor track record, but Guterres deserves praise for finally drawing attention to a subject that has been debated for decades by human rights groups, the military and experts working for the world’s biggest AI companies, most of them based in the United States.

One of them, Anthropic, is involved in a high-profile dispute with the US Department of Defence that mirrored the concerns voiced by Guterres’s UN conference. Anthropic, along with Google, OpenAI and xAI, won government contracts last year worth around $200 million to accelerate AI adoption for national security.

Unlike the other AI companies, Anthropic insisted that its most sophisticated model must not be used by the government for fully autonomous weapons; i.e. robots, and mass domestic surveillance.

To emphasise fears that technology is outrunning human control, Anthropic’s CEO, Dario Amodei, called for a global, coordinated slowdown in artificial intelligence development. AI systems, he warned, are advancing with such speed that they soon could build and improve their own successors, without human control.

Other AI-developing companies reacted coolly. The Pentagon reacted with anger. It ordered all federal agencies to stop using Anthropic technology and declared the company a “supply chain risk” – a designation that is now the subject of a battle in court.

It is more than a contract dispute – it is a fundamental disagreement over who controls the use of AI in weapons or other aspects of life.

When you talk to techies, sooner or later the different types of control over aerial, ground and maritime unmanned vehicles come up.

Also read | Pope Leo warns humanity risks repeating ancient mistake: What is the Tower of Babel and what does it have to do with AI?

There is “in the loop”, meaning that a human operator directs the autonomous system and decides whether or not to kill the target. There is “on the loop” when a human operator does not directly control the automated system but can override its decision. Finally, “off the loop” indicates that the system operates independently and makes life-or-death decisions.

For an illustrative example of an “in the loop” operation, let’s look back to the January 3, 2020 assassination of Major-General Qassem Soleimani, one of Iran’s most powerful military leaders, in a drone strike ordered by US President Donald Trump.

Soleimani’s movements had been closely tracked for days, and it became known that he was about to visit Baghdad. On arrival, he was greeted by the leader of a pro-Iranian Iraqi militia who escorted him to two cars waiting outside. As they began pulling away, they were hit by 100-pound Hellfire missiles launched from MQ9 Reaper drones high overhead.

The cars exploded into fireballs, incinerating Solimani, his host and eight of their aides.

Typical for an “in the loop” operation, the American Air Force officer who pushed the button to launch the missile sat in a stationary “cockpit” at Creech Air Force base near Las Vegas, almost 12,000 kilometres from Baghdad.

In the six years since the assassination – or “targeted killing” as the US called it- drones have become a central weapon in the wars between Ukraine and Russia, the US and Iran, Sudan, and most of all Israel against Hamas, a campaign which turned Gaza into a field of ruins and killed more than 70,000 Palestinians.

Israel’s war on Gaza showed the limits of “in the loop” operations given the green light by military and political leaders. One of the main purposes of using AI in war is to process huge amounts of data to identify suspected enemies. Left to human intelligence officers, that can be a protracted process.

It took weeks to follow Solimani and finally pinpoint him at the door of Baghdad airport. In contrast, Israeli AI programmes, based on cellphone data, facial recognition and locations known to frequent Hamas militants, produced a list of 37,000 targets within days of the October 7, 2023 massacre by Hamas of more than 1,200 Israelis.

One user of the system, named Lavender, told the Israeli/Palestinian +972 news outlet that “I would invest 20 seconds for each target and do dozens of them every day. I had zero added-value as a human, apart from being a stamp of approval” (of the AI-generated list).

Though you could argue that the Israelis followed “in the loop” procedures, simply pressing a fire button in response to a target provided by a computer, without the time to double-check, does not amount to human control.

AI-generated target lists are part of the explanation of the high Palestinian death toll in Gaza. Human rights organisations say most of those killed were civilians, or in military jargon “collateral damage.”

It is not often that the chief of an AI company and the chief of the United Nations publicly share worries about how speedily technology spreads. But addressing the UN meeting on Artificial Intelligence this month, Antonio Guterres gave an example.

“The Internet took 15 years to reach a billion people,” he said. “AI got there in two. And these systems are no longer tools awaiting instruction – they are writing code, acting online and making choices with less and less human oversight.”

Numbers from the two wars the United States fought in Iraq and Afghanistan provide another illustration of the astonishing speed of the development of robotic warfare. The US forces that stormed into Iraq in 2003 had no robots on the ground. There were none in Afghanistan either.

Just six years later, the two wars were waged with the help of an estimated 12,000 ground-based robots and 7,000 drones.

While there is growing understanding that technology is outpacing both scientific understanding and the ability of governments to adapt, the prospects of universal agreement on a solution are remote.

Last November, the UN General Assembly issued a resolution on the risks AI can pose to international humanitarian law and an emerging arms race.

The vote was 156 in favour and five against. These five were the United States, Russia, North Korea, Belarus and Israel. There were eight abstentions, including China, which is competing with the US on accelerating AI development.

A major problem in pushing ahead, according to a report in the MIT Technology Review, is what the computational neuroscientist Uri Maoz calls “Black Boxes. “

He wrote: “Having studied intentions in the human brain for decades and in AI systems more recently, I can attest that state-of –the-art AI systems are essentially Black Boxes.

“We know the inputs and outputs, but the artificial “brain” processing them remains opaque. Even their creators cannot fully interpret them or understand how they work.”

That will be even more true once the countries that are now using drones – Ukraine has emerged as a leader in its fight against the Russian invasion – begin to use large swarms rather than individual drones. This is why scientists have intensified their study of bees.

Like bees, drone swarms operate in large numbers, adapt their behaviour in real time, share tasks, communicate with each other and continue with their tasks even after suffering losses.