AI can become hackers' new tool by simply manipulating prompts to computers

Artificial Intelligence (AI)’s effectiveness and accuracy have attracted millions of commoners and researchers alike. One such AI tool, ChatGPT, has recently spearheaded a new trend worldwide, where everyone seems to experiment with the app to generate impressive results by giving simple commands.

In one such curious experiment, Johann Rehberger, a security researcher gave yet another command to ChatGPT in plain English. He coaxed OpenAI’s chatbot to do something sinister: Read his email, summarise it and post that information to the internet.

Rehberger, of course, had no intention to read another person’s email but was using its response as research.

But in the hands of a criminal, this technique could have been used to steal sensitive data from someone’s email inbox, Rehberger said.

The attack wouldn’t have affected most ChatGPT accounts. It worked because Rehberger was using a beta-test feature of ChatGPT that gave it access to apps such as Slack, Gmail and others.

‘Prompt injection’: A new form of cyberattack

What Rehberger did with ChatGPT is called ‘prompt injection’ which is a technique of a new class of cyberattacks that are increasingly important as technology companies place a new generation of AI software into their businesses and consumer products.

Prompt injection involves manipulating or injecting malicious content into prompts to exploit the system.

Watch |Gravitas: Is AI going to replace your brain?

All the AI models are driven by plain language instructions, called prompts that help them create answers that are remarkably accurate.

Some of these instructions tell the AI systems not to do bad things, like reveal sensitive information or say offensive things, but hackers like Rehberger have found unexpected ways to override them.

There has been a surge in prompt-injection attacks since ChatGPT’s release last November. People have used the technique to trick the chatbot into revealing details about how it operates, saying disturbing or embarrassing things.

Tampering with AI-models

With the increasing use of AI models, hackers are also improvising themselves to break into these models.

One such form of attack is data poisoning, which is becoming more common and disinformation experts are wary about them. In this method, a hacker tampers with data used to train AI models, causing misleading results.

An AI model that was created to ease human burden can become a new-age tech weapon of hackers, with increasing use of generative-AI systems by technology products.

As Eliezer Yudkowsky, one of the leading AI researchers from the US, once famously said, “The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else.”

(With inputs from agencies)

WATCH WION LIVE HERE