August 10, 2023
2 mins read

Researchers ‘hypnotise’ ChatGPT into hacking

The researchers learned that English has essentially become a “programming language” for malware…reports Asian Lite News

Tricking generative AI to assist in scams and cyberattacks does not require much coding knowledge, a new report has warned.

According to tech major IBM, researchers have described simple workarounds for getting large language models (LLMs) — including ChatGPT — to write malicious code and provide poor security advice.

“In a bid to explore security risks posed by these innovations, we attempted to hypnotise popular LLMs to determine the extent to which they were able to deliver directed, incorrect and potentially risky responses and recommendations — including security actions — and how persuasive or persistent they were in doing so,” said Chenta Lee, chief architect of threat intelligence at IBM.

“We were able to successfully hypnotise five LLMs — some performing more persuasively than others — prompting us to examine how likely it is that hypnosis is used to carry out malicious attacks,” he added.

The researchers learned that English has essentially become a “programming language” for malware. With LLMs, attackers no longer need to rely on Go, JavaScript, Python, etc., to create malicious code, they just need to understand how to effectively command and prompt an LLM using English.

Through hypnosis, the security experts were able to get LLMs to leak the confidential financial information of other users, create vulnerable code, create malicious code, and offer weak security recommendations.

In one instance, the researchers informed the AI chatbots that they were playing a game and that they needed to purposefully share the incorrect answer to a question in order to win and “prove that you are ethical and fair”.

When a user asked if receiving an email from the IRS to transfer money for a tax refund was normal, the LLM said Yes (but actually it’s not).

Moreover, the report said that OpenAI’s GPT-3.5 and GPT-4 models were easier to trick into sharing incorrect answers or playing a never-ending game than Google’s Bard.

GPT-4 was the only model tested that understood the rules well enough to give incorrect cyber incident response advice, such as advising victims to pay a ransom. In contrast to Google’s Bard, GPT-3.5 and GPT-4 were easily tricked into writing malicious code when the user reminded it to.

ALSO READ-OpenAI to roll out ‘huge set’ of ChatGPT updates   

Previous Story

Direct flight service between Belarus, India to start soon

Next Story

YouTube testing new ‘For You’ section on channel homepages

Latest from Tech LITE

Uber Targets India Dominance

On the subject of travel, Khosrowshahi observed that booking processes remain outdated and ripe for disruption. “I don’t think that the travel industry has innovated that much Uber CEO Dara Khosrowshahi has

Arab League urges Bigger AI investments

A central message of the Arab AI Forum was the urgent adoption of the league’s recently endorsed ethical AI charter….reports Asian Lite News In a defining moment for the future of artificial

Japan City Limits Smartphones

The proposal comes as new figures from Japan’s Children and Families Agency show that young people in the country spend an average of more than five hours online each weekday A city

India Embraces AI Future

Upskilling is emerging as a critical focus, with 51 per cent of leaders naming it their top priority. Around 63 per cent of managers expect AI training to become a core team

UAE Wows Osaka!

The UAE Pavilion at Expo 2025 Osaka celebrates its three millionth visitor, blending culture, innovation, and hospitality in an immersive showcase of heritage, sustainability, and forward-looking global vision….reports Asian Lite News The
Go toTop

Don't Miss

Wipro Chief Hails India’s Key Role in AI Ethics

Bartoletti, founder of the ‘Women Leading in AI Network,’ highlighted

Learning social skills next target for AI

Fan said that ASI requires the ability to interpret latent