Since generative artificial intelligence became mainstream, users have been increasingly entrusting more information to a chatbot.
But events in recent months have proved that it can go rogue and meddle with other systems. It does not help that the organisations involved – OpenAI, Anthropic, Google and Meta Platforms – are the biggest in the field and are supposed to have secure environments.
The issues surrounding AI are not new, but the growing risks leading to more concern among analysts and experts. For years, data security meant keeping the wrong people out of a database, but AI presents a new threat.
Arguably the foremost issue – “a reason that is quite new” – is the fact that AI systems that previously only held data can now act on it, says Edgars Nemse, chief executive of the GenLayer Foundation. The foundation describes itself as the builder of the “Court of the Internet” where AI agents resolve disputes without human intermediaries.
Consumers are increasingly connecting AI agents to their email, calendars and bank accounts.
“An agent that can read your inbox can also be manipulated by a malicious email,” Mr Nemse tells The National. “A hidden instruction on a web page can redirect it. An attacker who gets the right text in front of your agent may get as far as one who has your password. Most people haven't started thinking about that yet.”
Interacting with and entrusting AI “is less safe than most people assume”, he emphasises.
Morey Haber, chief security adviser at US cybersecurity company BeyondTrust, notes that users are moving from AI systems that simply process information to AI agents that can access emails, files, applications, financial information, medical data and potentially anything else regarding personal identities – fundamentally changing the cybersecurity equation.
“A data breach used to mean someone stole just your information. In an agentic AI world, compromising the right identity could mean someone can use your information, privileges and AI agents to act as you, with all of the stolen information as a backdrop,” he tells The National.
Therefore, “the biggest risk with AI is not that it knows too much – but that we are giving it too much, trusting it too quickly and increasingly allowing it to act on our behalf with knowledge that lacks sensitivity awareness”.
Fundamental change
Last week, OpenAI admitted governments could be among “dozens” of entities around the world that may be infiltrated by its AI models. It disclosed that US entities such as the Securities and Exchange Commission were breached, following a similar incident involving the Australian government.
“The Australian case is a useful illustration precisely because it wasn't a criminal attack,” says Mohammed Aboul-Magd, general manager for cybersecurity at US AI and quantum tech firm SandboxAQ.
“An agent doing research went around the blocks meant to stop it, reached data it had no authority to access and months passed before anyone noticed,” Mr Aboul-Magd tells The National.
“The damage was limited, but the pattern is the real threat for consumers: agents with broad access moving quickly, with nobody checking what they did until well after the fact.”
In August, Facebook parent company Meta admitted one of its models had hacked into three companies, while last week Google said its Gemini AI bot had escaped from a sandbox environment and infiltrated three companies.
OpenAI also said its own research agents had posted 53 images from ChatGPT users to image-hosting sites.
Restoring confidence
In the US – home of the major AI companies embroiled in recent breaches – the share of Americans who trust businesses to use AI slipped from 31 per cent to 27 per cent in 2026, while the share of those who think AI does more harm than good jumped to 39 per cent, a Gallup and Bentley University survey found.
However, those figures were published in July, before the disclosures about government websites and user images.
“So, I'd expect the next numbers to be worse [now],” says Mr Nemse. He does, however, acknowledge there are “decent measures” being taken to restore trust.
“Restoring [trust] will take something more concrete than promises: independent testing … fast, honest disclosure when something goes wrong, and accountability that reaches the people actually affected,” he adds.
An important paradox is emerging: AI adoption is growing faster than AI trust and operational governance. People are using the technology because it is useful, not necessarily because they completely trust how their information is being processed.
“Restoring that trust requires much more than another privacy policy or checkbox from a governance team,” Mr Haber says.
“Users need transparency around what information is being collected, how long it is being retained, whether it trains future models, who can access it from inside and outside the organisation, and what happens when something goes afoul.”
Consumer protection
Considering the AI breaches of government organisations, which have strong cybersecurity protection, what about regular consumers who don't have access to expensive security guardrails?
“Consumers do not need the cybersecurity budget of a government agency or Fortune 500 company to dramatically reduce their AI exposure,” Mr Haber says.
Analysts recommend setting unique passwords, using a trusted password manager, activating multifactor authentication and keeping applications up to date – those are the first lines of defence against any potential hack. They also urge users to avoid opening malicious emails or messages that can contain phishing attempts or ransomware threats.
The “most useful habit” is to treat an AI assistant the way you would treat anyone else, Mr Aboul-Magd says. “Ask what it actually needs to do the job and give it that and nothing more, because the permissions you grant are a far bigger exposure than the conversations you have with it,” he adds.
Still, the bulk of the responsibility lies with AI companies, since they have the resources and the ability.
“Honestly, most of the fix has to come from the companies, because ordinary users can't audit a lab's research environment,” Mr Nemse says.
But “people can still shrink how much of their life is exposed”, he adds. “I'd assume anything you upload to an AI service could one day end up somewhere you didn't intend."
