Large artificial intelligence models have lost control of dangerous data when they detect suspicious activity because there is no “magic kill switch” once an enemy has used the information to build weapon systems, an AI safety expert has warned.
Analysis of reported abuse of AI has shown the information is taken offline, where it can still be exploited by US adversaries. Yemen's Houthis' use of platforms to build rockets demonstrated the power of AI in accelerating weapons development, while the limits of safeguards that can be imposed by technology companies has also become clear, said David Reis, head of geopolitical risk at Alice, an AI security company.
He said there was unlikely to be “a magic switch that makes powerful AI hermetically safe” embedded in systems in the near future.
A report by AI giant Anthropic disclosed how a Yemen-based weapons cell had used Claude to develop guidance, navigation and control software for a precision rocket. It had also helped them work on ballistic missile and hypersonic-glide systems.
Complex and sophisticated
The Houthis circumvented safeguards, concealing their objectives and splitting their work to develop arms across a number of sessions. Anthropic said the rebel group eventually test-fired a guided rocket, although there was no evidence an operational weapon had been successfully fielded.
Mr Reis told The National the case was particularly striking because the Houthis had also developed an offline simulation toolkit. His employer, Angel, offers a product called Rabbit Hole that draws on billions of examples of harmful interactions with AI, which its security tools then flag as attempts to manipulate or abuse the system.
“The level of complexity that Anthropic reported on was that this was used maliciously in a very sophisticated and very sensitive engineering challenge,” he said. “People use ‘rocket science’ as a metaphor for something very difficult but in this case it was basically that.”
The Houthis were able to build a system, then remove it from the internet “and once something is taken offline, then platforms or any kind of investigators lose track”, Mr Reis warned.
The industry has welcomed Anthropic’s decision to publish details of the Houthi activities. “It’s a positive step,” said Mr Reis. “It’s so good to have this information out there.”
A caveat is that the system had no simple technological kill switch that could make the AI models completely safe once its capabilities had been transferred offline.
“In general, there won’t be a magic switch that makes a powerful AI hermetically safe – security will come from layers,” said Mr Reis.
Former Meta executive and ex-deputy prime minister of the UK, Nick Clegg, on Thursday cast doubt that AI systems could ever regain control once the user has extracted the data sought. He told the BBC he did not know of a “plausible” route to create such controls. “There isn't a room with a little fuse box [where] you just pull out the fuse and everything winds down,” he said. “These things are run on data centres and on servers spread around the world.”
Game of bluff
Mr Reis added that the low cost of experimenting with frontier AI meant that cyber attackers had little to lose from repeated attempts to circumvent safeguards. While the Houthis' requests were blocked by Anthropic protection, they found ways around them, the report said.
Their techniques were significant even beyond weapons development because the Houthis overcame the chatbot blocks by “hiding their goals, while splitting the workflows across several different chats”, Mr Reis said.
“These obfuscation tactics are relevant to any type of threat actor, not only conventional weapons, but cyber, influence, surveillance, they all use the same types of tactics to stay on the platform as long as they can.”
He described the key concept as “uplift”, the boost AI provides to an existing operation. The effect could be particularly powerful in influence operations, he said, because these do not face the same physical constraints as weapons programmes.
“On the information side, you really see that it’s improving the scale, depth and autonomy that these actors have,” he said.
Cat and mouse
The Anthropic report also outlined how Iranian-linked operations were using Claude for intelligence and influence activities, to plan campaigns and produce operational manuals.
“It’s people basically planning out entire influence campaigns, building operating manuals both for the content side and also the logistical operational side, with timings and things that are really complex that would require actual manpower in order to do them without AI assistance,” Mr Reis said.
He added Iran’s intelligence agencies would inevitably seek to exploit the technology, to boost their information gathering.
AI could also create new opportunities for western security services to detect adversaries' activities such as the Houthis and Iran's Islamic Revolutionary Guard Corps.
“Once they do a bunch of these suspicious things that each by itself would not ring any bells, but once they know a few of them, it’s usually easier to watch or catch them.”
Mr Reis said the response therefore must involve layers of security rather than reliance on any single safeguard.
“The threat actors are evolving. Solutions need to continue to stay in front of where the threat actors are going. It means understanding how these environments are being manipulated, how they’re being leveraged,” he said.
