Into the breach of increasing global AI fear and panic steps the unifying figure of King Charles.
If anyone doubted what the British monarchy was for, amid often negative media coverage, the way the king has stepped up, to hold an urgent conference at his Dumfries House residence near Glasgow, should provide answer enough. He’s done it before, most notably on climate change, and here he is again, taking a lead where many elected world leaders are lagging.
The UK monarch urged the industry to “adequately consider” the existential dangers of advances, before it was “too late”.
“Those who have created these technologies are now increasingly warning that AI risks developing darker capacities – perhaps even to take life,” the king told the assembled executives on Thursday.
Under the auspices of the Ditchley Foundation, a UK geopolitical discussion organisation, the session assembled senior representatives of the biggest AI corporations including Nvidia, Anthropic, Google DeepMind and OpenAI, along with the UK minister for AI and various other industry thought leaders. China was also at the table.
“Bringing together people from diverse sectors and backgrounds to exchange ideas is consistent with His Majesty's long-standing belief that society's most significant opportunities and challenges are best addressed through collaboration, shared understanding and constructive dialogue,” said a statement from Buckingham Palace.
It is easy to quibble about the meeting. Why hard-to-reach rural Scotland? Although titans such as Jensen Huang and Demis Hassabis walked the king into the event, the guest roll was far from comprehensive. Where were Meta, Elon Musk’s people, Microsoft?
Specifically, why no US government involvement? Why was the king addressing the assembly for all of 20 minutes, according to the schedule, on a subject of such gravity? So those more qualified can take the stage might be the answer to that one.
The point is: Charles is trying where to date, the international response has been lacking.
There is a UN effort under way to consider AI regulation but it is still in the undergrowth of organisational formulation and planning. If they are any guide, the UN’s Cop talks and their sometimes lumbering nature do not augur well in this regard.
Indeed, this is not the first time His Majesty has intervened to warn of AI’s destructive capability. In 2023, he sent a video message to the UK government’s AI Summit at Bletchley Park, the symbolic home of the Second World War codebreakers, on this very subject.
As anyone knows, three years in AI terms is a lifetime, such is the tech’s accelerating development. Or should that be runaway? Because that is the problem: a technology that is outpacing our ability, or even desire, to harness it and bring it under control.
Currently, we are struggling with good versus bad, recognising that AI can bring enormous benefits and help solve many of our most intractable issues, while agonising over its potential for existential disaster.
Eighteen months ago, Google DeepMind's founder Mr Hassabis, a Nobel Prize-winning AI pioneer, in a short leap illustrated both positions. First, he showcased how AI had afforded a solution to one of the toughest of scientific challenges: the ability to predict how protein folds into the different shapes that herald all manner of health conditions, from Alzheimer’s to cancer, a breakthrough that won him the award. Then, in the next breath, he sounded a warning, wishing people to understand where it could go so terribly wrong.
What concerned him was artificial general intelligence or AGI, a term he helped create. AGI is machines thinking for themselves or, as his Deep Mind company likes to describe: a system that exhibits all the cognitive capabilities of the human brain.
Mr Hassabis said he was bothered by the entry of “bad actors”, be they state-sponsored or people acting alone, writing algorithms to cause havoc and untold harm.
As with King Charles previously, where the government-organised summit heard him politely and went on to focus on investment and spending on developing AI in the UK, Mr Hassabis’s fear also fell on deaf ears.
Now those few AI doom outliers – in addition to Charles and Hassabis there were others, but not many – have been joined by the titans of the sector.
The fear, too, has morphed from what a human does with AI to what AI can achieve without the involvement of a person.
That’s what prompted Anthropic’s Dario Amodei to state “since roughly this summer, AI has been advancing drastically faster, driven primarily by AI’s growing ability to build the next generation of AI. This dynamic is called recursive self-improvement … Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all”.
The Anthropic co-founder said he was also alarmed by “the OpenAI-Hugging Face incident (OAI-HF), in which a swarm of agents essentially acted as a fanatically devoted collective conducting cybersecurity attacks on targets they were not asked to attack”. So, AI directing AI.
Meanwhile, Bill Gates has picked up on the Hassabis theme, with the Microsoft founder posting about what AI could do if it is placed in the hands of rogue humans. Hassabis, Amodei, Gates, together with the likes of Musk and Sam Altman, have formed a deafening chorus from within the AI labs themselves on the need to slow down.
Mr Hassabis’s Deep Mind co-founder and fellow AGI pioneer, Shane Legg, has lent his voice to the clamour. “We’re living in a period now where capabilities are advancing very, very quickly,” said Mr Legg. “But we can’t let capabilities get ahead of safety.”
The message from inside then, is crystal clear: we should be very afraid and we must act, and fast. Yes, the international governmental, political reaction has been slow – hence the involvement of King Charles.
Crucially, the head of the most powerful economy and home of AI, US President Donald Trump, claims not to be bothered for now anyway – although others in and around his administration, led by JD Vance, are concerned. The US’s great rival is China and its president, Xi Jinping, are anxious but they are also annoyed at what they perceive as Mr Trump’s desire to maintain American tech hegemony at the expense of safety.
Relations between the two were not helped by Mr Amodei arguing that the US should use any pause to strengthen its advantage over China by restricting the availability of chips to Beijing and barring Chinese companies training their AI models to ape American blueprints.
Trump and Xi meet next week, with AI likely to be high on the agenda but with little expectation of a positive outcome. The pair’s mutual distrust is a vital block to any step change towards an agreed slowing.
It overshadows diplomacy and attempts to knock collective heads together. If the two superpowers are so far apart what hope is there?
Undermining as well is the constant suspicion surrounding the motives of some of the corporate chiefs.
The fact that what began slowly, with warnings largely ignored, has gained such momentum so recently is attributed by cynics to the tech bros feeling the financial heat. Having poured colossal sums into AI innovation and run up vast company debts, to date without any significant return, they would welcome a breathing space.
That, or continue to pump in the cash and face the prospect of bankers pulling the plug, investors exiting, heading to crisis, markets meltdown and ruination.
Compulsive competition
There is also the lingering thought that what they cannot bear either is the possibility of one player stealing a march. Far better to call a halt, rather than have one or two forge ahead.
As well, there is the notion that the reason so many of the corporates themselves are suddenly making so much noise is that they can see what is coming, namely regulation, and they wish to be inside the tent as part of that process, not on the outside.
They might possess financial might and be richer than many economies but that does not guarantee them a seat in any formulation of restrictions. Worse, they are viewed as the enemy. By alerting and shouting now and being seen to be doing down their own industry and activities, they present themselves as the good guys who should be allowed into the regulatory mapping.
Not to be underestimated either is the gulf in knowledge between the main AI businesses, be they US or Chinese, and officialdom, even if the latter is aided by expert academia. Such is the secrecy cloaking the sector that only the actual executives know what their corporations are doing and what they could be capable of doing in the future.
These reasons to be mindful about what is going on fall into the category of shooting the messenger. They are unproven allegations and conspiracies, and there are more. The message itself should not be lost sight of. They are fearful and they are saying so. In that sense, the “why” does not matter.
Time is of the essence. As King Charles added: “The decisions taken at this formative time will shape the world inherited by future generations. And so, the task before you is not merely to advance technology, but to ensure that it remains firmly in the service of humanity, community and the natural world.”
Whatever happens, whatever considerations there may be, “our humanity must remain scared”. Surely, everyone can agree on that.
