King Charles warns of AI dangers as Jensen Huang pushes back on social media-style regulation

King Charles III warned the AI industry Thursday of existential risks to humanity posed by the rapid advance of artificial intelligence and urged that human control be preserved.

According to Reuters and the BBC, Charles hosted the gathering at Dumfries House in Scotland, welcoming Nvidia CEO Jensen Huang, Google Chief Scientist and DeepMind co-founder Demis Hassabis, and OpenAI Chief Financial Officer Sarah Friar, among others.

In his opening remarks, Charles said the pace and nature of AI's development were "as exciting as they are deeply concerning," adding that "our humanity must remain sacred" and that the public needed assurance "that we will not lose control of our destiny, our soul."

He noted that developers were increasingly warning of AI's darker potential — including the possibility of taking lives — and said there was "an urgency to examine the existential risk that such technology, in the wrong hands, could be used in ways that are deeply destructive."

"Of course, we must have sufficient controls in place before it is too late," Charles said. "The decisions made in this formative period will determine the world that future generations inherit." He urged those present that their task was "not merely to advance technology, but to ensure that it firmly serves humanity, our communities and the natural world."

The meeting came amid a growing chorus of calls within the industry for a slowdown in AI development. Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman have both raised or endorsed arguments for moderating the pace of development. The debate has spilled into politics, with President Donald Trump criticizing the slowdown argument, while some have questioned whether the leading players are advocating deceleration primarily to protect their own competitive advantage.

Huang, who attended the summit, struck a notably different tone from Amodei and others. A consistent opponent of AI development regulation, he argued at the meeting for what he called "responsible optimism" — the idea that developers themselves should bear accountability. "Those of us building AI must rigorously test our systems," he said. "When they are not safe enough, we should hold them back."

Huang also told reporters that "AI is very different from social media," according to Politico Europe. "Social media is a product, but AI is the technology behind every product, every technology — so regulate the product, not the AI technology," he said.

He said regulators should monitor AI products for failures and add new laws to existing frameworks where needed, stressing that "we must properly and rigorously test products before they are released to the public."

Hassabis told the summit that his primary concern was the misuse of autonomous AI systems by "bad actors" causing those systems to behave wrongly, Bloomberg reported. He said, however, that he "firmly believes in human ingenuity" and was confident the industry would resolve such concerns.

The summit produced no formal agreement. Reuters reported that Paolo Benanti, a papal adviser who has called for a slowdown in AI development, also attended.

Charles has long championed the climate change agenda but is not known as a technology expert. The gathering is widely seen as an effort by a leading Western figure to drive global discussion on one of the most pressing issues of the day.