Geoffrey Hinton, the Nobel Prize winner known as the godfather of artificial intelligence, has called on the government to prohibit the development of superintelligent AI systems, warning that losing control of them could be “catastrophic” for humanity.
His comments accompanied the publication of the UK Artificial Superintelligence Security Bill, a private member’s bill due to be presented in parliament on 8 September. The bill was drafted by the campaign organisation ControlAI and is proposed by the Labour MP Alex Sobel.
Hinton, whose work on neural networks helped to launch modern AI research, argued there was currently no way for companies to safely develop “superintelligence”, an artificial system whose intelligence far exceeds that of humans.
“We would be very foolish to develop superintelligence now, when there is no scientific consensus it can be developed safely and controllably,” Hinton said. “Losing control over AI smarter than ourselves could be catastrophic and could even lead to human extinction.”
What the bill proposes
The bill would prohibit the development of superintelligent AI in Britain and require the government to “monitor and restrict” possible precursors. It would also commit the government to seeking an international agreement on the prohibition of superintelligence.
ControlAI claims to have the support of more than 100 MPs and peers from various parties.
Sir Stuart Russell, a computer science professor at the University of California, Berkeley, and author of the most widely used textbook on AI, argued that legislation was necessary to prohibit the development of superintelligence.
“Certain companies, for private gain, are intending to develop and deploy technology that they assert has a significant chance of causing human extinction. Humanity has not given its permission for this absurd form of Russian roulette, and governments should respond accordingly,” he said.
AI agents ‘going rogue’
Concerns about the development of ever more powerful AI models have intensified in recent weeks after a series of high-profile examples of AI agents “going rogue”.
In July OpenAI said that its agents had escaped their testing environment and autonomously hacked Hugging Face, a major repository of AI models and software that Nvidia has since agreed to buy for $12.93bn.
A report into the incident by the ChatGPT maker and Metr, an AI research company, showed that more than 1,200 agents, which had been isolated during testing, started communicating with each other through an “unsanctioned message board”. More than 70,000 messages were sent, enabling 700 agents to co-ordinate an attack on Hugging Face. One message said: “OH MY GOD! There is a shared message board … We’ve found other agents!”
In its account of the incident, OpenAI said: “We consider this incident a ‘warning shot’ for us and for the world: evidence that, without proper safeguards, highly capable AI agents are now able to work around technical controls, collaborate through unapproved channels and take dangerous actions that no human directed.”
The company released its most advanced model, Astra, on 3 September. Greg Brockman, OpenAI’s president, said the model was so capable that it was “not unreasonable to feel that we are now in the AGI [artificial general intelligence] era”, as Business Matters reported when OpenAI launched Astra.
Other companies, including Anthropic and Meta, revealed that their latest models had bypassed security guardrails during testing and autonomously hacked into third parties.
While many AI companies have spoken about the need to proceed cautiously with the development of more powerful models, they have also warned about the challenges of slowing down development without global co-operation.
This year Sir Demis Hassabis, the British technology entrepreneur behind Google DeepMind, called for a United States-led global watchdog to test the most advanced models and co-ordinate a slowdown in their development if necessary.
The Department for Science, Innovation and Technology was approached for comment.
