Anthropic did not submit its most powerful artificial intelligence model to the UK government’s AI Security Institute for testing before releasing it last week, the first time the company has bypassed the British body.
Claude Mythos 5.1 was made available only to US vetting organisations. The model, designed specifically for cybersecurity and life sciences, is distributed through what Anthropic calls “trusted access programs” rather than released publicly.
In a blog post last week, the company said the model was “only available to a set of US organisations, though we’re co-ordinating with the US government to expand access to a broader set of domestic and international partners as quickly as possible”. The story was reported by the Financial Times.
Anthropic said access runs through two schemes: a Cyber Verification Program for defensive security professionals, and a Life Sciences Verification Program that it said was developed in partnership with the US government and has enrolled its first participants.
An earlier model in the same family, Claude Mythos, prompted crisis meetings among finance ministers and central bankers over fears the technology could be turned on the global financial system. Canada’s finance minister, François-Philippe Champagne, told media the model was “serious enough to warrant the attention of all the finance ministers”.
Export controls and access fears
The Trump administration imposed an export ban on the earlier generation of the model over the summer, after the US Department of Commerce flagged concerns that guardrails built into the models could be bypassed. Access was subsequently restored, but the move prompted fears that access to the world’s most advanced models could be cut off without warning.
The AI Security Institute was established in November 2023 by Rishi Sunak, who was prime minister at the time. Now a directorate of the Department for Science, Innovation and Technology, it tests and monitors the latest developments in AI models “to equip governments with a scientific understanding of the risks posed by advanced AI”, according to the institute’s own account of its work.
It is widely seen as a world leader in AI safety and has helped to shape international approaches to AI safety and security. Last week it tested OpenAI’s Astra model, the most advanced offering from the maker of ChatGPT, which was released last Thursday and which OpenAI classifies as having a “critical” level of cybersecurity capability.
The Cabinet Office said: “The AI Security Institute continues to collaborate closely with industry partners, including Anthropic, to make models safer.”
Researcher resigns over safety
Jacob Coxon, a researcher at Anthropic specialising in training new AI models, has resigned from the company, warning about the dangers of developing ever more powerful models.
Coxon said he had worked for three years at both OpenAI and Anthropic. “Neither company is acting with responsibility. They are racing straight to self-improving superintelligence and gambling with our lives,” he said on X.
Responding to Coxon’s resignation on X, Evan Hubinger, who leads Anthropic’s research on ensuring AI is aligned with human values and intentions, said there was a more than 10 per cent chance that AI could “kill all humans” in the next decade.
“I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to,” he said.
Separately, the UK Artificial Superintelligence Security Bill, a private member’s bill drafted by the campaign group ControlAI and proposed by the Labour MP Alex Sobel, was due to be presented in parliament yesterday. The bill would prohibit the development of superintelligent AI in Britain and require the government to monitor and restrict possible precursors.
Anthropic was contacted for comment.
