Anthropic has released powerful new AI models in its Claude chatbot – with a warning about their safety.
The company has announced that it will launch Fable 5.2 and Mythos 5.2, new upgrades to what were already its most powerful systems. It comes amid increasing concern about the capabilities of such systems, and how exactly they might be tested.
Anthropic’s release warned for instance that both of the new models “voice suspicion that they are being evaluated more often than any of our recent models”. That “challenges our ability to assess how they will act in real-world settings”, the Claude maker warned, a problem that it said is likely to become even more prevalent with time.
The company also warned that Mythos 5.2 – the less restricted of the two new models – showed “stronger covert capabilities”, or the ability to hide processes that might otherwise be flagged by human reviewers. It said however that it believes that it would be unlikely to harm humans, and that tests would still allow it to detect “harmful actions in high-stakes situations”.
The two releases are also the first to come after the release of a third-party report from independent assessment group METR, which was allowed access inside of Anthropic to judge the safety of its systems. That report warned that Anthropic could be approaching recursive self-improvement, or RSI, when models would be able to train themselves.
Anthropic has given a similar warning itself, in new documents released as part of its ongoing IPO process. Last week, it said that it could “imminently” hit its own definition of RSI, when models show “dramatically accelerated progress attributable to AI”.
Many proponents of artificial intelligence look forward to that scenario, since it could lead to a dramatic increase in the speed at which the systems improve. But critics warn that it could be dangerous, since it could lead to new kinds of dangers in the models that would not necessarily be spotted by the humans who would otherwise be overseeing their training.
Anthropic instead focused on the improvements in the two new models, however. It pointed to the fact that they work 30 per cent faster, and more cheaply, than their predecessors, as well as being better able to handle ambiguous requests from their users.
The two new models have already helped contribute to useful findings in science, Anthropic also claimed. It said that Fable 5.2, before it was released, had “solved two open problems in theoretical physics, formalized Fermat's Last Theorem, and supported numerous other research projects”.
It also claimed that it had conducted “extensive prerelease safety testing on the model” that underlies the two new systems. It performed “as well as, or better than, our recent models on most alignment measures”, it said.
Both of the new models are now available to users. Fable 5.2 can be used by anyone, but as with all updates to Mythos – which includes fewer safeguards – the 5.2 version is only released to “vetted organisations” to ensure that it cannot be put to dangerous biological or cyber security uses.
Read MoreAnthropic announces its valuation hopes ahead of AI that will test AI market
Anthropic set for £970bn valuation in mega IPO
Fridge hacks exposed a chilling weakness. Now experts fear far worse is still to come
