Anthropic announced this week that it has restricted the release of its latest model, called Mythos, because it is too good at finding security vulnerabilities in the software used by users around the world.
Instead of making Mythos available to the public, Frontier Labs plans to share it with a group of large companies and organizations that operate critical online infrastructure, from Amazon Web Services to JPMorgan Chase.
OpenAI is reportedly considering similar plans for its next cybersecurity tool. The ostensible idea is to preempt these large companies from malicious attackers who might leverage advanced LLM to compromise their secure software.
But the “e-word” in the sentence above suggests there may be more to this release strategy than just cybersecurity, or hype about the model’s capabilities.
Dan Lahav, CEO of AI Cybersecurity Institute Irregular, told TechCrunch in March, before the release of Mythos, that finding vulnerabilities with AI tools is important, but the specific value of weaknesses to attackers depends on many factors, including how they are combined.
“The question that’s always on my mind is, have they found something that they can exploit in a very meaningful way, individually or as part of a chain?” Rahab said.
Anthropic says Mythos can exploit far more vulnerabilities than its previous model, Opus. But it’s not clear whether Mythos is actually the end-all of cybersecurity models. Aisle, an AI cybersecurity startup, said Anthropic was able to replicate much of what it claims Mythos accomplished using a small, promiscuous model. Ayle’s team argues that these results show that there is no single deep learning model for cybersecurity, but that it depends on the task at hand.
Given that Opus was already considered a cybersecurity game-changer, there’s another reason Frontier Labs wants to limit its release to large enterprises. Opus creates a flywheel for contracts with large companies while making it difficult for competitors to copy the model using distillation. Distillation is a technique that leverages frontier models to cheaply train new LLMs.
“This is a marketing cover for the fact that top-of-the-line models are currently restricted by corporate contracts, making small labs unavailable for distillation,” suggested David Croshaw, software engineer and CEO of startup exe.dev, in a social media post. “By the time you and I can use Mythos, there will be a new top-end product that is exclusive to Enterprise. That treadmill will help keep Enterprise funding (the bulk of the dollars) flowing by relegating distillation companies to second place,” Crawshaw said.
This analysis is consistent with what we are seeing in the AI ecosystem. That is, a competition between Frontier Laboratories, which develops the largest and most capable models, and companies like Aisle, which rely on multiple models and see open source LLMs, often made in China and allegedly developed by distillation, as a path to economic advantage.
Frontier Labs has taken a tougher stance on distillation this year, with Anthropic publicizing attempts by Chinese companies to copy its models and Anthropic, Google and three major open AI labs working together to identify and stop distillers, Bloomberg reports.
Distillation is a threat to Frontier Labs’ business model because it eliminates the benefits of using large amounts of capital to scale. So while blocking distillation is already a worthwhile endeavor, a selective release approach to doing so also provides labs with a way to differentiate their enterprise offerings, as the category is key to profitable deployment.
It remains unclear whether Mythos and other new models truly threaten the security of the Internet, and carefully deploying this technology is the responsible way forward.
Anthropic did not respond to our questions at the time of writing about whether this decision was also related to distillation concerns, but the company may have found a smart approach to protecting the internet and its revenue.
