From Silicon Valley to Washington DC, the tech industry is obsessed with distilling AI

Earlier this year, Google AI head Jeff Dean went on a podcast to discuss distillation, a concept that at the time was little talked about outside of the volatile tech world.

Speaking about the development of Google’s AI models, Dean said he and his colleagues discovered the art of distilling artificial intelligence as Google sought to improve the performance of its systems without relying on one large image recognition model.

“Distillation is an important technology for improving the functionality of small models, but to distill it into small models, we need to have frontier models,” Dean said in February.

Five months later, distillation suddenly became a hot topic from Silicon Valley to Washington, DC. Technologists and lawmakers have debated whether distillation poses a national security threat and could allow China to catch up with the United States in a high-stakes AI race. Concerns grew late last week after Chinese research institute Moonshot AI released Kim K3, and users quickly realized it could compete with the best AIs on the market from Anthropic and OpenAI.

Unlike big U.S. AI companies that sell access to their proprietary models, Moonshot and other Chinese labs offer so-called open-weight models that allow users to download the technology, tweak it, and run it wherever they like.

Some government officials attribute Moonshot’s ability to catch up so quickly to distillation, specifically the theft of U.S. intellectual property by incorporating Anthropic’s Frontier Fabre model.

“We have information that the Moonshot AI extracted the Anthropic fable for the development of the K3 model,” White House adviser Michael Kratsios wrote on X on Wednesday. “To do this, they have developed a sophisticated internal platform to perform large-scale distillation on the US model, which allows them to quickly switch between multiple access methods to avoid detection.”

Broadly speaking, distillation refers to using the answers from a chatbot or the work product of an advanced AI model to train another model. This practice is controversial. Because model developers may be able to create competitive products simply by using the work of companies that have invested millions or billions of dollars in developing the most sophisticated training technologies, depending on how they are used.

“It’s like someone attended a lecture, read a textbook and did their homework hard,” said Pukal Hamal, founder of AI security company Security Pal. “Then other students say, ‘Hey, I didn’t do that. Can I copy your work?'”

Whether it was Kratsios’ post or anything else, executives from some of the biggest technology companies on the planet came together on Friday in an unprecedented way to make their position clear. After a series of social media posts throughout the week, tech giants Nvidia, Microsoft, Meta, and Palantir joined with more than 20 other companies to release a letter urging policymakers to avoid “premature restrictions” on promiscuous AI models that would “inhibit competition or drive innovation abroad.”

“Distillation, the practice of using the output of one model to help train or improve another model, is a widely used technique for improving, evolving, and validating models,” they write.

The emergence of distillation technology poses a challenge for U.S. policymakers, who have long been concerned about Chinese technology both from the perspective of intellectual property theft and national security concerns.

Colin Shea-Bleimeier, a researcher at Georgetown’s Center for Security and Emerging Technologies, said the U.S. government is trying to figure out its position.

The government could argue that Chinese and Russian companies “have an unfair advantage because they have leveraged the fruits of the industrious American model to improve their own performance,” Schiabreimayer said.

box CEO Aaron Levy was one of the signatories of Friday’s letter. In an interview, Levy said that to remain competitive, U.S. companies need access to the best technology, regardless of where it was developed.

“In general, the more innovation we see coming from the US, China, etc., the more advances we should expect in AI, and generally AI will move towards lower costs and more efficiency over time,” Levy said.

While much of the current discussion centers on China’s open-weight AI models like Kimi K3, many companies are incorporating distillation techniques in creating their own models, said Shashi Bellamkonda, research director at Info-Tech Research Group. For example, Nvidia used distillation as part of the training process for its Llama Nemotron series of models, as detailed in an accompanying research paper.

“Training a smaller, cheaper model based on the output of a larger model is a legitimate and very valuable technique, and is practiced all the time,” Bellamkonda said.

But Anthropic has a different take. Because the company understands how its model is being used and has a fast-growing business to protect. The company announced in February that its Claude functionality was being distilled on an “industrial scale” by China’s DeepSeek, Moonshot and MiniMax, using approximately 24,000 fake accounts and generating 16 million exchanges.

Anthropic, which has a market capitalization of nearly $1 trillion and aims to go public in the near future, said stopping illegal distillation is a matter of national security.

“U.S. companies like Anthropic are building systems to prevent state and non-state actors from using AI to develop biological weapons and conduct malicious cyber activities,” the company said in a February post. And stopping it will require “swift and concerted action among industry players, policymakers, and the global AI community.”

OpenAI and Anthropic prohibit distillation in their terms of service. Bellamkonda said they are essentially suggesting that using their large models without permission could be intellectual property theft.

But as the cost of AI continues to rise, companies will do everything they can to increase efficiency.

Hamal said that Security Pal, which uses AI to automate security assessments, could use Chinese open-class models like the Kimi K3 without any problems. That would save a lot of money, he says.

“We make sure there are no malicious backdoors in the code,” Hamal said. “But once you’ve done the evaluation, why host it on your own infrastructure?”

One of the big problems with Anthropic and OpenAI trying to claim IP theft is that both companies rely on other content sources to build their models and are being sued for it.

Max Pritt, an attorney at Boies Schiller Flexner who represents book authors in copyright lawsuits against AI companies, said the government is in the same position.

“While the administration has focused, at least publicly, on protecting the intellectual property of technology companies, it has remained largely silent about the intellectual property of creators and individuals used without their permission,” Pruitt said.

WATCH: Chinese AI companies are finding ways to monetize their models even when they remain open