Nvidia A number of tech giants on Monday launched new AI safety initiatives focused on open models, as the fallout from cyberattacks from fraudulent OpenAI models continues.
Last week, it was revealed that the startup targeted in the attack, Hugging Face, was unable to use the US’s key frontier model to protect itself because guardrails did not differentiate between attackers and defenders. Instead, it turned to the self-hosted open-weight China model, which is not bound by the same restrictions.
The tech giants have launched initiatives aimed at building and sharing open AI tools, as U.S. lawmakers begin to consider how to curb the growing adoption of China’s AI models, the most advanced of which is open weight.
Open models can be downloaded, modified, and self-hosted, as opposed to closed models (including Frontier systems built by Anthropic and OpenAI) that are only accessible through specific infrastructure.
“The Open Secure AI Alliance strives to use open technologies to remediate and disclose vulnerabilities,” Nvidia said in a statement. “The recent Hugface security incident was a stark reminder that cyber defenders need open frontier agent systems to protect themselves.”
In addition to Nvidia, other members of the alliance include: microsoft, space x, Palantirand dozens of other technology companies in the United States and Europe.
China’s efforts to curb AI
There are growing calls for measures to restrict access to models built by Chinese AI companies. Chinese AI companies have been accused of running campaigns to extract information from the systems of American rivals, a process known as “distillation,” where one model extracts knowledge from a better-trained model.
Treasury Secretary Scott Bessent last week threatened sanctions against Chinese companies that have carried out distillation attacks on American companies.
Chris McGuire, senior fellow for China and emerging technologies at the Council on Foreign Relations think tank, told CNBC that “there is a real possibility that the U.S. government will impose restrictions on the Chinese model.”
This could include a ban on transactions involving the models, such as US companies that buy tokens via APIs or host models on the cloud and charge customers inference fees, he said.
“In Washington, this is not a debate about open source versus closed source, it’s a debate about whether we tolerate Chinese intellectual property theft or not,” McGuire said. “Any action would focus on Chinese companies, not the open source ecosystem.”
But the most capable open source models are largely built by Chinese companies, so there are concerns about limitations.
Last week, Nvidia, Microsoft, metaPalantir and more than 20 other companies published a letter asking policymakers to avoid “premature restrictions” on promiscuous AI models that “thwart competition or promote innovation abroad.”
Nvidia said the OpenAI-Hugging Face incident showed a “realistic truth.” “If defenders cannot inspect, adapt, and run advanced AI on their own infrastructure, they will be limited in their ability to respond in the moments when speed matters most.”
