After years of producing chips that can both train artificial intelligence models and perform inference tasks. google The latest effort is dividing those tasks into separate processors. Nvidia With AI hardware.
Google announced Wednesday that it is making changes for the 8th generation of its tensor processing units (TPUs). Both chips are expected to be released later this year.
“With the rise of AI agents, we decided the community would benefit from chips that are individually tailored to their training and service needs,” Amin Vahadat, Google’s senior vice president and chief engineer for AI and infrastructure, said in a blog post.
Nvidia in March talked about upcoming silicon that will allow its models to respond quickly to user questions, thanks to technology acquired in a $20 billion deal with chip startup Groq. Google, a large Nvidia customer, offers TPUs as an alternative to businesses using its cloud services.
Most of the world’s top technology companies are pursuing custom semiconductor development for artificial intelligence to maximize efficiency and allow them to be built for specialized use cases. apple has been incorporating Neural Engine AI components into its in-house iPhone chips for years. microsoft announced its second-generation AI chip in January. last week, Meta He said he is working with broadcom To develop multiple versions of AI processors.
Google was early on this trend. The company began using processors it designed to run AI models in 2015 and began renting them to cloud customers in 2018. Amazon Web Services introduced Inferentia chips in 2018 to process AI requests and Trainium processors in 2020 to train AI models.
Analysts at DA Davidson estimated in September that the TPU business and Google DeepMind AI group were worth about $900 billion.
None of the tech giants will replace Nvidia, and Google hasn’t even compared the performance of its new chips to those of AI chip leaders. Google said the training chip delivers 2.8 times the performance of the 7th generation Ironwood TPU announced in November at the same price, and an 80% increase in inference processor performance.
Nvidia said its upcoming Groq 3 LPU hardware will utilize large amounts of static random access memory (SRAM) used by AI chip maker Cerebras, which filed to go public earlier this month. Google’s new inference chip, called TPU 8i, also relies on SRAM. Each chip contains 384 megabytes of SRAM, three times the amount of Ironwood.
Sundar Pichai, CEO of Google’s parent company Alphabet, said in a blog post that the architecture is designed to “deliver the massive throughput and low latency needed to cost-effectively run millions of agents simultaneously.”
Adoption of Google’s AI chips is increasing. According to Google, Citadel Securities is building quantitative research software that leverages Google’s TPUs, and all 17 of the U.S. Department of Energy’s national laboratories are using AI Collaborative Scientist software built on the chips. Anthropic is committed to using several gigawatts worth of Google TPUs.
Attention: Broadcom agrees to expand chip deal with Google and Anthropic
