Artificial intelligence has turned out to be much more expensive than anyone expected, and CFOs at major US companies now face a cruel new trade-off: token vs. human.
That was the picture the two enterprise AI CEOs at the center of the build described to CNBC this week. Their account of what goes on inside Fortune 500 companies paints a vivid picture of the threat rising costs pose to the AI industry. This is a risk that the market has yet to recognize as it hits record highs and creates new trillion dollar companies like: micron.
Arvind Jain, CEO of enterprise AI company Glean, told CNBC that the biggest topic for every company right now is overinflating AI budgets.
“Companies are telling us that their AI budgets are going to be used up in a month or two. This is an annual budget,” he said.
That’s because the cost of AI hasn’t fallen as much as buyers expected. In fact, it has gone up. Each new model released by Frontier Labs costs roughly twice as much per token as the model it replaces, putting enterprise AI on what Jain calls “a currently unsustainable path.”
“This is the first time I remember that the cost of technology is the same as the cost of labor. You’re comparing technology versus people,” he said. “Historically we haven’t had that conversation because technology is only a small part of the total cost of operating a business.”
Increasing AI budgets are increasingly replacing future headcount increases, he says.
Arvind Jain, CEO of Glean, takes to the SaaS Monster stage during day 1 of Web Summit 2022 at Altice Arena in Lisbon, Portugal on November 2, 2022.
Harry Murphy | Sports File | Getty Images
Matan Grinberg, CEO of Factory AI, which routes engineering work across all frontier AI models, described this change as a clear resource allocation issue currently underway within the leadership team.
“Companies say, if there’s one thing you can optimize, is it the number of employees or the AI spend per employee?” Greenberg said.
Greenberg said the company went through three different stages in about a year. The first was for the board to request that the CEO do something about AI. Then came so-called tokenmaxing, the use of AI by any means necessary regardless of cost. In the third phase, management is reevaluating their needs for the premium model.
“Do we need to use Opus-level intelligence for every task?” Greenberg said. “There’s no need for that.”
pay more than the repayment amount
The root of the squeeze is that the technology is working, but we are not yet being paid for it.
“The way AI works today is very powerful, but very inefficient,” Jain said. “Currently, the value of AI is less than the cost that companies are incurring.”
A big part of the problem is inefficiency in model selection. According to Jain, approximately 95% of enterprise AI usage is still performed on the most expensive frontier models, even for tasks that can be handled by cheaper alternatives.
The solution is simple. It’s about routing easy tasks to cheaper tiers. Jains say it is the easiest fruit.
“You can actually achieve 10x savings by routing the right model to the front,” he said.
This is also the argument behind Factory AI, which automatically sends each task to the best model for that task. The key, Greenberg says, is to realize that a job rarely actually requires a top-level job. He likened the gap between the latest frontier models to two veteran academics.
“Opus 4.7 and Opus 4.8 are like the difference between a professor who has been a professor for 13 years and a professor who has been a professor for 15 years,” Greenberg said. “For a layperson, it’s really difficult to tell the difference.”
The entire AI deal is based on a bet that historic demand will continue, with buyers largely indifferent to costs. But views from within Fortune 500 companies suggest that demand may be much more price-sensitive than the industry assumes.
Read more about what AI pricing means for the valuations of OpenAI and Anthropic, whose business models are built on premium pricing.
Watch: Full CNBC interview with Altimeter’s Pauline Yang
