Wasted tokens and “chaotic” system

Despite executives’ enthusiasm for artificially intelligent agents that can handle administrative tasks such as sleepless interns, the underlying technology remains unstable and potentially costly.

That was made clear at two separate events in Silicon Valley this week. There, executives and engineers discussed the current excitement and challenges surrounding AI agents.

Kevin McGrath, CEO of AI startup Meibel, said during the session that “the biggest problems we’re currently dealing with in AI” include the mistaken belief that everything needs to be handled by large-scale language models (LLMs).

“If you give all your tokens and all your money to an AI Claw bot, you’re just wasting millions of tokens,” McGrath said, explaining that companies need to be more careful when deciding which tasks are best suited for AI agents.

The tech industry has been pushing AI agents as the next big thing ever since the recent rise of OpenClaw, a so-called “harness” that allows developers to create and manage fleets of digital assistants using different AI models.

Nvidia CEO Jensen Huang told CNBC’s Jim Cramer in March that this is “definitely the next ChatGPT.”

But on Wednesday, the Generative AI and Agenttic AI Summit in San Jose featured technical staff from companies including: google and its DeepMind AI unit; Amazon, microsoft and meta It became clear that creating and operating an AI agent is not an easy task.

The session, led by Google software engineer Deep Shah, focused on new technologies aimed at helping manage the operational costs of running large numbers of AI agents.

Running AI agents is expensive, and poorly designed and maintained digital assistants and the systems that monitor their behavior can burn cash instead of saving it.

“If you think about machine learning systems or multi-agent systems, you’re going to find multiple challenges when you try to deploy those systems at scale,” Shah said. The first is inference cost.

Ravi Bulusu, CEO of startup Synchtron, pointed to the issue of complexity, noting the different ways companies organize data, choose technology platforms, and build and run software and their employees.

Because running an AI agent touches so much on all of these points, Bulusu said, “no single dimension can be solved in isolation, and interdependencies make the problem difficult and even chaotic in practice.”

The theme of AI agent complexity continued Thursday at the AI ​​event in Mountain View, California. The event featured ThinkingAI and MiniMax, both headquartered in Shanghai, China.

ThinkingAI recently rebranded as an AI agent management platform, moving away from its original mobile game analytics company known as ThinkingData.

As part of the rebranding, ThinkingAI partnered with MiniMax, which went public in Hong Kong in January. It is one of China’s leading AI research institutes, releasing powerful models for free to the open source community, making it one of the country’s so-called “AI Tigers.”

Chris Han, co-founder of ThinkingAI, said the move to AI agent management technology is part of an effort to expand from the video game space to other industries that are excited about AI agents but lack the expertise.

Han also said that despite OpenClaw’s growing popularity in China, it is too complex and has too many security flaws for enterprises.

“OpenClaw is a great tool for personal use, but it will never reach the enterprise level,” says Han. “At the enterprise level, you need to understand a lot of things: how to manage memory, agents, teams, and communication.”

Han declined to comment on national security concerns over China’s AI models that could affect ThinkingAI’s strategy, but said the service could also support AI models from companies such as OpenAI. google.

Han joked that if the U.S. government were to ban Chinese-made promiscuous AI models domestically, that might be taken as a good sign.

“If that happens, we’ll probably be successful,” Han said.

Note: AI demand metrics are broken and only human beings are realistic.