Much of the conversation around AI today focuses on building cloud capacity and large data centers to run models. Companies like Apple and Qualcomm are in the early stages of making on-device AI more useful. Meanwhile, Mirai’s 14-person technical team, based in London, is working to improve the way models run on mobile phones and laptops.
Backed by a $10 million seed round led by Uncork Capital, Mirai was founded last year by Dima Shvets and Alexey Moiseenkov. Both founders have experience building scalable consumer apps. Shvets co-founded the face-swapping app Reface with support from a16z. Mr. Schwetz later became a scout for the venture company. Moisenkov was the CEO and co-founder of the viral AI filter app Prisma over the past decade.
Shvets said that as consumer developers, both of them were thinking about AI and machine learning on devices even before generative AI became popular.
“When we met in London, we started talking about technology, and realized that with all the hype about GenAI and further AI adoption, everyone was talking about the cloud, about servers, about the arrival of AGI. But what was missing was on-device (AI) in consumer hardware,” he told TechCrunch.
Shvets and Moiseenkov started Mirai because they wanted to create a pipeline that could use AI to perform complex tasks on phones. When we asked other people developing consumer apps, we heard that many wanted to optimize costs and increase margins per token spent.
Mirai is currently developing a framework to improve model performance on devices. The company has built an inference engine for Apple Silicon that optimizes on-device throughput. The upcoming SDK will allow developers to integrate the runtime into their apps in just a few lines, the company said.
“One of the visions when we founded the company was that we wanted to give developers this Stripe-like eight-line code (integration) experience where they basically access our platform, integrate keys, and start working on summarization, classification, and other use cases,” Shvets said.
The startup is building this engine in Rust, which they claim can increase model generation speed by up to 37%. The company said that when adjusting models for platforms, it does not tinker with model weights to ensure the quality of the output is not compromised.
Mirai’s stack is currently focused on improving text and voice modalities on the platform, with plans to support vision in the future. The team has begun working with frontier model providers to tailor the model for edge applications and is in talks with various chip manufacturers. Later, we plan to bring that engine to Android as well.
Additionally, Mirai aims to release on-device benchmarks to allow model makers to test on-device performance. However, Shvets acknowledges that not all AI work can be done on the device. To enable mixed-mode operation, the team is building an orchestration layer to send requests that cannot be fulfilled on the device to the cloud.
The startup isn’t directly working with apps yet, but its engine is said to have the potential to power assistant, transcription, translation, and chat apps on devices.
Andy McLoughlin, managing partner at Uncork Capital, said he has been investing in edge machine learning companies for the past 10 years. He said the company sold the business early on and eventually sold the business to Spotify. He thinks things are different in today’s world.
“When you look at the cost of cloud inference, something has to change…Right now, venture capitalists are content to continue funding rocket ship companies by spending exorbitant amounts of money on cloud inference. But that won’t last long. At some point, people will look at the underlying economics of these businesses and realize something needs to change,” he said. “Every model maker seems to want to run some of their inference workloads at the edge, and we feel Mirai is very well positioned to capture this demand.”
Mirai’s seed round includes Dreamer CEO David Singleton, YC partner Francois Chaubard, Snowflake co-founder Marcin Żukowski, Eleven Labs co-founder Mati Staniszewski, former Google AdSense product manager and Coinbase board member Gokul Rajaram, Groq investor Scooter Braun, Turing.com CTO Vijay Krishnan, Theory Forge Individuals such as Ben from Ventures also participated. Parr and Matt Schlicht, and former Netflix technical lead Aditya Jami.
