Quick question: Would you like the AI to be so well trained that it could help a husband (or wife, for that matter) plan the outright murder of their spouse? Probably not, right? The intuitive reaction feels like no. I didn’t think it was a particularly difficult question.
But there are many diverse perspectives in America, and one such perspective was shared over the weekend by Comma AI founder and longtime jailbreak escapee George Hotz.
This post comes in response to a series of big-picture AI collaboration plans, most recently the AI Futures Institute’s policy document “AI 2040: Plan A.” The paper envisions a world in which researchers around the world collectively choose to delay AI development for 14 years for the benefit of humanity. But of course, not everyone who reads this paper will agree with its premises or conclusions.
Hotze, a member of the campaign, disagrees. In his post, he argues that the fast-takeoff scenario (the hypothesis that AI rapidly acquires superhuman abilities) doesn’t make much sense. I agree with much of what he says here! For Hotz, the best approach to AI coordination and safety is to focus on locally controlled AI models that closely align with user interests.
This is a great idea. In particular, it’s a reminder of how much of today’s AI is built around centrally managed services like Claude and ChatGPT. There are infrastructure-related reasons why AI services were developed this way. Hosting these large, cutting-edge models is expensive, and most people don’t use them enough throughout the day to justify truly personal AI. But as technology develops, those factors become less important. One of the things that’s so exciting about OpenClaw is this experimental DIY approach, and it’s great to see more AI products trying to recapture that.
But Hotz is a provocateur by nature, so he doesn’t stop there. He likens personalized AI to a gun that won’t complain (!) when you use it to kill your stepmother. (I feel like there are other rules for this, though?) A truly tuned AI would be able to order a meth lab device from Amazon Prime and teach you how to use it if that’s what you want and seek, he says. (Again, I don’t think AI will be the limiting factor here.) Hotze even says he would risk his life to uphold this principle, but it’s hard to imagine the chain of events that would lead to that.
“We either live in a world with freedom or we don’t,” Hotz writes. If there is such a choice, a world of freedom seems better. Still, I don’t understand.
Freedom isn’t everything, right? Structures that involve many people (society, markets, businesses, etc.) require a balance of fairness and the linking of individual needs to networks of interdependent preferences and responsibility systems. And anyone deploying a mass-market high-tech product probably needs to think about that entire network. That means seriously considering the interests of unmurdered spouses and in-laws around the world.
The freedom Hotz experiences is actually a space of possible futures made possible by collective enterprise. If we all start acting like little AI-powered Napoleons, those futures will disappear overnight. As the meme says, we live in a society.
It sure sounds cool that local AI is willing to take on the corporate world for my benefit. I can’t wait for the review unit.
If you buy through links in our articles, we may earn a small commission. This does not affect editorial independence.
