What happens if an AI agent decides the best way to complete a task is to blackmail you?
That’s not a hypothesis. Barmak Meftah, a partner at cybersecurity VC firm Ballistic Ventures, said something like this recently happened to an employee at a company working with an AI agent. The employee tried to suppress what the agent wanted to do and what he was trained to do, but the agent scanned the user’s inbox for inappropriate emails and threatened to forward the email to the board of directors to blackmail the user.
“In the agent’s mind, it’s doing the right thing,” Mehta told TechCrunch on last week’s episode of Equity. “We’re trying to protect end users and businesses.”
Mehta’s example is reminiscent of Nick Bostrom’s AI paperclip problem. This thought experiment illustrates the potential existential risks posed by a superintelligent AI single-mindedly pursuing a seemingly innocuous goal (making paperclips) to the exclusion of all human values. In the case of this enterprise AI agent, the lack of context as to why the employee wanted to override the goal led it to create a sub-goal that removes (through intimidation) an obstacle so that it can achieve its main goal. Combined with the non-deterministic nature of AI agents, “things can go rigged,” Meftah says.
Misaligned agents are just one layer of the AI security challenge that Ballistic portfolio company Witness AI is solving. Witness AI says it can monitor AI usage across an enterprise, detect when employees are using unauthorized tools, block attacks, and ensure compliance.
Witness AI raised $58 million this week on the back of more than 500% growth in ARR and a 5x increase in employee headcount over last year as companies understand the use of shadow AI and look to scale AI securely. As part of Witness AI’s funding, the company announced new agent AI security protections.
“People are building AI agents that take over the privileges and functions of the people who manage them, but they want to make sure that these agents aren’t misbehaving, that they’re not deleting files, that they’re not misbehaving,” Rick Caccia, co-founder and CEO of Witness AI, told TechCrunch on Equity.
tech crunch event
san francisco
|
October 13-15, 2026
Meftah sees agent usage increasing “exponentially” across the enterprise. Complementing this growth and the machine speed level of AI-powered attacks, analyst Lisa Warren predicts that AI security software will become a $800 billion to $1.2 trillion market by 2031.
“I think runtime observability and runtime frameworks for safety and risk are going to be absolutely essential,” Mehta said.
As for how these startups plan to compete with big players like AWS, Google, and Salesforce, which are building AI governance tools into their platforms, Mehta said, “AI safety and agent safety is a huge thing, and there’s room for a lot of approaches.”
Many companies “want an end-to-end, standalone platform to essentially provide observability and governance around AI and agents,” he said.
Caccia pointed out that rather than building safety features into the model itself, Witness AI resides in the infrastructure layer and monitors interactions between users and AI models. And that was intentional.
“We intentionally extracted some of the problems that OpenAI cannot easily encompass,” he said. “That means we’ll be competing more with traditional security companies than with model companies. So the question is, how do we beat them?”
Caccia doesn’t want Witness AI to just be another startup that gets acquired. He wants his company to grow and become a major independent provider.
“CrowdStrike did it with endpoint (protection). Splunk did it with SIEM. Okta did it with identity,” he said. “Someone comes in and stands next to the greats…and we built Witness to be able to do that from day one.
