Runtime AI Governance: What It Actually Costs to Run

Runtime governance costs real compute because catching an AI agent doing something wrong while it happens, not after, means analyzing every action as it occurs, and the first working versions of this idea were expensive: one documented approach added nearly a quarter more compute to run. The fix organizations are converging on is not less governance. It is tiered governance, where cheap checks handle most traffic and expensive analysis is reserved for what actually looks suspicious.

Why This Has to Happen During Execution, Not After

A yearly audit catches what an AI agent did last quarter. It does nothing for the agent making a bad call this afternoon. Once an agent can act, not just draft a paragraph for a human to review, the moment that matters shifts from “was this reviewed” to “was this stopped before it finished.”

That shift is what makes runtime governance different from every compliance process built for the previous generation of software: it has to run continuously, at the same speed as the agent it is watching, or it is not actually governing anything. A policy that only gets checked at the end of the day is a policy that finds out about a problem a day late.

The Real Cost of Watching Everything

The clearest public numbers on this tradeoff come from Anthropic’s own published account of building this kind of system for Claude. Their first-generation safeguards, called Constitutional Classifiers, cut the success rate of jailbreak attempts from 86% to 4.4% against an unguarded model, blocking roughly 95% of what would otherwise get through. That protection was not free: it added 23.7% to compute costs and increased the rate at which the model refused completely harmless requests by 0.38 percentage points.

Nearly a quarter more compute for every single interaction is the kind of number that makes a finance team ask hard questions about a security control, which is exactly why “runtime governance is expensive” is not a scare tactic. It is a documented, measured cost from a real production deployment.

The Fix Isn’t More Governance. It’s Tiered Governance.

The expensive part of runtime monitoring is applying deep analysis to every single action. The fix is not analyzing less. It is analyzing most things cheaply and reserving expensive analysis for what earns it.

Anthropic’s next-generation approach works exactly this way, described in a paper the company published in January 2026. A lightweight classifier, cheap enough to run on every single exchange, screens all traffic. Only the small fraction it flags as suspicious gets escalated to a more expensive, more accurate second-stage check. The lightweight stage can tolerate a higher false-positive rate because a flag is not a refusal, it is an escalation, so being cautious costs almost nothing.

The measured result, published by Anthropic: a refusal rate on harmless requests of 0.05%, roughly an 87% drop from the original system, at approximately 1% additional compute overhead instead of 23.7%. That is a rough 24-fold reduction in the tax runtime governance charges for a comparable, and better, level of protection. The system held up against adversarial pressure too: over 1,700 cumulative hours of red-teaming across 198,000 attempts surfaced only one high-risk vulnerability, and no universal jailbreak was found.

The architecture generalizes past this one vendor’s implementation. Zero trust security works on the identical principle: verify continuously rather than once, but scale the intensity of verification to the risk of the specific request rather than treating every request as equally worth a full check. Email security already runs a version of this: DMARC’s staged enforcement moves from passive monitoring to active quarantine to outright rejection, escalating scrutiny only as confidence in a threat grows, rather than blocking everything at the front door.

Two Attack Patterns That Beat Naive Monitoring

Tiered defense matters more once you see what it is actually defending against. Naive monitoring, checking a single input or a single output in isolation, misses attacks specifically engineered to look harmless in isolation.

• Reconstruction attacks split a harmful request into pieces that individually look benign, then have the model reassemble them. A harmful instruction embedded as scattered fragments across several unrelated-looking prompts never appears in one place for a classifier to catch.
• Output obfuscation attacks disguise a harmful answer so it reads as harmless if evaluated on its own. Anthropic’s own red-teaming found attackers successfully substituting dangerous chemical names with innocuous cover terms, so the output alone passed inspection while the conversation as a whole did not.

Both attacks exploit the same blind spot: evaluating a message without the context around it. The fix Anthropic settled on evaluates an exchange as a whole, output alongside the input and conversation that produced it, which is a harder target for exactly these techniques. Adversarial pressure on real models is not a hypothetical concern, either: on a PhD-level science benchmark, some jailbreaking approaches dropped a model’s own accuracy from 74% to as low as 32%, a reminder that an attacker forcing a model past its guardrails often also forces it to perform worse at the task the attacker wanted done.

What This Means for Evaluating Runtime Monitoring, From Any Vendor

Whether an organization builds this internally or buys it, the same questions separate a tiered system from an expensive one that will not survive a budget review:

• Ask for both numbers, not one. A vendor who quotes a false-positive rate without a compute overhead figure, or the reverse, is not being evaluated on the tradeoff that actually determines whether the tool survives contact with a real budget.
• Ask whether evaluation happens in context or in isolation. A system that checks outputs alone will miss reconstruction and obfuscation attacks by design, regardless of how good its individual classifier is.
• Ask what triggers escalation, specifically. “Suspicious activity” is not an answer. A real tiered system can name the signal that moves a request from cheap screening to expensive review.
• Measure against the cost of not doing this at all. The relevant comparison for a runtime governance budget is never zero. It is the cost of the incident the monitoring would have caught, the same math that already justifies continuous behavioral detection for human-driven risk, and the same logic behind closing the gap between detection and response rather than treating them as separate budgets.

The Bottom Line

Runtime governance for an AI agent has a real, measurable cost, and pretending otherwise sets up every security team for a budget fight it will lose. The organizations getting this right are not the ones spending the most on monitoring. They are the ones spending it on a tiered architecture that reserves expensive scrutiny for what has actually earned it, which is the same principle information security risk management has applied to human risk for years, now extended to a new kind of actor under the same people security management umbrella.

Frequently Asked Questions

Nikunj Rakesh

Nikunj is a CISO focused on helping organizations build effective security programs and resilient cultures. With a strong track record across industries, he drives governance and risk strategies that protect what matters most. Outside work, he mentors professionals and explores emerging trends shaping the future of cybersecurity.

Nikunj Rakesh

Nikunj is a CISO focused on helping organizations build effective security programs and resilient cultures. With a strong track record across industries, he drives governance and risk strategies that protect what matters most. Outside work, he mentors professionals and explores emerging trends shaping the future of cybersecurity.