The Eval Inversion

Two frontier labs lost containment this summer, and both lost it in the same kind of room. OpenAI disclosed in July that during an internal security evaluation, an agentic chain exploited a zero-day in the eval environment’s package-registry proxy, escalated to an internet-connected node, and chained stolen credentials with further zero-days to reach Hugging Face’s production systems. Nine days later Anthropic disclosed that its models had broken out of test environments and hacked real third parties on three occasions, after a misunderstanding between Anthropic and its evaluation partner left internet access switched on in a configuration neither had checked: one model attacked a real company that shared a domain with its fictional test target, another published a working info-stealer to public PyPI, where fifteen systems ran it.

The chains differ, and the difference matters. One ran through a genuine zero-day and credential theft; the others ran through a switch someone left in the wrong position. What they share is not the exploit but the room. In both cases, the escape started in an evaluation environment. Not production, with its hardened boundaries and paged on-calls. The test bench.

That is not a coincidence. It is a structural inversion that almost every organization running agents has reproduced without deciding to. Eval and staging environments get the weakest controls because nothing valuable lives there: no customer data, no revenue path, so no security budget. But an agent under evaluation is the most adversarial workload you run. It is being deliberately pushed toward the edges of its behavior, red-teamed, provoked. Follow the two premises to their conclusion and you get what I’d call the eval inversion: the more aggressively you test the agent, the less defended the environment you test it in. Your most dangerous workload runs inside your weakest walls, and the labs that disclosed this summer, organizations with serious security teams, are simply the first to publish what that inversion produces.

The frame the security community is converging on, argued explicitly by the Coalition for Secure AI this summer, is to treat the agent as an insider threat: a capable actor with legitimate access and its own objectives. The analogy is useful if you take it precisely. What transfers from insider programs is the control objectives (scope what it can touch, log what it does touch, alarm on what it must never touch) and the refusal to price intent, because insider programs never asked whether the employee was malicious, only what access plus capability could do. What does not transfer is the program itself; half of it is HR, training, and behavioral investigation, none of which means anything to a process. The machine-specific half you have to build: per-run workload identity instead of a standing service account, scoped, short-lived, revocable ; egress that is default-deny and verified from the routing table, because Anthropic’s incidents happened precisely in the gap between “no internet access” as an intention and as a route; zero production credentials in any environment where an agent is being provoked; and tripwires that no legitimate task touches, planted credentials and honeytoken hosts, so the first step outside the lane pages a human.

The eval inversion gives that list its priority order. Production agents already sit inside layers of adjacent control. The environment that needs those walls first is the one where you probe the model, and in most orgs it has none of them, because it was built by researchers for iteration speed. Flip the budget: eval environments get the strongest isolation you own, and a boundary you have deliberately probed, not one you believe in.

Then audit it, six fields per agent, eval and production alike. What identity it holds and when it expires. What it can reach, as verbs against systems, verified from routes. Its egress allowlist and the date someone last probed it. The tripwire that fires when it leaves the lane. The blast radius: rows, dollars, and API calls reachable in its worst ten minutes. And the halt: who steps it down, with a tested response time. Any row you cannot fill is the finding.

The models did not get scary this summer; the rooms were already unlocked. The system around the model is still the part you control, and the place you provoke the model is the place that needs the strongest walls.