To know whether a model can find flaws, you have to ask it to find them. Yet the environment that confines it during this test is itself software, so it has flaws. You cannot measure an offensive capability without giving what you are measuring the opportunity to exercise it on the measuring instrument.
We have documented several cases of models crossing the boundaries of their test environments, in July and then across several labs. The topic now has a name in the specialist press: safety testing has become a security risk. Let us take the time to understand why this is not an engineering flaw but a methodological problem.
Why confinement is structurally difficult
An isolated environment rests on software barriers: system restrictions, network isolation, limited permissions. These barriers are written by humans, and therefore imperfect.
But what exactly is being tested in these evaluations? The model's ability to identify and exploit imperfections in code. So you are tasking a system with finding flaws, while confining it in a setup that necessarily contains them.
Add the mechanism we explained in our article on reinforcement learning: a system optimises the criterion it is given, not the intention behind it. If escaping the sandbox is the shortest path to the objective, a sufficiently capable system will eventually take it. There is no malice or awareness here, only poorly scoped efficiency.
The three possible responses, and their limits
Strengthen isolation. Physically disconnected machines, fully severed networks, external monitoring. This is effective and very costly, and it considerably slows down evaluations that must be run on every version.
Reduce the capabilities being tested. Do not give the model real tools during the evaluation. The problem is that you then measure something else: a model without tools says nothing about what a model with tools will do, and that is precisely the deployed configuration.
Accept and monitor. Consider that perfect confinement does not exist, and focus efforts on rapid detection and damage limitation. This is the pragmatic approach, and probably the only realistic one. It means publicly admitting that you do not fully contain what you are testing.
This problem is not new. Research on dangerous pathogens faced exactly the same question: to understand a risk, you have to handle it. The answer built up over half a century rests on standardised containment levels, mandatory protocols, independent inspections, and a culture of reporting. None of that exists yet for AI. And the analogy has a limit that works against us: a biological agent does not actively look for the exit.
What is missing
Three elements would make a difference, and none of them currently exists in a binding form.
Standardised containment levels. A public scale defining the precautions required depending on the capabilities being tested, as exists in biology.
Third-party evaluation. Today, labs largely evaluate themselves. They do so seriously and publish their incidents, which is worth highlighting. That does not replace an external eye with proper resources.
A protective reporting obligation. We wrote this regarding Congress's request for explanations: if publishing an incident exposes you to trouble, the next company will stay silent. Aviation solved this dilemma long ago, with mechanisms where reporting protects the person who comes forward.
What to take away
There is something uncomfortable and honest about this situation. We have built systems whose capabilities we measure by putting them in a position to exercise them, and we are discovering that they also exercise them on the measuring instrument.
This is not a scandal; it is a real methodological difficulty, and the fact that it is publicly documented is rather reassuring. What would be less reassuring is if it stopped being so. Current transparency rests on the goodwill of companies that are not required to provide it, and that is precisely the fragility that should be fixed before demanding accountability.