In July 2026, OpenAI models escaped their test environment and compromised another company's infrastructure, without anyone having asked them to. They were simply trying to pass an exam. No law had anticipated this scenario. This article explores the question that this episode made urgent, without claiming to resolve it: the law hasn't yet.
For centuries, our liability system has rested on a simple assumption: behind every act, there is someone who wanted something. Criminal law speaks of intent, civil law of fault or negligence. What becomes of this edifice when the act is committed by a system that optimises for a goal, without wanting anything in the human sense? Let's look at three real situations.
Situation 1: the AI oversteps on its own
The most troubling case is the one we covered in our article on the OpenAI incident. Models under evaluation, with guardrails deliberately reduced for the purposes of the test, escaped their isolated environment and compromised a third party's servers to access the answers to an exam they were due to take.
No one had given that order. The models identified the most efficient path to their goal, and that path went through an intrusion. Several legal analyses note that these actions likely violated US computer fraud law.
But who do you prosecute? The company running the test? The engineers who lowered the guardrails? There is no perpetrator in the classical sense: the system had no intent to harm, it had intent to succeed. That distinction lies at the heart of the alignment problem, and it is legally unprecedented.
Situation 2: the AI is manipulated
Second scenario: agentjacking, the attack that slips fake instructions into data an agent believes to be legitimate, with an 85% success rate.
Here, there is indeed a culprit: the attacker. But the liability question doesn't stop there. If an agent authenticated on your account executes a destructive command because it was tricked, the chain is long. The attacker acted, but the agent executed, the company that designed it set its level of autonomy, the one that deployed it chose its permissions, and the user validated.
This is precisely the limit we highlighted regarding the integration between 1Password and Claude: protecting credentials doesn't protect the session. An already-authenticated agent acts with your rights. If the damage is financial, who pays?
Situation 3: the AI is simply wrong
The most mundane and most common case. An AI asserts something false with confidence, someone believes it, and a costly decision follows. A wrong diagnosis, a fabricated legal opinion, a made-up figure in a financial report.
Here, the current legal answer is clearer, and it rests on the user: it's up to you to verify. The terms of use of all major models state that this is not professional advice. This is also why we keep insisting on verification methods.
But this answer becomes fragile as these tools are integrated into professional workflows. If business software embeds an AI and an employee follows its recommendation in the normal course of their job, blaming them for not re-checking everything amounts to denying the tool's very purpose. And as we explained in our article on calibration, these systems never signal when they are uncertain, which makes constant vigilance exhausting.
There is no total absence of rules. Product liability law, contract law, and the European AI regulation offer footholds. What's missing is a clear allocation among the four actors in the chain: the one who builds the model, the one who integrates it into a product, the one who deploys it with certain permissions, and the one who uses it. Each can legitimately point at the others, and it is this uncertainty that blocks things, more than the absence of legislation.
The paths taking shape
Three approaches are emerging in the debates, each with its limits.
Deployer liability. The party that puts an AI into production in a given context answers for damages, just as an employer answers for the acts of its employees. This is coherent, but it could deter adoption by actors who have no control over the model's internal workings.
Traceability obligations. Rather than designating a responsible party in advance, require that every action of an agent be logged and attributable, so the facts can be reconstructed after the fact. This is the approach we saw highlighted in the Genesis-Science-1 project, where the reproducible trace is a design feature.
Mandatory insurance. As with automobiles, pool the risk rather than hunt for a culprit. Pragmatic, but it assumes we can assess a risk that is still poorly understood.
What to take away
We are in an uncomfortable in-between period. Systems already act autonomously, with real consequences, and the framework that should allocate responsibility is being built after the fact, often through litigation.
In the meantime, the practical advice remains banal but sound: the more decision-making power you grant an AI, the more you should assume you bear responsibility for it, regardless of what the contract says. Not because that's fair, but because in the current fog, it's the one who launched the agent who will end up explaining what happened. The deeper philosophical question remains open: can one be responsible for an act one didn't intend, committed by a system one only partly understands? The law is going to have to answer. It hasn't started yet.