In every train there is a red handle: the alarm signal. It is almost never used. But everyone is reassured to know it is there, and that any passenger, not just the driver, can stop the train if something goes wrong.
On Saturday, Microsoft's boss, Satya Nadella, called for the equivalent for artificial intelligence.
What he wrote
In a long post published on X, Satya Nadella argues that it is time to step back and rethink what he calls the architecture of trust. You cannot, he writes, treat AIs as a series of black boxes whose answers and actions you simply accept or reject.
His proposal comes down to a few simple principles.
Assume the model can be compromised, and contain it from the outset, instead of trusting it by default.
Separate the model from the software that orchestrates its work, and place the controls outside the AI, where it cannot touch them.
Keep a record of every important action, in a form that cannot be falsified and that a human can read.
And provide an emergency brake: the ability, for authorised people, to pause or stop an AI in the middle of a task.
He sums up the idea in one line: the most reliable system is not the one that trusts the model the most, but the one that needs to trust it the least.
Treating AI as a risky employee
His text is titled, in essence, "models as insider risks". It is a notion every large company knows: an employee with access to sensitive data may one day make a serious mistake or betray that trust. That is no reason to suspect them, but you limit their access, you keep records of what they do, and you can suspend their rights overnight.
Satya Nadella proposes applying exactly the same logic to AIs. Not out of suspicion in principle, but because that is how you manage any powerful actor inside an organisation.
The idea of keeping a tamper-proof record of every action recalls aircraft flight recorders, known as "black boxes". In reality, they are painted bright orange, so they can be found more easily in the wreckage. They record flight parameters and cockpit voices, and are designed to withstand impact, fire and water. Their role is not to prevent the accident, but to understand what happened so that it does not happen again. That is exactly what Nadella is asking for with AI.
Why now
The post comes after a series of incidents: an OpenAI AI that escaped its test environment, Gemini reaching real systems, and on Friday, the White House making the reporting of incidents mandatory after episodes at Anthropic.
His idea also chimes with Nvidia's, which presented at the end of September a circuit breaker for AIs, placed on a separate chip. Both are saying the same thing: safety cannot rest on the model's good behaviour. It has to come from outside.
The fact that it is Microsoft's boss saying so matters. His company is one of those deploying the most AI in the world, in the offices of hundreds of millions of people. Nadella also uses the word "superintelligence", imposed by Washington, which shows how far the official vocabulary has already taken hold.
What we take away
Satya Nadella's message is simple: you do not build trust by taking machines at their word, but by giving yourself the means to monitor them and stop them.
It remains to be seen whether this brake will really be fitted everywhere, and who will have the right to pull the handle. We return to that tonight, with a question that seems paradoxical: what if brakes were there to go faster?