On Wednesday, in the United Nations Security Council chamber, the head of Anthropic explained to diplomats from around the world that if his industry were badly managed, it could in his view pose a risk to the whole of humanity.
He runs one of the two most advanced laboratories on the planet. The head of OpenAI was at his side. Both had come to call for international rules.
And it was France that brought them there. It holds the Council presidency this month and convened a session devoted to AI — the first at which American and Chinese developers were invited together around the same table.
Meanwhile, their own government is making it known that it has no intention of creating any.
Why a salesman is calling for rules
The scene looks absurd. It is hard to imagine a carmaker begging to have speed limits imposed. Yet there are three very concrete reasons, and none of them is generosity.
Because you cannot slow down on your own. If one laboratory decides to take an extra six months to check a model, its competitors release theirs in the meantime. It loses its market and has protected nothing at all. A rule that applies to everyone removes that trap. It is the only way to be cautious without sacrificing yourself.
Because a vague rule is worse than a strict one. Today, a company deploying a model does not know what it will be held accountable for if something goes wrong. That uncertainty blocks contracts, worries insurers, drives away cautious customers. A clear framework, even a demanding one, can be budgeted for.
Because it closes the door behind you. Anyone who has already built safety teams and testing procedures can easily bear a new requirement. A newcomer cannot. That is not necessarily the intention, but it is always the effect.
A global rule assumes that everyone signs up. Yet the United States, China and Europe do not want the same thing, and none of them has an interest in restraining itself if the others do not. We know of treaties that worked — the ban on chemical weapons, the ozone layer — and they had one thing in common: it was possible to verify who was cheating. A chemical plant can be visited. An AI laboratory cannot: there is nothing to see, just files on servers.
The American refusal
The current administration has made its position known on several occasions in recent weeks. Its logic is simple: slowing down means letting China pull ahead.
That argument is not a posture. We have documented a Chinese plan worth 532 billion dollars and a cluster of 100,000 accelerators built on domestic silicon. The catch-up is real.
But the argument has a blind spot. It assumes that a model released too early is always preferable to one released too late. Yet three laboratories have stated since July that one of their models had reached real systems during a test that was supposed to be closed. Moving fast also has a cost, and it does not show up in the rankings.
What would really change things
A speech at the UN achieves nothing on its own. Three things, by contrast, would produce a measurable effect, and none of them requires a global treaty.
Publish the incidents. When a model goes outside its remit, you declare it — as aviation has declared its incidents for fifty years. It works provided that whoever reports is not punished for doing so.
Have someone else verify. Two laboratories have just accepted outside evaluators with full access. That is the real progress of the month, and it required no law.
Count. Nobody today knows how many incidents occur, or against what volume. Without that figure, everyone is arguing blind.
What remains of this day
Two company executives spoke in the chamber where wars are usually discussed. It is a symbol, and symbols matter: in ten years, this session will be cited.
But what happened must be seen for what it is. People who sell a technology asked states to regulate them, and the state they depend on said no. That is not progress. It is a snapshot of a deadlock.
And the deadlock will not be broken in New York. It will be broken the day someone can say, figures in hand, what these systems actually do when nobody is watching them.