Since 15 September, Cloudflare has by default been blocking so-called mixed-use bots on advertising-funded pages: those that combine indexing for a search engine, collection for model training and retrieval for agents. It is an infrastructure lever, and publishers had none until now.
Throughout the year we have documented the imbalance between those who produce content and those who exploit it, from publishers' lawsuits to the collapse of referral traffic. Here is a change that does not go through a court.
The mixed-use problem
This is the heart of the matter, and it is more subtle than a simple block.
A publisher has an obvious interest in being indexed: it brings visitors. It has a much less obvious interest in seeing its content used to train a model that will then answer questions without sending anyone its way.
Until now, these two uses often went through the same bot. Refusing one meant refusing the other — that is, disappearing from search results. This coupling left publishers facing an impossible choice, and it was not accidental.
Decoupling the uses changes the balance of power: you can agree to be indexed and refuse to feed a training run.
Why this is a different lever from a lawsuit
The legal actions we cover share a common flaw: they are long, costly, and their outcome arrives years after the events. We saw this with the Sony and Warner complaint and the disputes around Suno.
A technical block acts immediately, and it does not require being right in law. It is a de facto power, exercised by whoever controls the infrastructure.
This shifts the centre of gravity of the matter: negotiation no longer takes place only between a lab and a rights holder, but goes through a technical intermediary that protects a considerable share of the web.
A private actor that unilaterally decides which bots get through and which are blocked wields considerable power over the flow of information. Today, that power is exercised in favour of publishers, which corrects a real imbalance. It could be exercised differently tomorrow. It is the same question we raised about the acquisition of a distribution platform: when a technical layer becomes a mandatory crossing point, whoever controls it decides for everyone.
What this changes in practice
For a publisher, it is a new and immediate means of pressure. A lab that wants access to a corpus will have to negotiate rather than collect. This is exactly the mechanism that produced licensing deals in music: the block created the negotiation.
For a lab, access to fresh data becomes more complicated. We reported that clean data had become the scarce resource. A share of the web closing off makes that scarcity more acute.
For a user, the effect is ambiguous. A model deprived of recent sources answers less well on current affairs. But an ecosystem in which content producers are paid is the one that keeps producing content.
What to take away
This change illustrates something we observe often: balances of power are rarely settled where we look for them.
While lawsuits drag on and regulators draft texts, a configuration decision taken by an infrastructure provider alters the conditions of access to a considerable share of the web, overnight.
It is effective, and it displaces the problem rather than solving it. The balance of power between publishers and labs is now arbitrated by a third actor that nobody elected and whose own interests do not necessarily coincide with those of the other two.