Skip to content

How can you tell if a text was written by an AI?

The benchmarks everyone cites are largely wrong. Here is what actually works, what does not, and why you should be wary of them either way.

Advertisement
The problem with this question 🔍
Everyone wants a reliable method, and none exists. What does exist are probability signals that regularly get it wrong, and whose naive use does more damage than the problem they claim to solve. Here is what more or less works, what does not work at all, and why caution is needed on both sides.

We have covered a watermark bypassed in four hours and European transparency obligations. Here is the practical question everyone is asking.

The signals that prove nothing

Let us start by clearing the ground, because these beliefs circulate widely and cause unfair accusations.

The em dash. This is the most widespread myth. This punctuation mark is simply typographically correct, used by professional writers forever. Its presence indicates a well-configured keyboard or a word processor that auto-corrects, nothing more.

Certain words. You read lists of supposedly revealing terms. These lists are mostly made up of formal vocabulary that many people use. Using them as evidence amounts to suspecting anyone who writes well.

The absence of mistakes. A flawless text may have been proofread, corrected, or written by someone careful. It is the opposite of a signal.

Bullet lists. A format is not a signature.

What constitutes a real signal

Three signals have value, and you need to combine them rather than rely on a single one.

Excessive regularity. This is the strongest signal. Human text has variation: long sentences then short ones, a weaker paragraph, an awkward transition. Generated text tends toward unusual homogeneity, each paragraph having the same density and the same length.

The lack of anchoring. Human text almost always contains something specific: a precise date, an anecdote, an odd figure, a detail no one would invent. Generated text often stays at a general level, because it produces what is plausible rather than what happened.

The reversible conclusion. Generated text frequently ends with a cautious balance where both positions have merit. This is not wrong, it is simply what a system trained not to take sides produces.

Why detectors are dangerous ⚠️
Detection tools produce a probability score, and this score is systematically interpreted as a verdict. Yet false positives hit identifiable profiles first: those for whom it is not the native language, those who write in a highly structured way, those who use a limited vocabulary. Accusing someone on this basis is reproducing a bias while believing you are applying a method. And a generated text that has been reworded gets through.

The question to ask instead

Here is the useful shift, and it applies at school as much as at work.

Asking was this text written by an AI is a question that cannot be answered. Asking can this person defend this text is a question that can be answered in five minutes.

Why this choice rather than another? What did you discard? Where does this figure come from? Someone who knows their subject answers immediately, whatever their tool was. Someone who copied an output without reading it does not answer.

This is the criterion we proposed in our article on authenticity, and it has the advantage of being verifiable, unlike detection.

What to remember

Three things, and the third is the most important.

The popular signals are largely false and mainly serve to accuse innocent people. The real signals exist but remain probabilistic, and degrade as models improve. And above all: this race is lost in advance, because it is always easier to blur a signal than to preserve it.

The direction that makes sense is not to unmask the generated, but to be able to prove the authentic, and to hold accountable whoever publishes. The rest is a game of cat and mouse where the cat starts with a structural handicap.

Advertisement