A demonstration is not a product.

The video is impressive. The system understands, acts, succeeds. Six months later, no one has used it. That's not dishonesty, it's the normal gap between a demo and a product, and it can be measured.

The four questions to ask of any demo

1. Is it available, and to whom? "Announced," "in preview," "for a few partners," "waitlist," and "available" are five different states. Only the last one counts.
2. Is the video sped up or edited? The note is often in small print. A system that takes forty seconds to respond is never shown in real time.
3. How many attempts for that result? One successful example says nothing about the success rate. It's the most decisive piece of information and the least often given.
4. Who chose the task? A demo always shows the most favourable ground. That's fair game, as long as you know it.

The signal worth a thousand press releases

Wait until people with no connection to the vendor use it for real, on their own work, and talk about what breaks. That feedback always comes, and it comes fast: a few days, a week.

The practical rule: don't form an opinion on the day of the announcement. Waiting costs nothing, and the information is far better seven days later.

The special case of agents

Agent demos, systems that chain actions on their own, are the most misleading. A successful ten-step chain makes a great video. But if each step succeeds nine times out of ten, the whole chain succeeds about once in three. The demo shows the third that worked.

Do this right now: pick an announcement that impressed you. Look for the success rate. If it's nowhere to be found, you know what that means.