What breaks, and how to spot it

An automation doesn't stop by making noise. It keeps going, and it's wrong. That's what makes it riskier than a poorly done manual task.

The four classic failures

The input format changes. The supplier alters its invoice template, a column shifts. Nothing signals the error: the extracted data simply becomes wrong.

The edge case arrives. A message in another language, a negative amount, an empty field. The pipeline was designed for the normal case.

The model responds differently. You expect "URGENT" or "NORMAL", it replies "This message seems urgent". The next step doesn't know what to do with it.

The service on the other end changes. An interface evolves, an authorisation expires. The pipeline stops, sometimes without warning.

The three safeguards

Impose a strict output format. "Reply ONLY with one of these three words: URGENT, NORMAL, IGNORE. No other text." And plan for the case where it replies with something else.

A log. Every run leaves a trace: input, output, timestamp. Without a log, you'll discover the problem through its consequences, weeks later.

An alert on the abnormal. Zero runs today when there are usually twenty, that's a signal. Silence is not good news.

Periodic review

Once a month, open the log and check five runs at random. Ten minutes. It's the only way to catch a slow drift, the one that breaks nothing but skews everything.