A crashed system is loud. Users complain, alerts fire, someone fixes it. The truly expensive failure is quiet: an automation that simply stops — and nobody notices for days.
What a silent failure looks like
The nightly export runs into a changed field and ends without an error. The invoice workflow loses its login and stops sending. The price import has been processing an empty file for a week. Nothing crashed. Nothing turned red. The business just slowly drifts out of sync with reality.
Alert on what did not happen
Classic monitoring waits for an error. Silent failures do not produce one. The answer is to turn the logic around: every important job reports that it ran successfully — a heartbeat. If the heartbeat does not arrive in time, that absence is the alert.
Each block is one night. Grey: the job stopped and nobody knew. Orange: the moment someone finds out.
Beyond the heartbeat, plausibility checks catch the subtler cases: an import that usually handles 2,000 rows and suddenly handles 0, a daily revenue figure that is exactly the same as yesterday.
Where to start
- List the five automations the business would miss first.
- Give each one a heartbeat and an expected time window.
- Add one plausibility check per job: volume, totals or freshness.
- Send the alert to an owner — see The Automation Nobody Owns for why that matters.
The question is not whether an automation will stop one day. It is how long it takes until someone knows.
Want to find the silent failures before your customers do? We build, run and monitor systems ourselves — and we know where automation pays off and where it becomes a risk.