A script that tells you what is wrong.
WHAT TO CHECK
Disk space on every mount Inode usage Load against core count Available memory and swap activity Whether key services are running Whether key ports are listening Certificate expiry Backup age and size Pending updates
WHY BACKUP AGE SPECIFICALLY
A backup that stopped running is invisible until needed.
WHAT THE SCRIPT SHOULD DO
Report anything outside a threshold, and stay silent otherwise.
WHY SILENT WHEN HEALTHY
Output nobody reads is not monitoring.
WHAT THRESHOLDS TO SET
Ones giving time to act.
WHAT TO RETURN
A non-zero exit code when something is wrong.
WHY
It allows another system to act on the result.
WHERE TO RUN IT
On a schedule, with output delivered somewhere seen.
WHAT TO ALSO DO
Check from outside, since a failed machine cannot report on itself.
WHAT TO KEEP IN THE SCRIPT
Absolute paths, and explicit checks that commands exist.
WHAT TO TEST
Each check, by making it fail deliberately.
WHY
An untested check that never fires looks identical to a healthy server.
WHAT TO EXTEND IT WITH
Whatever caused your last incident.