AV Services · avservices.in · Linux Infrastructure, Mumbai · Since 1999
“A cron job that runs is not the same as a cron job that works. The log entry proves it ran. Only a verified output proves it did anything useful.”
— AV Services — field observation, recurring across managed server audits
The MD of a logistics company in Andheri called me on a Friday afternoon.
Not panicking. Just confused. His IT person had left 3 months ago and he had hired a freelancer to keep an eye on things. The freelancer had set up a cron job to back up the MySQL database every night at 2am. Job was running. No complaints.
He wanted me to do a quick check before onboarding as a retainer client.
I logged in. Pulled the cron log. The job had last run successfully on the 14th. It was the 28th.
The freelancer had not set up log rotation. The cron job was still firing — but writing to a log file that had hit the disk limit 2 weeks ago. When the log filled up, the job started failing silently. No alert. No email. No red flag anywhere.
14 days of missing backups. Nobody knew.
Cron jobs do not call you when they stop working. They just stop. And unless someone checks, you find out the hard way — usually when you need the backup most.
Most business owners assume their IT person is checking. Most IT people assume the green status in the scheduler means everything is fine. Neither is wrong exactly. They are just not asking the right question.
The right question: when did this job last actually complete successfully, and how do I know?
Here is what I check on every server audit.
First, the cron log. On most Linux servers it is at /var/log/syslog or /var/log/cron. A quick grep shows every execution:
grep CRON /var/log/syslog | tail -20
This tells you the job ran. It does not tell you the job succeeded.
Second, the output. If your cron job is not redirecting output somewhere, you have no record of what it actually did. Every cron job should write to a log file:
0 2 * * * /usr/local/bin/backup.sh >> /var/log/backup.log 2>&1
That 2>&1 captures errors too. Without it, errors vanish.
Third, the timestamp on the output file. If your backup creates a file, check when it was last modified:
ls -lh /backup/latest/
If it is 2 weeks old and the job supposedly runs nightly, something broke 2 weeks ago.
Three checks. Five minutes. Most servers I audit have never had any of them done.
The logistics company is on a retainer now. We run these checks monthly, plus automated alerts if a backup file has not been updated within 25 hours. The freelancer was not careless — he set it up and moved on. Nobody asked him to verify it.
That is the gap a retainer fills. Someone who comes back every month and asks: did this actually work?
The Andheri logistics company had a log file that grew without limit until it filled the disk. This is the most common cron failure pattern I see on unmanaged servers. The fix is logrotate — a standard Linux tool that trims log files on a schedule so they never fill the disk.
Create a logrotate config for your backup log:
/var/log/backup.log {
daily
rotate 30
compress
missingok
notifempty
copytruncate
}
Save that to /etc/logrotate.d/backup. Logrotate runs automatically via cron. It keeps 30 days of compressed logs and truncates the active file so the cron job keeps writing to it without filling the disk. Test it with:
logrotate --debug /etc/logrotate.d/backup
Debug mode shows what it would do without actually doing it. Run it once, confirm the output looks right, then leave it alone.
Most backup scripts I find on unmanaged servers are 3-line shell scripts with no error handling. They run, they may or may not produce a valid backup, and they exit silently either way. Here is a more complete pattern — a MySQL backup script that logs what happened, checks whether the output file was actually created, and sends an alert if something went wrong.
#!/bin/bash
BACKUP_DIR="/backup/mysql"
LOG="/var/log/backup.log"
DATE=$(date +%Y-%m-%d)
OUTFILE="$BACKUP_DIR/db-$DATE.sql.gz"
ALERT_EMAIL="arun@yourdomain.com"
mkdir -p "$BACKUP_DIR"
echo "[$DATE $(date +%H:%M:%S)] Backup started" >> "$LOG"
mysqldump --all-databases | gzip > "$OUTFILE"
EXIT_CODE=$?
if [ $EXIT_CODE -ne 0 ]; then
echo "[$DATE $(date +%H:%M:%S)] ERROR: mysqldump failed with exit code $EXIT_CODE" >> "$LOG"
echo "MySQL backup failed on $(hostname) at $(date)" | mail -s "BACKUP FAILED" "$ALERT_EMAIL"
exit 1
fi
if [ ! -s "$OUTFILE" ]; then
echo "[$DATE $(date +%H:%M:%S)] ERROR: Output file is empty or missing" >> "$LOG"
echo "MySQL backup produced empty file on $(hostname) at $(date)" | mail -s "BACKUP FAILED - EMPTY FILE" "$ALERT_EMAIL"
exit 1
fi
SIZE=$(du -sh "$OUTFILE" | cut -f1)
echo "[$DATE $(date +%H:%M:%S)] Backup complete: $OUTFILE ($SIZE)" >> "$LOG"
# Keep only last 30 days
find "$BACKUP_DIR" -name "db-*.sql.gz" -mtime +30 -delete
echo "[$DATE $(date +%H:%M:%S)] Old backups cleaned" >> "$LOG"
Three things this script does that most backup scripts do not. It captures the exit code from mysqldump and treats a non-zero exit as a failure — not a completed job. It checks whether the output file actually has content. And it sends an email if either check fails, so you find out that night rather than two weeks later when someone needs the restore.
The mail command requires a working MTA on the server — postfix or sendmail. If you do not have one configured, write the alert to a separate file and check it in your morning monitoring sweep instead.
The backup script above catches failures at runtime. This is a second check — a separate cron job that runs every morning and verifies the backup file from last night actually exists and is recent. Two independent checks are better than one.
#!/bin/bash
BACKUP_DIR="/backup/mysql"
MAX_AGE_HOURS=25
ALERT_EMAIL="arun@yourdomain.com"
LATEST=$(find "$BACKUP_DIR" -name "db-*.sql.gz" -mmin -$((MAX_AGE_HOURS * 60)) | sort | tail -1)
if [ -z "$LATEST" ]; then
echo "No backup file found in last ${MAX_AGE_HOURS} hours on $(hostname)" |
mail -s "STALE BACKUP ALERT" "$ALERT_EMAIL"
fi
Add this to cron to run at 7am every morning:
0 7 * * * /usr/local/bin/check-backup-age.sh >> /var/log/backup-check.log 2>&1
25 hours gives you a one-hour buffer past the 2am backup window. If the file is not there by 7am, something failed overnight. You find out before the business day starts, not during a recovery attempt.
The Andheri logistics server now has both scripts running. The backup job writes to a log with error handling. The morning check confirms the file exists. Logrotate keeps the log files under control. In 8 months since the retainer started, the morning check has fired once — a disk space issue on the backup destination that was resolved before the MD arrived at the office.
AV Services manages Linux servers on monthly retainer for businesses in Mumbai and across India. Every retainer includes monthly cron job verification and backup restore testing. If you want someone to check whether your scheduled jobs are actually doing what you think, start with a free 30-minute audit call — no access needed, no commitment. Cron verification is part of the broader system care and maintenance process.
Is your cron job running — or just running silently into nothing?
Free 30-minute audit call. No access needed. No commitment.
⚡ Check Your Risk Book Free Audit Server Down? Call NowLooking for ongoing management instead? See retainer pricing →