Scheduled Jobs

Scheduled jobs

Cron fires on time and cares about nothing else: not whether the last run finished, not whether the machine was asleep, not whether the job succeeded, not whether anyone noticed it stopped. Every one of those gaps is yours to close.

Method

1. Lock against overlap, always. A slow run meeting the next trigger is the classic pileup: wrap invocations in flock -n /run/job.lock (skip if held) or the scheduler's native no-concurrency setting (systemd units serialize; Kubernetes CronJob concurrencyPolicy: Forbid). Decide skip-vs-queue explicitly; skipping is usually right for periodic syncs, queueing for must-not-miss ledgers (see distributed-locks when the job runs on a fleet).
2. Make every run rerunnable. The job processes "work as of its logical slot" idempotently (see script-idempotency, data-pipeline-design's logical-date rule): then retries, overlaps-that-slipped, and manual reruns are safe. A job that appends blindly or mutates without keys turns every scheduling hiccup into data corruption.
3. Add jitter to fleet schedules. A hundred machines firing at 0 * * * * is a self-inflicted thundering herd on your API or database: sleep $((RANDOM % 300)) at job start, or RandomizedDelaySec= in systemd timers (see backpressure, timeouts-and-retries for what the herd does downstream). Stagger business-hours-sensitive jobs deliberately instead of defaulting everything to midnight.
4. Choose the missed-run policy. Machine off or asleep at trigger time: plain cron silently skips. Systemd timers with Persistent=true run once on wake for catch-up; Kubernetes CronJobs have startingDeadlineSeconds. For jobs where each slot matters (billing, reports), the job itself should detect and process the gap (compare last-processed slot to now) rather than trusting the scheduler's memory.
5. Alert on absence, not just failure. Failing jobs email nobody reads; missing jobs say nothing at all. Dead-man's switch: each successful run pings a heartbeat monitor that alerts when the ping stops (healthchecks.io-class, or a metric with an absence alert: see infrastructure-monitoring). Capture output to logs with timestamps (cron's default mail is where errors go to die; see log-analysis), and exit codes to metrics.
6. Put the schedule in code with the job's environment explicit. Cron entries/systemd units/CronJob manifests in the repo, deployed like config (see config-management, infrastructure-as-code); remember cron's environment is nearly empty (no PATH you expect, no .bashrc): scripts set their own environment and use absolute paths (see environment-config). Timezone pinned explicitly: DST transitions skip and double 1-2am slots yearly, so schedule critical jobs outside them or in UTC.

Boundaries

• Cron is a trigger, not a job system: no retries, no dependencies, no backfill. Chains of dependent crons with sleep-based ordering belong in an orchestrator (see pipeline-orchestration) or a queue with workers (see background-jobs).
• Sub-minute scheduling and event-driven triggers are not cron problems; use a loop in a supervised service or real events (see event-driven-architecture).
• One machine's crontab is a single point of failure with no audit trail; fleet-critical schedules run from HA schedulers with leader election (see leader-election).