Skip to content

How it works

backpulse is a dead-man’s switch. Instead of watching logs or exit codes, it watches for the check-in each scheduled job owes it — and raises the alarm when the check-in doesn’t arrive or reports failure.

Each check has a schedule (a cron expression plus timezone and a grace window). The job — directly or through backupctl — tells backpulse three things:

  • /start: the run began.
  • /success: the run finished and the result verified.
  • /fail: the run finished but something was wrong.

From those three signals plus the schedule, the detector derives everything:

SignalMeaning
No /start before the expected timethe scheduler or host failed
/start with no completionstuck, or the host disappeared
/failthe command or its result verification failed
Completion past the grace windowlate
Two /starts with no completion between themoverlapping or duplicate runs

The critical ordering: backupctl verifies the result before it reports /success. A restic adapter demands a fresh snapshot; a file adapter demands a recent, non-empty artifact. If verification fails, the run reports /fail even though the wrapped command exited zero. And the command stays authoritative — if backpulse itself is unreachable, pings only warn; a monitoring outage never fails your backup.

Detection and delivery are separate slices that never import each other. Detection enqueues one notification per channel; the delivery worker claims due rows under a lease, sends them, and records the outcome guarded so a stale worker can’t overwrite a recorded result. A crash between send and record can therefore deliver twice — receivers must tolerate duplicates. Failed sends retry with backoff, then dead-letter after the configured attempts so a broken destination stays visible instead of silently retrying forever.