Heartbeat monitoring guide

What is heartbeat monitoring?

Heartbeat monitoring (a dead-man’s switch) inverts normal checks: your job pings a unique PageLantern URL each time it runs, and PageLantern alerts you when the ping does not arrive on schedule — catching silently dead cron jobs, queue workers, and backups.

What is the grace period?

The grace period is extra time allowed after the expected ping before the heartbeat counts as missed, absorbing normal scheduling jitter. If you leave it blank, PageLantern defaults it to a quarter of the interval, with a minimum of 60 seconds and a maximum of one hour. Setting an explicit value (including 0) overrides the default.

How do I ping the heartbeat from a cron job?

Call the URL at the end of the job, and only on success. The single most common mistake is pinging unconditionally — from a wrapper script that always runs, or from a trap — which reports the job as healthy on the exact runs where it failed.

Use a short timeout and a couple of retries so a transient network blip on our side does not raise a false missed-run, and make sure a failure to ping cannot fail the job itself.

#!/bin/sh
set -e   # any failure below exits before the ping

/usr/local/bin/nightly-backup.sh

# Reached only on success. Never move this into a trap or run it unconditionally:
# a dead-man’s switch that always fires is not a switch.
curl -fsS -m 10 --retry 3 "$PAGELANTERN_HEARTBEAT_URL" > /dev/null

How do I ping from Python or Node?

Same rule: after the work, only on success, and never let the ping raise. A heartbeat that crashes your worker because our endpoint was briefly unreachable has made your reliability worse, not better.

import os
import urllib.request

def ping_heartbeat():
    url = os.environ.get("PAGELANTERN_HEARTBEAT_URL")
    if not url:
        return
    try:
        urllib.request.urlopen(url, timeout=10).read()
    except Exception:
        # A failed ping must never fail the job. A missed heartbeat is a
        # false alarm you can explain; a crashed worker is an outage.
        pass

run_nightly_import()   # raises on failure, so the ping below is skipped
ping_heartbeat()

What interval and grace should I choose?

Set the interval to how often the job is scheduled, not how long it takes. A backup that runs at 02:00 daily has an interval of 24 hours regardless of whether it takes four minutes or four hours.

Set the grace to cover the job’s worst realistic runtime plus queueing. A daily job that usually finishes in ten minutes but occasionally takes ninety needs a grace of at least two hours, or you will be paged every time it has a slow night.