Skip to main content

System status

Settings → System Status answers one question: is the background machinery running? It shows one card per leader-elected component — Main Scheduler, Domain Checker, Discovery Checker — each with a health badge, the current leader, and a table of that component's periodic sweeps with their last result. Backed by GET /api/v1/settings/scheduler-status (admin role). Auto-refreshes every 60 seconds.

What this page is not

It is deliberately scoped to scheduler/sweep health. There are no host metrics here — no disk, memory, DB pool, or queue-depth panels. For real monitoring, scrape the Prometheus endpoint and use /healthz (liveness) and /readyz (MongoDB-ping readiness) — see Observability. Per-process details (uptime, heartbeats, versions) live on Cluster instances.

Component cards​

Each card header shows:

  • Health badge — computed server-side on every request:

    StatusMeaning
    unknownNo sweep records yet (fresh install, or first cycle hasn't run).
    criticalNo active leader lock — nothing is running this component's sweeps.
    warning2+ panics in the last 5 runs of any sweep, or the last run is older than 3× the component's expected interval.
    healthyNone of the above.

  • Leader — the instance currently holding the component's lock, as name (mode@host). Locks written by older binaries lack that metadata and fall back to the opaque owner string.

The sweep table​

One row per sweep (hover the info icon for a description of what it does; the full catalog is in Scheduler status & sweeps):

ColumnMeaning
Statuscompleted, or panic (hover for the error message).
DurationWall time of the last run.
CountersWhat the run touched: scanned, matched, enqueued, updated, cleaned, skipped, failed. 0 items is normal when nothing was due.
Last RunRelative time of the last completion (hover for the exact timestamp).

Expanding a row loads the recent runs history (last 10) for that sweep — status, duration, which instance ran it, and when. Run history is retained for 7 days (also queryable via GET /api/v1/settings/scheduler-status/history?component=<name>&sweep=<name>&limit=N, max 100).

An internal license-validation sweep runs on the Main Scheduler but is hidden from this page.

How to read it​

  • A card shows "Awaiting first cycle" — the component's leader is up but hasn't completed a sweep yet. Main-scheduler cycles default to hourly, so this can legitimately persist up to an hour after a fresh start.
  • critical — no active leader — no live process is running that component. Check that at least one instance runs in scheduler or all mode (Runtime modes) and see Cluster instances for who's alive. After a leader crash, takeover can take ~2 minutes.
  • warning — stale last run — the leader exists but hasn't swept in 3× the expected interval. Check the leader instance's logs and MongoDB connectivity.
  • panic rows — the sweep crashed; the error message is preserved on the row. A single panic self-heals on the next cycle; repeated panics flip the component to warning and warrant a bug report with the message.
  • Counters that never move — e.g. Renewal Check always 0 items: nothing is due, which is normal outside the renewal window. Enqueued counts should correlate with jobs appearing in Jobs.

See also​