KEK versions page
Settings → KEK Versions is the read-only monitoring surface for envelope-encryption key state. It shows which KEK versions exist, which one new envelopes are wrapped with, how many live instances have each version loaded, and the progress/history of rotations. There is no Rotate button — every transition is operator-initiated via the CLI (certautopilot kek rotate). For rotation mechanics and the runbook, see KEK rotation.
The page auto-refreshes every 30 seconds; a Refresh button forces a reload.
Summary card
| Field | Meaning |
|---|---|
| Provider | The KEK provider in use (env or pkcs11). |
| Current version | The version this API server wraps new envelopes with. Shown in green normally; warning-colored when it disagrees with the keystore (see drift alerts). |
| Fleet readiness | N/M on v<current> — how many live instances are on the current version out of all live instances. Green when everyone agrees, warning-colored when instances lag. |
| Active rotation | v<from> → v<to> when a rotation is in progress, otherwise none. |
Versions table
One row per KEK version recorded in the keystore (kek_versions collection):
| Column | Meaning |
|---|---|
| Version | v1, v2, … |
| State | active (wrap target), retired (kept for decrypt fallback after rotation), removed (dropped via kek remove). |
| Source | Where the key material comes from. |
| Created / Retired / Removed | Lifecycle timestamps. |
| Loaded by | count / total — how many live instances currently have this version's key material loaded, from the heartbeat roster. |
How to read the Loaded by colors:
- active + full load — green: healthy.
- active + partial load — warning: fleet drift, some hosts are missing the active version. Sync
secrets.env(env provider) or verify HSM reach (pkcs11) on the lagging hosts and restart them. - retired + any count — neutral: expected during the retention window after a rotation; use
kek removewhen you're ready to drop it. - removed + count > 0 — warning:
kek removeran but the env var / HSM key is still present on some hosts — clean up and restart. - removed + count = 0 — green: fully drained.
Drift and mismatch alerts
Three alert banners can appear above the tables:
- Version pinned by config override (info) — the process was started with
encryption.current_version_override, which pins the wrap target and bypasses the keystore. Expected during disaster-recovery pinning; remove the override and restart to return to normal. - Keystore drift (error) — this API server wraps new envelopes with a different version than the keystore's active row. The keystore is the source of truth; restart the process to align, or set
encryption.current_version_overrideas a deliberate temporary pin. - Fleet mismatch (warning) — one or more live instances lag behind the fleet's current version. Per-host details are on the Cluster instances page. Resolve before starting the next rotation — the rotation preflight will otherwise refuse.
Active rotation card
While a rotation runs, a progress card shows: processed / skipped / failed record counters, start time, the initiating operator, and a progress bar (warning-colored if any records failed). Immediately after kek rotate returns there is a brief window where the total record count is still 0 — the page shows a spinner ("Waiting for first batch count…") instead of a misleading 0% bar.
Monitor or cancel from the CLI:
certautopilot kek status
certautopilot kek rotate --cancel
Rotation history
The last 10 rotations, each with: from → to versions, status (completed, completed_with_errors, failed, cancelled, in_progress, pending), start/completion time, duration, processed/skipped/failed counters, and who started it.
--from-version defaults to the keystore's active row, but passing it explicitly keeps the audit log unambiguous.
Checking readiness before a rotation
Before running kek rotate:
- Fleet readiness in the summary card must read
N/N— every live instance on the current version. - The fleet mismatch banner must be absent. If it lists lagging instances, fix key material on those hosts first (details per host on Settings → Cluster).
- The target version must show as Loaded by
N/Nin the versions table — every instance needs the new key loaded before rotation starts. - Confirm from the CLI with
certautopilot kek verify --target=<N>— see Fleet readiness for exit codes and laggard interpretation.