Rollback
Which modules can restore the previous certificate (SSH, Kubernetes, Vault), how manual and automatic rollback work, where rollback metadata lives, and backout patterns for non-reversible modules.
Rollback support matrix
| Module | Rollback | Mechanism |
|---|---|---|
| SSH | Full | Execute writes a .bak copy before overwriting each file; rollback restores the .bak and removes files that did not exist before. |
| Kubernetes | Conditional | Requires the store_content opt-in in the module's rollback config. Execute then stores the previous tls.crt/tls.key in the rollback metadata; rollback writes them back. Secrets that Execute created are deleted. Without the opt-in only create/delete is reversible. |
| Vault | KV v2 only | Rollback reads the previous secret version and writes it back as a new version. KV v1 has no version history — those targets are skipped, and if all targets are KV v1 the rollback reports not available. |
| Webhook | No | The receiver did whatever it did — CertAutoPilot cannot un-call it. |
| NetScaler | No | Certkey update is applied in place; no framework-level restore. |
| F5 BIG-IP | No | NO-DELETE policy — old cert/key objects stay on the device for manual re-attach. |
| Huawei Cloud | No | Cloud-side update is not reversible. |
| MerlinCDN | Conditional | Re-binds the CNAME to the previous certificate. Available only when delete_old_cert was off (the old certificate still exists); with it on, the old cert is deleted after the re-bind and rollback is unavailable. |
| IIS | No | The old thumbprint stays in the certificate store — re-bind manually in IIS Manager if needed. |
| WinRM (generic) | No | Non-reversible by design: the ActionSet can do anything (restart services, import into a JKS, custom logic), so a framework-level rollback is unsafe. Handle backout inside the ActionSet. |
| SMTP | No | Email cannot be recalled. |
| AWS ACM | No | A re-import overwrites the ACM certificate in place and AWS has already propagated it — there is no safe automatic restore. |
Rollback metadata & the RollbackAvailable flag
When Execute succeeds on a rollback-capable module, it returns rollback metadata (backup file paths, previous Secret content, previous KV version) which is persisted on the distribution document (last_rollback_meta). The API's computed rollback_available field is simply "rollback metadata is present" — the UI shows the Rollback button only when it is true.
Batch and retry children of a fan-out execution deliberately do not write rollback metadata (parallel children would race to overwrite each other's data). Rollback applies to normal single-job runs.
Manual rollback
- Open the certificate → Distribution tab.
- On a distribution row with rollback available, click Rollback and confirm.
API (project Operator role or higher; rate-limited to 5 distribution triggers per minute per project):
POST /api/v1/projects/{project}/certificates/{id}/distributions/{distId}/rollback
The call fails immediately if the distribution has no rollback metadata. Otherwise it enqueues a distribution_rollback job (up to 3 attempts) which:
- Takes the same per-cert/module lock as Execute — rollback never races a running distribution.
- Reads
last_rollback_metaand calls the module's Rollback with it. - Records the rollback status on the distribution:
success,failed, ornot_available.
Repeated triggers are idempotent: a job that already completed the rollback with a terminal status is skipped, and manual triggers share one idempotency key per distribution.
Validation endpoints are intentionally not run after a rollback — the targets now hold the previous certificate, so a fingerprint check against the rolled-back-from certificate would always fail.
Auto-rollback on failure
When attaching a distribution to a certificate, rollback-capable modules (SSH, Kubernetes, Vault) offer an Auto Rollback on Failure toggle (off by default). When enabled and an execution ends failed or partial with rollback metadata available, the worker automatically enqueues a rollback job (system/auto-rollback, idempotent per execute-job). If the enqueue itself fails, the distribution's rollback status is set to enqueue_failed and a notification is emitted.
Auto-rollback is a blast-radius limiter for mixed fleets: if 3 of 40 hosts got the new cert before the run failed, rollback returns them to the previous, still-consistent state.
Backout patterns for non-reversible modules
| Module | Operator backout |
|---|---|
| WinRM | Build backout into the ActionSet: Copy-Item the current file to .bak before overwriting, and document the Move-Item revert command next to it. |
| IIS | The previous certificate is still in the store — open IIS Manager and re-bind the site to the old thumbprint. |
| F5 BIG-IP | Old cert/key objects are never deleted — re-attach them to the client-ssl profile. |
| NetScaler | Re-add/link the previous certkey pair (the old files remain on the appliance unless you cleaned them up). |
| AWS ACM | Re-import the previous certificate PEM into the same ARN (download the prior artifact from CertAutoPilot), or temporarily switch the CloudFront/ALB listener to another ACM certificate. |
| Webhook / SMTP | Application-specific — the receiver owns the state. |
Troubleshooting
Rollback button missing
rollback_available is false: the module is non-reversible, the last run was a fan-out, the last execution failed before producing metadata, or (Kubernetes) store_content was not enabled at execute time. Metadata is captured during Execute — enabling an option now does not make a past run reversible.
Vault rollback fails with "no previous version to rollback to"
The secret was created by the distribution (there is no version to return to). Delete the secret manually if you need it gone.
Rollback status not_available after triggering via API
The metadata was cleared or never written between your check and the job run (e.g. a newer execution replaced it). Inspect the distribution's job history on the Distribution tab.