Changing KEK provider — fresh install required
CertAutoPilot's KEK provider (env vs
pkcs11) is chosen at install time and locked
immutably in the MongoDB kek_install singleton.
There is no in-place migration path: changing the provider
requires a fresh install against a new MongoDB. This page
explains why, and walks through the migration procedure.
01Why no in-place migration
Each provider's envelope wrap/unwrap behaviour differs (key material in process memory vs. key material resident in HSM). A running system with mixed-provider state would need code to:
- Route every Wrap to one provider and every Unwrap to both (by envelope's
providertag). - Hold both providers in memory simultaneously with different failure modes.
- Track per-version source so rotation knows which provider owns what.
- Coordinate cluster-wide rollout across provider boundaries.
We deliberately don't write that code. It's a large surface area, rarely exercised, and easy to get wrong in ways that lose data. Fresh install + data re-import is the supported path.
02What you can change without migration
- KEK versions within the same provider — rotation (
kek rotate --from-version=M --to-version=N) is designed for this. - Adding a new HSM key to an existing pkcs11 install for rotation (
kek pkcs11-init --version=N+1). - Changing the HSM module path or token label — the lock tracks
provideronly, not module-specific details. Edit config and restart;BuildRegistry's capability probe will flag incompatibilities.
03When you DO need to migrate
env→pkcs11to move KEK into an HSM for compliance.pkcs11→env(rare; might happen if an HSM is decommissioned).- Swapping one HSM family for another (e.g. SoftHSM2 dev → AWS CloudHSM production) — technically still
pkcs11but the key material is in a different physical device, so you need a fresh DB + new keys.
04Migration procedure
1. Back up everything on the current install
mongodump --uri=$MONGO_URI --out=/backup/pre-migration-$(date -I)
# Plus your secrets.env or K8s Secret, kept in a secure off-host
# location.
2. Export data that needs to survive
CertAutoPilot's encrypted records (certificate private keys, DNS credentials, module secrets, etc.) are tied to the current KEK. If you need to preserve certificates:
- Download each cert's PEM bundle via the API or UI while the service is still running. See Download.
- Export DNS / module credentials via the same route, documenting which belongs to which project.
3. Provision a fresh MongoDB
Different cluster, different DB name, or a clean db.dropDatabase() — whichever your operational policy allows.
4. Install with the new provider
Standalone — one command, PIN inline or via a mode-0600 file:
curl -fsSL https://raw.githubusercontent.com/CloudNativeWorks/certautopilot-archive/main/get.sh \
| sudo bash -s -- --version=1.4.0 --mongo=local \
--kek-provider=pkcs11 \
--pkcs11-module=/opt/cloudhsm/lib/libcloudhsm_pkcs11.so \
--pkcs11-token-label=<label> \
--pkcs11-pin='<hsm-user-pin>'
K8s: update values.yaml with secrets.encryption.provider: pkcs11 + pkcs11.* settings, helm install against the new namespace or cluster.
5. Re-import data
Via the API or UI. Create projects, re-upload DNS credentials, re-issue certificates, etc. The new install wraps everything under the new provider from scratch.
6. Retire the old install
Once the migration is verified. The old MongoDB + secrets are the only way to decrypt the old envelopes, so keep backups until you're confident the new install covers all your workloads.
05Install-lock error messages
When you run kek pkcs11-init --version=1 against a fresh MongoDB, it writes the lock as {provider: "pkcs11"}. From that moment on, any binary invocation pointing at this MongoDB with encryption.provider: env will fail fast at BuildRegistry:
KEK provider mismatch: installed="pkcs11" (since 2026-05-01T...)
but config says "env". Provider is immutable after install.
Similarly, running pkcs11-init against an env-locked MongoDB fails immediately without touching the HSM:
refusing to run: this database is locked for provider="env"
(since 2025-08-15T09:00:00Z by service). pkcs11-init cannot run
against a "env"-locked installation.
These guards prevent operators from accidentally contaminating one installation with another's data or keys — the implicit coupling of envelope tags + per-provider wrapping logic would otherwise allow silent corruption.
06Rollback
If a migration attempt fails, stop the new service, restore the old MongoDB + secrets from backup, start the old service. The old install is untouched by the migration procedure (step 3 creates a new DB, not reconfigures the existing one), so rollback is "use the old DB again".
07Troubleshooting
I forgot to export a cert before migration
The old install still has its DB and secrets. Start it again against the original infra (you kept the backups in step 1), download the missing cert, then re-upload to the new install.
Can I just kek rotate to move to the HSM?
No. kek rotate re-wraps envelopes with a new KEK version of the same provider. It cannot cross the provider boundary. See Why no in-place migration.