Skip to main content
2 video tutorials

Kubernetes module

Deploy a certificate as a kubernetes.io/tls Secret in one or more clusters. Works with in-cluster service-account auth (when CertAutoPilot itself runs on Kubernetes) or with inline kubeconfig for external clusters. Multi-cluster is supported via multiple targets.

Overview​

  • Resource produced: Secret of type kubernetes.io/tls with exactly two keys — tls.crt (cert + chain) and tls.key (private key). A separate ca.crt key is not produced; a consumer that needs the chain alone should read it out of tls.crt.
  • Auth: inline kubeconfig string (stored in a module credential), or the in-cluster service-account token (target Mode = in_cluster; the backend must run in that cluster — outside a pod the deploy fails with K8S_VALIDATION).
  • Which Secrets: the ones a target lists (or an automatic cap-<token>-<domain> Secret), or — with Find by certificate names — every TLS Secret in the chosen namespaces that already serves the certificate's names, plus a rollout of the workloads using them.
  • Namespace: a plain literal string per target (and per secret) — no templating. For multi-tenant reuse, use separate targets or per-distribution secret overrides.
  • Rollback: supported — a rollback re-applies the previous retained certificate version into the TLS Secret(s) and re-runs the restart targets (Rollback).

Prerequisites​

  • A module credential holding the kubeconfig (for external clusters) or a ServiceAccount binding (for in-cluster).
  • The credential's identity needs the rights under Minimum access.
  • The namespace must exist before distribution — the module does not create namespaces.

Minimum access​

Give the kubeconfig user (or, in in_cluster mode, the pod's ServiceAccount) exactly this:

  • Always: get, create, update on secrets — a Role + RoleBinding in every namespace a target writes a Secret to. That is the whole minimum for the default placement (listed Secrets).
  • Only with restart_targets: patch on the apps kinds you list (deployments, statefulsets, daemonsets).
  • Only with Find by certificate names: get, list, update on secrets in each searched namespace (it never creates a Secret); list + patch on deployments, statefulsets, daemonsets when a rollout is on (the default); list on ingresses is optional (display). With no namespaces named, also list on namespaces and the same rules cluster-wide (ClusterRole).
  • Only for orphan cleanup (a Secret this certificate wrote was renamed away): delete on secrets and list on pods, ingresses, deployments / statefulsets / daemonsets / replicasets, jobs / cronjobs. Without them the old Secret is kept with a warning.
  • Only for the live pickers: list on namespaces (ClusterRole), secrets, and the three workload kinds.

Not needed: cluster-admin, any ClusterRole for listed Secrets, delete on Secrets for normal deploys, creating namespaces, or anything for the health check (it calls only /version). Ready-to-apply YAML is under Minimum RBAC.

Create a K8s target​

  1. Settings → Distribution → Targets → New. Module type: Kubernetes.

  2. Connection mode:

    • In Cluster (in_cluster) — a plain option, always offered: uses the ServiceAccount token mounted in the CertAutoPilot pod and reaches only the cluster CertAutoPilot runs in. No credential is needed (the form disables and clears the field); outside a pod the deploy fails with K8S_VALIDATION.

    • Kubeconfig — pick a credential that stores the kubeconfig bytes. Kubeconfigs are envelope-encrypted.

      The kubeconfig must be self-contained: credentials and CA inline as client-certificate-data / client-key-data / certificate-authority-data or token. A kubeconfig that names anything on the CertAutoPilot host — exec or auth-provider plugins, tokenFile, or file paths in client-certificate, client-key, certificate-authority — is refused, at save (the credential form and the API return the offending entries) and again at use, because those fields instruct the Kubernetes client to run a program or read a file as the CertAutoPilot process. Every user and cluster entry is checked, not only the selected context. This also applies to a kubeconfig delivered by an external secret store, which is checked when it is read.

  3. Namespace: a plain literal string, e.g. ingress. The module does no namespace templating — default_namespace and each secret's namespace are used verbatim. For multi-tenant reuse, create a separate target per tenant or override the secrets per distribution.

  4. Secrets: optional. Leave the list empty and each certificate deploying through the target gets its own auto-created Secret named cap-<token>-<domain> (token: 8-char hash derived from the certificate, stable across renewals) in the Default Namespace — recommended for multi-certificate targets. Add explicit {namespace, name} entries only to pin names (e.g. ingress-nginx-tls expected by an existing Ingress). Per certificate, the secret list (and the restart-target list) can also be overridden from the certificate's Distributions → Overrides drawer; an override replaces both lists entirely.

    Behavior change

    Restart-only targets (no secrets configured) now also create the auto-generated per-certificate Secret.

    Deleting the certificate in CertAutoPilot leaves the cap-<token>-<domain> Secret in the cluster — remove it with kubectl delete secret when decommissioning a certificate.

    CertAutoPilot does delete in one case: when a Secret it previously wrote for this certificate is no longer in the desired list (you renamed it, changed a per-distribution override, or the domain slug changed), the old Secret is removed. The delete is guarded — the object must still carry CertAutoPilot's managed-by label and this certificate's id annotation, nothing verifiable may reference it (Ingress, Pod, Deployment, StatefulSet, DaemonSet, ReplicaSet, Job, CronJob), and it must not have changed since it was inspected. Anything unverifiable (an RBAC failure, a cluster using the Gateway API) is kept with a warning. Treat a rename as a real cutover: the old Secret may be gone before you repoint the consumer.

  5. Save → health check reaches the API server (GET /version) — it does not exercise any RBAC, so a missing permission surfaces at deploy time, not here.

Connection fields not covered above​

FieldMeaning
contextWhich kubeconfig context to use. Leave it unset at your peril on a multi-context kubeconfig: without it the file's current-context wins, so the certificate can land in a different cluster than you intended.
api_server_overrideTalk to a different API-server URL than the kubeconfig names. Kubeconfig mode only — an in-cluster target always dials the API server its service-account token was issued for, and saving this with in_cluster is refused.
insecure_skip_tlsSkip API-server certificate verification — and note it also discards any CA bundle the kubeconfig carries. Kubeconfig mode only.
proxy_urlRoute API traffic through an HTTP proxy. Kubeconfig mode only.

Targets run in parallel (up to 10 by default, hard cap 50), and because results are collected from goroutines their order in the run detail is not deterministic.

Choosing from the cluster (live pickers)​

The target form and the per-certificate override drawer can read the cluster and offer what is actually there instead of asking you to type names from memory:

  • Namespaces — every namespace (a Terminating one is flagged).
  • TLS Secrets — the namespace's kubernetes.io/tls Secrets, each labelled with the certificate it holds right now (subject, expiry), which Ingress serves it, and who claims it: this certificate, another CertAutoPilot certificate (both would rewrite the Secret on every renewal), cert-manager (it would reclaim the Secret), Helm (the next helm upgrade may revert it), or an owner reference. Picking an existing Secret means the deploy updates it in place and replaces its labels and annotations, so pick a Secret owned by another controller only after retiring that owner. An immutable Secret is flagged — it cannot be updated at all. To create a new Secret, switch the field to manual entry and type the name.
  • Workloads — the namespace's Deployments, StatefulSets and DaemonSets, each showing which Secrets its pod template mounts (volumes, projected sources, envFrom, env) and its ready/desired count — so a restart target that never consumes the Secret is visible as such.

The listings are read-only (list calls only — grant the extra verbs shown under Minimum RBAC, otherwise a picker reports K8S_FORBIDDEN and you type the value instead), never return Secret data (the certificate is summarised server-side; the key never leaves the cluster), and are capped at 500 entries. Nothing is read until you open a list. Typing always works: every picker has a manual-entry toggle and falls back to a plain input when the API server can't be reached.

Who may browse: listing through a saved target is a project operator action; browsing from a target form that hasn't been saved yet sends the connection details on screen and is restricted to project admins, as is pairing a saved target with a credential other than the one it was saved with. An in_cluster target browses with the pod's own service account — no credential is involved. In the override drawer the "Default (all targets)" row and the group rows span several clusters, so their fields stay manual-entry.

Execute​

  1. Attach a Kubernetes distribution to a cert.
  2. Dry-run reads each desired Secret and reports one planned action per Secret — create (absent), update (present, different fingerprint annotation) or already current (same fingerprint) — plus the restart targets that would be bounced. It does not diff the Secret's data.
  3. Execute creates the Secret, or replaces it with a plain PUT (fieldManager certautopilot, using the resourceVersion it just read). Any consumer that mounts the Secret as a Volume re-reads it on kubelet's sync interval (~60s); consumers using env vars require a pod restart.

Restart targets​

CertAutoPilot can restart the consuming workloads for you — you do not need a third-party reloader. A target's restart targets list names Deployment, DaemonSet or StatefulSet objects, and the module patches their pod-template certautopilot.io/restartedAt annotation, the same mechanism as kubectl rollout restart.

Two behaviours worth knowing:

  • Restarts are not covered by the fingerprint skip. They fire on every successful execution of the target, including a re-run that wrote nothing and a rollback.
  • They are suppressed when any Secret write on that target failed, so a partial deploy does not bounce your workloads.

Leave the list empty and nothing is restarted. Pods that mount the Secret as a volume and re-read it lazily need no entry; anything that loads the certificate at startup or from env vars does.

The health check does not test RBAC

A restart needs patch on the workload kind you list (the Role under Minimum RBAC includes it). Because the health check never exercises RBAC, a missing permission shows up as a failed target during a live renewal rather than at setup.

Find the Secrets by certificate names​

Instead of listing Secrets, a target (or one distribution's override row) can be set to Find by certificate names (placement.mode: match). Each run then searches the chosen namespaces and decides per kubernetes.io/tls Secret:

The SecretWhat happens
was written by this certificate (its certautopilot.io/cert-id annotation)updated
holds a certificate whose every DNS name this certificate covers (wildcards count, one label deep)updated in place — its own labels, annotations, owner references and other data keys stay
was written by a CertAutoPilot certificate that has since been deletedjudged by its names like any other Secret
already holds this certificate versionleft as is (current)
shares a name, but this certificate does not cover all of themleft alone — updating it would stop serving the other names
was written by another, existing CertAutoPilot certificateleft alone
is managed by cert-manager, Helm (app.kubernetes.io/managed-by) or an owner referenceleft alone, unless the target allows it (Also update Secrets cert-manager or Helm manage) — the controller may write its own certificate back
is immutableleft alone
shares no name with the certificate, or is not kubernetes.io/tlsnot considered

Nothing is created and nothing is deleted. Each Secret is judged again right before it is written: one that another certificate's run (or cert-manager) took over meanwhile is left alone and reported as skipped. On a multi-node install, use this setting only once every node runs 1.5.57 or later — an older node does not know it and writes the automatic Secret instead. Search in namespaces limits the search (empty = every namespace the credential can list); Except skips namespaces. A namespace the credential cannot read is skipped and reported (K8S_NAMESPACE_FORBIDDEN); a credential that cannot list namespaces at all must name them. More than 5000 TLS Secrets in scope stops the run before writing (K8S_SCOPE_TOO_LARGE). When nothing matches, the run succeeds without writing and warns K8S_NOTHING_MATCHED.

Roll out what uses them​

Every Deployment, StatefulSet and DaemonSet whose pod template uses an updated Secret (volume, projected volume, envFrom or env secretKeyRef) is selected for rollout by default. A workload rolls out only when one of its Secrets actually changed in that run — a re-run that writes nothing rolls nothing — and never after a failed Secret write. A rollout a run could not do (a Secret write failed, the rollout itself failed, the run ended early) is remembered and done by the next run, even though its Secret then already holds the certificate; the preview and the dry run mark it left over. A namespace whose workloads the credential cannot list is not rolled out and the run warns K8S_ROLLOUT_SKIPPED. Narrow it with:

  • Roll out → Every workload using it (default), narrowed by only in namespaces, never in namespaces, the kinds and a label selector, minus the workloads you remove (the switch in the preview, saved as rollout.exclude);
  • Only listed workloads — just the workloads you switch on in the preview (rollout.targets);
  • Nothing — no rollout.

A paused Deployment and a StatefulSet / DaemonSet with the OnDelete update strategy are listed but not rolled (the restart annotation does not move them) — the run warns K8S_ROLLOUT_SKIPPED. Pods, Jobs and CronJobs are not rolled.

See it before it runs​

Preview the Secrets and rollouts (target form and each target row of a distribution's override drawer) reads the cluster with the same rules a run uses and lists every Secret above with what it holds now, the Ingress that serves it and its verdict, then every workload using one and whether it would roll out. In the override drawer it judges the distribution's certificate and follows your unsaved edits; the switch beside each workload removes or adds it. In the target form (no certificate — a target serves many) it lists every TLS Secret in scope without a verdict. The dry run reports the same plan: one action per Secret written and per workload rolled out, every skipped Secret as a warning with its reason.

Rollback​

Supported via previous-version re-deploy: a rollback re-applies a previous retained certificate version into the TLS Secret(s) and re-runs the restart targets — the module's normal deploy path, fed older material. The former rollback.store_content opt-in is removed; rollback works without any snapshotting because the previous version comes from CertAutoPilot's own artifact history. Eligibility, the version picker, and auto-rollback: Rollback.

Multi-cluster deployments​

Create one target per cluster (each with its own kubeconfig credential) and group them. Attaching a single distribution to the group fans out across every cluster. Fan-out follows the same threshold / batch-size settings as the SSH module.

Minimum RBAC for an in-cluster install​

The bundled Helm chart creates the ServiceAccount but no Role — grant one like this (per target namespace) and bind it to the ServiceAccount the backend pod uses. The same verbs apply to the identity in a kubeconfig credential. Only the first rule is always needed; delete the blocks marked OPTIONAL you do not use.

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: certautopilot-distributor
namespace: <target-ns>
rules:
# REQUIRED — deploy, rollback, dry run: read, create and replace the TLS Secret
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "create", "update"]
# OPTIONAL — restart_targets: pod-template annotation patch (only the kinds you list)
- apiGroups: ["apps"]
resources: ["deployments", "statefulsets", "daemonsets"]
verbs: ["patch"]
# OPTIONAL — Secret picker
- apiGroups: [""]
resources: ["secrets"]
verbs: ["list"]
# OPTIONAL — orphan cleanup (a Secret CertAutoPilot wrote was renamed away):
# delete it once nothing in the namespace references it
- apiGroups: [""]
resources: ["secrets"]
verbs: ["delete"]
- apiGroups: [""]
resources: ["pods"]
verbs: ["list"]
- apiGroups: ["networking.k8s.io"]
resources: ["ingresses"]
verbs: ["list"]
- apiGroups: ["batch"]
resources: ["jobs", "cronjobs"]
verbs: ["list"]
# OPTIONAL — orphan cleanup (also replicasets) and the workload picker
- apiGroups: ["apps"]
resources: ["deployments", "statefulsets", "daemonsets", "replicasets"]
verbs: ["list"]

Find by certificate names across namespaces needs these verbs in every namespace it searches (a ClusterRole + ClusterRoleBinding for "all namespaces"): list / get / update on secrets, list on ingresses (display only, optional), and list + patch on deployments, statefulsets, daemonsets when a rollout is on (the default; not needed with rollout Nothing) — plus list on namespaces unless the namespaces to search are named. It never creates or deletes a Secret, so create and delete are not needed.

The namespace picker lists namespaces cluster-wide, which a namespaced Role cannot grant — add a ClusterRole with list on namespaces bound with a ClusterRoleBinding. Without it the namespace field still accepts a typed name; only that picker is affected. Note that list on secrets lets the identity read every Secret in the namespace — CertAutoPilot asks the API server for TLS-typed Secrets only and returns a certificate summary, never data, but the grant itself is broader than what leaves the cluster.

Troubleshooting​

Fails with "forbidden: secrets cannot be updated"​

ServiceAccount lacks update verb in the target namespace. Fix the RoleBinding. The code is K8S_FORBIDDEN (also used for a 401 — an expired kubeconfig token); it is not retried.

K8S_VALIDATION — the client could not be built​

The kubeconfig credential is missing, empty or unparseable, the context does not exist in it, proxy_url is not a valid URL, or the target is in_cluster while the backend is not running in a pod. Nothing was dialed and nothing is retried: fix the credential, context, proxy or mode. An unreachable API server is a different code, K8S_CONNECT (retried).

K8S_APPLY_FAILED — "type: Invalid value … field is immutable" or "field is immutable when immutable is set"​

The Secret you named already exists but is not a kubernetes.io/tls Secret (an Opaque one holding tls.crt/tls.key, say), or was created with immutable: true. Kubernetes never lets an update change a Secret's type or an immutable Secret's data, so the deploy fails without touching it. Use a new name (the module creates the Secret as kubernetes.io/tls) or delete and recreate the existing one. The Secret picker lists only TLS-typed Secrets and flags immutable ones for this reason.

"namespaces \"X\" not found"​

The module does not create namespaces. Create the namespace first, then retry.

Ingress still serves the old cert​

Ingress controllers cache certs. Check the controller's reload story — nginx-ingress reloads on Secret watch (fast), HAProxy Ingress reconciles on a timer. If in doubt, restart the controller pods; the Secret is already updated.

See also​

Video tutorials

Short, hands-on videos showing this feature working for real. English audio, English and Turkish subtitles. All video tutorials →

Deliver Certificates to Kubernetes as TLS SecretsYouTube ↗
Kubernetes — Find Secrets by Certificate Name and Roll Out WorkloadsYouTube ↗