Jobs

Every asynchronous operation — issuance, renewal, distribution, discovery, notifications — runs as a job in the MongoDB-backed queue. The Jobs page is where you watch them run, read their logs live, retry the failed ones, and cancel the stuck ones.

Open it from the top navigation: Jobs. The page is project-scoped — the project selector determines which jobs you see.

The job list

The page header shows the Job Queue title with an Auto-refreshing badge (the list refetches every 10 seconds), a Go to Job ID… search box that jumps straight to a job detail page, and a Refresh button for an immediate refetch.

Status tabs

A tab bar above the table filters by lifecycle state: All · Pending · Active · Completed · Failed · Dead · Cancelled. Switching a tab resets to page 1.

note

There is no type dropdown in the UI — the type filter is driven by the ?type= URL parameter. Links from the dashboard's job-queue widget arrive pre-filtered; you can also edit the URL by hand (e.g. /jobs?type=distribution_execute).

Columns

Column	Content
Type	Job type with underscores rendered as spaces. Fan-out batch children carry a small child tag.
Status	Coloured tag per state. A completed job whose result_status is partial shows a partial warning tag instead.
Certificate	Certificate name (or ID) the job belongs to, if any.
Triggered By	User display name, or a system trigger (renewal sweep, fan-out).
Attempts	Attempt counter.
Progress	Per-target progress bar for jobs that report totals (distribution fan-out). - otherwise.
Created	Enqueue timestamp.
Duration	completed_at − started_at, in seconds. Empty while running.
Error	Last error message as a red tag (truncated; open the detail for the full text).
Actions	Retry (failed/dead jobs) and Cancel (pending/active jobs) icon buttons.

Clicking anywhere on a row opens the job detail. Pagination uses the default page size from Settings → General (UI preferences).

Job lifecycle

State	Meaning
pending	Enqueued, waiting for a worker to pick it up.
active	A worker holds the lock and is executing it.
completed	Finished successfully. May carry result_status: partial (some targets succeeded, some failed — shown as a partial tag).
failed	The attempt errored; the queue will retry until max_attempts is exhausted.
dead	All attempts exhausted — dead-lettered. Requires a manual Retry.
cancelled	Cancelled by an operator before or during execution.

Job detail

Search a job ID or click a row to open Jobs → detail. The header shows a Back button, the job ID, and a Live tag while the job is pending/active (the page then refetches every 2 seconds).

Summary card

A bordered descriptions table with: Type, Status (plus partial tag where applicable), Attempts (current / max), Duration, Triggered By, Certificate (links to the certificate detail), Created / Started / Completed timestamps, Parent Job (fan-out children link back to the parent), Idempotency Key, and the full Error text (copyable).

Below common error patterns, the page appends a context hint automatically — e.g. DNS errors suggest checking propagation and credential permissions; SSH errors suggest reachability and firewall rules; WinRM/IIS errors suggest Windows credentials and binding configuration; timeouts suggest network connectivity.

ACME workflow progress

For issue_certificate, renew_certificate, and validate_manual_dns jobs, a Workflow Progress card renders a vertical step list: Initialized → Awaiting Validation → DNS Presenting → DNS Propagating → Validating → Finalizing → Downloading → Storing → Completed. The current step is highlighted; a failed job marks its step red.

Two extra panels appear when relevant:

• Manual DNS Challenges — while the job waits at Awaiting Validation, each pending TXT record is listed with its FQDN and a copyable value. See Manual DNS challenge.
• Presented Records — which TXT records were pushed, via which DNS provider, and when.

MSCA workflow

MSCA jobs show an MSCA Workflow card: current step, certificate template, CA request ID, poll count, renewal mode, subject summary, terminal reason (issued or an error), and start time.

Target progress

Distribution jobs that report per-target totals render a Target Progress bar (Targets: current / total) — active while running, red on failure.

Fan-out batch jobs

Large distributions are split into batch child jobs (see Distribution). The detail page annotates these:

• A failed batch job whose failed targets are retried automatically shows an info alert: failed targets are retried as a separate retry job — the remaining attempts on this job are not used.
• Automatic retry jobs are labelled as such, with a Parent Job link back to the originating batch.

Logs viewer

The Logs card lists every log line with a level tag, the message, and a timestamp. Levels are colour-coded: INFO (blue), WARN (orange, amber row background), ERROR (red, red row background). Lines with extra detail (command output, SOAP faults, stack context) expand into a scrollable monospace block. Bracketed tags and IP:port patterns in messages are rendered bold for scanning.

While the job is active the panel shows an auto-refresh badge and streams new lines incrementally (cursor-based, every few seconds), auto-scrolling to the bottom — scroll up to pause auto-scroll, scroll back down to resume. When the job finishes, a final full refetch catches any missed lines.

Retrying a job

1. Find a job in failed or dead state (list row action or the Retry button on the detail page).
2. Click Retry. The job resets to pending with its error cleared and re-runs on the same job document (the idempotency key is preserved).

• Requires the Admin project role or higher.
• Rate-limited to 20 retries per minute.
• Only failed and dead jobs are retryable — the API rejects anything else.

Cancelling a job

1. Find a job in pending or active state.
2. Click Cancel (red icon in the list, or the button on the detail page).

• A pending job is cancelled immediately.
• An active job gets a cancel request; the worker aborts at its next checkpoint — the UI confirms with "Job cancel requested" rather than an instant state change.
• Requires the Admin project role or higher.

Common job types

Type	What it does
issue_certificate	Full ACME issuance (order, DNS-01 challenge, finalize, store).
renew_certificate	ACME renewal of an existing certificate.
revoke_certificate	Revokes a certificate at the CA.
validate_manual_dns	Validates operator-created TXT records on the manual DNS path.
cleanup_dns	Removes challenge TXT records after validation.
msca_issue_certificate	Microsoft AD CS enrollment via CES/CEP.
msca_renew_certificate	AD CS renewal.
msca_poll_pending	Polls the Windows CA for a pending (manager-approval) request.
certificate_expiration_check	Periodic sweep that flags expiring certificates.
domain_expiration_check	WHOIS + DNS health check for a tracked domain.
distribution_execute	Delivers a certificate to targets (also the fan-out batch/retry child type).
distribution_rollback	Rolls a distribution back where the module supports it.
notification.send	Delivers one notification to one channel.
discovery_execute	Runs a discovery scan against a source.

Troubleshooting with job logs

1. Open the failing job and read the Error row first — the inline hint often names the failing subsystem.
2. Scan the logs for the last ERROR line; expand its detail block for raw output (DNS responses, SSH/WinRM output, SOAP faults).
3. For ACME jobs, check which workflow step went red — a failure at DNS Propagating is a DNS problem, at Finalizing a CA problem.
4. For distribution fan-out, follow the Parent Job / retry-job links to find the job that holds the per-target results.
5. Fix the cause, then Retry (failed/dead) — or re-trigger from the certificate or distribution page.

See also

• Operations — scheduler, queue internals, runbook
• Main dashboard — job queue widget
• Renewal
• Distribution
• Manual DNS challenge