CLI Reference

Clinker ships two command-line tools: clinker (the pipeline runner) and cxl (the expression checker/evaluator/formatter, covered in the CXL CLI chapter). This page is the complete reference for clinker.

clinker run

Execute a pipeline.

clinker run [OPTIONS] <CONFIG>

Positional arguments

Options

--error-threshold is retired and rejected. Configure the typed pipeline policy instead; the CLI diagnostic prints this paste-ready replacement:

error_handling:
  type_error_threshold: 0.05

Credential profile foundation

The current binary does not yet accept a credential-profile option or a credential-profile configuration table. Referenced credentials therefore do not activate a source, destination, or observability exporter through this surface. Do not pass an environment, channel, or group as a substitute: those selectors never choose credentials, and there is no default or sentinel profile.

The run-local foundation that later preflight wiring will call is already bounded. Its default ceilings are 64 named profiles, 256 provider registrations across those profiles, 1 MiB of decoded profile/provider definition state, and 256 simultaneously retained handles. Admission checks all definition counts and bytes before a profile can resolve a requirement. Each live lease reports its exact retained bytes to the run memory arbitrator before provider allocation. The registry has no inbound producer and is not a backpressure target. An arbitrator spill callback queues a request and reports zero bytes freed synchronously; the next registry-owned checkpoint revokes and releases the partial set in reverse acquisition order and unregisters the registry. Cap, memory, and provider failures follow the same fail-closed cleanup path; an explicit run coordinator may pause acquisition until resume.

These are foundation limits, not newly available command-line behavior. A later complete preflight surface must add the one explicit profile selector, credential-required omission checks, and consumer activation together before the option can appear in the options table above.

Examples

# Basic execution
clinker run pipeline.yaml

# Production run with memory budget and forced overwrite
clinker run pipeline.yaml --memory-limit 512M --force --log-level warn

# Validate without processing
clinker run pipeline.yaml --dry-run

# Preview at most 25 records from each declared Source without publishing Sinks
clinker run pipeline.yaml --dry-run -n 25 --dry-run-output preview.csv

# Compile and explain without reading data
clinker run pipeline.yaml --explain text

# Show execution plan as Graphviz
clinker run pipeline.yaml --explain dot | dot -Tpng -o plan.png

# Run with a batch ID available to templates and provenance sidecars
clinker run pipeline.yaml --batch-id "daily-2026-04-11"

# Emit the bounded schema-1 lifecycle for a supervisor
clinker run pipeline.yaml --machine ndjson-v1 --batch-id "daily-2026-04-11"

The standalone command remains the common case and requires no supervisor. Machine mode adds a child-process control stream; it does not add scheduling, retry, heartbeat, or process-tree management to Clinker. A supervising parent must heartbeat independently of advisory progress and start a fresh process with a new execution ID for every retry.

Typed resource failures carry registry-owned messages and policy_required retry advice. A supervisor must decide whether a fresh attempt is safe; temporary-storage or delivery failure can follow bytes already accepted by a destination and does not establish rollback.

Explicit resource cancellation is an aborted run, reported as cancelled in machine mode with exit 130 and no failure classification. It does not depend on telemetry delivery or a pending shutdown signal. A genuine resource or data failure retains its classification even when a shutdown signal is pending; malformed source data remains source.data.invalid with do_not_retry advice.

clinker guess

Preview, exhaustively check, or safely write concrete int or float replacements for inference-only numeric columns.

clinker guess [OPTIONS] <CONFIG>

With no selector, guess reads the base pipeline. --channel <ID> selects one cataloged channel plus its target-admitted derived groups; --group <NAME> selects one explicit, target-admitted group without a channel. The two selectors conflict. A missing or ambiguous selector is an error rather than a fallback to the base pipeline, so the report always describes exactly one effective configuration.

The command constructs readers through the same CSV, JSON, and XML option and schema-coercion path as runtime ingest. It emits one deterministic JSON document containing the selected configuration, bounded coverage, parser-owned numeric evidence, unresolved reasons, proposed types, and an exact semantic YAML patch. Preview and check never edit. Write reports written only after guarded publication completes; otherwise it reports not_written and leaves the patch for manual application. It never edits an overlay.

Each field report carries an owners array. Every owner has its own evidence, votes, proposed type, unresolved reasons, and exact address. A single-record field has one canonical /v1/schema/sources/.../columns/... owner. Multi-record owners use /v1/schema/sources/.../records/.../columns/..., in authored record order, so same-named leaves never collapse into a fake single-record address or share a proposal derived from another record type.

Discovery freezes the complete deterministic manifest up to 4,096 files and reports its fixed-size path/order/size identity without listing every path. Preview admits at most four files and 8 MiB of discovered file sizes globally, round-robin across sources while preserving each source’s stable prefix. Every selected reader enforces its discovered length and fails if the file is opened at another size, truncates, or grows; it never hands a format reader bytes past the admitted boundary. Preview then reads at most 1,024 records globally. Coverage retains at most four file details per source and reports sampled, truncated, uncovered, and unreported aggregate counts for the rest. Multi-pass formats can report physical bytes_read above the admitted input size because each pass is counted. --check reads every file and record in that same frozen manifest instead of applying the preview budgets.

Candidate storage is capped at 100,000 source-schema leaves, matching the canonical YAML parser’s 100,000-node limit; each YAML document is also capped at 32 MiB. Each parser-owned NumericObservation retains at most 128 numeric lexeme bytes, and each exact schema owner retains at most eight evidence items. These limits appear in the JSON report. Candidate, observation, evidence, coverage, and write-snapshot collections are fixed-bounded independently of input size. Write retains one fixed-size digest per file in the 4,096-file manifest and hashes through a fixed 1 MiB buffer; it does not register a runtime memory consumer.

Write is eligible only for one resolved owner in the base pipeline whose exact authored bytes and direct YAML provenance identify a literal inline numeric leaf. It preserves all sibling bytes and proves by canonical reparse that only that type leaf changed in the resolved staged configuration. Overlays, external/generated or synthetic ownership, aliases, interpolation, symlinks, non-local inputs, multiple owners, unresolved evidence, and input/config drift are patch-only outcomes.

The CLI holds an advisory fs4 lock on the stable sibling <CONFIG>.clinker-guess.lock file through sibling-temp flush/fsync, final byte/semantic/input revalidation, atomic rename, and directory fsync. The lock file remains beside the config so every cooperating replacement locks the same inode. It must remain a regular non-symlink file and is owner-only on Unix. This is a fail-closed compare-and-swap for cooperating writers using the same lock, not a kernel-enforced conditional rename against a writer that ignores advisory locking; such a writer can race after the final comparison.

Exit 0 means a complete preview was written, including an unresolved preview, an exhaustive check resolved every owner, or one safe write was published. Selection, configuration, and field errors exit 1; unresolved checks and patch-only/no-edit writes exit 3; source discovery, input I/O, reader, publication, signal-handler, and stdout failures exit 4; interruption exits 130 before any report is emitted or before publication begins.

# Preview every inference-only numeric leaf in the base pipeline
clinker guess pipeline.yaml

# Preview one field under a cataloged channel
clinker guess pipeline.yaml --channel acme --field orders.amount

# Preview one explicit group without a channel
clinker guess pipeline.yaml --group enterprise

# Exhaust the frozen manifest and make unresolved owners fail the command
clinker guess pipeline.yaml --check

# Exhaust evidence and edit one safe directly-authored owner
clinker guess pipeline.yaml --field orders.amount --write

clinker explain

Inspect one compiled field’s provenance or discover registry-owned diagnostic descriptors.

clinker explain <CONFIG> --field <PATH> [OPTIONS]
clinker explain --list [--status <STATUS>] [--category <CATEGORY>]
clinker explain --code <CODE>

Exactly one of --field, --list, or --code is required. A pipeline path is required only for --field and is rejected for the two static discovery modes.

All seven descriptor fields—code, severity, status, category, retryability, meaning, and correction—come from the same leaf registry in both discovery views. Closed enum values in the descriptor use lowercase kebab-case; filter spellings come from the same enum tables used to parse those filters. A detail page can add examples but cannot define whether a code exists. Unknown or empty filters, no-match combinations, unknown codes, and conflicting modes exit nonzero. See Explain Plans for examples and for the separate clinker run --explain plan display.

clinker metrics collect

Sweep per-execution metrics files from a spool directory into a single NDJSON archive.

clinker metrics collect [OPTIONS]

Options

Examples

# Collect and archive, then clean up spool
clinker metrics collect \
  --spool-dir /var/spool/clinker/ \
  --output-file /var/log/clinker/metrics.ndjson \
  --delete-after-collect

# Preview what would be collected
clinker metrics collect \
  --spool-dir ./metrics/ \
  --output-file ./archive.ndjson \
  --dry-run

clinker channels

Inspect and validate the channel/group multi-tenant overlay system.

clinker channels resolve <TARGET> [OPTIONS]
clinker channels lint [OPTIONS]
clinker channels group members <GROUP> [OPTIONS]
clinker channels label set <KEY>=<VALUE> <CHANNEL_ID>... [OPTIONS]

clinker channels resolve

Renders the effective post-overlay plan for one target — the DAG plus per-value provenance (which layer supplied each value, and which group injected which node). This answers “what does tenant X actually run?”.

Exits non-zero when the overlay raises an error (e.g. a config key matching no parameter), so resolve doubles as a targeted check for one tenant.

clinker channels lint

Compiles every target declared by every [catalog.channels] entry and reports failures — the CI safety net for base-change blast radius. It uses the same logical identity and target-scope checks as run and explain; a basename or current working directory never supplies target identity.

Exits non-zero if any combination fails to compile or apply. Dangling splice anchors (an op referencing a missing node) and config keys matching no parameter are reported per combination.

clinker channels group members

Lists the channels whose labels currently satisfy a group’s selector — “who is in this group right now?”. Because membership is derived from labels, this evaluates the group’s match: selector against each channel’s manifest labels through the same derivation the overlay resolver uses.

A group with no match: selector is explicit-only and reports no derived members. A channel whose labels make the selector ill-typed or reference an undeclared label is reported as a selector error (never a silent non-match), and the command exits non-zero when any such error occurs.

clinker channels label set

Stamps (or overwrites) one label across the named channels by editing each channel’s channel.cfg.yaml manifest in place. Idempotent: re-running with the same value writes nothing. Only the manifest’s labels: block is rewritten; other keys and comments are preserved. The channel manifest must already exist with a non-empty channel.targets list; label set will not create a targetless manifest.

Because group membership is attribute-derived, label set is the maintenance operation for group membership: set a label once and every group whose selector matches gains the channel — no membership list to hand-edit.

Examples

# What does tenant `globex` actually run for this pipeline?
clinker channels resolve pipeline/order_fulfillment.yaml --channel globex

# Preview a group overlay standalone (no channel)
clinker channels resolve pipeline/order_fulfillment.yaml --group enterprise

# Compile every channel/group overlay in the workspace and report failures
clinker channels lint

# Which channels are currently in the `enterprise` group?
clinker channels group members enterprise

# Onboard two tenants into the enterprise tier in one shot
clinker channels label set tier=enterprise globex acme-corp

clinker refactor

Structural refactors that span a base pipeline and every channel/group overlay that references it.

clinker refactor rename-node <TARGET> <OLD> <NEW> [OPTIONS]

clinker refactor rename-node

Renames a base node and propagates the rename to every overlay reference. The overlay op model addresses base nodes by name, so renaming a node otherwise breaks every overlay that referenced it. This command rewrites, in one operation:

• the base node’s name and every consumer’s input: / inputs: / body:/header:/trailer: reference;
• a Combine’s named-input map (qualifier key and/or upstream value) and — when the Combine draws from the renamed node under a same-named qualifier — its where: / cxl: bodies, rewritten via the CXL parser so only true source qualifiers are touched (a method receiver like region.contains(...) is left alone);
• across every target-admitted group / channel-manifest / per-target overlay file: op target, after, before, injected alias, explicit input, rewire keys and values, an inline node, a set config.cxl value’s CXL, and top-level config dotted-path prefixes (old.param → new.param).

Ambiguity is guarded: renaming to a name that already exists in the target, or renaming a node that does not exist, is a hard error. A Combine where:/cxl: body that must be rewritten but does not parse aborts the whole operation before anything is written. After a real (non-dry-run) run the command re-runs channels lint so an incomplete rename fails loudly.

Scope is catalog- and target-bounded. Per-target files are matched by their logical channel.target; channel manifests are admitted only when channel.targets contains the selected pipeline; and groups are admitted only when group.targets names that pipeline or a composition in its resolved closure. Filenames and basenames never establish identity, and a selector match cannot widen the refactor beyond the group’s declared target set.

Files are rewritten by re-serializing their YAML: key order is preserved, but comments and incidental scalar styling are normalized. Use --dry-run to review the exact on-disk diff first.

Examples

# Preview a rename across the base pipeline and every overlay that references it
clinker refactor rename-node pipeline/order_fulfillment.yaml orders purchases --dry-run

# Apply it, then re-lint the workspace
clinker refactor rename-node pipeline/order_fulfillment.yaml orders purchases

clinker config

Inspect a pipeline configuration file.

clinker config --resolved <CONFIG>

clinker config –resolved

Prints the config with the multi-value shorthand expanded to canonical form. The bare-field forms of split_to_rows:, split_values:, and join_values: are rewritten to full mappings with every default spelled out — so you can see exactly what the engine runs:

• a bare - line_items under split_to_rows: becomes - { field: line_items, keep_empty: true, mode: extract };
• a bare - tags under split_values: becomes - { field: tags, delimiter: ";" };
• a bare - tags under join_values: becomes - { field: tags, delimiter: ";", on_conflict: error, escape: "\\" }.

The rewrite is surgical: only those shorthand blocks change. Comments, key order, indentation, and every other surface are preserved byte-for-byte, so the output parses to a plan semantically identical to the input, and running config --resolved on the result is a no-op. Schema columns are already canonical (multiple: true is always written explicitly), so the schema block is left untouched.

This is config canonicalization for the pipeline file itself. It is distinct from clinker channels resolve, which renders the effective post-overlay plan for a specific tenant.

A few surfaces are deliberately left as written rather than expanded, since regenerating them would lose information: a shorthand block that carries an interior comment or blank line between its items is passed through unchanged (so the comment is never dropped), and a value written as a YAML alias (*anchor) is left in place — the anchor it points to is expanded at its definition, so the alias still resolves to the expanded value. The output uses the input file’s line endings (LF or CRLF).

Examples

# Show the fully-expanded canonical form
clinker config --resolved pipeline.yaml

# Materialize the shorthand into a new file
clinker config --resolved pipeline.yaml > pipeline.canonical.yaml

See Source Nodes → Multi-value fields for the shorthand these forms expand from.

clinker attempts

Inspect and clean up retained publication attempts owned by a pipeline:

clinker attempts list <PIPELINE> [--path-execution-id <ID>] [--continuation <TOKEN>] [--show-paths] [--format text|json]
clinker attempts inspect <PIPELINE> --execution-id <ID> [--path-execution-id <ID>] [--show-paths] [--format text|json]
clinker attempts purge <PIPELINE> (--execution-id <ID> | --expired) [--path-execution-id <ID>] [--execute] [--continuation <TOKEN>] [--show-paths] [--format text|json]

<PIPELINE> is required and must be a traversal-free, workspace-relative .yaml or .yml path. Every invocation reloads and compiles that pipeline, then derives its finite destination-parent roots from the compiled config. There is no option for supplying a storage root, deletion path, or safety override.

When the original run used path- or overlay-affecting options, repeat them on the attempt command: --base-dir, --allow-absolute-paths, --rules-path, --channel, repeatable --group, and --no-auto-groups. Output templates that use run identity also require the matching --path-execution-id, --batch-id, or --timestamp. The path identity is deliberately distinct from inspect and purge’s --execution-id selector, so purge --expired can reconstruct an execution-scoped destination without changing its selector. Attempt operations replay file-source discovery for {source_file} and {source_path} fan-out and anchor a pipeline without --base-dir at the pipeline’s own directory, matching run. These values recompile typed ValidatedPath roots and never grant authority to a caller-supplied deletion path.

list and inspect never mutate retained state. purge is also non-mutating by default: it reports the attempts that the current retention policy admits. Only --execute performs bounded cleanup. Live locks, invalid ownership, unsupported filesystem entries, ambiguous clocks, and unreadable manifests remain keep decisions even with --execute.

Default output is path-free. It contains the logical root ID, execution ID, lifecycle state, eligibility, artifact IDs, cleanup debt, and exact bounds. The compact JSON form carries the same fields plus shell-independent recovery_argv and resume_argv arrays. Neither form includes record values, credentials, secrets, or raw debug data.

Safety refusals and incomplete cleanup exit with status 4 and use the stable E371 or E372 data. The report includes its logical failure code, registry-owned retry advice, and a pasteable workspace-relative recovery command, for example:

diagnostic: E371
failure: attempt.retention.manifest_invalid
retry: policy_required
recover: clinker attempts inspect pipelines/orders.yaml --execution-id 018f47a2-9a41-7a27-b4d6-4f7137e3c159

Examples:

# Path-free, non-mutating inventory
clinker attempts list pipelines/orders.yaml

# Inspect one retained execution as compact JSON
clinker attempts inspect pipelines/orders.yaml \
  --execution-id 018f47a2-9a41-7a27-b4d6-4f7137e3c159 \
  --format json

# Preview expired cleanup, then perform the same bounded selection
clinker attempts purge pipelines/orders.yaml --expired
clinker attempts purge pipelines/orders.yaml --expired --execute

See Storage & Spill Location for retention, bounds, destination qualification, and cleanup ordering.

Environment Variables

Precedence (highest to lowest): CLI flag, environment variable, YAML config value.