CLI Reference | kapitan compile

kapitan compile

Merges inventory and inputs and produces generated files in the output folder (/compiled by default)

Compile all targets

kapitan compile

click to expand output

Compiled mysql-generator-fetch (0.18s)
Compiled vault (0.25s)
Compiled pritunl (0.22s)
Compiled gke-pvm-killer (0.05s)
Compiled examples (0.30s)
Compiled mysql (0.08s)
Compiled postgres-proxy (0.06s)
Compiled echo-server (0.06s)
Compiled global (0.03s)
Compiled guestbook-argocd (0.08s)
Compiled tutorial (0.13s)
Compiled kapicorp-project-123 (0.03s)
Compiled kapicorp-demo-march (0.03s)
Compiled kapicorp-terraform-admin (0.03s)
Compiled sock-shop (0.32s)
Compiled tesoro (0.09s)
Compiled dev-sockshop (0.32s)
Compiled prod-sockshop (0.38s)
Compiled argocd (2.29s)

Selective compilation

Using target names

Compiles one or more targets selected by name using --targets or -t

kapitan compile -t mysql tesoro

click to expand output

Compiled mysql (0.06s)
Compiled tesoro (0.09s)

Using labels

Compiles one or more targets selected matching labels with --labels or -l

Info

This works if you have labelled your targets using the following syntax:

parameters:
  ...
  kapitan:
    ...
    labels:
      customer: acme

see Labels for more details

$ kapitan compile -l customer=acme
Compiled acme-project (0.14s)
Compiled acme-pipelines (0.10s)

Fetch on compile

Use the --fetch flag to fetch External Dependencies.

kapitan compile --fetch

This will download the dependencies according to their configurations By default, kapitan does not overwrite an existing item with the same name as that of the fetched inventory items.

Use the --force-fetch flag to force fetch (update cache with freshly fetched items) and overwrite inventory items of the same name in the output_path.

kapitan compile --force-fetch

Use the --cache flag to cache the fetched items in the .dependency_cache directory in the root project directory.

kapitan compile --cache --fetch

Embed references

By default, Kapitan references are stored encrypted (for backends that support encription) in the configuration repository under the /refs directory.

For instance, a reference tag ?{gpg:targets/minikube-mysql/mysql/password:ec3d54de} would point to a phisical file on disk under /refs like:

refs/targets/minikube-mysql/mysql/password

data: hQEMA8uOJKdm07XTAQgAp5i [[ CUT ]] BwqYc3g7PI09HCJZdU=
encoding: base64
recipients:
- fingerprint: D9234C61F58BEB3ED8552A57E28DC07A3CBFAE7C
type: gpg

The --embed-refs flags tells Kapitan to embed these references on compile, alongside the generated output. By doing so, compiled output is self-contained and can be revealed by Tesoro or other tools.

kapitan compile --embed-refs

See how the compiled output for this specific target changes to embed the actul encrypted content, (marked by ?{gpg: :embedded} to indicate it is a gpg reference) rather than just holding a reference to it (like in this case ?{gpg:targets/minikube-mysql/mysql/password:ec3d54de} which points to ).

click to expand output

diff --git a/examples/kubernetes/compiled/minikube-mysql/manifests/mysql_app.yml b/examples/kubernetes/compiled/minikube-mysql/manifests/mysql_app.yml
[[ CUT ]]
apiVersion: v1
data:
-  MYSQL_ROOT_PASSWORD: ?{gpg:targets/minikube-mysql/mysql/password:ec3d54de}
-  MYSQL_ROOT_PASSWORD_SHA256: ?{gpg:targets/minikube-mysql/mysql/password_sha256:122d2732}
+  MYSQL_ROOT_PASSWORD: ?{gpg:eyJkYXRhIjogImhR [[ CUT ]] gInR5cGUiOiAiZ3BnIn0=:embedded}
+  MYSQL_ROOT_PASSWORD_SHA256: ?{gpg:eyJkYXRhI [[ CUT ]] eXBlIjogImdwZyJ9:embedded}

Fast YAML emission with rapidyaml

For YAML-heavy compiles (large Helm-style component trees, big Kubernetes manifests, hundreds of targets, etc.) the YAML emitter itself can become a meaningful share of total compile time. Kapitan ships an opt-in fast path backed by rapidyaml (a C++ YAML library), enabled with --yaml-use-rapidyaml.

pip install kapitan[rapidyaml]
kapitan compile --yaml-use-rapidyaml

On a realistic Kubernetes Deployment manifest the rapidyaml emitter is around 6x faster than the default PyYAML emitter.

Note

• Compiled output is deterministic and semantically equivalent to the default PyYAML output: mapping keys are still sorted alphabetically and every scalar round-trips to the same Python object.
• Cosmetic differences are possible (e.g. rapidyaml may single-quote a few scalars that PyYAML leaves plain, like image: 'mysql:latest'). Existing fixtures comparing exact bytes may need regenerating.
• If the rapidyaml package is not installed, the flag is ignored with a one-time startup warning and PyYAML is used.
• Documents containing ASCII control characters automatically fall back to PyYAML for that single document (rapidyaml does not escape them in double-quoted scalars).

Flags

The table below is generated from Kapitan's argument parser at docs-build time, so it always matches the installed version. See also the global flags accepted by every command, and the .kapitan dotfile to set any of these permanently.

Flag	Default	Choices	Description
--inventory-backend	reclass	reclass, reclass-rs, omegaconf	Select the inventory backend to use (default=reclass)
--migrate	False		Migrate your inventory to your selected inventory backend.
--compose-target-name	False		Create same subfolder structure from inventory/targets inside compiled folder
--enable-class-wildcards	False		Enable glob pattern expansion in classes: lists (default: off). Patterns such as clusters., dev- and * are expanded to all matching class names discovered under inventory/classes/. This flag is intentionally opt-in: inventories that contain literal glob metacharacters in class names (e.g. config[html]) or Reclass references that include ? would be incorrectly treated as patterns if expansion were always on.
--search-paths, -J	[".", "lib"]		set search paths, default is ["."]
--jinja2-filters, -J2F	lib/jinja2_filters.py		load custom jinja2 filters from any file, default is to put them inside lib/jinja2_filters.py
--verbose, -v	False		set verbose mode
--prune	False		prune jsonnet output
--quiet	False		set quiet mode, only critical output
--output-path	.		set output path, default is "."
--fetch	False		fetch remote inventories and/or external dependencies
--force-fetch	False		overwrite existing inventory and/or dependency item
--force	False		overwrite existing inventory and/or dependency item
--validate	False		validate compile output against schemas as specified in inventory
--parallelism, -p			Number of concurrent compile processes, default is min(len(targets), available CPU count)
--indent, -i	2		Indentation spaces for YAML/JSON, default is 2
--refs-path	./refs		set refs path, default is "./refs"
--reveal, --no-reveal	False		reveal refs (warning: this will potentially write sensitive data)
--embed-refs	False		embed ref contents
--inventory-path	./inventory		set inventory path, default is "./inventory"
--inventory-pool-cache	True		enable passing inventory object cache to process pool, setting to False might speed up compilation for large inventories, default is True
--cache, -c	False		enable compilation caching to $XDG_CACHE_HOME/kapitan, default is False [this is EXPERIMENTAL]
--ignore-version-check	False		ignore the version from .kapitan
--use-go-jsonnet	False		use go-jsonnet
--schemas-path	./schemas		set schema cache path, default is "./schemas"
--yaml-multiline-string-style, -L	literal	literal, folded, double-quotes	set multiline string style to STYLE, default is 'literal'
--yaml-dump-null-as-empty	False		dumps all none-type entries as empty, default is dumping as 'null'
--yaml-use-rapidyaml	False		use the rapidyaml emitter, fallback to PyYaml if rapidyaml not installed, default is False
--targets, -t			targets to compile, default is all
--labels, -l			compile targets matching the labels, default is all

Share this page