Skip to content

Deploying Keda with Kapitan

We have worked hard to bring out a brand new way of experience Kapitan, through something that we call generators

Although the concept is something we've introduced in 2020 with our blog post Keep your ship together with Kapitan, the sheer amount of new capabilities (and frankly, the embarassing lack of documentation and examples) forces me to show you the new capabilities using a practicle example: deploying Keda.

Objective of this tutorial

We are going to deploy Keda using the helm chart approach. While Kapitan supports a native way to deploy helm charts using the helm input type, we are going instead to use a generator based approach using the "charts" generator.

This tutorial will show you how to configure kapitan to:

  • download a helm chart
  • compile a helm chart
  • modify a helm chart using mutations

The content of this tutorial is already available on the kapitan-reference

Deploying KEDA

Define parameters

## inventory/classes/components/keda.yml
parameters:
  keda:
    params:
      # Variables to reference from other places
      application_version: 2.11.2
      service_account_name: keda-operator
      chart_name: keda
      chart_version: 2.11.2
      chart_dir: system/sources/charts/${keda:params:chart_name}/${keda:params:chart_name}/${keda:params:chart_version}/${keda:params:application_version}
      namespace: keda
      helm_values: {}
...

Override Helm Values

As an example we could be passing to helm an override to the default values parameters to make the operator deploy 2 replicas.

  helm_values:
    operator:
      replicaCount: 2

Download the chart

Kapitan supports downloading dependencies, including helm charts.

When Kapitan is run with the --fetch, it will download the dependency if not already present. Use --force-fetch if you want to download it every time. Learn more about External dependencies

## inventory/classes/components/keda.yml
...
  kapitan:
    dependencies:
      # Tells kapitan to download the helm chart into the chart_dir directory
      - type: helm
        output_path: ${keda:params:chart_dir}
        source: https://kedacore.github.io/charts
        version: ${keda:params:chart_version}
        chart_name: ${keda:params:chart_name}
...

Parameter interpolation

Notice how we are using parameter interpolation from the previously defined keda.params section. This will make it easier in the future to override some aspects of the configuration on a per-target base.

Generate the chart

## inventory/classes/components/keda.yml
...
  charts:
     # Configures a helm generator to compile files for the given chart
    keda:
      chart_dir: ${keda:params:chart_dir}
      helm_params:
        namespace: ${keda:params:namespace}
        name: ${keda:params:chart_name}
      helm_values: ${keda:params:helm_values}

Compile

Before we can see any effect, we need to attach the class to a target. We will create a simple target which looks like

# inventory/targets/tutorials/keda.yml
classes:
- common
- components.keda

Now when we run kapitan compile we will see the chart being donwloaded and the manifests being produced.

./kapitan compile -t keda --fetch
Dependency keda: saved to system/sources/charts/keda/keda/2.11.2/2.11.2
Rendered inventory (1.87s)
Compiled keda (2.09s)

kapitan compile breakdown

  • --fetch tells kapitan to fetch the chart if it is not found locally
  • -t keda tells kapitan to compile only the previously defined keda.yml target
ls -l compiled/keda/manifests/
total 660
-rw-r--r-- 1 ademaria root 659081 Aug 29 10:25 keda-bundle.yml
-rw-r--r-- 1 ademaria root     79 Aug 29 10:25 keda-namespace.yml
-rw-r--r-- 1 ademaria root   7092 Aug 29 10:25 keda-rbac.yml
-rw-r--r-- 1 ademaria root   1783 Aug 29 10:25 keda-service.yml

Using mutations

Now let's do a couple of things that would not be easy to do with helm natively.

You can already notice that the content of the chart is being splitted into multiple files: this is because the Generator is configured to separate different resources types into different files for convenience and consistency. The mechanism behing it is the "Mutation" of type "bundle" which tells Kapitan which file to save a resource into.

Here are some example "mutation" which separates different kinds into different files

        mutations:
          bundle:
            - conditions:
                kind: [Ingress]
              filename: '{content.component_name}-ingress'
              ...
            - conditions:
                kind: [HorizontalPodAutoscaler, PodDisruptionBudget, VerticalPodAutoscaler]
              filename: '{content.component_name}-scaling'
            - conditions:
                kind: ['*']
              filename: '{content.component_name}-bundle'

Catch-all rule

Notice the catchall rule at the end that puts everything that has not matched into the bundle.yml file

bundle mutation

Currently most of the keda related resources are bundled into the -bundle.yml file Instead, we want to separate them into their own file.

Let's add this configuration:

  charts:
     # Configures a helm generator to compile files for the given chart
    keda:
      chart_dir: ${keda:params:chart_dir}
      ... 
      mutations:
        bundle:
          - conditions:
              # CRDs need to be setup separately
              kind: [CustomResourceDefinition]
            filename: '{content.component_name}-crds'

Upon compile, you can now see that the CRD are being moved to a different file:

ls -l compiled/keda/manifests/
total 664
-rw-r--r-- 1 ademaria root  11405 Aug 29 10:56 keda-bundle.yml
-rw-r--r-- 1 ademaria root 647672 Aug 29 10:56 keda-crds.yml
-rw-r--r-- 1 ademaria root     79 Aug 29 10:56 keda-namespace.yml
-rw-r--r-- 1 ademaria root   7092 Aug 29 10:56 keda-rbac.yml
-rw-r--r-- 1 ademaria root   1783 Aug 29 10:56 keda-service.yml

patch mutation

As we are using Argo, we want to pass a special argocd.argoproj.io/sync-options annotation to the CRD only so that ArgoCD can handle them properly.

For this we are going to use the patch mutation:

...
      mutations:
...
        patch:
          - conditions:
              kind: [CustomResourceDefinition]
            patch:
              metadata:
                annotations:
                  argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true,Replace=true

Upon compile, you can now see that the CRDs have been modified as required:

diff --git a/compiled/keda/manifests/keda-crds.yml b/compiled/keda/manifests/keda-crds.yml
index 2662bf3..9306c3a 100644
--- a/compiled/keda/manifests/keda-crds.yml
+++ b/compiled/keda/manifests/keda-crds.yml
@@ -2,6 +2,7 @@ apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
   annotations:
+    argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true,Replace=true
     controller-gen.kubebuilder.io/version: v0.12.0

Summary

With this tutorial have explored some capabilities of Kapitan to manage and perform changes to helm charts. Next tutorial will show how to make use of Keda and deploy a generator for Keda resources