Kustomize

Kustomize traverses a Kubernetes manifest to add, remove or update configuration options without forking. It is available both as a standalone binary and as a native feature of both oc and kubectl.

Exploring Kustomize

The principles of kustomize are:

Purely declarative approach to configuration customization
Manage an arbitrary number of distinctly customized Kubernetes configurations
Every artifact that kustomize uses is plain YAML and can be validated and processed as such
As a "templateless" templating system; it encourages using YAML without forking the repo it.

Exploring the Kustomize CLI

The kustomize CLI should have been installed as part of the lab setup. Verify that it has been installed.

kustomize version

This should display the version, it should look something like this.

v5.1.0

Kustomize, at its core, is meant to build on top of native Kubernetes manifests based on YAML while leaving the original YAML intact. It achieves this in a "template-less" templating format. This is done by providing a kustomization.yaml file.

We will be focusing on two Kustomize sub-commands: the build command and the edit command.

Kustomize `build`

The build command takes the YAML source (via a path or URL) and creates a new YAML that can be piped into oc create. We will work with an example in the content/modules/ROOT/examples/kustomize-build directory within the repository you cloned.

cd ~/openshift-gitops-workshop/content/modules/ROOT/examples/kustomize-build

Here you should see two files, a welcome.yaml file and a kustomization.yaml file, let’s have a look at them.

cat ./welcome.yaml

welcome.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: welcome-php
  name: welcome-php
spec:
  replicas: 1
  selector:
    matchLabels:
      app: welcome-php
  strategy: {}
  template:
    metadata:
      labels:
        app: welcome-php
    spec:
      containers:
      - image: quay.io/redhatworkshops/welcome-php:latest
        name: welcome-php
        resources: {}
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
      securityContext:
        runAsNonRoot: true
        seccompProfile:
          type: RuntimeDefault

This file shows nothing special. Just a standard Kubernetes Deployment manifest.

Now what if, for example, we wanted to add a label to this manifest without editing it? This is where the kustomization.yaml file comes in.

cat ./kustomization.yaml

kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ./welcome.yaml
patches:
- patch: |-
    - op: add
      path: /metadata/labels/testkey
      value: testvalue
  target:
    group: apps
    kind: Deployment
    name: welcome-php
    version: v1

As you can see in the output, we only need a resources and a patches section to accomplish this:

The resources is an array of individual files, directories, and/or URLs where other manifests are stored. In this example we are just loading in one file.
The patches is where we add our label to this manifest.

You can read about what options are available for patching in the official documentation site

Build this manifest by running:

kustomize build

You can see that the new label got added to the manifest!

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: welcome-php
    testkey: testvalue # Our new label
  name: welcome-php
spec:
  replicas: 1
  selector:
    matchLabels:
      app: welcome-php
  strategy: {}
  template:
    metadata:
      labels:
        app: welcome-php
    spec:
      containers:
      - image: quay.io/redhatworkshops/welcome-php:latest
        name: welcome-php
        resources: {}
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
      securityContext:
        runAsNonRoot: true
        seccompProfile:
          type: RuntimeDefault

Kustomize `edit`

You can use the kustomize edit command to make manifest changes instead of manually writing YAML. For example, you can change the image tag the Deployment above uses from latest to ffcd15 by running the following:

kustomize edit set image quay.io/redhatworkshops/welcome-php:ffcd15

This will update the kustomization.yaml file with an images section.

cat kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ./welcome.yaml
patches:
- patch: |-
    - op: add
      path: /metadata/labels/testkey
      value: testvalue
  target:
    group: apps
    kind: Deployment
    name: welcome-php
    version: v1
images:
- name: quay.io/redhatworkshops/welcome-php
  newTag: ffcd15

Now when you run:

 kustomize build .

You should see not only the new label, but also the new ffcd15 image tag!

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: welcome-php
    testkey: testvalue
  name: welcome-php
spec:
  replicas: 1
  selector:
    matchLabels:
      app: welcome-php
  strategy: {}
  template:
    metadata:
      labels:
        app: welcome-php
    spec:
      containers:
      - image: quay.io/redhatworkshops/welcome-php:ffcd15
        name: welcome-php
        resources: {}
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
      securityContext:
        runAsNonRoot: true
        seccompProfile:
          type: RuntimeDefault

You can now see how you can take existing YAML and modify it for your specific environment without the need to copy or edit the original.

Kustomize can also be used to write a new YAML file and pipe it into the oc (or kubectl) command for immediate execution in your environment. Example:

kustomize build . | oc apply -f -

Exploring Kustomize with `oc`

The OpenShift CLI (oc) has support for Kustomize built in. It inherits this from kubectl which has had this same support since Kubernetes 1.14.

You can see this by running:

oc kustomize --help

You can run the kustomize build command by doing:

oc kustomize

Although you can use oc kustomize and pipe it into the oc apply command, you don’t have to. The oc apply command has the -k option that will run the build before it applies the manifest.

oc kustomize and kubectl kustomize implement a subset of the kustomize featureset. For example, the edit command is not implemented.

Now you will apply your kustomize-d Deployment manifest into one of your projects:

Ensure that you are in the ~/openshift-gitops-workshop/content/modules/ROOT/examples/kustomize-build directory

oc apply -n USER_PLACEHOLDER-bgdk -k .

You should see the following letting you know the deployment was successful

deployment.apps/welcome-php created

You can pass not only directories, but URLs as well. The only requirement is that you have a kustomization.yaml file in the path.

Now that the deployment is created, you should see the pods running in the namespace with

oc get pods -n USER_PLACEHOLDER-bgdk

The console should return something along the lines of

NAME                          READY   STATUS    RESTARTS   AGE
welcome-php-9474fc448-sthzr   1/1     Running   0          34s

You can also see the deployment was created with the additional labels

oc get deployment welcome-php -o jsonpath='{.metadata.labels}{"\n"}' -n USER_PLACEHOLDER-bgdk

Resulting in

{"app": "welcome-php","testkey": "testvalue"}

Finally, check that the image was updated based on the customization we made

oc get deploy welcome-php -n USER_PLACEHOLDER-bgdk -o jsonpath='{.spec.template.spec.containers[].image}{"\n"}'

The output should return

quay.io/redhatworkshops/welcome-php:ffcd15

As you can see kustomize can be a powerful tool.

You can delete the applied resources with oc delete and kustomize, too

oc delete -n USER_PLACEHOLDER-bgdk -k .

Once you run this command, check that you see the following to confirm deletion

deployment.apps "welcome-php" deleted

Deploying Kustomized Application

In Chapter 2 - GitOps Basics, you learned that in a GitOps workflow the entire application stack (including infrastructure) is reflected in a Git repo. The challenge is how to do this without duplicating YAML.

So now that you’ve explored kustomize, let’s see how it fits into Argo CD and how it can be used in a GitOps workflow.

Before proceeding, move back into the examples home directory:

cd ~/openshift-gitops-workshop/content/modules/ROOT/examples

Kustomized Application

Argo CD has native support for Kustomize. You can use this to avoid duplicating YAML for each deployment. This is especially helpful if you have different environments or clusters you’re deploying to.

Take a look at this Application definition:

cat ./bgdk-app.yaml

bgdk-app.yaml

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: bgdk-app
spec:
  destination:
    namespace: $USER-bgdk
    server: https://kubernetes.default.svc
  project: default
  source:
    path: documentation/modules/ROOT/examples/bgdk
    repoURL: https://github.com/openshiftdemos/openshift-gitops-workshop
    targetRevision: master
  syncPolicy:
    automated:
      prune: true
      selfHeal: false

As you will see below, this Application definition refers to the same base resources you used in Chapter 2. It references a directory path (content/modules/ROOT/examples/bgdk) which contains a new kustomization.yaml file specific to this application. This concept is called an overlay. Where a base set of manifests are overlay-d with your customizations.

Click here to learn more about Kustomize bases and overlays,

Now take a look at this Kustomization definition:

cat ./kustomization.yaml

kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../bgd-base
patches:
- patch: |-
    - op: replace
      path: /spec/template/spec/containers/0/env/0/value
      value: yellow
  target:
    group: apps
    kind: Deployment
    name: bgd
    version: v1

This kustomization.yaml takes the base application located at content/modules/ROOT/examples/bgd-base and patches the Deployment manifest so that you get yellow dots instead of blue ones.

Now deploy this application:

sed 's/$USER/USER_PLACEHOLDER/' ~/openshift-gitops-workshop/content/modules/ROOT/examples/bgdk-app.yaml | oc apply -n USER_PLACEHOLDER-argocd -f -

The Argo CD Web Console

Switch back to the Argo CD Web UI, you may be presented with the Argo CD login screen again.

You can login using your the same credential you used for OpenShift which were provided by the workshop administrator.

This should now show you two apps on the Argo CD UI.

Open the Route for this application. It’s in the USER_PLACEHOLDER-bgdk Project. Remember you can get this from the topology view, or from the CLI:

oc get route -n USER_PLACEHOLDER-bgdk bgd -o jsonpath='{"http://"}{.spec.host}{"\n"}'

You should see yellow balls flying around. This is the same application you used previously, and so you know that by changing the environment variable on the deployment you can change the ball color.

Argo CD deployed the application with your kustomize-ations! To review what we just did:

Deployed an Application called bgd with a blue square.
Deployed another Application based on bgd called bgdk
The Application bgdk was deployed in its own namespace, with deployment customizations.
ALL without having to duplicate YAML!