Kustomize for configuration management

 Like most code, we most likely want to ultimatyely store the YAML files we use to issue commands to Kubernetes in a version control system such as Git. This leads to some cases where this format might not be ideal: for example, in a machine in a machine learning pipeline, we might perform hyperparameter searches where the same application is being run with sightly dirreent parameters, leading to a glut of duplicate command files.

Or, we might have arguments, such as AWS account keys, that for secuyrity reasons we do not want to store in a text file. We might also want to increase reuse by splitting our command into a base and additions; for example, in the YAML file show in Code 2.1, if we wanted to run ngnix alongside different databases, or specify file storage in the different cloud object stores provided by Amazon, Google, and Microsoft Azuere.

For these use cases, we will make use of the Kustomize tool(https://kustomize.io), which is also available through kubectl as:

kubectl apply -k <kustomization.yaml>

Altenatively, we could use the Kustomize command-line tool. A kustomization. yaml is a template for a Kubernetes application; for example, consider the following template for the training job in the Kubeflow example respositiory (http://github.com/kubeflow/pipelines/blob/master/mainfests/kustomize/sample/kustomization.yaml):

apiVersion: kustomize.config.k8s.io/v1beta1

kind: Kustomization

bases:

    # Or

# github.com/kubeflow/pipelines/manifests/kustomize/env/gcp?ref=1.0.0 

    - ../env/gcp

    # Kubeflow Pipelines servers are capable of 

    # collecting Prometheus metrics.

    # If you want to manitor your Kubeflow Pipelines servers

    # with those metrics, you'll need a Prometheus server

    # in your Kubeflow POipelines cluster.

    # If your Kubeflow Pipelines cluster.

    # If you don't already have a Prometheus server up, you

    # can uncomment the following configuration files for Prometheus.

    # If you have your own Prometheus server up already

    # or you don't want a Prometheus server for monitoring,

    # you can comment the flollwing line out.

    # - ../third_party/prometheus

    #- ../third_party/grafana

# Identifier for application manager to apply ownerReference.

# The ownerFeference ensures the resources get garbage collected

# when application is deleted.

commonLabels:

    application-crd-id: kubeflow-pipelines

    #Used by Kustomize

    configMapGenerator:

        - name: pipeline-install-config

        env: params.env

        behavior: merge

    

    secretGenerator:

        -name: mysql-secret

        env: paras-db-secret.env

        behavior: merge

    # !!! If you want to customize the namespcae,

    # please also update

    # sample/cluster-scoped-resources/kustomization.yaml's

    # namespace field to the same value

    namespace: kubeflow

    ### Customizaiotn ###

    # 1. Change values in params.env file

    # 2. Chage values in rarams-db-secret.env

    # file for CloudSQL username and apssword

    # 3. kubectl apply -k ./

    ###

We can see that this file refers to a base set of configurations in a separate kustomization.yaml file located at the relative path ../base. To edit variables in this file, for instance, to change the namespace for the application, we would run:

kustomize edit set namespace mykube

We could also add configuration maps to pass to the training job, using a key-value format, for example:

kustomize edit add configmap configMapGenerator --from-

literal=myval-myval

Finally, when we are read to execute these commands on Kubernetes, we can build the necessary kubectl command dynamically and apply it, assuming kustomization. yaml is in the current directory.

kustomize build . |kubectl apply -f-

Hopefully, these exampoles demonstrate how Kustomize provides a flexible wazy to generate the YAML. we need for kubectl using a template; we will make use of it often in the process of parameterizing our workflows later in this book.

Now that we have coverd how Kubernetes manages Docker applications in the cloud, and how Kustomize can allow us to flexibly reuse kubectl yaml commands, let's look at how these components are tied together in Kubeflow to run the kinds of experiments we will be undertasking later to create generative AU model in TensorFlow.