Kustomize for configuration management

 Like most code, we most likely want to ultimatyely store the YAML files we use to issue commands to Kubernetes in a version control system such as Git. This leads to some cases where this format might not be ideal: for example, in a machine in a machine learning pipeline, we might perform hyperparameter searches where the same application is being run with sightly dirreent parameters, leading to a glut of duplicate command files.

Or, we might have arguments, such as AWS account keys, that for secuyrity reasons we do not want to store in a text file. We might also want to increase reuse by splitting our command into a base and additions; for example, in the YAML file show in Code 2.1, if we wanted to run ngnix alongside different databases, or specify file storage in the different cloud object stores provided by Amazon, Google, and Microsoft Azuere.

For these use cases, we will make use of the Kustomize tool(https://kustomize.io), which is also available through kubectl as:

kubectl apply -k <kustomization.yaml>

Altenatively, we could use the Kustomize command-line tool. A kustomization. yaml is a template for a Kubernetes application; for example, consider the following template for the training job in the Kubeflow example respositiory (http://github.com/kubeflow/pipelines/blob/master/mainfests/kustomize/sample/kustomization.yaml):

apiVersion: kustomize.config.k8s.io/v1beta1

kind: Kustomization

bases:

    # Or

# github.com/kubeflow/pipelines/manifests/kustomize/env/gcp?ref=1.0.0 

    - ../env/gcp

    # Kubeflow Pipelines servers are capable of 

    # collecting Prometheus metrics.

    # If you want to manitor your Kubeflow Pipelines servers

    # with those metrics, you'll need a Prometheus server

    # in your Kubeflow POipelines cluster.

    # If your Kubeflow Pipelines cluster.

    # If you don't already have a Prometheus server up, you

    # can uncomment the following configuration files for Prometheus.

    # If you have your own Prometheus server up already

    # or you don't want a Prometheus server for monitoring,

    # you can comment the flollwing line out.

    # - ../third_party/prometheus

    #- ../third_party/grafana

# Identifier for application manager to apply ownerReference.

# The ownerFeference ensures the resources get garbage collected

# when application is deleted.

commonLabels:

    application-crd-id: kubeflow-pipelines

    #Used by Kustomize

    configMapGenerator:

        - name: pipeline-install-config

        env: params.env

        behavior: merge

    

    secretGenerator:

        -name: mysql-secret

        env: paras-db-secret.env

        behavior: merge

    # !!! If you want to customize the namespcae,

    # please also update

    # sample/cluster-scoped-resources/kustomization.yaml's

    # namespace field to the same value

    namespace: kubeflow

    ### Customizaiotn ###

    # 1. Change values in params.env file

    # 2. Chage values in rarams-db-secret.env

    # file for CloudSQL username and apssword

    # 3. kubectl apply -k ./

    ###

We can see that this file refers to a base set of configurations in a separate kustomization.yaml file located at the relative path ../base. To edit variables in this file, for instance, to change the namespace for the application, we would run:

kustomize edit set namespace mykube

We could also add configuration maps to pass to the training job, using a key-value format, for example:

kustomize edit add configmap configMapGenerator --from-

literal=myval-myval

Finally, when we are read to execute these commands on Kubernetes, we can build the necessary kubectl command dynamically and apply it, assuming kustomization. yaml is in the current directory.

kustomize build . |kubectl apply -f-

Hopefully, these exampoles demonstrate how Kustomize provides a flexible wazy to generate the YAML. we need for kubectl using a template; we will make use of it often in the process of parameterizing our workflows later in this book.

Now that we have coverd how Kubernetes manages Docker applications in the cloud, and how Kustomize can allow us to flexibly reuse kubectl yaml commands, let's look at how these components are tied together in Kubeflow to run the kinds of experiments we will be undertasking later to create generative AU model in TensorFlow.

Important Kubernetes commands

 In order to interact with aKubernetes cluster running in the cloud, we typically utlize the Kubernets command-line tool(kuberctl). Instructions for installing kubectl for your operating system can be found at(https://kubernetes.io/docs/tasks/tools/install-kubectl/).To verify that you have successfully installed kubectl, you can again run the help command in the terminal

kubectl --help

Like Docker, kubectl has manay commands; the important one that we will use is the apply command, which, like docker-compose, thakes in a YAML file as input and communicates with the Kubernetes control plane to start, update, or stop pods:

kubectl apply -f <file.yaml>

As an example of how the apply command works, let us look at a YAML file for deploying a web server(nginx) application:

apiVersion: v1

kind: Service

metadata:

    name: my-nginx-svc

    labels:

        app: nginx

    spec:

        type: LooadBalacer

        ports:

            - ports: 80

        selector:

            app: nginx

---

apiVersion: apps/v1

kind: Deployment

metadata:

    name: my-nginx

    labels:

        app: nginx

    spec:

        replicas: 3

        selector:

            matchLabels:

                app: nginx

        template:

            metadata:

                labels:

                    app: nginx

            spec:

                containers:

                    -name: nginx

                    image: nginx:1.7.9

                    ports:

                        - containerPort: 80

The resources specified in this file are created on the Kubernetes cluster nodes in the order in which they are listed in the file. First, we create the load balancer, which routes external traffic between copies of the nginz web server. The metadata is used to gag these applications for querying later using kubectl. Secondly, we create a set of 3 replicas of the nginx pod, using a consistent container (image 1.7.9), which users port 80 on their repective containers.

The same set of physical resources of a Kubernetes cluster can be shared among serral virtual clusters using namespaces-this allows us to segregate resources among multiple users or groups. This can allow, for example, each team to run their own set of applications and logically behave as if they are the only users. Later, in our discussion of Kubeflow,we will see how this feature can be used to logically partition projects on the same Kubeflow instance.

Kuberanetes: Robust management of multi-container applications

 The Kubernetes project - sometimes abbreviated as k8s- was born out of an internal container management project at Google known as Borg. Kubernetes comes from the Greek word for navigator, as denoted by the seven-spoke whell of the project's log. Kubernetes is written in the Go protgramming language and provids a robust framework to deploy and manage Docker container applications on the underlying resources managed by cloud proviers(such as Amazon Web Service(AWS), Microsoft Azure, and Google Cloud Platform(GCP)).

Kubernetes is fundamentally a tool to control applications composed of one or more Docker containers deployed in the cloud: this collection fo containers is known as a pod. Each pod can bave one or more copies (to allow redundancy), which is known as a replicaset. The two main components of a Kubernetes deployment are a control plane and nodes. The control plane hosts the centralized log for ldeploying and managing pods, and consists of (Figure 2.4):

- Kube-api-server: This is the main application that listens to commands from the user to deploy or update a pod, or manages external access to pods via ingress.

- Kube-controller-manager: An application to manage functions such as controlling the number of replicas per pod.

- Cloud-controller-manager: Manages functions particular to a cloud provider.

- Etcd: A key-value store that maintains the environment and state variables of different pods.

- Kube-scheduler: An application that is responsibile for finding workers to run a pod.

While we could set up own control plane, in practice we will usually have this function managed by our cloud provider, such as Google Kubernetes Engine(GKE) or Amazon's Elastic Kubernetes Services(EKS). The Kubernetes nodes-the individual machines in the cluster - each run an application known as a kubelet, which monitors the pod(s) running on that node.

Now that we have a high-level view of the Kubernetes system, let's look at the important commands you will need to interact with a Kubernetes cluster, update its components, and start and stop applications.

Connecting Docker containers with docker-compose

 So raf we have only discussed a few basic Docker commands, which would allow us to run a single servic ein a singel container. however, you can probably appreciate that in the "real world" we usually need to have one or more applications running concurrently-for example, a website will have both a web application that fetches and processes data inresponse to activity from an end use and a database instance to log that information. In complex applications, the website might even be composed of multiple small web applications or microservices that are specialized to particular use case such as the fron end, user data, or an order management system. For these kinds of applications, we will need to have more than one container communicating with each other. The docker-compose tool(https://docs.docker.com/compose/)is written with such application in mind: it allows us to specify serverl Docker containers in an application file using the YAML format. For example, a configuration for a website with an instance of the Redis ddatabase might look like:

version: '3'

service:

    web:

        build:

        port:

            -"5000:5000"

        volumes:

            -.:/code

            -logvolume01:/var/log

        links:

            -redis

        redis:

            image: redis

    volumes:

        logvolume01: {}

The tow application containers here are web and the redis database. The file also specified the volumes (disks) linked to these two applications. Using this configuration, we can run the command:

docker-compose up

This starts all the containers secified in the YAML file and allows them to communicate with each other. However, even though Docker containers and docker-compose allow us to construct complex applications using consistent execution environments, we may potentially run int issues with robustness when we deploy these services to the cloud. For example, in a web application, we connot be assured that the virtual machines that the application is running on will persist over long periods of time, so we need processes to manage self-healing and redundancy. This is also relevant to distributed machine learning pipeline, in which we do not want us to have backup logic to restart a sub-segment of work. Also, whuile Docker has the docker-compose functionallity to link together serveral containers in an application, it does not have robust rules for how communication should happen among those containers, or how to manage them as a unit. For these purposes, we turn to the Kubernetes library.

Important Docker commands and syntax

 To understand how Docker works, it is useful to walk through the template used for all Docker containers, a Dockerfile. As an example, we will use the TensorFlow container notebook example from the Kubeflow project(https://github.com/kubeflow/kubeflow/blob/master/components/example-notebook-servers/jupyter-tensorflow-full/cpu.Dockerfile).

This file is a set of instructions for h ow Docker should take a base operating environment, add dependencies, and execute a piece of software once it is packaged:

FROM public.ecr.aws/jlr09q0g6/notebook-servers/jupyter-tensorlflow:master-abf9ec48

# install - requirements.txt

COPY -- chown=jovyan:users requirements.txt /tmp/requirements.txt

RUN python3 -m pip install -r /tmp/requirements.txt --quiet --no-cache-dir \

    && rm -f /tmp/requirements.txt

While the exact commands will differ between containers, this will give you a flavor for the way we can use containers to manage an application - in this case running a Jupyternotebook for interactive machine learning experimentation useing a consistent set of libraries. Once we have installed the Docker runtime for our particular operating system, we would execute such a file by running:

Docker build -f <Dockerfilename> -t <image name:tag>

When we do this, a number of things happen. First, we retrieve the base filesystem, or image, from a remote repository, which is not unlike the way we collect JAR files from Artifactory when using Java build tools such as Gradle or Maven, or Python's pip installer. With this filesystem or image, we then set required variables for the Docker build command such as the username and TensorFlow version, and runtime environment variables for the container. We determine what shell program will be used to run the command, then we install dependencies we will need to run TensorFlow and the notebook application, and we specify the command that is run when the Docker container is started. Then we save this snapshot with an identifier composed of a base image name and one or more tags (such as version numbers, or, in many cases, simply a timestamp to uniquely identify this image). Finally, to actually start the notebook server running this container, we would issue the command:

Docker run <image name:tag>

By default, Docker will run the executable command in the Dockerfile file; in our present example, that is the command to start the notebook server. However, this does not hazve to be the case; we could have a DockerFile that simply builds an execution envirnment for an application, and issue a command to run within that environment. In that case, the command would look like:

Docker run <image name:tag> <command>

Docker push <image name:tag>

Note that the iamge name can contain a reference to a particular registry, such as a local registry or one hosted on one of the major cloud providers such as Elastic Container Server(ECS) on AWS, Azure Kubernetes Service(AKS), or Google Container Registry. Publishing to a remote registry allows developers to share images, and us to make containers accessible to deploy in the cloud.

Docker: A lightweight virtualization solution

 A consistent challenge in developing robust software applications is to make them run the same on a machine different than the one on which they are developed. These differences in environments could encompass a number of variables: operating systems, programming language library versions, and hardware such as CPU models.

Traditionally, one approach to dealing with this heterogeneity has been to use  Virtual Machine(VM). While VMs are useful to run applications on diverse hardware and operating systems, they are also limited by being resource-intensive(Figure 2.3): each VM running on a host requires the overhead resources to run a completely separate operating system, along with all the applications of dependencies within the guest system.

However, in some cases this is an unnecessary level of overhead; we do not necessarily need to run an entirely separate operating system, rather than just a consistent environment, including libraries and dependencies within a single operating system. This need for a lightweight framework to specify runtime envirnments prompted the creation of the Docker project for containerization in 2013. In essence, a container is an environment for running an application, including all dependencies and libraries, allowing reproducible deployment of web applications and other programs, such as a database or the computations in a machine learning pipeline. For our use case, we will use it to provide a reproducible Python execution environment (Python language version and libraries) to run the steps in our generative machine learning pipelines.

We will need to have Docker installed for many of the examples that will appear in the rest of this chapter and the projects in this book. For instructions on how to install Docker for your particular operating system, please refer to the directions at (https://docs.docker.com/ install). To verify that you have installed the applications successfully, you should b able to run the following command on your terminal, which will print the available options:

docker run hello-world