Installing Kubeflow in AWS

 In order to run Kubeflow in AWS, we need a Kubernetes control plane available in the cloud. Fortunately, Amazon provides a amanged service called EKS, which provides an easy way to provision a control plane to deploy Kubeflow. Follow the following steps to deploy Kubeflow on AWS:

    1. Register for an AWS account and install the AWS Command Line Interface

This need to interact with the various AWS services, following the instructions for your platform located at https://docs.aws.amaxon.com/cli/latest/userguid/cli-chap-install.html. Once it is installed, enter:

aws configure

to setup your account and key information to provision resources.

    2. Install ekstl

This command-line utility allows us to provision a Kubernetes control plane in Amazon from the command line. Follow instructions at https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html to intall.

    3. Install iam-authenticator

To allow kubectl to interact with EKS, we need to provide the correct permissions using the IAM authenticator to modify our kubeconfig. Please see the installation instructions at https://docs.aws.amazon.com/eks/latest/userguide/install-aws-iam-authenticator.html.

    4. Download the Kubeflow command-line tool

Links are located at the Kubeflow releases page(https://github.com/kubeflow/kubeflow/releases/tag/v0.7.1).Download one of these directories and unpack the tarball using:

tar -xvf ktctl_v0.7.1_<platform>.tar.gz

    5. Build the configuration file

After entering environment variables for the Kubeflow application director (${KF_DIR}), the name of the deployment (${KF_NAME}), and the path to the base configuration file for the deployment ($(CONFIG_URI}), which is located at https://raw.githubusercontent.com/kuveflow/manifests/v0.7-branch/kfdef/kfctl_aws.0.7.1.yaml from AWS deployments, run the following to generate the configuration file:

mkdir -p ${KF_DIR}

cd ${KF_DIR}

kfctl build -V -f ${CONFIG_URI}

This will generate a local configuration file locally named kfctl_aws.0.7.1.yaml. If this looks like Kustomize, that's becuase kfctl is using Kustomize under the hood to build the configuration. We also need to add an environment variable for the location of th local config file, ${CONFIG_FUILE}, which in this case is:

export CONFIG_FILE={KF_DIR}/kfctl_aws.0.7.1.yaml

    6. Launch Kubeflow on EKS

Use the following commands to lunch Kubeflow:

cd {KF_DIR}

rm -rf kustomize/

kfctl apply -V -f ${CONFIG_FILE}

It will take a while for all the Kubeflow components to become available;

you can check the progress by using the following command:

kubectl -n kubeflow get all

Once they are all available, we can get the URL address for the Kuberflow dashboard using:

kubectl get ingress -n istio-system

This will take us to the dashboard view shown in the MiniKF examples above.

Note that in the default configuration, this address is open to the public; for secure applications, we need to add authentication using the instructions at https://www.kubeflow.org/docs/aws/authentication/.