Deploying

Intro

If you want to install kubernetes on anything but your local machine, but have never used kubernetes… stop. There are countless technologies which may be used to deploy Kubernetes in all sorts of clouds and on-premise setups. People get stuck on the installation, and the resulting kubernetes clusters are often divergent from what kubernetes was meant to be. Worse yet, a novice may not notice. For example, on-premise deployments of kubernetes, given the constraints of a corporate datacenter, may be missing persistent storage and loadbalancers which most applications built for kubernetes typically depend on.

Learn via Progression

Expect to crawl before you walk. Do not skip steps.

Learning Kubernetes is a progression, and we have curated the steps, documentation, and tools to get you up to speed quickly with good practices.

If you are new to Kubernetes:

  • Do not try to install kubernetes from scratch on your own. Here’s why.
  • Install kubernetes on your local machine by following the KDF Getting Started guide for Mac or Windows. This will create a healthy kubernetes cluster for development.
  • The Getting Started guides will also step you through deploying a few basic services such as logging, monitoring, and dashboards within Kubernetes, and then introduce you to their usage and accessing their dashboards.
  • On your own, proceed to deploy a few applications beyond the tutorial using helm charts.
  • Once you understand basic kubernetes concepts and have experienced working on a healthy cluster, you may attempt to deploy your own kubernetes cluster

Afterwards, any attempts to deploy Kubernetes in the cloud or on-premise will be better understood given the environmental constraints.

Deployment Considerations

On-premise, bare-metal, and single node kubernetes deployments are often missing the following infrastructure, which decrease the benefits of Kubernetes and make it difficult for most applications which expect this basic Kubernetes functionality.

  • Persistent network storage and storage class provisioners
    • Without this, pods which rely on persistent storage are stuck on a single worker node and cannot migrate to another worker node. Its persistent volume is local to a node and cannot be network mounted on a different worker node. This drastically decreases the resilience of any stateful application in Kubernetes upon node failures.
  • Load-balancer integration
    • Without this, applications within kubernetes will not be auto-wired to be reachable from outside of kuberentes.

There are less resilient and unpalatable workarounds for the above, for example:

  • As a workaround for the lack of persistent network storage, we have run OpenEBS, but found them difficult due to skillset requirements and operational efforts. We created proof-of-concepts with CephRBD, but found you really need a dedicated team to operate it.

  • As a workaround for the lack of load-balancer integration, we have run Ingress as a Nodeport Dameonset, and added all node records to an DNS A-record. This works, but does not auto-wire upon node failures or additions such as during autoscale events.

Local Devenv

Kubernetes deployments on your development machine can be made to feel very much like a Gold Standard Kubernetes cluster, if you jump through a few hoops. We’ve created tools and a turnkey set of instructions to help you get this going with good practices and patterns. Documentation may be found in our KDF Getting Started documentation.

Recommendations

Mac

Windows

Linux

  • Minikube
    • We’ve had success with Minikube on linux.
  • Kubeadm
    • The Kubeadm project has made a lot of progress recently. It is the core installer technology used by other installers such as Kubespray. We haven’t tried it in a while, but would revisit it.
  • Microk8s
    • Microk8s is a newcomer. We haven’t tried it yet, but it is definitely worth considering.

Single Node Cluster

There are quite a few technologies for setting up a single node kubernetes cluster for production.

Recommendations

  • Kubespray

    • Kubespray is an Ansible-based installer, which may be directed at a single node or multiple nodes. It takes care of OS level configurations, and will eventually be default-configured to drive Kubeadm for the actual installing of kubernetes services.

  • Kubeadm

    • Seems to be where the momentum is for kubernetes core installation functionality.

  • Minikube is NOT recommended. The project is targeted only for development, and by default will not persist state between reboots.

Google GKE

This is our #1 choice for Kubernetes in the cloud.

AWS

AWS with Kops is our #2 choice for Kubernetes in the cloud.

  • Kops
    • We have years of experience deploying with Kops. Kops has been around for quite a while as the only option for AWS, and it has functioned solidly. It’s a highly-recommended open-source project.
  • Amazon Elastic Container Service (EKS)
    • Amazon EKS is a newcomer. We haven’t tried it yet, but it is definitely worth considering.

Bare Metal / Pre-Provisioned

Bare Metal and other Pre-Provisioned machines can be installed with Kubespray.

  • Kubespray
    • Kubespray is an Ansible-based installer, which may be directed at a single node or multiple nodes.

Openstack

This installation process is the same as Bare Metal, however first the machines must be pre-provisioned using Terraform

Vsphere

This installation process is the same as Bare Metal, however first the machines must be pre-provisioned using Terraform