In my first blog, I discussed containerization in general, and introduced the concept of an OCI-compliant image that, once built, could be easily deployed to any platform where an OCI-compliant runtime can execute that image as a container.

This technology has been widely adopted by many companies as they transform their application development by deploying DevOps methodologies for faster release cycles while improving the quality of the software. At the same time, they also benefit from increased flexibility that allows them to easily deploy their applications on premises and also in the cloud without going through a lengthy install and configuration process.

But all this doesn't come by itself, right? Because I work in IT-operations, my interest is to look behind the curtains and understand how IT-personnel operate containers in practice. So, in this article, I want to take a look at the topic of Kubernetes and what makes it the de-facto standard for container orchestration today.

Dealing with the swarm

You might think operating a single container is easy. You take your CLI of choice and start a container, for instance using the Docker command docker run helloworld. In a similar way, you can also stop the container and hence freeze its current execution state from where it can resume execution at any time. Alternatively, you can remove the container completely from the runtime in which case the execution starts from the very beginning the next time you run the container. The z/OS analogy of operating this way is comparable to using MVS START, RESET QUIESCE and STOP commands on a system console, targeted to a started task.

In another example, you might have two containers, one running a client-facing web server and the other running the application server. Like in the previous example, each container could be operated, separately. However, the web-server is meaningless without the application server. In fact, unnecessary error responses are returned by the web-server when the requests cannot be served correctly. That is why you want to start the application server first and then the web server. For this, the container industry has developed a simple solution: Docker Compose. Here, a single docker-compose.yaml file describes how the multi-container application is built and how the individual services depend on each other. Docker Compose considers such dependencies, if specified, when starting this application.

Docker Compose is limited to multi-container applications running on the same host. But if there is a need to build networks of services across hosts, a solution is needed that can allow you to orchestrate and potentially scale out containers across multiple nodes in a cluster.  This is where Docker Swarm, or swarm mode as it is called, now comes to play allowing you to orchestrate and operate containerized applications at scale. For instance, it has a declarative service model where you can describe the desired state of the services the application is composed of. Swarm monitors the state and also reconciles differences between the actual and desired state. Swarm provides role-based scheduling and offers load balancing for services that are available across multiple nodes in a cluster.

It might seem that with swarm, everything is in place to enable you to operate containers at enterprise scale, but let’s now look at Kubernetes.

Introduction to Kubernetes

Kubernetes is a solution for deploying and orchestrating the container workloads across multiple nodes in a cluster, similar to swarm. In fact, Kubernetes and Docker Swarm have many characteristics in common but there are also differences. I found a quite useful comparison in this article that summarizes the difference as "simplicity vs. complexity and completeness." One of the pros on Kubernetes' side in my mind is the support of the Cloud Native Computing Foundation (CNCF) that brings a huge community that drives continuous improvement of Kubernetes and under which a large ecosystem of open source tools has arisen.

It is not possible to provide a comprehensive deep dive of Kubernetes in a short blog article like this, but allow me to highlight a few of the most important aspects to the operations of containerized workloads.

Cluster overview

At a high level, a Kubernetes cluster looks like this:

The cluster is comprised of one or more nodes which can be virtual or physical machines. Local on each node is an agent, called kubelet through which Kubernetes manages the deployment of so-called Pods on the nodes. That means, the kubelet is responsible for ensuring that the expected containers are running in a Pod and that they are healthy. However, the actual instantiation of containers, i.e. reserving storage, wiring the network and starting the process, is the job of the underlying container runtime. In many cases (e.g. with Red Hat OpenShift), the kubelet uses CRI-O as the default implementation of the so-called Kubernetes CRI (Container Runtime Interface) which includes runC that we have already seen in the first issue of this blog.

The (worker) nodes in a cluster are connected to the control plane which typically consists of multiple (control plane) nodes for high availability purposes. Administrators can use the command-line tool kubectl to control and manage the Kubernetes cluster. The tool leverages one of the services offered by the control plane, an API-service, that acts as a front end to the control-plane.

Deployments

Pods are the smallest deployable units of compute that you can create and manage in Kubernetes. A Pod consists of one or more containers that share the same network and storage resources. Typically, you have a single-container application, that means only one container runs in a Pod. But, if there is the need to tightly couple different services so they are scheduled and located on the same node, then a Pod can also contain multiple containers.

Kubernetes knows from the so-called pod spec, a piece of yaml-text, what containers are needed. Here is a snippet of such a specification which refers to the image nginx:1.14.2 and tells the container called nginx to listen on port 80.

Typically, Pods are not created directly. Rather, they are created as part of a workload resource, for instance as part of a Job for batch execution or within a Deployment.

Deployments are what I would call the most typical kind of a workload resource. Their specification declares the “to-be” state of the contained ReplicaSet and Pods. Deployments go together with a so-called controller. When the desired state of a Deployment changes, the controller adjusts the actual state to the desired state.

The Kubernetes documentation on the internet does a very nice job describing what a Deployment is and how the specification is to be read. Because it matters for operations, I'd like to reference this description. The example uses the following specification for this Deployment, called nginx-deployment:

You can see the previous Pod specification at the bottom. In addition, the Deployment spec also contains the specification of a ReplicaSet that tells Kubernetes to start 3 nginx Pods. It helps to read the specification from bottom to top: The template attribute gives the Pod a label, here app:nginx. The ReplicaSet then selects matching Pods using a matching label of app:nginx.

This might look complicated when you see this the first time, but you get quickly used to this way of describing resources in a Kubernetes environment because every resource, not just Pods, or Deployments that we have seen here, is described in a very similar way. The beauty of this declaration is that you now can create a deployment with powerful out-of-the-box capabilities that I’d like to highlight below.

Operating deployments

Now, the final step required to start this Deployment is to apply this specification. If we pretended that this specification is in a yaml file called my-nginx-deployment.yaml you can use the kubectl tool to apply it:

kubectl apply -f my-nginx-deployment.yaml

From this point on, Kubernetes takes control and manages the lifecycle of the Pods in this Deployment. If a node fails and hence the Pod no longer runs, Kubernetes can detect this and then instructs the deployment controller to start a new Pod on another Node. This way, the actual number of Pods will be adjusted to meet the desired number of Pods as specified in the deployment specification.

Kubernetes has a very powerful scheduler that selects candidate nodes based on organizational requirements, for example nodes assigned to different teams, nodes with specialized hardware, or maybe based on general availability and utilization. Once the scheduler has selected a node the kubelet on that node is asked to start the Pod.

As an administrator, you can also change the number of replicas at any time to reduce or increase the number of active Pods in the ReplicaSet. It suffices to change the deployment spec accordingly and run the kubectl apply command again.

A feature that I am really excited about is the ability to update the Pod spec within the deployment. Imagine, that the nginx server was updated from version 1.14.2 to 1.16.1 and that you want to roll-out that new version across all the Pods in the deployment. You can easily accomplish that using kubectl and set the new image you want to have for this deployment. Once this command is issued, Kubernetes one by one stops each Pod running an older container image and replaces it with a Pod using the new container image. That’s it. This is really powerful and simple. Additionally, rolling back images where there is a problem is just as easy. This is just the matter of a kubectl call to undo the previous rollout. I save the detailed commands here but if you are interested, you can find a detailed description of this process here.

So, what this brief explanation shows also characterizes the highlights of a Kubernetes-based solution.

1. With deployments you get high-availability out-of-the box.
2. The declarative management of Kubernetes allows you to specify the level of availability you expect. And Kubernetes takes care to manage the actual number of Pods to the desired number of Pods.
3. The workloads can be easily scaled up or down by changing the deployment specification. This works even automatically in combination with an autoscaler that uses resource utilization metrics to decide when to increase or decrease the Deployment.

Summary

I have to admit, Kubernetes is a fascinating technology, and, like any technology, it has some built-in complexity which requires specialized skills to successfully master it. Fortunately, you can get started with small steps. You can take one of the many online tutorials on the internet or install your own personal minikube environment on any PC or Mac.

This article focused on how Kubernetes orchestrates containers across the nodes in a cluster. In my next article, I'd like to dive into traditional IT-operations, and I’ll also revisit how operations is done on z/OS today.