This article was originaly published in 2017.07.27
Kubernetes Blue-Green Deployments Working Example
Contents
Introduction
Prerequisites
Creating a Kubernetes Cluster in IBM Bluemix
Create Hello World Containers
Creating UrbanCode Deploy Components
Creating an UrbanCode Deploy Application
Allowing UrbanCode Deploy to Access Bluemix
Deploy Load Balancer and First Version of the Webpage
Verify the Application is Running
Importing Container Version Information
Deploy the Second Version of the Webpage
Update the Load Balancer
Deploy the Third Version of the Webpage
Update the Load Balancer Again
Conclusion
Introduction
You may have heard how UrbanCode Deploy simplifies the management of Kubernetes blue-green deployments. If not, read this article and watch this video:
This document will walk through setting up the working example of a Kubernetes blue-green deployment on IBM Bluemix seen in the video above.
Prerequisites
UrbanCode Deploy (now DevOps Deploy) version 6.2.2 or later with the following plug-ins installed:
The following templates should be installed to UrbanCode Deploy. Templates may be found on GitHub in the UrbanCode Deploy templates repository.
Other requirements:
- An IBM Bluemix account (Note: you must be able to create a standard Kubernetes cluster. Trial accounts do not have the ability to create standard Kubernetes clusters).
- The Bluemix CLI
- The Kubernetes CLI
- Docker with a Docker ID (must be able to access Docker Hub)
- An UrbanCode Deploy agent should be installed on the machine where the Bluemix CLI and Kubernetes CLI have been installed.
Creating a Kubernetes Cluster in IBM Bluemix
Let’s start by creating a standard Kubernetes cluster in IBM Bluemix. Please note that trial accounts are not able to create standard clusters.
- Login to the IBM Bluemix website and, from the menu on the left, click Apps, then Containers.
- Click the Create Cluster button.
- Under Cluster type, select Standard.
- Enter a name for your cluster in the Cluster Name field and click the Create Cluster button.
- Your cluster begins to deploy and the Gain access to your cluster page appears. Click Clusters to go back to the Clusters page.
- Refresh the page until the State of your cluster shows as Ready (this may take a few minutes).
Your Kubernetes cluster has been created and is ready for use.
Create Hello World Containers
For our working example, let’s create a few versions of a Hello World container we will deploy in a blue-green fashion. We will store our containers on Docker Hub. If you would like to skip this section, you may use the containers created by the author of this article, however there is no guarantee these containers will preserved or maintained. If the author’s containers exist, they may be found here. Example files we will use later in this walk-through use the author’s images by default.
To create our Hello World container, let’s follow the example in the Docker documentation with some slight changes (we will leave out the Redis piece and make different versions of our containers).
First, create a directory on the machine where Docker is installed. Create a file named Dockerfile in that directory. The contents should be:
# Use an official Python runtime as a parent image
FROM python:2.7-slim
# Set the working directory to /app
WORKDIR /app
# Copy the current directory contents into the container at /app
ADD . /app
# Install any needed packages specified in requirements.txt
RUN pip install -r requirements.txt
# Make port 80 available to the world outside this container
EXPOSE 80
# Define environment variable
ENV NAME World
# Run app.py when the container launches
CMD ["python", "app.py"]
Next, create a file named requirements.txt and place it in the directory. Its contents should simply be:
Flask
Finally, create a file named app.py in the directory. Set its contents to be:
from flask import Flask
import os
import socket
app = Flask(__name__)
@app.route("/")
def hello():
html = "<h3>Hello {name}!</h3>" \
"<b>Hostname:</b> {hostname}<br/>" \
"<b>Version:</b> v1"
return html.format(name=os.getenv("NAME", "world"), hostname=socket.gethostname())
if __name__ == "__main__":
app.run(host='0.0.0.0', port=80)
Your directory should contain only these three files.
Now, we will create the Docker image by running the build command. Let’s name our image bgdemo. Run the command below (note the period at the end).
docker build -t bgdemo .
Our container has been created. Let’s share it with the world via Docker Hub.
First, log in to the Docker public registry by running this command and completing the prompts.
docker login
Next, tag the image with the following command, replacing username with your Docker ID.
docker tag bgdemo username/ucdbgdemo:v1
Now it’s time to publish the image. Run the following command, replacing username with your Docker ID:
docker push username/ucdbgdemo:v1
Congratulations, your container is being shared with the world.
Let’s make a second version of our container. Edit the app.py file in our directory and change the line:
"<b>Version:</b> v1"
To:
"<b>Version:</b> v2"
Save your changes.
Build this new version of the container with:
docker build -t bgdemo .
Tag the container with this command (replacing username with your Docker ID):
docker tag bgdemo username/ucdbgdemo:v2
Publish the new version of the container with this command (replacing username with your Docker ID):
docker push username/ucdbgdemo:v2
Let’s follow these steps yet again to create a third version of our container.
Edit the app.py file in our directory and change the line:
"<b>Version:</b> v2"
To:
"<b>Version:</b> v3"
Save your changes.
Build this new version of the container with:
docker build -t bgdemo .
Tag the container with this command (replacing username with your Docker ID):
docker tag bgdemo username/ucdbgdemo:v3
Publish the new version of the container with this command (replacing username with your Docker ID):
docker push username/ucdbgdemo:v3
You now have three versions of your container in Docker Hub.
Creating UrbanCode Deploy Components
We will now create two components inside UrbanCode Deploy. One will represent our load balancer, which will direct traffic to either our blue or green deployment of our webpage. The other component will represent our webpage. Note that we do not need to have separate components for the blue and green deployments; we can manage both with a single YAML file.
Let’s start by creating a YAML file to represent our load balancer. On the machine where your UrbanCode Deploy agent resides, create a directory structure of /tmp/loadbalancer/blue. Create a file in the blue directory named loadbalancer.yaml with these contents:
apiVersion: v1
kind: Service
metadata:
name: my-load-balancer
spec:
type: LoadBalancer
ports:
- port: 80
selector:
color: blue
Note the selector is set to the color blue.
Next we will create the component in UrbanCode Deploy that will manage our load balancer YAML file. In UrbanCode Deploy, go to Components and click the Create Component button. In the dialog that opens, enter a name for your component (such as bgdemo Load Balancer). In the Component Template field select Kubernetes. In the Source Configuration Type field, select File System (Versioned). In the Base Path field, enter /tmp/loadbalancer. Verify the Copy to CodeStation checkbox is checked. Under Default Version Type, select Import new component versions using a single agent, then in the Agent for Version Imports field, select the agent on the machine where you created your YAML file. Your completed dialog should look something like this:
Click the Save button to create your component.
Next, we will add our loadbalancer.yaml file to our component as a component version artifact. Click the Versions tab for our component, then click the Import New Versions button. Refresh your page until a version named blue appears.
Click on blue to examine the version. The loadbalancer.yaml file should appear as an artifact.
We now have a version of our load balancer that may be deployed to direct traffic to our blue deployment. Next, let’s create a version that directs traffic to our green deployment.
Rename the directory that contains our loadbalancer.yaml file from /tmp/loadbalancer/blue to /tmp/loadbalancer/green. Edit the loadbalancer.yaml file changing the text blue to green. Save your changes. Your loadbalancer.yaml file should now look like this:
In UrbanCode Deploy, go to your component’s Versions tab again and click the Import New Versions button. Refresh until your green version appears.
Click on the green version and verify it contains the loadbalancer.yaml file as an artifact.
Finally, let’s add a tag to our component. Click on the Components tab and find the component we created. Place your cursor over the component and click the Add Tag icon that appears. Click the Create Tag link and name your new tag Load Balancer. Pick Tangerine Yellow as the color and click the Save button. You component should now show the Load Balancer tag.
Our load balancer component is complete. We will now go through a similar process to create a component that will manage our deployed webpage.
Again, let’s start by creating a YAML file to represent our deployed webpage. On the machine where your UrbanCode Deploy agent resides, create a directory structure of /tmp/webpage/1.0. Create a file in the 1.0 directory named webpage.yaml with these contents:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: webpage-@color@
spec:
replicas: 3
template:
metadata:
labels:
color: @color@
spec:
containers:
- name: bgdemo
image: amatthew99/ucdbgdemo:v1
resources:
requests:
cpu: 100m
memory: 100Mi
env:
- name: GET_HOSTS_FROM
value: dns
ports:
- containerPort: 80
If you created your own containers and placed them on Docker Hub, change amatthew99 to your Docker ID.
Notice that the color label is set to @color@. This is a token that will be replaced with the value blue or the value green depending on which deployment we are executing. This allows us to share the same YAML file for both the blue and green deployments. Other values may be converted to tokens if those values change for each deployment. Notice the @color@ token also appears in our deployment name. Our deployments will have the names webpage-blue and webpage-green once running. You may be wondering why our container version is hardcoded as v1 in the YAML file and we are not replacing it with a token. That is because the Kubernetes UrbanCode Deploy plug-in may change that value at deploy time for you, allowing you to change container image versions without editing the YAML file.
Similar to what we did for the load balancer, we will now create the component in UrbanCode Deploy that will manage our webpage YAML file. In UrbanCode Deploy, go to Components and click the Create Component button. In the dialog that opens, enter a name for your component (such as bgdemo Webpage). In the Component Template field select Kubernetes. In the Source Configuration Type field, select File System (Versioned). In the Base Path field, enter /tmp/webpage. Verify the Copy to CodeStation checkbox is checked. Under Default Version Type, select Import new component versions using a single agent, then in the Agent for Version Imports field, select the agent on the machine where you created your YAML file.
Click the Save button to create your component.
Next, we will add our webpage.yaml file to our component as a component version artifact. Click the Versions tab for our component, then click the Import New Versions button. Refresh your page until a version named 1.0 appears.
Click on 1.0 to examine the version. The webpage.yaml file should appear as an artifact.
Finally, let’s add a tag to our component. Click on the Components tab and find the component we created. Place your cursor over the component and click the Add Tag icon that appears. Click the Create Tag link and name your new tag Kubernetes. Pick Bondi Blue as the color and click the Save button. Your component should now show the Kubernetes tag.
Creating an UrbanCode Deploy Application
We will now create an UrbanCode Deploy application to manage our blue-green deployments.
In UrbanCode Deploy, click the Applications tab, then click the Create Application button. Enter a name for your application such as Blue-Green Demo. In the Application Template field, select Kubernetes Blue-Green Deployment Application.
Click the Next button. In the Kubernetes section, from the Add Component drop-down, select the component that represents our webpage. In the Load Balancer section, from the Add Component drop-down, select the component that represents our load balancer.
Click the Next button. On this page in the wizard, we describe the environments to be created for our application. We will create three environments: one to represent our blue deployment, one to represent our green deployment, and one to represent our load balancer. Notice the Blue and Green environments have a required property named Color. The Blue environment is set to blue by default, and the Green is set to green by default. Remember the @color@ token in our webpage.yaml file? That token will be replaced by this property depending on which environment we are deploying from.
Also notice our environments are named Blue 1, Green 1, and Load Balancer 1. For the demo, we will only use three environments. Because of that, I like to remove the trailing 1, but that’s a personal preference.
Click the Next button. On this page in the wizard, we select which UrbanCode Deploy agent to use in each environment. Click and drag the agent that is on our Kubernetes CLI machine to each environment’s Agent Prototype.
Click the Create button to finish creating the application.
Allowing UrbanCode Deploy to Access Bluemix
Before we may begin deploying, there are a few more steps we have to follow to allow UrbanCode Deploy to access our Bluemix Kubernetes cluster.
First, we need to download the Kubernetes configuration files for our cluster. See this page for more information.
On the machine with the Bluemix CLI installed, run a command to login. For example:
bx login -u myUserId -p myPassword -a https://api.ng.bluemix.net
Next, initialize the Bluemix Container Service plug-in with this command:
bx cs init
Finally, get the Kubernetes configuration file for your cluster by running this command, replacing clusterName with the name of your Bluemix Kubernetes cluster:
bx cs cluster-config clusterName --admin
As the command completes, it will display the name and location of the Kubernetes config file.
In this example, my config file is /root/.bluemix/plugins/container-service/clusters/myStandardCluster-admin/kube-config-ams03-myStandardCluster.yml. Since all of the environments (Blue, Green, Load Balancer) in my UrbanCode Deploy application will use this Kubernetes configuration file, let’s set this value as an application property.
In UrbanCode Deploy, click on Applications, then click on the application we created. Click the Configuration tab, then click Application Properties. Click the Add Property button. Name our property KubeConfigFile and set the value to be the location of the Kubernetes configuration file for your cluster. For example:
Click the Save button. The property has been created.
We now need to tell the deploy process to use this configuration file while executing commands.
In UrbanCode Deploy, click on the Components tab at the top of the page, then click the Templates tab. Click on the Kubernetes template, then click on the Processes tab. Click on the Process and Apply YAML File process to edit the process.
The final step in our process is named Apply Resources. This step runs a kubectl apply command. We need to tell this step to use our Kubernetes cluster configuration file when doing so. Click the pencil icon on the step to edit its properties. Check the Show Hidden Properties checkbox. In the Global Flags field, enter this value:
--kubeconfig=${p:application/KubeConfigFile}
This will add the –kubeconfig option to our kubectl command while specifying our KubeConfigFile application property.
Click the OK button to close this dialog with our changes.
Before we save our process, we have one more thing to do. We want to ensure we are logged in to Bluemix and that our Bluemix container service plug-in is initialized before we attempt to apply our YAML files. Let’s add a step to our process to do so.
Find the Shell step on the left and drag it to our process. Let’s place it right after the Clean working directory step.
Click the pencil icon on the step to edit its properties.
In the Name field, enter Login to Bluemix and Initialize Plug-in.
In the Shell Script field, enter two lines. The first line should be the command you use to login to Bluemix. This line will be unique to your Bluemix credentials. The second line is the command to initialize the container service plug-in. For example:
bx login -u myUserId -p myPassword -a https://api.ng.bluemix.net
bx cs init
Click the OK button to close this dialog, then click the Save button to save your changes.
Deploy Load Balancer and First Version of the Webpage
We are ready to begin deploying!
Click the Applications tab, then the name of our application. You should see your three environments displayed (Load Balancer, Blue, Green). Click the Request Process button next to our Load Balancer environment.
In the Run Process on Load Balancer dialog, uncheck the Only Changed Versions checkbox. In the Process field, select Deploy Load Balancer from YAML. Click the Choose Versions link. Click the Add… button, then check blue.
Click the OK button to close the Component Versions dialog.
Click the Submit button to begin the deployment of our Load Balancer. Wait for the process to complete.
If we go back to our application in UrbanCode Deploy and click the twisty next to our Load Balancer environment, we see that version blue has been deployed.
All traffic is now being sent to our blue deployment. We better deploy our blue deployment now.
Click the Request Process button for our Blue environment. Uncheck the Only Changed Versions checkbox. In the Process field, select Deploy Application from YAML. Click the Choose Versions link. Select version 1.0 of our webpage component.
Click the OK button.
Click the Submit button to begin the deployment.
Notice we didn’t select a version of our container to deploy. Since this is the very first time we are deploying from this YAML file, it will deploy the version specified in the YAML file.
The process completes. Our webpage should be live. Let’s verify it is in the next section.
Verify the Application is Running
On the machine where the Kubernetes CLI is installed, run the command:
kubectl get services
That will return something like this:
Note the ports for the my-load-balancer service. In this example, the value is 80:30453/TCP.
We now need to get the IP address of one of our Bluemix Kubernetes cluster worker nodes. On the Bluemix webpage, click on your cluster’s name. From the menu on the left, click Worker Nodes. Note the Public IP of one of your worker nodes.
In the example above, the IP address is 185.10.231.44.
In a web browser, enter the IP address, followed by a colon, followed by the port number that was returned earlier (leaving out the 80: and /TCP). In the example above, the address would be:
185.10.231.44:30453
Press enter to navigate to the site. We should see that version v1 of our webpage is running.
Importing Container Version Information
When we ran our blue deployment, UrbanCode Deploy automatically created a new component for us that will manage the container image used in our application. We need to make sure the component is aware of image versions that are available in the image registry. The good news is, depending on your UrbanCode Deploy server and how long it took you to verify that our deployment worked, the component may have already pulled in this information for us.
Back in UrbanCode Deploy, click the Components tab. You should see a new component created with a name in the format ApplicationName-ImageName. It should also have the tag kubernetes.image. Click this new component’s name, then click the Configuration tab.
Let’s look at the Version Source Configuration section. Notice the Source Configuration Type has been set to Docker. The Image Name field has also been set. The Registry field is blank, but if you hover your cursor over the help for this field, notice that a blank field defaults to Docker Hub.
Scroll down more and look for the Import Versions Automatically checkbox.
If this is checked, UrbanCode Deploy will periodically check your container registry for new versions. Let’s see if it has done so for us already. Click the Versions tab.
If UrbanCode Deploy has already checked the container registry for container versions, you should see the three versions of our image displayed here. If you only see one version, click the Import New Versions button and refresh the page until the versions appear. Note that the images themselves are not being downloaded to UrbanCode Deploy. Instead, UrbanCode Deploy is simply building a reference of available image versions that exist in the specified image registry.
Deploy the Second Version of the Webpage
Version v2 of our container image is ready. Let’s deploy it as part of our green deployment.
Click on the Applications tab, then your application name. Click the Request Process button for the Green environment. Uncheck the Only Changed Versions checkbox. In the Process field, select Deploy Application from YAML. Click the Choose Versions link. You should now see a component that represents your webpage application, and a component that represents your container version. Select version 1.0 for the webpage component, and version v2 for your ucdbgdemo container version.
Click the OK button, then the Submit button to begin the deployment.
The process completes, and your green deployment is live. Go back to our running webpage in your browser and refresh the page. Notice the version still says version v1. That’s because we haven’t instructed the load balancer to direct traffic to the green deployment. Let’s do that next.
Update the Load Balancer
Let’s look at our application inside UrbanCode Deploy. Click the Expand All link.
In this one view, we can see that our load balancer is pointing to our blue deployment (since the current version is blue), our blue deployment is running version v1 of our ucdbgdemo container, and our green deployment is running version v2. Our staging and verification of our green deployment is complete, and we now want to switch our traffic in that direction with no downtime.
Click the Request Process button for the Load Balancer environment. Uncheck the Only Changed Versions checkbox. In the Process field, choose Deploy Load Balancer from YAML. Click the Choose Versions link. Choose green as the version to deploy.
Click the OK button, then click the Submit button to begin the update of our load balancer. Wait for the process to complete.
Our application shows that load balancer is now directing traffic to our green deployment.
To verify this is correct, go back to the running webpage in your browser and click the refresh button. The version should now display v2.
Deploy the Third Version of the Webpage
Our developers have been hard at work, and now version v3 of our container is ready. Let’s deploy it to the blue deployment.
In UrbanCode Deploy, on our application screen, click the Request Process button for the Blue environment. Uncheck the Only Changed Versions checkbox. In the Process field, select Deploy Application from YAML. Click the Choose Versions link. Select version 1.0 of our webpage component, and version v3 of our ucdbgdemo container image.
Click the OK button, then the Submit button to begin the deployment. When the process completes, version v3 of our container will be running in our blue deployment, however the load balancer is still pointing to our green deployment.
Update the Load Balancer Again
Change the load balancer to point to the blue deployment by again following the steps in the Update the Load Balancer section, selecting to deploy the blue version this time. When the process completes, refresh the webpage and verify version v3 is now running.
Conclusion
Congratulations, you now have a working example of a Kubernetes blue-green deployment. Using UrbanCode Deploy, you may easily execute zero downtime blue-green deployments. You can manage which container versions are deployed without editing files or using the command line. Audit tracking shows what was deployed when and where. Continue to explore UrbanCode Deploy and see all that is has to offer!
#Kubernetes
#docker
#UrbanCodeDeploy