Join our two fictitious z/OS system programmers as they compare their different experiences of managing an application on IBM zSystems. They’ll talk about installing applications, managing user access, managing resources, stability, and security.
For a summary of why containers have become so significant and the options on IBM zSystems, see part 1.”
To make it kind of interactive, I decided to write this blog as an interview of two fictitious people:
-
“Josh”, is a young, ;-) , 55 years old , z/OS sysprog using the traditional way of working with applications,
-
“Deb”, is also a young, 28 years old, new z/OS sysprog working with applications deployed in containers running in zCX under z/OS.
My interactions with the personas are in italics. Josh and Deb dialogs are in bold italics.
Examples: Me : Here is an example. / Josh: “Here is an example”
We are all set to start our adventure and discuss about containers on IBM zSystems.
As you may know, the job of the z/OS system programmer is very complex and requires skills in many aspects of the system.
The role of the z/OS system programmer can vary from one installation to another, but usually includes installing, customizing, and maintaining the operating system. In most large z/OS installations, system programmers become specialists for only specific tasks.
Figure 2 : Some areas for a z/OS system programmer
So Josh, could you describe how you are deploying an application to z/OS ?
Installation of an application
Josh: “For each application, I have to perform some SMP/E tasks for each product required by the application (receive/apply/accept).
I have to customize each product for the target environment (dev/test, pre-prod, prod,…) and deploy modules in each of the required sub-systems. It can be DDL for DB2 or CICS programs and so on…
In the end, I have to perform a technical validation using a business function of an application involving all the elements previously prepared.”
Yes. This is what we found in our discussions at the IBM Client Engineering with our customers, most of them need to have several experts involved on z/OS to ensure the application is installed and running fine. Do you have the same steps for containers under z/OS, Deb ?
Deb: “Not exactly. First, a fundamental difference is I am deploying applications to Linux and not directly into z/OS as Josh. zCX is running as a z/OS started task and the only way to deploy an application is to connect using SSH to a specific container running an “sshd daemon” and then to interact with the docker engine through the docker CLI.
Let me show you a picture here:”
Figure 3 : Overview of a zCX instance running in a z/OS LPAR
Deb: “So from my desktop, I ssh into the zCX appliance using its DVIPA address (here 10.1.2.1). Once connected I can do a docker pull to retrieve the images required to deploy the applications. Usually those docker images are hosted on a corporate registry but it can be an external registry like hub.docker.com (docker) or quay.io (Red Hat).”
Josh: “But this is usually performed by Linux administrators ?!”
Deb: “ Yes, you are right, and that is one of the interests of the solution. I can provide access to a zCX environment so developers can deploy their application without having to get skills on z/OS. Then the key is that zCX is a z/OS started task so I can manage it as any of the traditional subsystems.”
Managing the user access
So I have a second question for both of you.
We have seen the high level process to deploy an application in both cases but Deb, you mention there will be external persons accessing the z/OS environment. From the experience in our customer engagements, there are always questions about the security of such environment as it is running under z/OS close to critical applications and data. How do you manage the security and access? Josh, I think you are using RACF or TopSecret ?
Josh: “ Yes , correct, RACF is the security manager used to control all the access”.
Deb: “ This is a good question. Effectively, remote Linux administrators can access the appliance with ssh. There are three ways to manage the authentication of users :
-
Use a local registry by directly interacting with the Linux kernel (see Figure 3),
-
Use a remote LDAP server (OpenLDAP, Active Directory or other…)
-
And my preferred way, use a z/OS LDAP (Directory Server) with RACF integration.”
Take a look at figure6.
Ok, so what about the application layer ? How are you managing it from a security point of view ?
Josh : “This is usual work, I am using RACF and the many mechanisms provided by each of the components or products supporting the application running on z/OS”.
Deb: “I have a limited number of controls on the application, as Linux admin may deploy new elements by themselves but the Linux kernel of a zCX appliance has hardened security. Some examples I can give you are: no way to directly ssh into the Linux kernel, the operations/commands you can execute are limited to the docker interface commands, the Docker containers running inside are isolated and they can also be isolated from other environments. As a system programmer I can also limit the amount of resources consumed by a zCX instance. So I would say, as a z/OS sysprog, I can effectively control this new type of resources from a z/OS standpoint.”
Managing the resources and scalability
In their projects, some of my zCX customers are using it to run Open Source databases like MongoDB or PostgreSQL, others are using an IBM MQ concentrator pattern to optimize their general processor consumption on z/OS. zCX is exploiting zIIP processors so when the workload is dispatched onto a zIIP processor, the processing cycles used do not contribute to the MSU count and therefore do not impact software usage charges. This makes adding new applications to the IBM zSystems extremely cost-effective especially compared to competing platforms such as distributed systems or public cloud.
So regarding the control of the resources, is there any difference in the way to manage the resources when using containers with zCX ? How do you scale the application in both environments ? Josh, what is your usual way ?
Josh: “ Hum, these are two vast questions...do you mean managing resources also at the z/OS LPAR level ?”
No, I am assuming these tasks will be performed using the Hardware Management Console (HMC) and a bunch of z/OS commands.
Josh: “Yes, that’s right, this is how I usually manage the hardware resources. Many operations are then dynamically possible as for instance: adding CPU resources to an LPAR. Under z/OS , the scalability of the application can be performed in a very precise way depending on the utilized subsystems. Generally, I will start by increasing the number of units of work in the same address space, then maybe increase the number of address spaces and ultimately increase the number of z/OS LPARs.”
Interesting , so you have several levels of actions here in the traditional way of managing the environment. So Deb, what about your way with containers ? How are similar actions possible ?
Deb: “First, as Josh said , he mainly has to manage subsystems inside one z/OS LPAR. From this point of view, managing zCX instances is very equivalent as each zCX is an z/OS address space. I can define WLM policies and priorities. The difference comes from the fact that inside each zCX, there will be Docker containers running.
It means there will be a tuning to perform inside zCX for all the containers that are required to run the application. It requires a different set of skills than the z/OS System Programmer ones. These are more Linux or docker admin skills as one application may span across multiple zCX instances. Usually to manage the resources, a Linux admin will scale the number of containers for an application. Docker provides a mechanism called Docker Swarm and Red Hat OpenShift is using Kubernetes”
Oh, I got lost now, so I understand from a z/OS point of view you can manage the z/OS address space as Josh is doing but you said you can span your containers across multiple zCX ? Can you please explain how this is working ?
Deb: “Ok, let me do another drawing for you to explain, for instance, the Docker Swarm mechanism”
Figure 4 : Application containers span across multiple nodes (zCX instances)
Josh: “ So Deb, this is equivalent to my second level where I mention I will increase the number of z/OS address spaces, right ?”
Deb:” Correct Josh, but I have, first, to prepare my environment for an equivalent capability. This is during the deployment of each zCX instance. I have some requirements to follow regarding the datasets and the zFS (used to hold zCX Appliance image, configuration file, etc…). Here is an example of some pre-requirements.”
Figure 5 : Planning to deploy multiples zCX instances in a z/OS Sysplex
Ok Deb ,now I understand we have an environment with multiples zCX instances. Can you continue on the Docker Swarm mechanism for managing resources ?
Deb: “Oh yes sure, so on figure 4 above when you access port 8080 on any zCX, the swarm load balancer routes your request to an active container. It means you can now scale you application easily. This is a way to add more containers and also ensure availability of your application. The scheduler in Swarm decides on which node, ‘a zCX instance’, a service can run. And this is out of the control of WLM. This is a pure Linux/Docker management mechanism. “
So If I want to add more CPU or memory to a zCX, this is still possible and WLM will be dispatching all its vcpus on the available zIIP (or GCPs) ?
Deb: “You are right.”
Josh: “So, finally , managing a zCX is not so different from all my usual tasks ?! The specific differences come from the containers running inside. Now, I am having a better understanding of running applications with containers with zCX but I have another question for you. We talked about the application layer security but we didn’t discuss configuration security. How is this handled ?
Security/ patching (whole versus more granular approach. Minimize risk)
Deb: “So let’s start with the standard security manager for z/OS: RACF. As you are used to , Josh, a zCX STC runs under a user identity. You need to define this user and define a protected user.”
Josh: “Ok and I will also manage the users who can provision the zCX instances using z/OSMF ?
Deb: ”Yes, I see you got it! But you will also have to secure the zCX configuration files sitting under the Unix System Services (USS). Then you will need to define access to the zFS and VSAM created by the z/OSMF workflows.”
Josh: “A best practice for security is to use Dataset encryption, can I use it ?”
Deb: “Yes of course, it is even recommended to always protect each layer of the solution”.
Josh: “ What do you mean by “layers of the solution” ?”
Deb: “Ok, let me draw you another picture to show the options for containers. Each zCX instance is running a Linux operating system but no one can take root access of this Linux OS as each remote connection is performed to a container running the ssh daemon. And you remember , we already discussed about the options to manage user access.”
Figure 6 : Security overview for containers using zCX
Josh: “Oh yes sure ! It is getting a bit clear to me even if I still have many questions !. Can we now talk about the value of containers and their benefits for running applications on z/OS compared to what I am used to ?”. I see some differences but I am not yet convinced about the reasons I would run this technology for my traditional applications.
Yes Josh, good question, looks like a natural follow-up !
Let’s take a break and explore this in the next and last part of this series of blogs.
Additional References: