DevOps Platform

 View Only

Bringing DevOps to the Mainframe using GitLab Ultimate and z DevOps Tooling

By Randall Langehennig posted Tue October 26, 2021 05:22 PM

  

Bringing DevOps to the Mainframe with GitLab Ultimate and z DevOps Tooling

Author: Randy Langehennig

Overview

In this blog post, we look at some exciting capabilities which can help transform your Mainframe development and release processes to help you move at a faster pace to keep up with the open systems side of the business.   This means moving away from a “waterfall” approach of development and release to a more “agile” approach which allows for intelligent builds, the incorporation of automated testing, and provide incremental release automation.   The result is better quality code that is released much more frequently helping your organization to work to meet customer demands and expectations.

What is GitLab?

IBM and GitLab announced an exciting partnership on January 12, 2021 to provide a seamless experience in the software delivery workflow across multiple environments and diverse toolchains including the mainframe.

GitLab is an enterprise-class, robust DevOps platform which provides a single pane of glass for your entire development workflow.    GitLab unifies all the necessary components to control the delivery of software from a central point.   With the solution, you can leverage built-in features including:

  • Repositories
  • Issues and Merge Requests
  • GitLab CI/CD Pipelines (for build and release management)
  • Value Stream Management (VSM)
  • Package Registry (for your artifacts)

GitLab can integrate with your existing investments as well including, Jira, Bugzilla, Redmine, Confluence, IBM EWM, and more.

What is GitLab Ultimate for IBM z/OS?

IBM recently announced a new “GitLab Ultimate for IBM z/OS” offering to help bring enterprise-wide DevOps automation for applications being developed to run on IBM z/OS.   This enables automated build for traditional z/OS applications that use COBOL and PL/I using the IBM Dependency Based Build (DBB) capability that comes alongside the GitLab offering.

Further, you can build upon this foundation and integrate with other IBM DevOps tools like IBM Developer for z/OS (IDz), IBM Z Virtual Test Platform, IBM UrbanCode Deploy (UCD) and much more as shown below: 


The remainder of this blog post will walk you through the day in the life of a mainframe developer working on a CICS-based COBOL application called “Catalog Manager” to help highlight the capabilities and benefits of this exciting offering.

Day in the Life of a Mainframe Developer

For this example, a member of the test team found a bug with the “Catalog Manager” program and has reported the issue.   Let’s look at the application and the issue using our Host Connection Emulator:

 

For the screen above, the user selects #1 to list all the items.  The next screen is shown:

 
The user then selects item “0010” to order.   The next screen is displayed as shown below:

 

Here is where we find the bug.   The description for item “0010” has a *DEBUG* message instead of the description we would expect.   As a result, using GitLab, the tester created a new issue for our project to help describe the issue as shown below:


Creating a Merge Request

In this example, the developer can open this issue and then examine notes provided by the tester for the defect.   The developer can then click the “Create merge request” button to begin working to correct the issue in his or her own isolated branch.

For this blog post, the developer has already merged his change into the feature branch.   Let’s review the code change by clicking the “Open in Web IDE” button as shown below:

 

A developer can use their favorite IDE including IBM Developer for z (IDz), VS Code, and more by checking out the branch of code used with this merge request.   There are benefits for using IBM Developer for z (IDz) including have a smart editor that understands COBOL, PL/I code and allowing one to pull up copybooks side-by-side with your COBOL code as examples.  

For this quick review of the capabilities, I am going to use the Web IDE provided in GitLab.  The developer edits the “DFH0XVDS.cbl” COBOL program and finds that on lines 345 to 353, there is a bug in the code as shown below which is inserting the DEBUG message in the description field:

 

Our developer comments out lines 345 to 353 by inserting a “*” and then clicks the “Commit…” button to commit the changes.   In the next dialog that appears, our developer commits the code to the feature branch that GitLab automatically created for us and provides a comment as shown below:


Let’s take a look at the GitLab CI/CD pipeline next.

The GitLab CI/CD Pipeline and z/OS DevOps

Since we just committed our code to our feature branch, we will now complete the merge request by merging the change back into the “development” branch which is our “main” for this blog post.    When the merge is approved, GitLab will run a CI/CD pipeline to build the code, run quality checks, and package and deploy the code to our Integration (INT) environment for additional testing.

To get started, we will go back to our merge request as shown below and open it:

 

As the developer, I will press the “Mark as ready” button as shown below to notify my team that the code is ready to be merged back into the “development” branch:

 

At this point, you could have someone assigned to approve the merge and you certainly have someone on your team that would have the ability to press the Merge button as shown below:

 

In this case, we click the Merge button to merge the code back into the development branch and our CI/CD pipeline will begin to run as shown below:

 

In this pipeline, a build phase will run which leverages IBM Dependency Based Build (DBB) to perform an impact build.   The build is intelligent in that it can detect the code that was modified, and it will only build those files that changed along with any other files that have a dependency on the files that you changed.    In the past, using legacy tooling, one would have to perform a “full build” that compiled all the files (e.g. 100 Cobol programs) for a project.    This build could take a long time to complete.   Using this modern DevOps tooling, we can develop our code and work in isolation to build and deploy solutions in a much more agile manner.   Here is the output of the build:

 

We can see that our build was clean and it completed in a short amount of time (only 24 seconds!).   Further, as part of this build phase, we are running some zUnit tests leveraging the IDz zUnit feature.   We can see the output here:

 

The zUnit tests that are run are in context of the code changes you committed.   In other words, based on the file changes made, it knows which z Unit tests should be executed.   In this case, the test for module “TDFH0XVD” passed.   What is also nice is that the results of these zUnit tests are transformed into a standard jUnit format so they can be presented to the user in the CI/CD pipeline results as shown below:

 

We are now in the Analysis phase as shown below:

 

In this phase, we are leveraging an IDz capability called “Code Review” to analyze our COBOL or PL/I program to ensure the code being committed is meeting our company standards or requirements.    To do this, the “Code Review” capability will check your code against “rules”.   Examples of rules include:

  • Cyclomatic Complexity
  • Avoiding language elements obsolete in Enterprise COBOL 5.1
  • Avoiding a “select *” in EXEC SQL

You can write your own rules as well as leverage the out-of-box rules.   Further, based on severity, you can fail the build if a rule is violated.   In this case, we did not have any critical or severe rule violations, so our build continues to the next phase called “Packaging” as shown below:

 

In the packaging phase, our CI/CD pipeline is leveraging IBM UrbanCode Deploy to package up the incremental changes so you can ultimately deploy them to a target environment.     The immutable artifact of the build is “pushed” to JFrog Artifactory which is our definitive asset repository as shown below: 

 

The packaging phase also leverages the “buztool” command to make a REST API call to the IBM UrbanCode Deploy server to make it aware of the new component version (see example below: "rel-1.0.1-3708") and include version links back to the GitLab pipeline, the code commit, and the DBB build result as shown below:

 

The buztool command leverages the output of the DBB build process to generate a “shiplist” of artifacts that can be deployed to your target environment.   Finally, the component version in IBM UrbanCode Deploy refers back to Artifactory to pull down the assets for deployment.

 

In our next phase, we deploy the incremental update to the Catalog Manager application in the Integration environment as shown below:

 

This phase is leveraging IBM UrbanCode Deploy to perform the automated deployment so that our team can test the updates in an Integration environment to ensure the application is working well alongside other programs.   IBM UrbanCode Deploy will reflect the deployment as it keeps track of inventory of what versions are deployed in each target environment as shown below:

 

This “incremental” deployment is very quick in comparison to the legacy solution we used in the past.   In the past, when we wanted to update our application, we had to promote every application from DEV to INTEGRATION and this introduced risk and was also time consuming.  The process was more aligned to “waterfall” processes.   When we are satisfied with the results of our testing in the Integration environment, we can then manually trigger the deployment to the “Acceptance” environment by clicking the “play” button in the pipeline as shown below:

 

The “Acceptance” environment has more production-like data, a more complete database, and more so we can take the next step to ensure the changes meet our needs.   In IBM UrbanCode Deploy, we can see the results of the deployment as shown below:


As a result of the deployment, we can connect to the CICS region and validate that the issue has been resolved as shown below:

Conclusion

This is an exciting new world for our mainframe team using more modern tooling to meet goals of quicker delivery with great quality.  

GitLab Ultimate for z/OS is valuable for the z development team because they are typically separate from the distributed team and they use decades old legacy tools for development.   The legacy tools are not agile and are a hindrance to continuous delivery.   Further, many members of the z dev team are retiring, so a more modern platform like GitLab will help with on-boarding new developers who can use the same tools as the distributed teams.   GitLab reduces tool-chain complexity, replaces expensive legacy tools, and unites the z developers with the distributed developers.   Both can be more agile. 

If you would like to learn more about GitLab Ultimate for z/OS, visit the IBM page.

Many thanks to @Mathieu Dalbin and the DevOps Acceleration Team for their assistance with the this blog.   The DAT team are deep experts in helping IBM customers modernize their z workloads.   If you would like their assistance with z DevOps, please check out their web site here: https://ibm.github.io/mainframe-downloads/DevOps_Acceleration_Program/devops-acceleration-program.html

Permalink