Using IBM UrbanCode Deploy’s buztool to package and push Mainframe Artifacts to JFrog Artifactory and Codestation
Author: Randy Langehennig
Version 1.0, 12/5/2022
Overview
The purpose of this blog post is to try to provide an example of how buztool (the z/OS Deploy Toolkit) is used to create a new component version for deployment to support the mainframe. We will also show how the artifact for deployment (a .zip file) can be pushed to JFrog Artifactory at the same time as pushing it to IBM UrbanCode Deploy's (now IBM DevOps Deploy) CodeStation repository as shown in the diagram below:
The Packaging Step in a Pipeline
Here is an example pipeline in GitLab for the build, packaging, and deployment of mainframe programs:
For this blog post, I am focusing on the ‘Packaging’ phase and step as shown above. This example seamlessly integrates with IBM Dependency Based Build (DBB) which is used in the Build and Test step of the process. If you were to examine the output log of the packaging step, you will find output that looks like this:
*** Parsing DBB build report /u/gitlab/gitlab-runner/zos/builds/retirementCalculator/master/BUILD-OUTPUT/BuildReport.json.
** Deployable files detected in /u/gitlab/gitlab-runner/zos/builds/retirementCalculator/master/BUILD-OUTPUT/BuildReport.json
GITLAB.RETCALC.MASTER.LOAD(EBUD01), LOAD
** Generate UCD ship list file
Creating general UCD component version properties.
Storing DBB Build result properties as general component version properties due to single build report.
Creating shiplist record for build output GITLAB.RETCALC.MASTER.LOAD(EBUD01) with recordType EXECUTE.
** Write ship list file to /u/gitlab/gitlab-runner/zos/builds/retirementCalculator/master/BUILD-OUTPUT/shiplist.xml
The packaging step will first parse the Dependency Based Build (DBB) build report (BuildReport.json) to detect the new deployable files that were the result of the code that was modified and built using DBB’s impact build. This step will create a “shiplist.xml” which is used by buztool to create a new deployable artifact.
As the process continues, here is an example call to the buztool command:
** Following UCD buztool cmd will be invoked
/var/ucd-agent/bin/buztool.sh createzosversion -c retirementCalculator-GitLab -s /u/gitlab/gitlab-runner/zos/builds/retirementCalculator/master/BUILD-OUTPUT/shiplist.xml -o /u/gitlab/gitlab-runner/zos/builds/retirementCalculator/master/BUILD-OUTPUT/buztool.output -prop /u/gitlab/gitlab-runner/zos/builds/retirementCalculator/master/retirementCalculator/application-conf/artifactory.properties -v "version-7146"
The createzosversion command is called by buztool.sh and the following parameters are important here:
-c = the name of the IBM UrbanCode Deploy Component where the new Component Version will be pushed
-s = the fully qualified path to the shiplist.xml that specifies the artifacts to be deployed
-prop = the path to the artifactory.properties file (more to come shortly)
-v = the version label used as the new Component Version found in both codestation and Artifactory
When the command completes, you should see output that looks like this:
** Create version by running UCD buztool
zOS toolkit config : /var/ucd-agent/ (7.2.1.0,20211011-0758)
zOS toolkit binary : /var/ucd-agent/ (7.2.1.0,20211011-0758)
zOS toolkit data set : RATCFG.UCD.V7R2M1 (7.2.1.0,20211011-0758)
Reading parameters:
....Command : createzosversion
....Component : retirementCalculator-GitLab
....Version : version-7146
....Shiplist file : /u/gitlab/gitlab-runner/zos/builds/retirementCalculator/master/BUILD-OUTPUT/shiplist.xml
....Buztool Properties File : /u/gitlab/gitlab-runner/zos/builds/retirementCalculator/master/retirementCalculator/application-conf/artifactory.properties
If you were to login to the IBM UrbanCode Deploy web console, you would find the new component version added to the Component (retirementCalculator-GitLab) as shown below:
If you click on component “version-7146”, you should see the details as shown below including the artifact and deploy type:
Artifactory Integration
In more recent release of IBM UrbanCode Deploy, the ability to push the artifact to an external repository was added with the buztool command. In order to push the artifact to JFrog Artifactory, you will need to create an artifactory.properties file that looks like this:
#Wed Oct 21 10:50:31 GMT+01:00 2020
EXTREPO.apiKey=pbe{hnfkb8REPLACEMEK54lGTbTREPLACEME/JGH/H/cGREPLACEME}
EXTREPO.url=http://10.x.x.x:8081/artifactory
EXTREPO.Repository_type=artifactory
EXTREPO.repository=RetirementCalculator
EXTREPO.username=admin
The properties file will include an apiKey needed to authenticate with your JFrog Artifactory instance, the external repository URL, the repository name, and the external repository user associated with the apiKey.
When the buztool command is executed, it will generate a buztool.output file that looks like this:
#UCDz toolkit output properties for creating version
#Wed Nov 30 19:20:36 CET 2022
version.url=https\://ucd-hostname\:8443//\#version/331100a5-5d3c-4257-b5bd-6223e1b4be36
version.repository.type=CODESTATION
version.id=331100a5-5d3c-4257-b5bd-6223e1b4be36
version.name=version-7146
component.name=retirementCalculator-GitLab
version.shiplist=/u/gitlab/gitlab-runner/zos/builds/retirementCalculator/master/BUILD-OUTPUT/shiplist.xml
Behind the scenes, a version-7146.zip file was pushed to Artifactory. You could find it at this example URL: http://10.x.x.x:8081/artifactory/RetirementCalculator/version-7146.zip
Back in IBM UrbanCode Deploy (UCD), if you view the new component version and click on the Configuration sub-tab, you will find the extRepoURL property is set as shown below:
For the deployment process, we will need the Artifactory API KEY to pull down this example artifact as well. Here is an example of the “Download Artifacts for zOS External Repo” step:
Summary
Integrating with an external repository like JFrog Artifactory is easily accomplished as we modernize the way we build and deploy our z/OS code for the Mainframe. I hope this example provides guidance to those looking to achieve this in their CI/CD pipelines. In the latest UrbanCode Deploy 7.3.0 release, you can now pass arguments to the buztool command to integrate with JFrog Artifactory as well without the need for the properties file if you prefer.
I would like to send a note of THANKS to the DevOps Acceleration Team (DAT) and specifically @Mathieu Dalbin and Shabbir Moledina for allowing me to use their live DAT environment to help build this blog post. The DAT team are deep experts in helping IBM customers modernize their z workloads. If you would like to learn more about what their team can offer to you, please checkout their web site to learn more about the DevOps Acceleration Program.
#UrbanCodeDeploy
#DevOps