You've been waiting for a long time, and now it's here!
Thanks to the z/OS Open Tools project, which is specialized in porting Open Source tools to the z/OS platform, the z/OS-native GitLab Runner is now available to the community, allowing a seamless integration of the z/OS environment into enterprise CI/CD pipelines.
The use of a native GitLab Runner on the target platform also unlocks interesting capabilities:
- The communication channel is initiated by the agent through an HTTPS-based protocol, which has become an industry standard now. As some users are reluctant to build communication and automation over the SSH protocol, the use of HTTPS channels removes this technical requirement that was present when using SSH executors with GitLab Runners running outside of z/OS.
- In pipeline definitions, the use of a native runner enables the `artifact` keyword which helps in uploading artifacts produced during the build phase of pipelines. As such, log files, build reports and test results can now easily be uploaded to the GitLab server, which becomes the single point of control for developers when checking for the execution of pipelines and analyzing the results of each pipeline stage.
This post is not meant to be a step-by-step configuration guide, instead it is aiming to give general guidance on:
Before we start, it is important to take note of the support structure for this z/OS-native GitLab Runner. At the time of writing this post, neither the z/OS Open Tools project team nor GitLab offer a support model for the z/OS-native GitLab Runner.
However, in case you encounter a technical problem with the ported runner itself, issues can be reported on the z/OS Open Tools GitHub repository's issues page. More information on support provided by the z/OS Open Tools project can be found on this page.
This port of the GitLab Runner on z/OS can be qualified as experimental, and its use is under your responsibility.
Now let's dive in!
Installing the GitLab Runner on z/OS
As mentioned, the port of the GitLab Runner to the z/OS platform has been performed by the z/OS Open Tools project, which also proposes a vast collection of popular Open Source tools for z/OS.
It all starts with the installation of the Package Manager, which provides the zopen utility. The instructions provided on this aforementioned documentation page will help you get the zopen environment properly set up and configured.
The Package Manager enables you to select which components need to be installed on z/OS. Typically, this utility downloads binaries on demand from z/OS Open Tools's github releases and this requires a direct connection to the internet (and more precisely to github.com) to work properly. However, an installation with local PAX files can be also done for air-gapped environments.
Starting the GitLab Runner installation
Once the Package Manager is correctly installed, the zopen list command will list all the utilities available and their associated statuses on your z/OS machine.
Here is the entry corresponding to the GitLab Runner, not installed yet, with the version available at the time of writing this post:
| Package |
Installed |
Available |
Latest Tag |
| gitlab-runner |
Not installed |
N/A |
runner.20240321_160142 |
The zopen install gitlab-runner command will start the installation of the GitLab Runner, which is fairly straightforward:
- Querying repo for latest package information.
Processing package: gitlab-runner
- Installing gitlab-runner...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 1583 100 1583 0 0 1490 0 0:00:01 0:00:01 --:--:-- 0
- Downloading gitlab-runner.20240321_160142.zos.pax.Z file from remote to zopen package cache...
- Download complete.
- Processing gitlab-runner.20240321_160142.zos.pax.Z...
After this operation, 202.10 MB of additional disk space will be used.
Do you want to continue? [y/n]
y
- Expanding gitlab-runner.20240321_160142.zos.pax.Z
/
- Integration complete.
- Checking for env file.
- .env file found, adding to profiled processing.
- Sourcing environment to run any setup.
Setting up gitlab...
Setup completed.
- Checking for runtime dependencies.
- No runtime dependencies found.
Successfully installed: gitlab-runner
The gitlab-runner executable is now available on your z/OS machine, in a subfolder of the z/OS Open Tool installation. In my case, it is located at /var/zopen-0.8.2/zopen/usr/share/zopen/prod.
This program is the GitLab Runner for z/OS itself, equivalent to the executables you would download from GitLab when installing an on-premise agent on any other platform. As a reference, this page of the official GitLab documentation details how the installation can be done for officially supported platforms. A similar approach will be used for the z/OS GitLab Runner.
Configuring the GitLab Runner
On z/OS, long-running tasks or daemons usually run as Started Tasks. Since the GitLab Runner is supposed to run as a background service, we will create a user with the necessary credentials to only run the pipeline tasks.
Commands described in Chapter 3 of this documentation can be used to create a user and register them to run a given Started Task. Your z/OS system programmer should be able to provide the right configuration for your environment. In my configuration, the user created to run the STC is called GITLABR.
In the HOME directory of the STC User (/u/gitlabr in my configuration), copy the gitlab-runner executable:
cp /var/zopen-0.8.2/zopen/usr/share/zopen/prod/gitlab-runner/gitlab-runner.20240321_160142.zos/bin/gitlab-runner /u/gitlabr
The GitLab Runner is built on Go, which has been available in the z/OS environment for the last few years. To be able to correctly run on z/OS, Go programs need to run a specific script that will remove duplicate values for the PATH environment variables. This script, called .env, is also shipped with the GitLab Runner and is located in a subfolder of the z/OS Open Tools installation directory.
You will need to copy this configuration file to the HOME directory of the user running the STC:
cp /var/zopen-0.8.2/zopen/usr/share/zopen/prod/gitlab-runner/gitlab-runner.20240321_160142.zos/.env /u/gitlabr
Make sure the ownership of these files is set to the user running the GitLab Runner, and their file permissions are set accordingly.
Running the executable should show the following output:
MDALBIN - /u/gitlabr > ./gitlab-runner
Runtime platform arch=s390x os=zos pid=67178 revision=HEAD version=development version
NAME:
gitlab-runner - a GitLab Runner
USAGE:
gitlab-runner [global options] command [command options] [arguments...]
VERSION:
development version (HEAD)
AUTHOR:
GitLab Inc. <support@gitlab.com>
COMMANDS:
exec execute a build locally
list List all configured runners
run run multi runner service
register register a new runner
reset-token reset a runner's token
install install service
uninstall uninstall service
start start service
stop stop service
restart restart service
status get status of a service
run-single start single runner
unregister unregister specific runner
verify verify all registered runners
artifacts-downloader download and extract build artifacts (internal)
artifacts-uploader create and upload build artifacts (internal)
cache-archiver create and upload cache artifacts (internal)
cache-extractor download and extract cache artifacts (internal)
cache-init changed permissions for cache paths (internal)
health-check check health for a specific address
read-logs reads job logs from a file, used by kubernetes executor (internal)
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--cpuprofile value write cpu profile to file [$CPU_PROFILE]
--debug debug mode [$RUNNER_DEBUG]
--log-format value Choose log format (options: runner, text, json) [$LOG_FORMAT]
--log-level value, -l value Log level (options: debug, info, warn, error, fatal, panic) [$LOG_LEVEL]
--help, -h show help
--version, -v print the version
The next step is to register the z/OS GitLab Runner to your GitLab server. In our setup, we're using an on-prem GitLab instance. Following the process described on this page, we will register this new runner using a runner authentication token.
You can choose to create a new Linux agent (we will cheat a little bit here!) and specify the zos-native tag:
Creating the runner in GitLab will provide a command to execute on z/OS to register the new runner with the provided token, for instance:
gitlab-runner register --url http://10.3.20.96 --token glrt-5p5S2whiH-w__K9uGtzn
During the registration of the agent on z/OS, you can specify a name for the agent and the type of executor. For this option, we will choose the shell executor which allows the execution of shell commands on z/OS Unix System Services. One important detail is that the GitLab Runner expects to use bash located at /bin/bash. If not already done, please check Chapter 2.1 of this document which provides useful information for this setup.
The configuration file is saved in the /etc/gitlab-runner/config.toml file when the configuration is done with a root user, but when running with a non-root user (which will be the case when running as a STC), the GitLab Runner searches for its configuration in ~/.gitlab-runner/. We just need to copy the file /etc/gitlab-runner/config.toml into the ~/.gitlab-runner/ folder of the user who will run the STC, with this command:
cp /etc/gitlab-runner/config.toml /u/gitlabr/.gitlab-runner
Inside this config.toml file, two lines must be added to allow the runner to correctly clone repositories to the right location.
Add these two lines in the runner section of the config.toml file:
[runners.custom_build_dir]
enabled = true
Also, make sure this configuration file is owned by the user who runs the STC.
To wrap the gitlab-runner executable in a shell script, we are providing this snippet that we will later use in the STC definition:
#! /bin/sh
PRGPATH="`dirname "$0"`"
cd $PRGPATH
source $PRGPATH/.env
$PRGPATH/gitlab-runner run
The GitLab Runner doesn't start automatically, which is a good thing as we want to run it through the STC that we defined previously. Here is a sample PROC that can be used to start the GitLab Runner as an STC:
//GITLABR PROC HOME='/u/gitlabr/',
// BASH='/bin/bash'
//RUNNER EXEC PGM=BPXBATSL,REGION=0M,TIME=NOLIMIT,
// PARM='PGM &BASH. &HOME./gitlab-runner.sh'
//STDOUT DD PATH='&HOME/stdout',
// PATHMODE=SIRWXU,
// PATHOPTS=(OWRONLY,OCREAT)
//STDERR DD PATH='&HOME/stderr',
// PATHMODE=SIRWXU,
// PATHOPTS=(OWRONLY,OCREAT)
//STDENV DD *
_BPXK_AUTOCVT=ON
//*
// PEND
You can now start the STC. If everything goes well, you should see the following unformatted output in the STDERR file:
Runtime platform [0;m arch [0;m=s390x os [0;m=zos pid [0;m=33622070 revision [0;m=HEAD version [0;m=development version
[0;33mWARNING: No service system detected. Some features may not work! [0;m
Starting multi-runner from /u/gitlabr/.gitlab-runner/config.toml... [0;m builds [0;m=0 max_builds [0;m=0
[0;33mWARNING: Running in user-mode. [0;m
[0;33mWARNING: Use sudo for system-mode: [0;m
[0;33mWARNING: $ sudo gitlab-runner... [0;m
[0;m
Configuration loaded [0;m builds [0;m=0 max_builds [0;m=1
listen_address not defined, metrics & debug endpoints disabled [0;m builds [0;m=0 max_builds [0;m=1
[session_server].listen_address not defined, session endpoints disabled [0;m builds [0;m=0 max_builds [0;m=1
Initializing executor providers [0;m builds [0;m=0 max_builds [0;m=1
And the runner should now be available in the list of runners on your GitLab server instance:
The installation part is now done, and the runner can be used in a CI/CD pipeline!
Building a CI/CD pipeline with the GitLab Runner on z/OS
For this part, we will leverage two components that were recently delivered to the community through the public dbb repository: the Common Backend Scripts and the GitLab CI pipeline template.
Implementing the pipeline steps with the Common Backend Scripts
Never heard about the Common Backend Scripts?
This set of scripts were designed to accelerate the implementation of a CI/CD pipeline for z/OS applications, by providing the foundations for building, packaging and deploying mainframe artifacts. These scripts can be used by any CI/CD orchestration platform.
To benefit from these scripts, the first step will be to make the dbb repository available to your development environment. Usually, you would host this repository in your own Git based central server - it can be GitLab of course - to better control its lifecycle.
Prior to using the scripts, some configuration must be performed in the pipelineBackend.config file. The configuration process is the not main course of today's post, you can refer to the README page for more information on how to set up these scripts.
As part of the configuration of the Common Backend Scripts, some environment variables must be defined (typically in the profile of the user) for the scripts to work properly. One of these variables is PIPELINE_WORKSPACE, and it defines where the Common Backend Scripts are expecting to find the root workspace where applications will be cloned. This variable is then used when computing the paths to the different files when running the different scripts.
In my case, this variable points to where the GitLab Runner will clone the applications' repositories:
export PIPELINE_WORKSPACE=/u/gitlabr/builds
This variable can be exported in the .profile file of the user that runs the GitLab Runner on z/OS (gitlabr in my configuration).
Once the Common Backend Scripts are configured correctly, we can then proceed with the creation of the CI/CD pipeline definition in GitLab.
Creating the pipeline definition in GitLab
Never heard about the Pipeline Templates?
The GitLab Pipeline Template was recently released, to help GitLab users quickly implement CI/CD pipeline in GitLab while they leverage the Common Backend Scripts. Another template exists for Azure DevOps while we are still working on one or two others.
The GitLab Pipeline template can be seen as a skeleton, that you can configure and tweak to implement your own steps as part of the pipeline process. It provides a good canvas for your own implementation, and showcases how the Common Backend Scripts can be integrated together for building, packaging and deploying z/OS components. The provided template makes use of the Zowe RSE API plugin to drive commands on the z/OS system where the pipeline is executing.
As we will use the z/OS-native GitLab Runner configured earlier, we will need to tweak this template a bit to remove all the occurrences of the Zowe RSE API plugin.
Defining an application in GitLab
The next step will be to define a project in GitLab where I already migrated the source code of my z/OS application. In my case, I'm using an (old version of an) application called CICS Banking Sample Application (CBSA). But this work can be done with any of your own applications.
You can read more about the migration process on this documentation page.
Integrating the GitLab CI/CD pipeline definition
As mentioned, the published GitLab Pipeline Template makes use of the Zowe RSE API plugin to connect to the target z/OS system where actions will be performed. As we are using a z/OS-native runner, we can skip the configuration of the Zowe RSE API plugin, and remove all the occurrences from the GitLab CI/CD definition to move to the native invocation of the Common Backend Scripts.
We can also remove some of the steps which are no longer required, for instance the steps that package the output log files into archives that are downloaded and attached to the executing pipeline in the GitLab server. To provide the same capability, we will use the artifact keyword in the CI/CD pipeline definition, to document the output files that we want to archive as part of the pipeline execution. Leveraging this z/OS-native GitLab Runner significantly simplifies the GitLab CI/CD pipeline definition.
You can find the sample .gitlab-ci.yml file at the bottom of this post. This sample can be integrated in your application's repository and customized accordingly, to enable GitLab CI/CD pipelines with the z/OS-native GitLab Runner.
Final words
Congratulations! You now have a working pipeline running natively on z/OS based on GitLab. This pipeline implementation follows the recommended branching principles for mainframe development!
This high-level guide showcased how to install, configure and use the z/OS-native GitLab Runner to build your own CI/CD pipeline with GitLab. Though not meant to be a step-by-step cookbook, this blog post should provide you with enough cursors to understand and plan the implementation of the z/OS-native GitLab Runner in a pipeline definition in your environment.
Again, keep in mind the experimental aspect of the GitLab Runner port to z/OS. However, it is a good exercise to experiment with this runner and start implementing your CI/CD pipeline using the Common Backend Scripts!
Sample GitLab CI/CD pipeline definition for z/OS-native GitLab Runner
variables:
GIT_STRATEGY: none
GIT_CHECKOUT: "false"
#Git strategy is set to "none" but overwrite later to "clone" in the jobs to create release candidate tag and release tag.
application: "CBSA"
wdEnvironmentFileIntegration: ${WAZI_DEPLOY_SAMPLES}/plum-samples/external-repos/environment-conf/python/EOLEB7-CBSA-Integration.yml
wdEnvironmentFileAcceptance: ${WAZI_DEPLOY_SAMPLES}/plum-samples/external-repos/environment-conf/python/EOLEB7-CBSA-Acceptance.yml
wdEnvironmentFileProduction: ${WAZI_DEPLOY_SAMPLES}/plum-samples/external-repos/environment-conf/python/EOLEB7-CBSA-Production.yml
packageName: ${application}.build-${CI_PIPELINE_ID}
# Working directory on USS
uniqueWorkspaceId: ${CI_PROJECT_DIR}
baselineReferenceFile: $CI_PROJECT_DIR/CBSA/application-conf/baselineReference.config
verbose: "-v"
pipelineType:
value: "build"
options:
- "build"
- "release"
- "preview"
description: "The pipeline type to indicate a build pipeline (build only with test/debug options) or a release pipeline (build for optimized load modules), or if it runs in preview mode. Set to 'build' by default."
releaseType:
value: "patch"
options:
- "major"
- "minor"
- "patch"
description: "The release type to indicate the scope of the release: a major, minor or patch (bug fixes) release"
stages:
- Setup
- Build
- Packaging
- Deploy Integration
- Deploy Acceptance
- Deploy Production
- Finalize
- Cleanup
# Note: All the jobs are set not to add to pipeline if there is TAG commit.
Clone:
stage: Setup
tags: [zos-native]
rules:
- if: $CI_COMMIT_TAG
when: never
- when: always
dependencies: []
variables:
GIT_STRATEGY: clone
GIT_CHECKOUT: "true"
GIT_DEPTH: 0
script: |
# Environment parameters
echo Environment parameters
echo [INFO] Unique workspace = ${uniqueWorkspaceId}
echo [INFO] Git repository URL = ${CI_REPOSITORY_URL}
echo [INFO] Branch = ${CI_COMMIT_REF_NAME}
echo [INFO] Application = ${application}
echo [INFO] Project directory = ${CI_PROJECT_DIR}
echo [INFO] Pipeline Type = ${pipelineType}
echo [INFO] Release Type = ${releaseType}
echo [INFO] Verbose = ${verbose}
1-Build application:
needs: ["Clone"]
stage: Build
tags: [zos-native]
rules:
- if: $CI_COMMIT_TAG
when: never
- when: on_success
dependencies: []
artifacts:
name: "buildReport-${CI_PIPELINE_ID}"
when: always
paths:
# Uncomment or comment the type of file that o not want to publish to Gitlab artifacts
- "logs/*.log"
- "logs/*.html"
- "logs/*.json"
- "logs/*.xml"
- "logs/*.txt"
script: |
dbbBuild.sh -w ${uniqueWorkspaceId} -a ${application} -b ${CI_COMMIT_REF_NAME} -p ${pipelineType} ${verbose}
RC=$?
echo [TESTING] RC ===== ${RC}
if [ ${RC} -eq 0 ]; then
echo [INFO] Build job passed.
else
if [ ${RC} -eq 4 ]; then
echo [WARNING] Build job failed due to no source code changes. Exit code: ${RC}.
else
echo [ERROR] Build job failed. Exit code: ${RC}.
exit ${RC}
fi
fi
2-Create release candidate tag:
needs: ["1-Build application"]
stage: Build
tags: [zos-native]
rules:
# The job only run if the pipelineType is "release".
- if: $CI_COMMIT_TAG
when: never
- if: $pipelineType != "release"
when: never
- when: on_success
dependencies: []
script: |
echo [INFO] Retrieve baseline reference information from ${baselineReferenceFile}.
# Retrieve the baseline reference from the baselineReferenceFile
if [ ! -f "${baselineReferenceFile}" ]; then
echo [ERROR] Applications baseline reference configuration file ${baselineReferenceFile} was not found.
exit 1
else
# Extract the current branch from GitLab envirnment
export mainBranchSegment=`echo ${CI_COMMIT_BRANCH} | awk -F "/" '{ print $1 }'`
export secondBranchSegment=`echo ${CI_COMMIT_BRANCH} | awk -F "/" '{ print $2 }'`
export thirdBranchSegment=`echo ${CI_COMMIT_BRANCH} | awk -F "/" '{ print $3 }'`
echo [INFO] Branch ${CI_COMMIT_BRANCH}
echo [INFO] Branch segment ${mainBranchSegment}, ${secondBranchSegment}, ${thirdBranchSegment}
# Find base line version of the current branch from the baseLineReferenceFile
case ${mainBranchSegment} in
"main")
export baselineRef=`cat "${baselineReferenceFile}" | grep "^${mainBranchSegment}" | awk -F "=" '{ print $2 }'`
;;
"release")
# echo [INFO] Extract baseline for release branch
export baselineRef=`cat "${baselineReferenceFile}" | grep -m 1 "^release/${secondBranchSegment}" | awk -F "=" '{ print $2 }'`
#echo [INFO] Baselinereference = ${baselineRef}
;;
esac
if [ -z "${baselineRef}" ]; then
echo [ERROR] No baseline ref was found for branch name ${Branch} in ${baselineReferenceFile}.
exit 1
else
echo [INFO] Baseline Reference: ${baselineRef}
fi
# Compute the name of the next release based on the releaseType
if [ "${releaseType}" == "patch" ]; then
export newVersion=`echo ${baselineRef} | sed 's/^["refs\/tags\/rel-]*//g' | sed 's/-[a-zA-Z0-9]*//g' | awk -F. -v OFS=. '{$3 += 1 ; print}'`
fi
if [ "${releaseType}" == "minor" ]; then
export newVersion=`echo ${baselineRef} | sed 's/^["refs\/tags\/rel-]*//g' | sed 's/-[a-zA-Z0-9]*//g' | awk -F. -v OFS=. '{$2 += 1 ; $3 = 0; print}'`
fi
if [ "${releaseType}" == "major" ]; then
export newVersion=`echo ${baselineRef} | sed 's/^["refs\/tags\/rel-]*//g' | sed 's/-[a-zA-Z0-9]*//g' | awk -F. -v OFS=. '{$1 += 1 ; $2 = 0; $3 = 0; print}'`
fi
export newVersionTag=$(echo "rel-${newVersion}")
echo [INFO] Computed tag for the next release: ${newVersionTag}
# Compute the name of the next release candidate
export releaseCandidate=$(git tag --list "${newVersionTag}_*rc*" --sort=-committerdate | head -n 1)
echo [INFO] Current release candidate tag: ${releaseCandidate}
if [ -z "${releaseCandidate}" ]; then
# initial version tag
export newVersionTag="${newVersionTag}_rc00"
echo [INFO] Updated release candidate tag: ${newVersionTag}
else
# bump up release candidate number
export newVersionTag=`echo ${releaseCandidate} | sed 's/^["refs\/tags\/rel-]*//g' | awk -F "rc" '{print "rel-"$1"rc"(-f2 $2+1)}' | tr -d \"`
echo [INFO] Updated release candidate tag: ${newVersionTag}
fi
# Creates the git tag using the CLI REST interface and the automation token
curl --request POST --header "PRIVATE-TOKEN: ${AutomationToken}" "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/repository/tags?tag_name=${newVersionTag}&ref=${CI_COMMIT_REF_NAME}"
RC=$?
if [ ${RC} -eq 0 ]; then
echo [INFO] Created release candidate tag: ${newVersionTag}.
else
echo [ERROR] Failed to create release candidate tag. Exit code: ${RC}.
exit ${RC}
fi
fi
Packaging:
needs: ["1-Build application"]
stage: Packaging
tags: [zos-native]
rules:
- if: $CI_COMMIT_TAG
when: never
- when: on_success
dependencies: []
script: |
packageBuildOutputs.sh -w ${uniqueWorkspaceId} -t ${application}.tar -a ${application} -b ${CI_COMMIT_REF_NAME} -p ${pipelineType} -v ${packageName}
RC=$?
if [ ${RC} -eq 0 ]; then
echo [INFO] Packaging job passed.
else
echo [ERROR] Packaging job failed. Exit code: ${RC}.
exit ${RC}
fi
1-Generate Deployment Plan:
needs: ["Packaging"]
stage: Deploy Integration
tags: [zos-native]
rules:
- if: $CI_COMMIT_TAG
when: never
- if: $pipelineType == "preview"
when: never
- when: on_success
artifacts:
name: "GenerateDeploymentPlanReport-${CI_PIPELINE_ID}"
when: always
paths:
- "deployPkgDir/deploymentPlan.yaml"
- "deployPkgDir/deploymentPlanReport.html"
script: |
wazideploy-generate.sh -w ${uniqueWorkspaceId} -i ${application}.tar
RC=$?
if [ ${RC} -eq 0 ]; then
echo [INFO] Generate deployment plan job passed.
else
echo [ERROR] Generate deployment plan job failed. Exit code: ${RC}.
exit ${RC}
fi
2-Deploy to Integration:
needs: ["1-Generate Deployment Plan"]
stage: Deploy Integration
tags: [zos-native]
rules:
- if: $CI_COMMIT_TAG
when: never
- if: $pipelineType == "preview"
when: never
- when: on_success
dependencies: []
script: |
wazideploy-deploy.sh -w ${uniqueWorkspaceId} -e ${wdEnvironmentFileIntegration} -i ${application}.tar -l deploy/evidences/evidence.yaml
RC=$?
if [ ${RC} -eq 0 ]; then
echo [INFO] Deploy to integration job passed.
else
echo [ERROR] Deploy to integration job failed. Exit code: ${RC}.
exit ${RC}
fi
3-Generate and publish deployment report for Integration:
needs: ["2-Deploy to Integration"]
stage: Deploy Integration
tags: [zos-native]
rules:
- if: $CI_COMMIT_TAG
when: never
- if: $pipelineType == "preview"
when: never
- when: on_success
dependencies: []
artifacts:
name: "integrationDeploymentReport-${CI_PIPELINE_ID}"
when: always
paths:
- "deployPkgDir/deploy/deployment-report.html"
- "deployPkgDir/deploy/evidences/evidence.yaml"
script: |
wazideploy-evidence.sh -w ${uniqueWorkspaceId} -l deploy/evidences/evidence.yaml -o deploy/deployment-report.html
RC=$?
if [ ${RC} -eq 0 ]; then
echo [INFO] Generated deployment evidence.
else
echo [ERROR] Failed to generate deployment evidence file. Exit code: ${RC}.
exit ${RC}
fi
4-Cleanup Development:
needs: ["3-Generate and publish deployment report for Integration"]
stage: Deploy Integration
tags: [zos-native]
variables:
FF_ENABLE_JOB_CLEANUP: 1
rules:
- if: $CI_COMMIT_TAG
when: never
- if: $pipelineType == "preview"
when: never
- when: manual
script: |
deleteWorkspace.sh -w ${uniqueWorkspaceId}
RC=$?
if [ ${RC} -eq 0 ]; then
echo [INFO] Clean up development job passed.
else
echo [ERROR] Clean up development job failed. Exit code: ${RC}.
exit ${RC}
fi
1-Deploy to Acceptance:
needs: ["2-Deploy to Integration"]
stage: Deploy Acceptance
tags: [zos-native]
rules:
- if: $CI_COMMIT_TAG
when: never
- if: $pipelineType == "preview" || $pipelineType == "build"
when: never
- when: manual
#- if: $CI_COMMIT_REF_PROTECTED == "true"
# when:
dependencies: []
script: |
wazideploy-deploy.sh -w ${uniqueWorkspaceId} -e ${wdEnvironmentFileAcceptance} -i ${application}.tar -l deploy-acceptance/evidences/evidence.yaml
RC=$?
if [ ${RC} -eq 0 ]; then
echo [INFO] Deploy to acceptnace job passed.
else
echo [ERROR] Deploy to acceptnace job failed. Exit code: ${RC}.
exit ${RC}
fi
2-Generate and publish deployment report for Acceptance:
needs: ["1-Deploy to Acceptance"]
stage: Deploy Acceptance
tags: [zos-native]
rules:
- if: $CI_COMMIT_TAG
when: never
- if: $pipelineType == "preview" || $pipelineType == "build"
when: never
- when: on_success
dependencies: []
artifacts:
name: "acceptanceDeploymentReport-${CI_PIPELINE_ID}"
when: always
paths:
- "deployPkgDir/deploy-acceptance/evidences/evidence.yaml"
- "deployPkgDir/deploy-acceptance/deployment-report.html"
script: |
wazideploy-evidence.sh -w ${uniqueWorkspaceId} -l deploy-acceptance/evidences/evidence.yaml -o deploy-acceptance/deployment-report.html
RC=$?
if [ ${RC} -eq 0 ]; then
echo [INFO] Generated deployment evidence.
else
echo [ERROR] Failed to generate deployment evidence file. Exit code: ${RC}.
exit ${RC}
fi
1-Deploy to Production:
needs: ["1-Deploy to Acceptance"]
stage: Deploy Production
tags: [zos-native]
rules:
- if: $CI_COMMIT_TAG
when: never
- if: $pipelineType == "preview" || $pipelineType == "build"
when: never
- when: manual
#- if: $CI_COMMIT_REF_PROTECTED == "true"
# when: on_success
dependencies: []
script: |
wazideploy-deploy.sh -w ${uniqueWorkspaceId} -e ${wdEnvironmentFileProduction} -i ${application}.tar -l deploy-production/evidences/evidence.yaml
RC=$?
if [ ${RC} -eq 0 ]; then
echo [INFO] Deploy to production job passed.
else
echo [ERROR] Deploy to production job failed. Exit code: ${RC}.
exit ${RC}
fi
2-Generate and publish deployment report for Production:
needs: ["1-Deploy to Production"]
stage: Deploy Production
tags: [zos-native]
rules:
- if: $CI_COMMIT_TAG
when: never
- if: $pipelineType == "preview" || $pipelineType == "build"
when: never
- when: on_success
dependencies: []
artifacts:
name: “productionDeploymentReport-${CI_PIPELINE_ID}"
when: always
paths:
- "deployPkgDir/deploy-production/evidences/evidence.yaml"
- "deployPkgDir/deploy-production/deployment-report.html"
script: |
wazideploy-evidence.sh -w ${uniqueWorkspaceId} -l deploy-production/evidences/evidence.yaml -o deploy-production/deployment-report.html
RC=$?
if [ ${RC} -eq 0 ]; then
echo [INFO] Generated deployment evidence.
else
echo [ERROR] Failed to generate deployment evidence file. Exit code: ${RC}.
exit ${RC}
fi
Create production version:
needs: ["1-Deploy to Production"]
stage: Finalize
tags: [zos-native]
rules:
- if: $CI_COMMIT_TAG
when: never
- if: $pipelineType == "preview" || $pipelineType == "build"
when: never
- when: manual
#- if: ($CI_COMMIT_REF_NAME == "main" && $CI_COMMIT_REF_PROTECTED == "true")
# when: on_success
script: |
echo [INFO] Retrieve baseline reference information from ${baselineReferenceFile}.
# Retrieve the baseline reference from the baselineReferenceFile
if [ ! -f "${baselineReferenceFile}" ]; then
echo [ERROR] Applications baseline reference configuration file ${baselineReferenceFile} was not found.
exit 1
else
# Extract the current branch from GitLab envirnment
export mainBranchSegment=`echo ${CI_COMMIT_BRANCH} | awk -F "/" '{ print $1 }'`
export secondBranchSegment=`echo ${CI_COMMIT_BRANCH} | awk -F "/" '{ print $2 }'`
export thirdBranchSegment=`echo ${CI_COMMIT_BRANCH} | awk -F "/" '{ print $3 }'`
echo [INFO] Branch: ${CI_COMMIT_BRANCH}
echo [INFO] Branch segment: ${mainBranchSegment}, ${secondBranchSegment}, ${thirdBranchSegment}
# Find base line version of the current branch from the baseLineReferenceFile
case ${mainBranchSegment} in
"main")
export baselineRef=`cat "${baselineReferenceFile}" | grep "^${mainBranchSegment}" | awk -F "=" '{ print $2 }'`
;;
"release")
export baselineRef=`cat "${baselineReferenceFile}" | grep -m 1 "^release/${secondBranchSegment}" | awk -F "=" '{ print $2 }'`
;;
esac
if [ -z "${baselineRef}" ]; then
echo [ERROR] No baseline ref was found for branch name ${Branch} in ${baselineReferenceFile}.
exit 1
else
echo [INFO] Baseline Reference: ${baselineRef}
fi
#Compute the current version
export oldVersionTag=`echo ${baselineRef} | awk -F "-" '{ print $2 }'`
echo [INFO] Old version: ${oldVersionTag}
# Compute the name of the next release based on the releaseType
if [ -z "${releaseType}" ]; then
export releaseType="patch"
fi
if [ "${releaseType}" == "patch" ]; then
export newVersionTag=`echo ${baselineRef} | sed 's/^["refs\/tags\/rel-]*//g' | sed 's/-[a-zA-Z0-9]*//g' | awk -F. -v OFS=. '{$3 += 1 ; print}'`
fi
if [ "${releaseType}" == "minor" ]; then
export newVersionTag=`echo ${baselineRef} | sed 's/^["refs\/tags\/rel-]*//g' | sed 's/-[a-zA-Z0-9]*//g' | awk -F. -v OFS=. '{$2 += 1 ; $3 = 0; print}'`
fi
if [ "${releaseType}" == "major" ]; then
export newVersionTag=`echo ${baselineRef} | sed 's/^["refs\/tags\/rel-]*//g' | sed 's/-[a-zA-Z0-9]*//g' | awk -F. -v OFS=. '{$1 += 1 ; $2 = 0; $3 = 0; print}'`
fi
echo [INFO] New version: ${newVersionTag}
# Update baselineReference
export remoteFound=`git remote | grep gitlab_origin | wc -l`
if [ $remoteFound -eq 1 ]; then
git remote remove gitlab_origin
fi
git remote add gitlab_origin ${CI_SERVER_PROTOCOL}://Automation:${AutomationToken}@${CI_SERVER_HOST}:${CI_SERVER_PORT}/${CI_PROJECT_PATH}.git
cat ${baselineReferenceFile} | sed "s/main=refs\/tags\/rel-${oldVersionTag}/main=refs\/tags\/rel-${newVersionTag}/" > ${baselineReferenceFile}.new
iconv -f ISO8859-1 -t IBM-1047 ${baselineReferenceFile}.new > ${baselineReferenceFile}
chtag -tc IBM-1047 ${baselineReferenceFile}
echo "release/rel-${newVersionTag}=refs/tags/rel-${newVersionTag}" >> ${baselineReferenceFile}
rm ${baselineReferenceFile}.new
git add ${baselineReferenceFile}
git commit -m "Add maintenance release to baselineReference.config file"
git push gitlab_origin HEAD:main -o ci.skip
curl --request POST --header "PRIVATE-TOKEN: ${AutomationToken}" "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/repository/tags?tag_name=rel-${newVersionTag}&ref=${CI_COMMIT_REF_NAME}"
RC=$?
if [ ${RC} -eq 0 ]; then
echo [INFO] Created release candidate tag: ${newVersionTag}.
curl --request POST --header "PRIVATE-TOKEN: ${AutomationToken}" "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/repository/branches?branch=release%2Frel-${newVersionTag}&ref=rel-${newVersionTag}"
RC=$?
if [ ${RC} -eq 0 ]; then
echo [INFO] Created release maintenance: ${newVersionTag}.
else
echo [ERROR] Failed to create release maintenance. Exit code: ${RC}.
exit ${RC}
fi
else
echo [ERROR] Failed to create release candidate tag. Exit code: ${RC}.
exit ${RC}
fi
fi
Cleanup:
needs: ["Create production version"]
stage: Cleanup
tags: [zos-native]
variables:
FF_ENABLE_JOB_CLEANUP: 1
rules:
- if: $CI_COMMIT_TAG
when: never
- if: $pipelineType == "preview" || $pipelineType == "build"
when: never
- when: manual
script: |
deleteWorkspace.sh -w ${uniqueWorkspaceId}
RC=$?
if [ ${RC} -eq 0 ]; then
echo [INFO] Cleanup job passed.
else
echo [ERROR] Cleanup job failed. Exit code: ${RC}.
exit ${RC}
fi
Please consider to customize this pipeline definition for your own environment!