Continuous Integration with Micro Services Builder (Jenkins) on ICP with IIB and MQ

This article and associated collateral will explore the deploying of IBM MQ and IBM Integration Bus on IBM Cloud Private (ICP) using the Micro Services Builder pipeline to automate the delivery of integration.
Scenario Overview Diagram

The scenario shown above simulates these coarse grained steps:

1. Multiple IIB developers working independently and unit testing on their local machines.

2. Once they are happy with their work they “check-in” (export) their BAR files (and MQSC scripts) into a source repository.

3. These updates are pushed to public a Github IIB-MQ repository set up to deliver an IIB and MQ System Integration Test (SIT) environment on IBM Cloud Private (Kubernetes container Service).

4. A GitHub webhook initiates a Micro Services Builder Pipeline (Jenkins) process on ICP that has been registered as an OAuth application for the IIB-MQ repository on GitHub.

5. The pipeline (re)builds and deploys the IIB and MQ SIT environment with all the latest IIB and MQ artefacts deployed and ready for socialization testing.

IIB MQ automated deployment scenario in detail

The “cookbook” style document, currently still in draft form, IBM-MQandIIB-IBMCloudPrivate-MicroServicesBuilder.pdf is available at the following URL:
https://github.com/DAVEXACOM/IIB-MQ-SupportingRepo/blob/master/IBM-MQandIIB-IBMCloudPrivate-MicroServicesBuilder.pdf

is focused on the IIB and MQ automated build pipeline using Micro Service Builder (Jenkins) pipeline to enable developers to go from local unit testing to a System Integration Test (SIT) environment. It will walk you through the follow steps:

1. Git Clone the DAVEXACOM\IIB-MQ repository to your GitHub organization
2. Explore the IIB-MQ file structure and the relationships such that you understand how an IIB/MQ Helm release repo for ICP hangs together, specifically:
a. The Docker file
b. The Jenkins file
c. The helm chart YAMLs
d. The .SH that executes when the container starts
3. Optionally change or add IBM MQ MQSC files or IIB BAR files in your copy of the IIB-MQ repo
4. Configure your IIB-MQ repo for OAuth application access from IBM MSB (Jenkins)
5. Configure and Deploy IBM MSB Pipeline (Jenkins) on IBM Cloud Private.
6. Kick off your first IIB/MQ deployment manually using Jenkins “Scan Repository Now”
7. Connect IBM MQ Web Console/IIB WebUI
8. Test the MyHTTPESQLTestFlow (hello world message flow) deployed on IIB
9. Optionally create a local git clone on your IIB Toolkit workstation of your IIB-MQ repo so you can save you IIB Development work and push to the GitHub master repository (you don’t have to do this you can use another mechanism or simply upload changed or new bar files).
10. Create and configure the WebHook on your IIB-MQ Github repo to “call” the Jenkins URL on the ICP MSB pipeline to perform a new build when changes are made to the IIB-MQ repo.
11. Optionally. As Webhooks are inbound to Jenkins if the ICP instance is on a private network and you are using public Github you’ll need something like the IBM Cloud (Bluemix) Secure Gateway configured to offer a public IP address and port number to the Jenkins Webhook URL.
12. Update the MyHTTPESQLTestFlow (Project Interchange supplied) to change the “Hello World” message and Build and Save the BAR file.
13. Upload or Git Push the BAR file to your IIB-MQ repo
14. Follow the IBM MSB pipeline (Jenkins) console logs
15. Reconnect the IIB Web UI and IBM MQ Console
16. Retest the MyHTTPESQLTestFlow and observe the new “Hello World” message

Video walk through demonstration of this scenario solution

Below is a set of video capture walk throughs for the solution components detailed above along with a demonstration of changing an IIB message flow as a developer and testing those changes in the SIT environment on ICP after the Jenkins has automatically rebuilt that SIT environment.

1. GitHub repository DAVEXACOM\IIB-MQ orientation video. Walk through of how the DockerFile, container entry point shellscript, JenkinsFile and Helm Chart files relate to each other and how to customize them.

2. Building an deploying the IIB and MQ SIT environment on ICP for the first time using Jenkins manually.

3. Enabling GitHub IIB-MQ repository for automated Jenkins builds. Rebuilding to SIT environment, trigger by new or updated BAR files pushed to GitHub.

All collateral and materials are available on gitHub at the following locations

https://github.com/DAVEXACOM/IIB-MQ
This repository delivers a combined IIB and MQ docker image. It contains the Helm charts for IIB and MQ and Jenkins file set up and ready to clone.

https://github.com/DAVEXACOM/IIB-MQ-SupportingRepo
This repository container the supporting collateral and documentation.
IBM-MQandIIB-IBMCloudPrivate-MicroServicesBuilder.pdf focuses on the IIB and MQ automated build pipeline using Micro Service Builder (Jenkins) pipeline to enable developers to go from local unit testing to a System Integration Test (SIT) environment.

IBM-MQandIIBonICP-DeploymentScenariosV3p.pdf captures every step of my journey of exploring a range of deployment scenarios for IIB and MQ on IBM Cloud private. It is in draft form as it was “written on the go” but covers the following:

1.simply deploying and single IIB and single MQ containers via the built in Helm Charts via the ICP console
2.deployment using command line Kubeclt from a GitHub docker repository for a custom combined IIB and MQ docker image
3.modifying the custom combined IIB and MQ docker image repository to enable for Helm chart deployment
4.using Micro Service Builder (Jenkins) pipeline on iCP to do the pull, build and deploy from the Helm chart version of the IIB and MQ docker image repository on GitHub
5.Webhook enabling the Github repository such that it “pokes” Micro Service Builder in iCP to do a rebuild and deploy when a new or updated BAR file of MQSC file is pushed to the Github repo

IIBMQicpDemoPI.zip A Project Interchange file containing an IIB “hello world” HTTP message flow.