An approach to build DevOps pipeline for ACE on Cloud Pak for Integration

Ensure that you have configured the build server by following the steps from 1 to 5 from the recipe mentioned below:

Building CI-CD Pipeline for IBM App Connect Enterprise on Cloud Pak for Integration

Note that the instructions in this blog do not use OpenShift plugin and Docker plugin, so you may skip to install these plugins in Jenkins; however docker and oc client must be installed on build server.

For this demonstration, we will create two additional folders inside the ACE project:

– Properties folder: This folder will contain properties files for different environments, e.g. DEV.properties and QA.properties. These properties files would contain the environment specific parameters to be used in mqsiapplybaroverride command to override node properties and UDPs

– Build folder: This folder will contain Jenkins pipeline script for the ACE project. So the build script for the ACE project is also checked-in to source control

For this demonstration, I have created two ACE projects which are checked-in at the location below:

Test.App: https://github.com/awasthan/Test.App.git

Sum_Service: https://github.com/awasthan/Sum_Service.git

Both these projects have two properties files (DEV.properties and QA.properties) inside the ‘properties’ folder. Let us examine the ‘jenkinsfile’ script for the Test.App project.

Now let us create the Jenkins build job for this Test.App project.

Create a Jenkins pipeline project with the name ‘Test.App’.

Specify to read pipeline script from SCM and enter the github project location, credential to access GitHub and path of the jenkinsfile in the project.

Click on Save.

Now build the project so that it uploads the BAR files on the repository and tags the code. The snippet below shows a successful build of the Test.App project.

Now let us look at the Nexus repository if BAR files for this version (version 10 here) have been uploaded. The snapshot below shows the zip file containing the version 10 BAR files.

Now let us look at Github if the Test.App source code has been tagged with version 10. Below snippet shows the tag.

Similarly configure the second ACE project (Sum_Service).

Now configure the deployment jobs for each environment. You may have a single parameterized job to deploy to a different CP4I environment; however due to security and compliance reasons, you would probably have different deployment jobs for respective environments.

Before the Integration Server starts, the container is checked for the folder /home/aceuser/initial-config.
Refer to https://github.com/ot4i/ace-docker to understand how ace images can be built dynamically.

We can have a deployment project containing configuration information, Dockerfile and Jenkinsfile as depicted in below diagram.

I have a sample deployment job that contains these two files:

• Dockerfile: This file will have instructions to build the image baked with BAR files
• Jenkinsfile: This file will have pipeline script to deploy to CP4I environment, create route, Horizontal Pod autoscalar policy etc.

ACE helm-chart version 3.1.0 (CP4I 2020.1.1) creates the route as well, so you may use that route unless you prefer a different hostname for the route.
To do other configurations on IntegrationServer, you must pre-install the configuration secret containing required configurations. A generic template for configuration parameters and an script to generate secrets can be downloaded from platform navigator while doing manual deployment.

The sample deployment job used in this example is checked-in to Github:
https://github.com/awasthan/Test.App.CP4I.Deployment.git

Another sample deployment job with configuration parameters to set-up connectivity with EventStreams instance is checked-into github as well. This deployment job shows how to do other required configurations for Integration Server in addition to baking BAR file on to base image.
https://github.com/awasthan/Kafka_APIs_CP4I_prod5_Deployment.git
The configuration folder in this project contains parameters in ‘server.conf.yaml’, ‘truststorePassword.txt’and ‘setdbparms.txt’. Other files are left blank as those parameters are not being used in this IntegrationServer configuration. ‘server.conf.yaml’ contains the fields that need to be overridden, ‘setdbparms.txt’ contains parameters for security credentials and ‘truststorePassword.txt’ contains password for truststore. Note that in real life, the sensitive information like passwords would not be stored in source control. You would put them in some protected place and have Jenkins to pull them dynamically while executing the pipeline.
This deployment job deploys BAR files from below two projects:
https://github.com/awasthan/Publisher_api.git
https://github.com/awasthan/Consumer_API.git

Let us examine the Dockerfile.

You can add other configuration parameters as well, for example to do SSL configuration etc. Follow the documentation at https://github.com/ot4i/ace-docker

Let us examine the parts of the jenkinsfile. This pipeline has been parameterized to accept some parameters dynamically. It does following tasks:

• Checks-out the deployment project
• Pulls the specified versions (ACE_APP1_VERSION and ACE_APP2_VERSION) of BAR files from Nexus repository for ACE_APP1 and ACE_APP2.
• Logs-in to OCP registry of specified CP4I instance
• Builds the docker image using the Dockerfile and DEV BAR file and pushes to OCP registry
• Performs helm deployment. Notice the name of ACE image (image.aceonly) in helm install command. Since we are using ace only image, so value of ‘imageType’ has been specified as ‘ace’. If you are using ACE with MQ client or ACE with MQ server, value of ‘imageType’ would be ‘acemqclient’ and ‘acemqserver’ respectively. Make sure to specify the image parameter based on imageType
• If it is a new helm release being installed, route and horizontal autoscalar policy (hpa) are being created
• If it is an upgrade to existing helm release, it only upgrades the helm release and route & horizontal autoscalar policy are left unchanged
• Performs the clean-up
  

Now let us look at the Jenkins pipeline Job for this.
Parameters have been defined for all the placeholders in above jenkinsfile.

Pipeline script has been configured to pull from Github.

Similarly you can configure deployment job for QA environment and so on.

Now let us deploy an IntegrationServer that contains version 15 of Test.App and version 16 for Sum_Service. Also note that, the base image of ACE being pulled in docker file is version 11.0.0.6.1. In the next step we will apply fix-pac 11.0.0.7 on this IntegrationServer.

Click on ‘Build with Parameters’ for the deployment job.

Supply the parameter values

Click on Build.
Below is the snippet of build logs.

Let us take a look at the ACE Dashboard.

The Test.App contains a simple ping flow. You can test the flow by hitting the url in web-browser. The url will be:
http://<route>/ping

You can get the route by using ‘oc get route’. Also notice the hpa.

You may need to apply patch to your IntegrationServer for various reasons. Some of the reasons could be:

• Apply new fix pac for ACE
• Deploying new version of an application
• Deploying another application on same IntegrationServer
• Removing a deployed application
• Changing configuration parameters

And so on…

In this scenario, let us assume that we want to apply new ACE fix pac. The current image has been built on ACE 11.0.0.6.1. So first of all you need to upload IBM provided new image for ACE fix-pac. Follow the step 10 in below article to upload ACE fix-pac image in OCP registry.

Modernizing Integration – Migration from IIB to App Connect running on IBM Cloud Pak for Integration(CP4I)

If you are pulling the images online from IBM entitlement registry, you don’t need to do this step.

After this is done, let us update the Dockerfile of the deployment job to pull new ACE fix-pac, in this case 11.0.0.7-r1

Now run the click on ‘Build with Parameters’ for the deployment job.

Notice that the ‘DeploymentType’ is selected as ‘upgrade’ and the name of existing helm release for upgrade has been provided. Selecting the value ‘upgrade’ for DeploymentType will execute else condition in jenkinsfile, which means it will not try to recreate route and hpa policy.

Now click on Build. Below is the log snippet of the deployment job execution.

Now the IntegrationServer is running on ACE 11.0.0.7.r1 fix pac.

Though as part of DevOps process, you can implement automatic rollback in case deployment failure or test failure. However in certain scenarios, you may be asked to rollback to a previous version due to some other reason.
To do that, you can go to ‘Cloud Pak Foundation’→Administer→Helm Releases

Find the Helm release and click on Rollback.

Select the release to rollback to and click on Rollback.

You can perform the roll back using command line as well.