This article will provide guidance to teams who are looking to build a simple CI/CD for IBM App Connect Enterprise (ACE) bar file deployment onto IBM Cloud Pak for Integration using GitHub, Jenkins, and Nexus Artifactory. The sample developed for this article is an ACE Toolkit REST API that receives payload and routes the payload to an MQ Queue. You can deploy Cloud Pak for Integration artifacts using latest OpenShift Tekton pipelines, but this article is based on the traditional Jenkins, Nexus Artifactory.
2. OpenShift Environment
This article will be using OpenShift cluster created on IBM Cloud, environment topology below.
|
Products
|
Version
|
|
OpenShift
|
v4.7 on IBMCloud
|
|
Cloud Pak for Integration
|
V2021.3.1
|
|
IBM MQ
|
v9.2.3.0
|
|
IBM App Connect Enterprise Toolkit
|
v12 .0.2.0
|
|
ACE Dashboard
|
V12.0.2.0-r1
|
|
ACE Integration Server Container Image
|
V12.0.2.0-r1
|
|
Jenkins
|
2.277.1
|
|
Nexus
|
3.29.2-02
|
2.1 Gather OpenShift Configuration
Login to the OpenShift Cluster.
Select “Copy login command”, and “Get token”.
Select “Copy login command”, and “Get token”.
Open a Terminal, and login to OpenShift Cluster from Command Line.
$ oc login --token=<token> --server=https://< OCP_API_URL your ocp cluster public api url>
Create default route if not created already.
$ oc patch configs.imageregistry.operator.openshift.io/cluster --patch '{"spec":{"defaultRoute":true}}' --type=merge
$ oc get route default-route -n openshift-image-registry --template='{{ .spec.host }}'
Copy, and save the default-route URL, the URL will be used later in the Jenkins Deploy job.
Note: The route will be used later in Jenkins job to push ACE images to OpenShift Registry.
3. IBM Cloud - Api Key
Create IBMCloud IAM Apikey if you don't have one already by navigating to IBM Cloud > Manage > Access (IAM) > API keys > “Create an IBM Cloud API Key”. This article will use the Api Key to login to the cluster but you can use OpenShift userid, and password combination as well.
Copy and Save the key for later use in Jenkins.
4. Setup IBM Cloud Pak for Integration MQ, App Connect Operators
If MQ, and App Connect operators are not installed already, then login to OpenShift Console, and Navigate to Operator Hub > search for IBM MQ and install the operator onto "All Namespaces". Similarly install App Connect Operator as well.
4.1 Create MQ Queue Manager "MQDV01"
In OpenShift console, create projects cp4i-mq, cp4i-ace.
Switch project to cp4i-mq.
Navigate to Operator Hub > Installed Operators.\
Click on IBM MQ Operator > Queue Manager tab > Create Queue Manager button.
4.2 MQ Configuration
From OCP Console > workloads > pods > change project to cp4i-mq. Click on mqdv01 pod. Click Terminal tab.
$ runmqsc mqdv01
DEFINE CHANNEL(SB.SVRCONN) CHLTYPE(SVRCONN)
SET CHLAUTH(SB.SVRCONN) TYPE(BLOCKUSER) USERLIST('nobody')
SET CHLAUTH(SB.SVRCONN) TYPE(ADDRESSMAP) ADDRESS(*) USERSRC(NOACCESS)
SET CHLAUTH(SB.SVRCONN) TYPE(ADDRESSMAP) ADDRESS('172.30.*') USERSRC(CHANNEL)
Note: Check your cluster configuration, and adjust ADDRESS value according to your cluster setup.
REFRESH SECURITY TYPE(AUTHSERV)
DEFINE QLOCAL(SB.INVOICE.IN) DEFPSIST(YES) MAXDEPTH(100000)
4.3 Create App Connect Dashboard
Login to OpenShift Console, Switch project to cp4i-ace..
Navigate to Operators > Installed Operators > IBM App Connect > All Instances > Create New > Dashboard.
Wait for the dashboard to be ready!
5.3 Docker Install
Use this link and install Docker on your PC https://docs.docker.com/desktop/mac/install/
Make sure Docker is installed successfully.
$ docker images
$ docker ps
6. App Connect Enterprise Toolkit
6.1 ACE Toolkit v12.0.2.0
Download App Connect Enterprise developer edition from https://www.ibm.com/docs/en/app-connect/12.0?topic=enterprise-download-ace-developer-edition-get-started
Note: ACE v12.0.2.0 is already packaged with GitHub plugins, no need to install separately.
6.2 Git Plugin setup
Navigate IBM App Connect Enterprise Toolkit (Menu) -> Preferences -> Team -> Git -> Configuration, create entries by pressing Add Entry... button:
user.email with value <your_github_account_email>
user.name with value <your_first_name> <your_last_name>
core.autocrlf with value true if you have MS Windows OS or input if you have Linux/mac OS
6.3 SSH Config
Navigate to Menu IBM App Connect enterprise toolkit -> Preferences -> General -> Network Connections -> SSH2 > Key Management tab. If you don't have a public key yet - click “Generate RSA key”...,
Copy the public key into the clipboard.
then Save Private Key..., Ok, Ok
Close Toolkit and reopen Toolkit.
7. GitHub configuration
Login to your github account, and add your public key generated at the previous step in the "Settings" menu in the right upper corner. Click SSH Public Keys in the menu on the left and Paste the public key generated in the above step.
7.1 Create GitHub Repository
Name whatever you like for example REST2MQ_API.
Save the git repository URL.
8. ACE Toolkit – Clone Github Repository
Navigate to Window > Open Perspective > Other > Git > Right click on blank area on left panel and select “Clone a Git Repository”.
8.1 ACE Toolkit - Import Sample Project
From App Connect Toolkit File > Import.
Import the App Connect Enterprise Toolkit sample project interchange (Attached file) into your workspace.
8.2 Explore Projects
8.2.1 SB_REST2MQ_INVOICE_API Project
8.2.2 SB_RESTS2MQ_API_MQ_POLICY Project
8.2.3 SB_RESTS2MQ_API_DEPLOY Project
Dockerfile
#Container Registry
#12.0.1.0-r4 https://www.ibm.com/docs/en/app-connect/containers_cd?topic=obtaining-app-connect-enterprise-server-image-from-cloud-container-registry
FROM cp.icr.io/cp/appc/ace-server-prod@sha256:f7a74de7e5cd3d1d56cabde1c11b174b1be643f48c3bce63ab5f344495877052
COPY *DEV*.bar /home/aceuser/initial-config/bars/
EXPOSE 7600 7800 7843 9483
ENV LICENSE accept
Add all projects to your github repository.
8.3 mq-policy.zip
Create mq-policy.zip by right clicking on SB_REST2MQ_INVOICE_API_MQ_POLICY > Export > General > Archive File. Select “To Archive File” as /Users/<user>/ibm/ACET12/workspace/SB_REST2MQ_INVOICE_API_DEPLOY/ConfigurationInputs/mq-policy.zip.
Check-in the projects to github, and they will be used later by the Jenkins jobs.
9. Jenkins Deployment
Login to Jenkins http://localhost:8080
9.2 Create Build Job
Create new Freestyle job, name SB_REST2MQ_API_BUILD.
Add Build Step: Execute Shell
Copy and paste below shell script, adjust according to your environment.
#!/bin/bash
echo “Creating bar file”
APP_NAME=SB_REST2MQ_INVOICE_API
GROUP_ID=com.ibm.esb
ACE_HOME=/Applications/IBM\ App\ Connect\ Enterprise.app/Contents/mqsi
echo “Setup mqsi command environment – mqsiprofile”
. "$ACE_HOME"/server/bin/mqsiprofile
echo `pwd`
echo $WORKSPACE
mkdir barfiles
echo “Creating bar file”
#"$ACE_HOME"/tools/mqsicreatebar -data $WORKSPACE -b $WORKSPACE/barfiles/${APP_NAME}.bar \
#-cleanBuild -deployAsSource -trace -v $WORKSPACE/createbartrace.txt
ibmint package --input-path . --output-bar-file $WORKSPACE/barfiles/${APP_NAME}.bar
mqsiapplybaroverride -b barfiles/${APP_NAME}.bar -p ${APP_NAME}/properties/dev.properties -o barfiles/${APP_NAME}_DEV_${BUILD_NUMBER}.bar -r
mqsiapplybaroverride -b barfiles/${APP_NAME}.bar -p ${APP_NAME}/properties/qa.properties -o barfiles/${APP_NAME}_QA_${BUILD_NUMBER}.bar -r
# Artifactory Upload
zip barfiles/${APP_NAME}_${BUILD_NUMBER}.zip barfiles/${APP_NAME}_DEV_${BUILD_NUMBER}.bar barfiles/${APP_NAME}_QA_${BUILD_NUMBER}.bar
curl -u admin:$NEXUS_ADMIN_PASSWORD --upload-file ./barfiles/${APP_NAME}_${BUILD_NUMBER}.zip \
http://localhost:8081/repository/generic/com/ibm/esb/${APP_NAME}/${BUILD_NUMBER}/${APP_NAME}.zip
# Git – tag the version
git tag -a ${BUILD_NUMBER} -m 'Tagged by Jenkins build'
git push git@github.com:xxxxxx/SB_REST2MQ_API.git --tags
echo “… END …”
9.3 Deploy Job
Create the second Jenkins Job to deploy Integration Server to IBM Cloud Pak for Integration.
Create new Freestyle job, name it REST2MQ_API_ICP4I_DEPLOY_DEV.
> Select “This project is Parameterized”.
Parameters:
|
Parameter Name
|
Type
|
Value
|
|
DeploymentType
|
Choice
|
install, upgrade
|
|
IAM_APIKEY
|
Password
|
Your IBMCloud IAM Apikey
|
|
OCP_API_URL (See section 2)
|
String
|
OpenShift Clusters Public service endpoint URL
|
|
OCP_DEFAULT_ROUTE
|
String
|
Your Openshift default route.
|
|
|
|
|
|
ICP4I_ENTITLEMENT_KEY_CP_ICR_IO
|
Password
|
Your IBM Cloud Pak for integration entitlement key.
|
Please visit https://ibm.github.io/datapower-operator-doc/features/entitled-registry/ to retrieve your ICP4I_ENTITLEMENT_KEY.
> Source Code Management section
Select github and paste the github repository URL into the “project URL” field.
> Add Build Step: Execute Shell
Copy/paste the below lines. Adjust according to your environment.
#!/bin/bash
echo $PARENT_BUILD_NUMBER
PARENT_APP_NAME=SB_REST2MQ_INVOICE_API
Namespace=cp4i-ace
# integration server, imagenames must be lowercase - can't have underscores for integrationserver.
IntegrationServer=sb-rest2mq-invoice-api
tag=latest
export PATH=$PATH:/usr/local/bin:
# Nexus Artifact download
curl -X GET -u admin:$NEXUS_ADMIN_PASSWORD \
"http://localhost:8081/repository/generic/com/ibm/esb/${PARENT_APP_NAME}/${PARENT_BUILD_NUMBER}/${PARENT_APP_NAME}.zip" \
-o ${PARENT_APP_NAME}.zip
unzip -o ${PARENT_APP_NAME}.zip
chmod -R 777 barfiles
# DEV bar file
rm $HOME/.jenkins/jobs/${JOB_NAME}/*.bar
ls -l barfiles
cp barfiles/*.bar $HOME/.jenkins/jobs/${JOB_NAME}
# Dockerfile
####cp $WORKSPACE/${PARENT_APP_NAME}/Dockerfile /var/lib/jenkins/jobs/${JOB_NAME}
cp SB_REST2MQ_API_DEPLOY/Dockerfile $HOME/.jenkins/jobs/${JOB_NAME}
# OpenShift Login
oc login -u apikey -p ${IAM_APIKEY} --server=${OCP_API_URL}
oc project ${Namespace}
# Build docker image with bar files
docker login cp.icr.io --username ekey --password ${ICP4I_ENTITLEMENT_KEY_CP_ICR_IO}
docker build -t ${IntegrationServer}:${BUILD_NUMBER} $HOME/.jenkins/jobs/${JOB_NAME}
docker tag ${IntegrationServer}:${BUILD_NUMBER} ${OCP_DEFAULT_ROUTE}/${Namespace}/${IntegrationServer}:${BUILD_NUMBER}
docker logout
# Push the image to OpenShift Registry
docker login https://${OCP_IMAGE_REGISTRY} -u $(oc whoami) -p $(oc whoami -t)
docker push ${OCP_DEFAULT_ROUTE}/${Namespace}/${IntegrationServer}:${BUILD_NUMBER}
#Create the IntegrationServer Configurations from CR yaml's
chmod -R 777 SB_REST2MQ_API_DEPLOY/ConfigurationInputs
chmod -R 777 SB_REST2MQ_API_DEPLOY/ConfigurationResources
# Create Custom Resource for the policy project
policy=$(base64 SB_REST2MQ_API_DEPLOY/ConfigurationInputs/mq-policy.zip)
sed -e "s/replace-with-namespace/${Namespace}/" -e "s~replace-with-policy-base64~${policy}~" SB_REST2MQ_API_DEPLOY/ConfigurationResources/mq-policy.yaml > mq-policy-temp.yaml
# Create CR for server configuration
serverconf=$(base64 SB_REST2MQ_API_DEPLOY/ConfigurationInputs/server.conf.yaml)
sed -e "s/replace-with-namespace/${Namespace}/" -e "s~replace-with-serverconf-base64~${serverconf}~" SB_REST2MQ_API_DEPLOY/ConfigurationResources/server.conf.yaml > server.conf-temp.yaml
# Update Integration Server CR
deptoken=`oc get secrets -n $Namespace | grep deployer-dockercfg | awk '{print$1}' | cut -d- -f3`
sed -e "s/replaceIntServerName/${IntegrationServer}/" \
-e "s/replaceNamespace/${Namespace}/" \
-e "s/replacewithdeployerdockercfgtoken/${deptoken}/" \
-e "s/replaceWithBakedImage/${IntegrationServer}:${BUILD_NUMBER}/" \
SB_REST2MQ_API_DEPLOY/ConfigurationResources/IntegrationServer.yaml > IntegrationServer-temp.yaml
# DEPLOYMENT
echo ${DeploymentType}
if [[ ${DeploymentType} == "install" ]]; then
echo "installing ......"
oc apply -f mq-policy-temp.yaml
oc apply -f server.conf-temp.yaml
oc apply -f IntegrationServer-temp.yaml
else
echo "upgrading ......"
oc replace -f mq-policy-temp.yaml
oc replace -f server.conf-temp.yaml
oc replace -f IntegrationServer-temp.yaml
fi
Save the job.
10. Jenkins Build & Deploy
10.1 Jenkins Build
Run SB_REST2MQ_API_BUILD job, and make sure the bar file is created and uploaded binary to Nexus Artifactory.
Run SB_REST2MQ_API_ICP4I_DEPLOY_DEV job, and check the output. Make sure the docker image is created on your computer, pushed the image to the OpenShift Image Registry, and the Integration is created succesfully.
$ docker images
Login to OpenShift Console, switch project to cp4i-ace, navigate to Builds > ImageStreams. Make sure the image is pushed.
Open App Connect Dashboard and check the integration server is created, and in Ready state.
From the App Connect Dashboard, select sb-rest2mq-invoice-api Integration Server. Click SB_REST2MQ_INVOICE_API Api. Copy the URL for addInvoice Operation.
$ curl --request POST --url <paste the url copied above> --header 'accept: application/json' --header 'content-type: application/json' --data '{"cnum":"100", "cname": "Hello World!"}'
Output should look like below.
Download the attached App Connect Toolkit Project Interchange.
I would like to thank Senior Solution Architect Andy Anderson, and Bruno Neves (Integration Tech Seller) for reviewing the article. Their feedback was extremely helpful for fine-tuning the article.