Message Image  

App Connect

Integration Development to Micro Services Principles on OpenShift – Part 2 

Mon August 03, 2020 07:35 AM

Introduction

Modern platforms, DevOps tooling and agile approaches have accelerated the rate at which organizations can bring new applications and business function to bare. At IBM we see no reason why integration developers cannot derive many of the benefits being enjoyed by application developers in those organizations in leveraging DevOps tools on modern, container based platforms to deliver agile integration.
For a full description of IBM’s position on Agile Integration please take a look at the following article which discusses Agile Integration and introduces the IBM Cloud Pak for Integration (ICP4i) based on RedHat OpenShift Container Platform.

Integration Modernization the Journey to Agile Integration – A view from the field

This is the second part of a three part article series. In this, Part 2 of the article along with it’s associated collateral we will explore build, deploy and test of a third ACE microservice that offers a RESTful interface to IBM MQ and also use a Tekton Pipeline to deploy a second copy of ACE Microservice 2 on RedHat Openshift.

The first part of the series of articles covers Build, deploy and run of IBM Middleware using RH Openshift OOTB features and can be found at:

Integration Development to Micro Services Principles on OpenShift – Part 1

The third part of this series of articles covers testing and stubbing of IBM Middleware with IBM Integration Tester and can be found at:

Integration Development to Micro Services Principles on OpenShift – Part 3

Delivering ACE and MQ services on RHOS – Overview

Figure 1. ACE and MQ on RHOS Integration Micro Services – Environment

We will leverage a base App Connect Enterprise (ACE) v11.0.0.n image that has a fixed configuration and deliver integration microservices “from” that image. These integration microservices perform a single integration function, are immutable and individually scalable.ACE Microservice 3 offers a REST interface to a client connected IBM MQ Queue Manager. The MQ Queue Manager will be based from the IBM MQ v9.1 base image but has a custom layer that includes fixed MQ configuration including queues, SVRCONN channels and TLS server side KeyStore files and certs such that “off-cluster” tools and applications can connect via TLS to the queue manager on RH Openshift. An RH Openshift based Tekton Pipeline will be implemented to build, deploy and run a standalone integration service ACE Microservice 2, a Restful “echo” style service.

This article is in an early form and I plan to update it with a greater level of detail.

The scenarios

In terms of tools and approaches this article and associated collateral will explore:

  • 5. RedHat OpenShift Out of the Box capabilities for deploy, run and test with ACE MicroService 3
  • 6. Tekton Pipelines(Tech Preview on RH OpenShift 4.2) to build, deploy and run ACE microservices

The Persona – A standalone/disconnected integration developer

In exploring ACE and MQ on RedHat OpenShift we will assume the persona of the disconnected developer. A developer working in a standalone unit test environment. Developing integration logic, building into images, deploying the resultant containers to run on RHOS and testing his work.

The Environment

Figure 2. Disconnected (standalone) Developer

Introducing the ACE Integration microservices

 

The Integration Micro Service 3 (ACE MS3) exposes a RESTFul (API) interface to an MQ Queue Manager. Integration Micro Service 2 (ACE MS2) can be used as a standalone RESTful test “echo” style service.

 

    Integration Micro Service 3

    RESTInput(HTTP) -> Mapping Node -> Manage Transport Headers -> MQPut -> RESTReply(HTTP)

Figure 3. RESTful Interface for ACE Microservice 3

Figure 4. Message Flow for ACE Microservice 3

Integration Micro Service 2

RESTInput(HTTP) -> Mapping Node Payload+"Hello from Integration Microservice 2" -> RESTReply(HTTP)

Integration Micro Service 2 simply returns a “Hello World” echo style message. Integration Micro Service 2 can be called directly.


Figure 5. REST Interface to ACE MS2

Figure 6. Mapping for ACE MS2


Figure 7. Message Flow for ACE MS2


The ACE Liveliness Probe

Our ACE Liveliness Probe service is another Restful service that we deploy into the ACE standard operating environment image, the base image from which Integration Micro Service 1 and Integration Micro Service 2 images are built. So it appears in all ACE Integration Micro Services containers. (this is not the service baked into the IBM cloud paks, we turn those off to demonstrate having a customer-centric base standard operating environment (SOE) image.


Figure 8. Rest Interface ACE Liveliness


Figure 9. Message Flow for ACE Liveliness

The Liveliness Probe uses ESQL in a compute node to return the execution group (integration server) label plus a current timestamp to the calling application: 

 Set OutputRoot.JSON.Data.Messages[1].item[1] = 'DA1:'||ExecutionGroupLabel||':' 

                                ||CAST(CURRENT_TIMESTAMP AS CHARACTER FORMAT 'yyyyMMdd-HHmmss');

 

There is more detail on how the ACE integration microservices and the ACE Liveliness probe the Supporting Materials and documentation on GitHub from an earlier article written from deployment of ACE on IBM Cloud Private: https://github.com/DAVEXACOM/ACEonICPIntSupportingMaterial.

 

The ACE Toolkit Source projects for all the ACE services can be found at: https://github.com/DAVEXACOM/tekton-ace-example/tree/master/ACEMicroservicesSrcProject.



Step by Step Guide for Scenarios 5 and 6

Scenario 5. RedHat OpenShift OOTB capabilities for deploy, run and test with ACE MicroService 3 with off-cluster MQ

note: The following instructions complete with visual screen captures of all the steps is available in document form at:

  • Git Repo: https://github.com/DAVEXACOM/tekton-ace-example/doc
  • Document: 2.Developer Experience for ACE MQ Testing With RIT v1.0.docx from page 78


Collateral – Published Repositories, registries and collateral to clone or copy

 

Introducing the ACE Integration Microservice 3

The Integration Micro Service 3 exposes a RESTFul (API) interface to an MQ Queue Manager

Figure 10. RESTful Interface for ACE Microservice 3

Figure 11. Message Flow for ACE Microservice 3

Initial First time set up of the ACE MS3 build and deployment artifacts on RH OpenShift


Log into RH OpenShift on the command line and create a new build based on the ACE Micro service 3 gitHub repository DockerFile and collateral.

oc login --token=icMK5QdKiseLa0-kREUcCvYHoK0hCrZr0W9QB6rUs_Y --server=https://api.cloudpak.ocp4.cloudnativekube.com:6443
oc new-build https://github.com/DAVEXACOM/ibm-ace-mqc-soe-ms3-build.git

 

Observe the results in RH OpenShift Console->Builds->Image Streams

Observe the results in RH OpenShift Console->Builds->Build Config

Observe the results in RH OpenShift Console->Builds->Builds

 

Create a new application based on the build config created in the previous step.

oc new-app ibm-ace-mqc-soe-ms3-build --env LICENSE=accept

 

Observe the results in the RH OpenShift Console->Workloads->Deployment Configs

 

Create a route to expose ACE Micro Service 3 with a public IP address

oc create -f c:\openshift\data\create-ace-route.yaml

kind: Route
apiVersion: route.openshift.io/v1
metadata:
name: ibm-ace-mqc-soe-ms3-build
namespace: da-build-project
labels:
app: ibm-ace-mqc-soe-ms3-build
spec:
host: >-
ibm-ace-mqc-soe-ms3-build-da-build-project.apps.cloudpak.ocp4.cloudnativekube.com
subdomain: ”
to:
kind: Service
name: ibm-ace-mqc-soe-ms3-build
weight: 100
port:
targetPort: 7800-tcp
wildcardPolicy: None

Setting up the IBM MQ Queue Manager

In this example we’ll set up queue manager called MyAPPQMGR on your local machine (i.e. Off cluster).

MyAPPQMGR listens on port 1515
MyAPPQMGR has a SVRCONN called IVT.SVRCONN created as follows: 

Channel creation for Linux based queue manager:

DEFINE CHANNEL(IVT.SVRCONN) CHLTYPE(SVRCONN) REPLACE
SET CHLAUTH(IVT.SVRCONN) TYPE(BLOCKUSER) USERLIST(nobody)
ALTER AUTHINFO(SYSTEM.DEFAULT.AUTHINFO.IDPWOS) AUTHTYPE(IDPWOS) CHCKCLNT(NONE) ADOPTCTX(YES)
SET CHLAUTH(IVT.SVRCONN) TYPE (ADDRESSMAP) ADDRESS(*) MCAUSER(‘mqm’)
REFRESH SECURITY TYPE(CONNAUTH)
Channel creation for windows based queue manager
DEFINE CHANNEL(IVT.SVRCONN) CHLTYPE(SVRCONN) REPLACE
SET CHLAUTH(IVT.SVRCONN) TYPE(BLOCKUSER) USERLIST(nobody)
ALTER AUTHINFO(SYSTEM.DEFAULT.AUTHINFO.IDPWOS) AUTHTYPE(IDPWOS) CHCKCLNT(NONE) ADOPTCTX(YES)
SET CHLAUTH(IVT.SVRCONN) TYPE (ADDRESSMAP) ADDRESS(*) MCAUSER(‘MUSR_MQADMIN’)
REFRESH SECURITY TYPE(CONNAUTH)

IBM Secure Gateway Service – IBM Cloud (Server side)

If your RHOS instance is not running locally on your machine/laptop i.e. it has been installed in the IBM Cloud for example then you will need a Secure Gateway service to facilitate a public IP address to services running on your laptop.

To set up an IBM Secure Gateway Service for connection to your local Queue Manager for connection by the ACE Micro Service 3 running in the IBM Cloud take the following steps.

 

Log into IBM Cloud

Create an IBM Secure Gateway Service following the service instructions on the IBM Cloud

Download the secure gateway client
Add a destination to point to your machine/laptop. This destination endpoint IP address may change depending on whether you are connecting from home, work or mobile.

The server side public IP address and port number will be in the format:
Cap-au-sg-prd-02.securegatway.appdomain.cloud:15558
Note this will be used in the ACE Micro Service 3->MQ Output Node->MQ Connection->Queue manager host name property.

IBM Secure Gateway Service – Client (Laptop end)

If you are running on Windows you will likely need to open specific ports in the firewall on your machine.
Configure Windows Defender Firewall with Advanced Security->Inbound Rules.
Add a rule called DAMQ1515 and open port 1515. In this example I open 8080 as we will reuse the security gateway in another, later test scenario.

Config the access control list for the IBM Secure Gateway client by opening a browser against http://localhost:9003/acl
Add IPAddress:PortNumber entries to allow access to your machine/laptop. You might need multiple entries if you reconnect at home, work, mobile.

Testing MQ via IBM Public IP – using RFHUTILC


Use RFHUTILC (client) version, part of supportpac MS03 to test the connection to your MyAPPQMGR using SVRCONN channel IVT.SVRCONN via the secure gateway public IP address.

 

The following connection string goes in the “Queue Manager Name (to connect to)” property:

 IVT.SVRCONN/TCP/cap-au-sg-prd-02.securegateway.appdomain.cloud(15558)

The queue name to test put/get to is TO.BACKEND.Q
Put a message to the queue with RFHUTILC

 

You can observe the traffic flowing into your machine/laptop’s queue manager in the secure connector client logs.
http://localhost:9003/logs

 

Check the “Current Queue Depth” for TO.BACKEND.Q on MyAPPQMGR.
You can use the MQExplorer for a graphical display

Or from the command line with runmqsc MyAPPQMGR

 dis ql(TO.BACKEND.Q) CURDEPTH
2 : dis ql(TO.BACKEND.Q) CURDEPTH
QUEUE(TO.BACKEND.Q) TYPE(QLOCAL)
CURDEPTH(1)

end

Updating ACE Microservice 3 to Put to MQ via IBM Public IP

Update the ACE Micro service 3 property to use the queue manager connection you tested above with RFHUTILC.

 

Use Cap-au-sg-prd-02.securegatway.appdomain.cloud:15558 in the ACE Micro Service 3->MQ Output Node->MQ Connection->Queue manager host name property.

This property can be updated in the IBM ACE Toolkit->ACE micro service 3 subflow source OR via the ACE Micro Service 3 BAR override parameters.

Build and Save the BAR file and push the updated BAR file to Github (your clone of) https://github.com/DAVEXACOM/ibm-ace-mqc-soe-ms3-build repo.

 

Next perform either an manual rebuild via:

RH OpenShift Console->Builds->Builds->ibm-ace-mqc-soe-ms3-build-1 from the actions pull down select rebuild.
Or set up the webhook in Github using the URL that can be retrieved from via:
RH OpenShift Console->Builds->Build Configs-> ibm-ace-mqc-soe-ms3-build-1 and use the “Copy URL with secret” for github in the Webhooks section at the bottom.

 

The webhook will look similar to the below and should be used in the Github repos->settings->webhooks.

https://api.cloudpak.ocp4.cloudnativekube.com:6443/apis/build.openshift.io/v1/namespaces/da-build-project/buildconfigs/ibm-ace-mqc-soe-ms3-build/webhooks/-tkgynHJZ8jdcNLTxnP-/github

Once the rebuild has completed you can test ACE MS3 and check the results on MQ.

Testing ACE Microservice 3 to Put to MQ via IBM Public IP

Get the Route for ACE MS3 via:
RH OpenShift Console->networking->routes->ibm-ace-mqc-soe-ms3-build
or
oc get routes

 

Use the route information to call ACE MS3

 

URL for ACE MS3 service: 

http://ibm-ace-mqc-soe-ms3-build-da-build-project.apps.cloudpak.ocp4.cloudnativekube.com/ivtrest/v1/doiIvt

  

with Data to test ACE MS3 service:

 {

"name":"anyname",

"phone":"123545454",

"email": "anyname@host.com"

}

 

Testing with a REST Client will produce an echo response with uppercasing:

{"name":"ANYNAME","phone":"123545454","email":"ANYNAME@HOST.COM"}

 

The same data should be present in a message on the TO.BACKEND.Q on MyAPPQMGR queue manager in IBM MQ.

Scenario 6. Tekton Pipelines(Tech Preview on RH OpenShift 4.2) to build, deploy and run ACE microservices

note: The following instructions complete with visual screen captures of all the steps is available in document form at:

  • Git Repo: https://github.com/DAVEXACOM/tekton-ace-example/doc
  • Document: 1.Developer Experience for ACEMQ with RHOS Tools and Tekton v1.1.docx from page 95

Introducing the ACE Integration Microservice 2

RESTInput(HTTP) -> Mapping Node Payload+"Hello from Integration Microservice 2" -> RESTReply(HTTP)

Integration Micro Service 2 simply returns a “Hello World” style message. Integration Micro Service 2 can be called directly or through Integration Micro Service 1.

Figure 12. Mapping for ACE MS2