Fine Grained Integration – IBM App Connect Enterprise on IBM Cloud Private with DevOps
In an article written earlier in 2018 I explored “foundational” approaches to running IBM Middleware on IBM Cloud Private Kubernetes clusters.
In this new article and its associated collateral we will explore a more cloud native, “componentized/decentralized” approach that demonstrates how to deliver integration with the App Connect Enterprise product on IBM Cloud private to microservices principles.
Figure 1. Integration Microservices Scenario
We will leverage a base App Connect Enterprise (ACE) v11.0.0.2 image that has a fixed configuration and deliver integration microservices “from” that image. These integration microservices perform a single integration function, are immutable and individually scalable. The two integration microservices are simplistic but designed to work together as a simple microservices application.
The scenario
The ICP Cloud represents an environment that exists in the standard Software Delivery Life Cycle (could be SIT, QA, Perf, Prod (or potentially one of those environments in an ICP cluster namespace).
We are delivering a microservices application consisting of multiple microservices, two of which have requirements that are integration-centric and ACE has been chosen as the runtime for these functions.
Our fictitious organization maintains a standard operating environment (container image) for microservice integration with a fixed configuration component on top of which the integration microservices are added and then the images deployed and run as a Helm release.
The primary mechanism for deployment will be Jenkins but we’ll use Microclimate, as it sorts out security on the ICP instance for us, and stands up the Jenkins toolchains.
The Personas – the team delivering our micro services application
- Developer 1 and Developer 2 have their own ACE Enterprise Toolkits and workspaces and are integration developers. They write ACE services and flows and build BAR files.
- One or more Build Specialists are responsible for maintaining the SOE image and building integration microservices images from the SOE image based on the BAR files created by the Developers.
- One or more Deployment Specialists are responsible for packaging and deployment of microservices applications that can include integration microservices. They use Helm and the ICP Devops tooling and pipelines to deploy.
This article is in an early form and I plan to update it with a greater level of detail.
There is more detail on how the ACE integration microservices and the ACE Liveliness probe work along with many screen captures of the final solution on ICP 3.1 in the Supporting Materials and documentation on GitHub: https://github.com/DAVEXACOM/ACEonICPIntSupportingMaterial.
The ACE Integration microservices
The Integration Micro Service 1 exposes a RESTFul (API) interface. This service will call Integration Micro Service 2 via it’s RestFul(API) interface
- Integration Micro Service 1
RESTInput(HTTP) -> Mapping Node -> RESTRequest (call Integration Microservice 2) -> RESTReply(HTTP)
- Integration Micro Service 2
RESTInput(HTTP) -> Mapping Node Payload+"Hello from Integration Microservice 2" -> RESTReply(HTTP)
Integration Micro Service 2 simply returns a “Hello World” style message. Integration Micro Service 2 can be called directly or through Integration Micro Service 1.
We have set up the integration in this way to position it for using Istio as a service mesh to route between integration micro services. However, in this initial example I will make use of the ICP DNS service that uses services names to resolve host names and port numbers. So the base URL on the RESTRequest Node in Micro Service 1 will call Micro Service 2 by its service name.
The ACE Liveliness Probe
Our ACE Liveliness Probe service is another Restful service that we deploy into the ACE standard operating environment image, the base image from which Integration Micro Service 1 and Integration Micro Service 2 images are built. So it appears in all ACE Integration Micro Services. (this is not the service baked into the IBM cloud paks, we turn those off to demonstrate having a customer-centric base standard operating environment (SOE) image.
The Docker images
The ACE standard operating environment image is built from:
- Ubuntu
- ACE v11.0.0.2
- IBM MQ v9.1 Client (not used in these scenarios)
- Fixed configuration – a BAR file with RESTful Service, a custom Liveliness Listener
The ACE integration microservices images build from this (base) SOE image.
Before you start – Jenkins Build Script customization on ICP
When using Microclimate and Jenkins we have found it best to customize the Jenkins build scripts (instead of using the Microclimate/Jenkins set up that is more aligned to Java and NodeJS microservices).
We made changes to the Jenkins pipeline scripts to get greater flexibility in the image build, deploy only, and image build and deploy. This allowed us greater control in demoing these scenarios. The instructions can be found here: https://github.com/cloudnativedemo/icp-notes/blob/master/microclimate_notes.md
Build it yourself – Build, Test and Demo Document
I have included a PDF document that contains instructions on how to perform the following tasks:
- Customizing the Jenkins Pipelines scripts.
- Creating and loading all the collateral onto IBM Cloud Private 3.1.x
- Running the build Pipelines to load the docker images onto IBM Cloud Private
- running the build Pipelines to perform the Helm Releases.
- Exploring and modifying the github source collateral
- Testing the Microservices application (Microservice 1 and 2) Helm Release.
Documentation for using the collateral: https://github.com/DAVEXACOM/ACEonICPIntSupportingMaterial/blob/master/documentation/ICP-AgileIntegration-ACE v1.1.pdf
Video tutorials / demonstrations
I have created two video tutorials that demonstrate the final solution:
Integration Microservices Unit Test – Using Helm to deploy and test an individual service
Integration Microservices Application Test – Using Helm to deploy all micro services as a single application for testing
Round up
As mentioned up front, I do plan to add a lot more detail on the process of building out this “integration to micro services principles solution”. However, with the scenario up, running, and proven, and the materials all publicly available, I wanted to make those materials available to the developer community that are already exploring ACE, Docker, and Kubernetes. See the supporting materials ICP-AgileIntegration-ACE v1.1.pdf for more detail.
Supporting Materials and documentation: https://github.com/DAVEXACOM/ACEonICPIntSupportingMaterial
Acknowledgments
I’d like to acknowledge Do Nguyen and Peter Jessup of the IBM Australia Hybrid Cloud Integration Technical Sales team for their excellent technical work in helping to produce these materials.
Summary of the GitHub repository collateral
The collateral is all available from the DAVEXACOM organization: https://github.com/DAVEXACOM
Summary of the Docker Hub collateral
The Docker Hub collateral is all available from the DAVEXACOM organization: https://cloud.docker.com/u/davexacom/repository/list.
You can use the Docker Hub images with ICP rather than the on board repository OR pull them to you workstations and play around with them in a docker only (non K8s) environment.