Co-Author - Carsten Boernert
This blog is part of a series. For the whole series list see here
When deploying IBM App Connect Enterprise integrations in containers there are options around how we deploy the runtime, the configurations, the integration flows and environment specific data. All the movable parts can either be “included in the image”, “configured at deploy time” or a mixture of the two. In this article we will explore these different deployment options, the benefits of each and considerations for taking each approach.
What changes when we deploy IBM App Connect Enterprise in containers?
From IBM App Connect v11 the integration server can be deployed on its own, without the need for an integration node to look after it. This is most relevant in a container scenario where the container platform itself takes on the management of the individual integration servers. Additionally, all the IBM App Connect Enterprise configuration required is defined alongside the container providing a more portable deployment of integrations. Container based deployment has many notable differences to a traditional deployment, such as:
- Deployment isolation: In container deployments we see groups of integrations being independently deployed into their own container runtime as opposed to a more traditional singular centralized ‘broker’ deployment. This allows for greater isolation, protecting each deployment from the other integrations.
- Product runtime versioning: Since the container image also includes the product runtime this provides an opportunity to, for example, run different versions of IBM App Connect Enterprise in different containers allowing each integration to independently choose when and how to upgrade. Previously in a shared server environment all integrations were running on the very same version of the product and had to be upgraded at the same time, with costly regression testing, and greater risk during the single upgrade.
- Memory and CPU: Container platforms enable explicit/declarative ways to specify how much CPU and memory each (IBM App Connect Enterprise) container requires, ensuring the specific needs of the integrations can be taken into account.
- Dedicated Operating System Resources: With VM/server-based IBM App Connect Enterprise deployments there are a few areas where resources just could not easily be dedicated and managed, meaning unrelated workloads could potentially impact or have adverse effects on each other. An example of this would be the integration node level listener or the maximum number of threads or processes on the VM/server.
- Load balancing: In a traditional VM/server-based deployment, load balancing across VM replicas has to be explicitly set up and maintained. In containers this is replaced with the dynamic routing which is automatically set up and managed by the container platform as containers are deployed.
- TLS termination: In containers this would potentially be handled and configured by the platform (i.e. outside of the container) which in traditional VM/server-based deployments the configuration would be in a configurable service. This can be managed through a ‘Service Mesh’ like Istio which you can read about in the O’Reilly book ‘Istio Explained’ - https://www.ibm.com/downloads/cas/XWN1WV9Q
- Declarative Configuration: The majority of the configuration is now declaratively defined within a configuration file (server.conf.yaml) rather than requiring post deployment IBM App Connect Enterprise commands to be run on the node and server. This is linked with container-based objects like Kubernetes ConfigMap that are read on start-up, ensuring that the integration server’s state is not changed at runtime. This means configuration is more portable and repeatable as the documented YAML defines the true state of a redeployed container.
What is a container image and what to include in it?
A container image is a read-only template which is used to create instances of containers. Anything not included in the image can then be sourced during the creation of the container. It is not part of the deployed image but will be available in the running container.
Therefore, when deploying IBM App Connect Enterprise into container environments there is a decision to be made on how much of the configuration (including external libraries, drivers etc.), flows and environment settings should be included as part of the image being deployed.
Figure 1 – Shows what is included in each container image type
Sometimes the following terms associated with the scale shown in Figure 1 are:
- ‘baked’ (furthest right column) as in everything is baked into the image,
- ‘fried’ (further left column) where everything is configured as it is needed.
Most organizations will fall somewhere between these two extremes, but most commonly we see people using the image types C and D from Figure 1.
All options represented in the diagram are valid ways of building and deploying integration containers. Generally, the decision as to which approach to take relates to secondary considerations such as image management, skills within a team, non-functional requirements (testing, security) and management of the live environment.
Using pre-built container images
IBM provides a pre-built container image for IBM App Connect Enterprise in the IBM Entitled Registry. This image contains pre-integrated database drivers, and the MQ Client. Note that this pre-built image can only be used in conjunction with the App Connect Operator. On deployment of the image, you can specify the location of a remote BAR file and links to configuration information, and the container will draw those in during start up. Used in this way it is therefore an example of Image type B in Figure 1. It is also possible to build new images based on the pre-built image, adding server configuration, a BAR file, or even environment configuration in order to create image types C, D, or E.
Whilst the pre-built image covers most scenarios, there will always be circumstances where teams would prefer to create a completely custom image. A sample is provided on github for this purpose. However, there are many advantages to using the pre-built image with the App Connect Operator as described in the following articles:
Images and Hierarchies
New images can be based on existing images.
Figure 2 – Basing Images on other images
Image 1 could be deployed, adding the BAR file and server configuration at deployment time. Alternatively, further images could be built by layering configuration (e.g. server.conf.yaml) and code (e.g. the BAR file) on top of the first image as in Image 2 in Figure 2. In this case we would only have to add the environment specifics at deployment time.
We could go even further and create an image for each environment that the integration will be deployed to as in Image 3 in Figure 2. In this case we could simply deploy the image and it would automatically contain all the environment specific information required.
These images are hierarchical, relying on each other, meaning a level of management is needed. Part of this management is the responsibility of ownership of the images, from the template owner through to the application teams owning environment specific images and containers.
Having a single container with everything deployed into it doesn’t mean just one image to manage. All three images shown in Figure 2 will need to be managed and consideration given when making changes, i.e. how a change in Image 2 might impact the top level of images (Image 3) with environment specific configurations.
The illustration below provides an example of an image hierarchy where images have been created down to the level of each environment. Let’s look at the implications of this;
Figure 3 – Example image hierarchy where environment specific images have been created
We can see that making a change to an upstream image (such as “Rest Flow”) potentially affects many downstream images and could cause a cascade of image updates to be required.
On the one hand this might be considered good for governance as it enforces that all images share a known base. On the flip side, small changes to images at the base of the hierarchy could result in sweeping changes, which detracts from the isolation and decoupling we’re trying to achieve in more fine-grained deployment.
Image template configurations
Certain groups of integrations will share common aspects. The IBM App Connect image from the IBM Cloud Container Registry provides a good base template containing database drivers and the MQ Client. Further templates can be built on top of that to include any additional elements and can be used as ‘Enterprise Templates’ to be managed and maintained by the organization for all future integrations of a given type.
Such elements found in Templates, are well-suited to be “included in the image” rather than “configured at deploy time”. Examples might include the addition of database drivers, certain client libraries and agents, and maybe even standardized sub-flows for things such as error handling.
The core App Connect Enterprise runtime configuration is detailed in the server configuration YAML file (server.conf.yaml). Certain settings in that server configuration can also be included as part of an image template, to implement organization wide standards and defaults. Examples might include administrative role mapping, configuration for offloading logs to an external system and more.