MQ

When we created the IBM MQ on AWS Developer Essentials Learning path we wanted it to be as simple as possible to deploy an IBM queue manager so that developers could focus on building messaging applications. The solution is container-based and uses docker compose to orchestrate the AWS deployment steps. The default configuration is just enough to get developers through the learning path and can be easily modified by editing the yaml files. The intention was to provide a production ready sample that bakes-in good practice following an infrastructure as code (IaC) approach.

We often get asked to provide a Terraform based deployment solution alongside docker compose. This would provide additional flexibility to integrate with our users’ existing pipelines. It sounds like a simple ‘cut and paste’ porting exercise to deliver the same AWS hosted runtime environment, just with a different orchestration layer.

There are, however, some significant differences. Docker compose abstracts much of the networking and infrastructure so users don’t need to define concepts such as VPC, Subnets, Security groups, Target groups etc. What you need gets generated for you at deployment time by the docker compose Elastic Container Service (ECS) engine, and most of the time this is good enough. Under the covers, docker compose makes use of AWS Cloud Formation to manage AWS resources. In contrast, Terraform uses the AWS provider APIs to manage AWS resources directly. One benefit of the docker compose abstraction is that a single configuration can be deployed to multiple cloud environments with minimal vendor specific changes. 

While Terraform provides finer-grained control, it does require solution designers to have a more detailed understanding of the target cloud technologies and how they interact. As a consequence, Terraform configurations are more verbose, although they are straightforward to navigate and each stanza is easy enough to follow.

We want the IBM MQ on AWS developer essentials learning path to be easy for new cloud developers to follow, so why are we being asked for Terraform examples? Our decision to use docker compose seemed obvious as it’s more developer centric; simple to understand, high level, can be used to run containers on a developer’s own machine, and is easily ported to multi-cloud environments. However, the higher degree of abstraction in docker compose provides less user control and can lead to runtime deployment variations over time. Even a small variation in behaviour is an anti-pattern within the DevOps community. Terraform configurations aim to do exactly what they say on the tin, providing greater control and deliver greater confidence in repeatable IaC deployments. As a user, the main difference in building Terraform configurations is that you need to understand exactly how you want to configure your cloud. While this makes getting started a little slower, it does force a conscious choice in decision making. With good encapsulation, Terraform components are easy to re-use and share, there are just more of them.

What does all this mean for the learning path? We’re pleased to announce Terraform support for the learning path is now available in our dev-patterns GitHub repository. We’ve added a new Terraform sample configuration to deploy an IBM MQ Advanced for Developers container image onto AWS. Just like our approach for docker compose, the sample config is just enough to complete the learning path and can be easily customized. It's made available as open source under an Apache 2.0 License so users can use it as a starting point for their own development. The result is rapid provisioning of a queue manger that can be easily deprovisioned when not needed. The samples additionally show how to place the queue manager in private subnets and how to configure external storage to provide message persistence. 

It is good practice to deploy applications within private subnets and our Terraform configuration samples show how to allow ingress traffic via a hardened network load balancer through the two MQ ports: 9443 for REST and console http traffic, and 1414 for messaging traffic. If you need messaging is to be restricted to applications running in private subnets, then these ingress rules can be removed.

To persist configuration and messaging state, the queue manager container requires an external storage volume. The storage volume must be prepared with the correct file structure and posix permissions, allowing the queue manager to read/write data as a non-root user.  This Initialization is achieved with a one-off task which we have shown how to accomplish with Terraform.

Most of the time all developers want is a queue manager, with no fuss, so they can create and test their applications. You may be already familiar with the tutorial Ready, set, connect!.  These Compose and Terraform IaC samples give more options to the Get an IBM MQ queue for development in IBM Cloud story.