IBM MQ

Using AWS CloudWatch to monitor IBM MQ queue managers 

Wed November 13, 2019 03:21 PM

Using AWS CloudWatch to monitor IBM MQ queue managers

Mark E Taylor |Aug 25 2016 Updated

In this blog entry I'm going to show how you can monitor MQ queue managers using Amazon's CloudWatch service.

Keeping the demo simple

For the purposes of this demonstration, I stayed away from running any actual code on AWS. That avoided having to deploy programs into images, or configure much security. The queue manager runs on my local Linux workstation, as does the grafana service. The data is pushed to, and retrieved from, CloudWatch running in AWS but that's all. In reality, someone using CloudWatch is likely to want to include the monitoring collection program in an EC2 image to run alongside the queue manager in that instance. And that in turn will require a security role to be configured with that instance.

Security configuration

For my simple setup, I had to have accessible credentials so that I could write to CloudWatch. Since the MQ service runs under the mqm account, I put a copy of a credentials file containing the AWS_ACCESS_KEY and AWS_SECRET_KEY into ~mqm/.aws/credentials. That file may also contain the user's default AWS region (eg "us-west-2") for connection; if not provided there, the region can be given as a parameter on the mq_aws command line. That same file was also put under the home directory of the grafana userid so it can use CloudWatch as a data source.

The MQ program

All of the MQ programs for monitoring are in the same github repository. Cloning that repository gives copies of all of the MQ source code, configuration scripts, and example Grafana dashboards. You will need to get a copy of an error-logging package using

      go get -u github.com/Sirupsen/logrus

To build the CloudWatch collector, mq_aws, you also need the AWS client packages:

    go get -u github.com/aws/aws-sdk-go/service

You can then build the collection program with

    go build -o mq_aws cmd/mq_aws/*.go

Just as for the other monitoring programs, configure the queue manager through the MQSC and shell scripts in that directory, so that the main program is started as an MQ Service.

Looking at CloudWatch data

Once the monitoring program is running correctly (you can look for errors in /var/mqm/errors/aws.out if you keep the same MQSC script), the metrics are sent to CloudWatch.

When you log into AWS and go to the CloudWatch Management Console you can see them in the Custom Metrics section, under the IBM/MQ namespace:

 

From there, metrics can be selected. You can see two filters, one for the queue-related metrics (which of course also includes the queue manager name as a field) and one for the queue-manager-wide metrics:

 

 

Selecting some of these metrics allows creation of a dashboard in CloudWatch.

Using Grafana

The CloudWatch dashboard is not very sophisticated graphically, but it does at least allow graphs to be created and shown. Better as a visualisation interface is Grafana. And for that, I created a dashboard similar to that used for all the other databases in this series of articles. The biggest difference is that there does not seem to be a way to directly query using wildcards for the queue or queue manager names ("dimensions" in CloudWatch terms); using Grafana templates might be a way to deal with that. The queries used in my sample dashboard explicitly name the queues to be monitored. Also, refresh times are set to be much less frequent because of the rates used in CloudWatch, and the pricing model that charges for queries and updates.

 

Summary

This series of articles has shown how MQ V9 can feed data into a variety of monitoring solutions, including those commonly-used for cloud deployments of queue managers.

I would welcome feedback on these tools. Please leave any feedback here, or in the GitHub issue tracker, whether bugs, enhancements, or thoughts on the value of the monitoring.

 

Entry Details

Statistics
0 Favorited
6 Views
1 Files
0 Shares
1 Downloads
Attachment(s)
docx file
Using AWS CloudWatch to monitor IBM MQ queue managers.docx   233K   1 version
Uploaded - Wed November 13, 2019

Tags and Keywords

Related Entries and Links

No Related Resource entered.