A floating IP or Virtual IP (VIP) allows you to use a single IP address within a DNS entry that will automatically move between servers depending on which host has the active service. This avoids the need to specify multiple addresses and have clients detect which host they should direct service requests to.
If you have ever used IBM MQ RDQM for high availability you are probably aware of the floating IP address or VIP capability that is included as part of the RDQM system, but did you know that users of IBM MQ running on Red Hat Enterprise Linux (RHEL) can implement a floating IP address for Native HA and Multi-instance queue managers using the Red Hat supported utility Keepalived.
Keepalived is a software-based routing solution designed for high availability, it uses the Virtual Router Redundancy Protocol (VRRP) to manage VIP failover between Linux servers. It ensures service availability by monitoring an application’s health or readiness and automatically switching traffic to a server where the active instance of the application is running. Keepalived can use a check script that exits with ‘zero’ if the application is active or ‘one’ if it is inactive. By using Keepalived with a script to monitor where a queue manager is running, you can implement a floating IP solution for Native HA or Multi-instance queue managers.
By implementing a very simple check script that calls ‘dspmq’ we can see if a queue manager has a status of ‘RUNNING’ and return zero to indicate to Keepalived that this is the server where the VIP should be. The server that has the active queue manager is given a higher priority than the servers where the script is not detecting a ‘RUNNING’ queue manager and returning a status of one (a replica or standby queue manager instance). If a server is down, it simply won’t have a say, and the decision is left to the remaining servers to decide who has the running queue manager.
In this sample I have taken the same approach used by the IBM MQ Operator to detect where the active queue manager is located. A key point to remember is that Native HA and Multi-instance queue managers will handle where the active queue manager is, Keepalived does not manage any resources other than the VIP, it simply observes and moves the VIP accordingly.
Note that we must execute the ‘dspmq’ command with the ‘-n’ option to ensure that output will always be generated in the same way without any language translation - this ensures that we always get the expected string of ‘RUNNING’.
Steps for setting up
You will need ‘root’ level access to setup Keepalived.
Create an IBM MQ check script for Keepalived
As user ‘root’ create a script in the ‘/usr/local/bin/’ folder and give it execute permission.
$ vi /usr/local/bin/mq-active.sh
Code sample:
#!/bin/bash
if /opt/mqm/bin/dspmq -n -m $1 | grep -q -F '(RUNNING)'; then
exit 0
fi
exit 1
$ chmod u+x /usr/local/bin/mq-active.sh
The -F option with grep tells it to use simple string matching, the -q is for quiet output.
You might want to enhance the script, but I have deliberately kept it succinct to show the pertinent commands.
Install Keepalived
Keepalived should be available in the base RHEL repository. As ‘root’ run the following command to install.
$ dnf install -y keepalived
Configuring Keepalived
After you have installed Keepalived on each server, configure it by editing the Keepalived configuration file found in the following folder:
/etc/keepalived/keepalived.conf
Sample:
vrrp_script check_service {
script "/usr/local/bin/mq-active.sh DEMO"
interval 2 # Check every 2 seconds
weight -20 # Reduce priority by 20 if script fails
fall 1 # Required failures before triggering
rise 1 # Required successes before resetting
}
vrrp_instance VI_1 {
state EQUAL
interface eth1
virtual_router_id 51
priority 100
advert_int 1
track_script {
check_service
}
virtual_ipaddress {
vvv.vvv.vvv.vvv/24
}
check_unicast_src
unicast_peer {
bbb.bbb.bbb.bbb
ccc.ccc.ccc.ccc
}
}
If your network allows multicast, you can leave out the ‘unicast_peer´ section. Alternatively, if it does not allow multicast then each server must be told about the other server’s IP address in the group (so server-a lists server-b and server-c, server-b lists server-a and server-c IP addresses and so on), this also has the advantage of acting like an allow list so only servers with the listed IP addresses will be allowed to participate. Refer to the Keepalived documentation for a full description of the configuration file settings.
This would be the configuration for server ‘a’ where vvv… is the floating IP address, and bbb… and ccc… are the IP addresses of servers ‘b’ and ‘c’ in a group of servers called [a, b, c].
To enable and start
$ systemctl enable keepalived
$ systemctl start keepalived
Should IBM MQ provide this capability?
In my opinion it’s a firm no but let me explain why. Linux system administrators do not like to give system-level access to application teams, for good reason, and application administrators don’t like to be dependent on having systems or networking skills to manage their application capabilities and operations.
RDQM replication is disk-based and uses DRBD for that, DRBD is underpinned by HA systems Pacemaker and Corosync, because of that RDQM can leverage those systems to provide a floating IP without getting into the weeds, but at some level it still requires ‘root’ level access.
Keepalived also requires ‘root’ level access and it’s for that reason I don’t believe IBM MQ should be controlling the management of a floating IP. DevOps has its place, but some system or network-level configuration should be left with the specialists in that area.
Disclaimer
The views and opinions expressed on this blog are solely those of the author and do not necessarily reflect the official policy or position of any other agency, organization, employer, or company. The samples in this article are provided "as is" without warranty any information in this article is used at your own risk.