Imagine an environment containing many MQ Queue Managers, where you use these Queue Managers to provide High Availability of MQ. Queue names are duplicated across all of the Queue Managers, so a message must be sent to a specific queue, but it can go to any Queue Manager. You may even have a priority order for which Queue Manager to send messages to. In this environment, you want your flows in App Connect Enterprise to put to one Queue Manager, and if that Queue Manager becomes unavailable, the flow should switch to using another Queue Manager.
The Solution: Client Connection Definition Tables
The MQ nodes in App Connect Enterprise can use a Client Connection Definition Table (CCDT) to define multiple client connections to the various Queue Managers in your environment. These can be given a priority order that App Connect Enterprise will follow when making the connection.
In the next section, we will go through an example with a simple flow that writes to one of two Queue Managers, and automatically switches queue manager when the current one is unavailable.
The components
- Two Queue Managers, let’s call them “Alice” and “Bob”, each with a queue called “OUT”. These also have connector and listener definitions, so clients can connect to them. We will also put these in a Queue Manager group called “QMGroup”.
- A single CCDT file that defines connections to both Queue Managers.
- An ACE Integration Node, or a standalone Integration Server, containing a flow that accepts a HTTP request and sends the message to an output queue (your flow can be more complex).
Setting up the output Queue Managers
First create and start a Queue Manager called “Alice”, then run the MQSC commands below against “Alice”, replacing YOUR_USER_ID with a user that has sufficient authority in MQ to write to the channel. It should be sufficient to be in the mqm group.
These commands do the following:
- Creates a queue to write to.
- Create and start a listener.
- Create a server connection channel, then set the channel authentication such that just the specified user can write to the channel. Other channel authentication settings may also work.
DEF QL(OUT)
DEF LISTENER(ALICELISTENER) TRPTYPE(TCP) CONTROL(QMGR) PORT(1414)
START LISTENER(ALICELISTENER)
DEF CHL(ALICEOUTCHAN) CHLTYPE(SVRCONN)
SET CHLAUTH(ALICEOUTCHAN) TYPE(ADDRESSMAP) ADDRESS('*') USERSRC(NOACCESS) WARN(NO) ACTION(ADD)
SET CHLAUTH(ALICEOUTCHAN) TYPE(BLOCKUSER) USERLIST('nobody') WARN(NO) ACTION(ADD)
SET CHLAUTH(ALICEOUTCHAN) TYPE(USERMAP) CLNTUSER('YOUR_USER_ID') USERSRC(CHANNEL) ACTION(ADD)
Then create and start a second Queue Manager called “Bob”, running the MQSC commands below against it. Again, replace YOUR_USER_ID with a user that has sufficient authority in MQ to write to the channel.
DEF QL(OUT)
DEF LISTENER(BOBLISTENER) TRPTYPE(TCP) CONTROL(QMGR) PORT(1415)
START LISTENER(BOBLISTENER)
DEF CHL(BOBOUTCHAN) CHLTYPE(SVRCONN)
SET CHLAUTH(BOBOUTCHAN) TYPE(ADDRESSMAP) ADDRESS('*') USERSRC(NOACCESS) WARN(NO) ACTION(ADD)
SET CHLAUTH(BOBOUTCHAN) TYPE(BLOCKUSER) USERLIST('nobody') WARN(NO) ACTION(ADD)
SET CHLAUTH(BOBOUTCHAN) TYPE(USERMAP) CLNTUSER('YOUR_USER_ID') USERSRC(CHANNEL) ACTION(ADD)
Creating the CCDT
You can create the CCDT file on either queue manager, but you must have all of the client definitions in one single file. This means that we must create all of the client connection channels in one queue manager; it doesn’t matter which one though. Run runmqsc against either Queue Manager and run the following commands:
DEF CHL(ALICEOUTCHAN) CHLTYPE(CLNTCONN) TRPTYPE(TCP) QMNAME('QMGroup') DESCR('Client-connection to ALICEOUTCHAN for the group QMGroup') CONNAME('127.0.0.1(1414)') CLNTWGHT(0)
DEF CHL(BOBOUTCHAN) CHLTYPE(CLNTCONN) TRPTYPE(TCP) QMNAME('QMGroup') DESCR('Client-connection to BOBOUTCHAN for the group QMGroup') CONNAME('127.0.0.1(1415)') CLNTWGHT(10)
Things to note:
- The names of the client connection channels must match the names of the server connection channels in each queue manager.
- We use the IP address 127.0.0.1 as the queue managers are local in this example, so these should change as needed in your environment. The ports should also match those set on the listeners on the queue managers.
- We set a weighting for each channel with the CLNTWGHT setting, where lower numbered weightings are selected in preference to higher ones. In this case we use “Alice” in preference over “Bob”.
- Both client channels have the QMNAME set to the name of the Queue Manager group we are creating. This is case-sensitive, so we surround the name in single-quotes.
This will crate a CCDT file that you can copy to where your Integration Node is running. On UNIX systems, you should be able to find this file at /var/mqm/qmgrs/{Queue Manager Name}/@ipcc/AMQCLCHL.TAB
Creating a message flow
First we need to create a message flow for the Integration Server to run. This will be a simple flow that can be called over HTTP and will send a message to one of our queue managers. You can use a more complex flow, as long as it uses an “MQPolicy” on the MQOutput node.
- Launch the ACE toolkit:
{path to ACE install}/ace toolkit
- Create a new application and add a new flow. The flow should contain the following message flow nodes:
Then we need to create the “MQPolicy” policy and set it to connect to our queue managers using our CCDT file:
- Create a new Policy project called “MyMQPolicy”, and create a policy in it of type “MQEndpoint” with the name “MQPolicy”.
- In the policy, set the following properties:
- Set “Connection” to “CCDT”.
- Set the “Queue Manager Name” to “*QMGroup” (remember that this is case sensitive)
- Set the “Security Identity DSN” to “QmGroupCreds”
Note that the “*” at the start of the Destination Queue Manager Name is important as it tells the client that the Queue Manager name does not have to match, and to use the group name. This is what lets the Integration Node send to Queue Managers with different names.
Finally, add both the application and the policy to a new BAR file.
Running the message flow
Once we have a message flow, we can run it in an Integration Server and start pushing messages through.
For a standalone Integration Server:
- Make sure you have sourced the mqsiprofile script:
. {path to ACE install}/server/bin/mqsiprofile
- Create a “work directory” for the Integration Server:
mqsicreateworkdir {path to workdir}
- Copy the CCDT file we created earlier into the workdir. This is not essential but it’s a good place to store it.
- Modify the
server.conf.yaml file in the work directory. Set the value for BrokerRegistry.mqCCDT to the path of the CCDT file.
- Create the security identity used by the MQPolicy:
mqsisetdbparms -w {path to workdir} -n mq::QmGroupCreds -u {your_user_id} entering your password when prompted.
- Deploy the bar file to the work directory:
mqsibar -a {path to bar file} -w {path to workdir}
- Start a standalone Integration Server:
IntegrationServer --name Server1 --work-dir {path to workdir}
For an Integration Node:
- Make sure you have sourced the mqsiprofile script:
. {path to ACE install}/server/bin/mqsiprofile
- Create a Node:
mqsicreatebroker Node1
- Start the Node:
mqsistart Node1
- Set the Integration Node to use the CCDT file:
mqsichangeproperties Node1 -o BrokerRegistry -n mqCCDT -v {path to CCDT file}
- Create an Integration Server in the node:
mqsicreateexecutiongroup Node1 -e Server1
- Create the security identity used by the MQPolicy:
mqsisetdbparms Node1 -n mq::QmGroupCreds -u {your_user_id} entering your password when prompted.
- Deploy the bar file to the Integration Server:
mqsideploy Node1 -e Server1 -a {path to bar file}
Testing the flow
Once the Integration Server is running, you should be able to call the flow. The simplest way is to use a web browser to go to the URL being server by the flow. If you are running the Integration Server on your local machine then you can call http://localhost:7800/SendToMQ (You can check the output logs from the Integration Server to confirm that it is listening on port 7800).
The flow will start sending messages to the Queue Manager "Alice". You can use the sample MQ applications to read from the queue: {path to MQ install}/samp/bin/amqsget OUT Alice.
If you stop and start "Alice" then call the URL, the flow will start sending messages to "Bob". If you are running on Linux you can run a simple loop that calls the flow repeatedly (something like while true; do curl http://localhost:7800/SendToMQ; sleep 1; done), then watch the Integration Server flip between the queue managers as you restart them.
Limitations
- Once the Integration Server is sending messages to a Queue Manager it will not switch until that Queue Manger goes down; i.e. it will not switch back to the original Queue Manager unless it is unable to connect to the second Queue Manager.
- You can only have a single CCDT file for the Integration Node or a standalone Integration Server, so if you have multiple flows using multiple target Queue Manager Groups, you need one CCDT file that contains definitions for all client connections for all queue manager groups.
Useful links