Hi Sumit,
the issue is that the IBM MQ implementation of pub/sub uses some of the fields in the MQMD (including the message id and correlation id).
If you put a message to a queue and get back the message id, the message id returned in the response to the MQPUT is the same as the message id received by the application that performs the MQGET.
If you do the MQPUT to a topic alias the message id received by the application performing the MQGET will be different to the one seen in the MQPUT response. The request/response convention is to use the message id of the request in the correl id of the response. If an application follows this convention and the message was published and replicated, then the sending application can't find the correct response message, because the correl id is wrong.
The stream queue capability resolves this problem, because both messages (the original and the streamq copy) have the same message id. Of course, if your streamq destination is a topic alias, the message id there will be wrong, but the original applications will keep working. If you changed a working application from using a queue to using a topic alias and delivering the request message via an administrative subscription, the applications would not deliver response messages correctly.
This was identified several versions ago and an RFE requesting enhancements to pub/sub was raised. Stream Queues address the requirement, but in a different way.
So if you want to replicate the bodies of messages, but aren't using request/response processing, then go ahead and use pub/sub if you would prefer to. If you need to capture your request messages then Stream Queues provide a solution.
Regards,
------------------------------
Neil Casey
Senior Consultant
Syntegrity Solutions
Melbourne, Victoria
IBM Champion (Cloud) 2019-21
------------------------------
Original Message:
Sent: Thu August 26, 2021 06:51 AM
From: sumit sinha
Subject: Why streaming queues can't be use for HA scenario specially for zero loss of message when the queue manager node is down
Thanks David.
"The rules you highlight would mean that you shouldn't also set up streaming on Q1 on QMGR2 as that would create a chain of connected streams. And the second one prevents you from enabling a stream queue on QMGR1's xmitq instead of Q1 itself."
I understand, technically i can but i shouldn't create a chain of connected streams & neither its supported.
can i ask one more question : any specifc reason for launching this feature when i could achieve the same thing ( replicating messages )with MQ pub-sub.
Thanks,
Sumit
------------------------------
sumit sinha
Original Message:
Sent: Thu August 19, 2021 02:47 AM
From: David Ware
Subject: Why streaming queues can't be use for HA scenario specially for zero loss of message when the queue manager node is down
Hi Sumit,
The configuration you propose is supported, as you are simply streaming from Q1 to RQ1 on QMGR1.
The rules you highlight would mean that you shouldn't also set up streaming on Q1 on QMGR2 as that would create a chain of connected streams. And the second one prevents you from enabling a stream queue on QMGR1's xmitq instead of Q1 itself.
We'll see if we can improve our documentation to make this clearer.
So if you are simply looking to duplicate the messages flowing through Q1@QMGR1 so that they also arrive at Q1@QMGR2 then you definitely can.
Depending on your needs, you could see this as an HA or DR mechanism, but it is very different to how MQ natively handles HA and DR. The main difference from an application's point of view is that only the production of the messages is being replicated with streaming queues, not the consumption of those messages, unlike something like RDQM which replicates all state changes, keeping multiple representations of the same queue in step.
So with streaming queues you'd need your own mechanism in place to be consuming messages from Q1@QMGR2 to ensure that the queue did not over fill, yet still holds enough messages to provide you benefit in the case of a fail over from QMGR1. This would obviously expose you to possibly processing the same message multiple times in the event of a switch from QMGR1 to QMGR2, but maybe you already have ideas on how to handle that within the application.
Thanks,
David
------------------------------
David Ware
STSM, IBM MQ Chief Architect
Hursley, UK
Original Message:
Sent: Wed August 18, 2021 10:37 AM
From: sumit sinha
Subject: Why streaming queues can't be use for HA scenario specially for zero loss of message when the queue manager node is down
Hi All,
As we have a new feature for streaming queues in V 9.2.3. The document does mention some of the sceanrios where streaming queues can be used . It also mention the restrictions for using streaming queues as mentioned in this link .
few restritions to highlight here for my question:
- Defining a chain of queues streaming to each other, for example, Q1->Q2, Q2->Q3, Q3->Q4
- Defining STREAMQ on a queue configured with USAGE(XMITQ)
So, as per documentation, these cases are not supported though i think, these are technically feasible. My question is : why can't we use streaming queues for message availability across queue managers / nodes , this may insure message availability across different queue manager in HA scenario & almost 0 RPO in case of DR.
App -> Q1 (QMGR1)-----RQ1(StreamQ & RemoteQ ) ------xmitQ -------------------------------------Q1(QMGR2)
I undersstand , we have other solutions for replicating messages across different queue manager nodes in HA & DR scenario ( RDQM ), but not all the queues in a queue manager are critical & if we can achive replication of message / duplicaion of message across queue manager nodes using streaming queue, that would add a granual level replication feature. ( though licensing & some performance overhead may come )
Thanks,
Sumit
------------------------------
sumit sinha
------------------------------