Hi, We are designing a system where we are getting all messages on MQ queue 'X' from external system. From this queue 'X' messages are further routing to the respective systems queues 'A', 'B', 'C' on same Queue Manager which are later processed by several applications.

Now, i need below recommendations to design this kind of system. Attached is the sample design.

1. What if queue 'A', 'B', or 'C' are full or unavailable. Will message rollback automatically back to the queue 'X' after some retries OR how is that ? Looking for best approach we should follow.

2. If we have millions of messages coming on queue 'X' and because of unsuccessful delivery of one message to another queues 'A', 'B' on same QM; how the delay of delivery of unsuccessful message can be handled when resources (queues 'A', 'B') are available again.

Please suggest some good approach we can follow considering performance as well.

Any assistance would be much appreciated. Thanks!

What you do when one (or more) of Queue A, B or C reject the MQPUT of a message onto it depends on a few criteria about your messages. Mainly ordering. Is it essential that the messages on Queue X remain in the same order when they are delivered downstream?

If order is not required, then one technique (utilised by MQ queue manager channels) is to retry a few times (just in case it is a transitory problem) and then to write the message to a side queue (called the Dead Letter Queue in the case of channels, or some applications use the Backout Queue for putting messages to one side) for later processing. Then you can get on with delivering messages to the queues that are still accepting messages.

You ask whether the message will rollback automatically to QueueX after some retries. Well that depends on what you code. Assuming that you get the messages from QueueX using Syncpoint (MQGMO_SYNCPOINT), then if you have a failure, you can roll the message back onto Queue X again using the MQBACK verb. However, if you are moving it to the side queue, then you won't need to roll back the message. I assume you are designing and writing the application that reads from QueueX and puts to Queues A,B&C? 

Alternatively I suppose you might be using Message Broker, aka IBM ACE to do this routing? In which case it has all this sort of processing built in.

As to the side queue, you would also need to process that. You could in your application, keep track of whether messages to Queue A, which was previously failing to accept messages, is now back and open for business again, and you could then go and read from the side queue and grab all the side-lined messages. Alternatively, you could have a process a bit like the Dead Letter Queue Handler which periodically retries all the failed puts.

What are you using to determine which message goes to which queue?

Is there any reason why you want this staging queue, rather than using MQ channels to do all the above for you by having the External system use the Queue A,B&C queue names instead of QueueX?

Cheers,
Morag

Hi, We are designing a system where we are getting all messages on MQ queue 'X' from external system. From this queue 'X' messages are further routing to the respective systems queues 'A', 'B', 'C' on same Queue Manager which are later processed by several applications.

Now, i need below recommendations to design this kind of system. Attached is the sample design.

1. What if queue 'A', 'B', or 'C' are full or unavailable. Will message rollback automatically back to the queue 'X' after some retries OR how is that ? Looking for best approach we should follow.

2. If we have millions of messages coming on queue 'X' and because of unsuccessful delivery of one message to another queues 'A', 'B' on same QM; how the delay of delivery of unsuccessful message can be handled when resources (queues 'A', 'B') are available again.

Please suggest some good approach we can follow considering performance as well.

Any assistance would be much appreciated. Thanks!

Hi, We are designing a system where we are getting all messages on MQ queue 'X' from external system. From this queue 'X' messages are further routing to the respective systems queues 'A', 'B', 'C' on same Queue Manager which are later processed by several applications.

Now, i need below recommendations to design this kind of system. Attached is the sample design.

1. What if queue 'A', 'B', or 'C' are full or unavailable. Will message rollback automatically back to the queue 'X' after some retries OR how is that ? Looking for best approach we should follow.

2. If we have millions of messages coming on queue 'X' and because of unsuccessful delivery of one message to another queues 'A', 'B' on same QM; how the delay of delivery of unsuccessful message can be handled when resources (queues 'A', 'B') are available again.

Please suggest some good approach we can follow considering performance as well.

Any assistance would be much appreciated. Thanks!

Hi Anurag
1)
No, messages will not automatically rollback to queue X. But it is an option
queue X does not know that queue A is full.
What happens when a message cannot be delivered:
https://www.ibm.com/support/knowledgecenter/SSFKSJ_9.1.0/com.ibm.mq.con.doc/q015720_.htm
Queue depth events
https://www.ibm.com/support/knowledgecenter/SSFKSJ_9.1.0/com.ibm.mq.mon.doc/q036420_.htm
My proposal
One can monitor queue events e.g. full by 25%, 75%.
Stop sending messages into the queue when it is full by 75%
Application X could work with a configuration file that states do not send Messages to A when almost full send it to A.ELSE
The "queue monitoring" would control the content of this configuration file.
2)
You could ask Application A to first consume messages from A.ELSE then from A
Or let another support tool move messages from A.ELSE to A

Queue Full of A,B or C is not your only risk.
What shall happen if Mainframe sends messages faster to queue X then your app distributes it from X to A,B,C

Hi, We are designing a system where we are getting all messages on MQ queue 'X' from external system. From this queue 'X' messages are further routing to the respective systems queues 'A', 'B', 'C' on same Queue Manager which are later processed by several applications.

Now, i need below recommendations to design this kind of system. Attached is the sample design.

1. What if queue 'A', 'B', or 'C' are full or unavailable. Will message rollback automatically back to the queue 'X' after some retries OR how is that ? Looking for best approach we should follow.

2. If we have millions of messages coming on queue 'X' and because of unsuccessful delivery of one message to another queues 'A', 'B' on same QM; how the delay of delivery of unsuccessful message can be handled when resources (queues 'A', 'B') are available again.

Please suggest some good approach we can follow considering performance as well.

Any assistance would be much appreciated. Thanks!

Thanks Matthias and sorry for delay , was on vacations.

Anyways we have this distribution of messages from queue X to queues A, B and C on the same queue manager & our java applications further process the messages from these A, B, & C queues respectively.

1. We will be putting the messages to backout queue (A.ELSE) as you mentioned if something goes wrong with A, B and C queues, (assuming they are full). But if application A starts consuming from A.ELSE & also the messages from queue X now will keep coming on A.ELSE always, because queue A will remain full all the time as application started processing messages from A.ELSE. How this can be handled?

2. Also, is it good practice to move messages from A.ELSE back to source queue 'X' ? If yes , can this be established as Queue to Queue transmission on same queue manager?

3. How i will be notified in case i set the Queue Depth events? Let's say i enable the Queue Depth High event to 80%, so how do i come to know when this limit is reached?

Hi Anurag
1)
No, messages will not automatically rollback to queue X. But it is an option
queue X does not know that queue A is full.
What happens when a message cannot be delivered:
https://www.ibm.com/support/knowledgecenter/SSFKSJ_9.1.0/com.ibm.mq.con.doc/q015720_.htm
Queue depth events
https://www.ibm.com/support/knowledgecenter/SSFKSJ_9.1.0/com.ibm.mq.mon.doc/q036420_.htm
My proposal
One can monitor queue events e.g. full by 25%, 75%.
Stop sending messages into the queue when it is full by 75%
Application X could work with a configuration file that states do not send Messages to A when almost full send it to A.ELSE
The "queue monitoring" would control the content of this configuration file.
2)
You could ask Application A to first consume messages from A.ELSE then from A
Or let another support tool move messages from A.ELSE to A

Queue Full of A,B or C is not your only risk.
What shall happen if Mainframe sends messages faster to queue X then your app distributes it from X to A,B,C

Hi, We are designing a system where we are getting all messages on MQ queue 'X' from external system. From this queue 'X' messages are further routing to the respective systems queues 'A', 'B', 'C' on same Queue Manager which are later processed by several applications.

Now, i need below recommendations to design this kind of system. Attached is the sample design.

1. What if queue 'A', 'B', or 'C' are full or unavailable. Will message rollback automatically back to the queue 'X' after some retries OR how is that ? Looking for best approach we should follow.

2. If we have millions of messages coming on queue 'X' and because of unsuccessful delivery of one message to another queues 'A', 'B' on same QM; how the delay of delivery of unsuccessful message can be handled when resources (queues 'A', 'B') are available again.

Please suggest some good approach we can follow considering performance as well.

Any assistance would be much appreciated. Thanks!