MQ

Hello dear MQ people,

we have the following setup at hand:

A client is sending messages with message grouping being connected to a transmissionqueuemanager, which then relays the messages to the target queuemanager via SCTQ into a local queue.

The problem arises after the messages have been put to the SCTQ and are moved to the local queue: The messages are committed one by one, which leads to a potential problem when the local queue ends up full and no complete message group has been put into the local queue. We tested and verified this, as the reading application will only read completed message groups, which then leads to a softlock.

Our assumption was, that the messages can only be committed once the last message in group has been transferred, but this is not the case here.

Here's a simple illustration:

Is this intended behaviour or are there ways to configure it so that the messages are only committed as a full group when using the above mentioned setup?

Kind regards

Sebastian

------------------------------
Sebastian Wilk
------------------------------

The queue manager knows if the sequence of messages (and segments) in the group is complete,  regardless of the order in which those messages actually arrive.

With message groups (and segments) you can choose how much help the queue manager gives you in managing the reassembly of messages and whether messages in incomplete groups can be consumed.  The option MQGMO_ALL_MSGS_AVAILABLE indicates whether all of the messages in the group must be available before any message in the group is returned.

Channels move message (and segments) individually, they are not concerned with the completeness of the group (for example the channel has no knowledge of how the message will be consumed).

You state "Our assumption was, that the messages can only be committed once the last message in group has been transferred", if this was the case the full set of messages would still need to be on the target queue, and hence MAXDEPTH would need to be high enough to permit this. 

So the obvious answer to your question is 'no there is no option to only transmit/commit whole groups', the more intersting thing is what you were actually hoping to achieve if there were ?

What am I missing ?? For example do you have multiple message groups accumulating on the target queue and were hoping not to need MAXDEPTH to be big enough for all of the groups concurrently (not possible I'm afraid). What issue do you anticipate through increasing the max depth on the target queue ?

Hello dear MQ people,

we have the following setup at hand:

A client is sending messages with message grouping being connected to a transmissionqueuemanager, which then relays the messages to the target queuemanager via SCTQ into a local queue.

The problem arises after the messages have been put to the SCTQ and are moved to the local queue: The messages are committed one by one, which leads to a potential problem when the local queue ends up full and no complete message group has been put into the local queue. We tested and verified this, as the reading application will only read completed message groups, which then leads to a softlock.

Our assumption was, that the messages can only be committed once the last message in group has been transferred, but this is not the case here.

Here's a simple illustration:

Is this intended behaviour or are there ways to configure it so that the messages are only committed as a full group when using the above mentioned setup?

Kind regards

Sebastian

------------------------------
Sebastian Wilk
------------------------------

Your statement is correct in regards to the maxdepth, and this is also the issue we ran into.

Our local queue has reached the maxdepth but only incomplete message groups were sitting in it, thus the reading application could not consume any of them, resulting in the putting job to park the messages in the SCTQ.

The takeaway we are looking for here is this scenario:

We have 700.000 messages to be put into the target queue, the queue has a capacity fo 50.000

How can we make sure that there are only complete groups in the target queue so that it does not lock itself up?

The obvious choice is to increase the maxdepth (which we already did) but it sounds like a weird design that sender and receivers do have to respect the message group but intraqueuemanagercommunication is able to put and commit a message one by one? This could always lead to the abovementioned situation if the messages exceed the maxdepth of the targetqueue.

I'll have a look at the MQGMO_ALL_MSGS_AVAILABLE option.

The queue manager knows if the sequence of messages (and segments) in the group is complete,  regardless of the order in which those messages actually arrive.

With message groups (and segments) you can choose how much help the queue manager gives you in managing the reassembly of messages and whether messages in incomplete groups can be consumed.  The option MQGMO_ALL_MSGS_AVAILABLE indicates whether all of the messages in the group must be available before any message in the group is returned.

Channels move message (and segments) individually, they are not concerned with the completeness of the group (for example the channel has no knowledge of how the message will be consumed).

You state "Our assumption was, that the messages can only be committed once the last message in group has been transferred", if this was the case the full set of messages would still need to be on the target queue, and hence MAXDEPTH would need to be high enough to permit this. 

So the obvious answer to your question is 'no there is no option to only transmit/commit whole groups', the more intersting thing is what you were actually hoping to achieve if there were ?

What am I missing ?? For example do you have multiple message groups accumulating on the target queue and were hoping not to need MAXDEPTH to be big enough for all of the groups concurrently (not possible I'm afraid). What issue do you anticipate through increasing the max depth on the target queue ?

Hello dear MQ people,

we have the following setup at hand:

A client is sending messages with message grouping being connected to a transmissionqueuemanager, which then relays the messages to the target queuemanager via SCTQ into a local queue.

The problem arises after the messages have been put to the SCTQ and are moved to the local queue: The messages are committed one by one, which leads to a potential problem when the local queue ends up full and no complete message group has been put into the local queue. We tested and verified this, as the reading application will only read completed message groups, which then leads to a softlock.

Our assumption was, that the messages can only be committed once the last message in group has been transferred, but this is not the case here.

Here's a simple illustration:

Is this intended behaviour or are there ways to configure it so that the messages are only committed as a full group when using the above mentioned setup?

Kind regards

Sebastian

------------------------------
Sebastian Wilk
------------------------------