MQ

MQ

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
Back to discussions
Expand all | Collapse all

How to set Message priority for a message on a queue

  • 1.  How to set Message priority for a message on a queue

    Posted Wed October 16, 2024 04:48 PM

    Hey All, 

    We have a queue : consider(ABC.AUTH)  On the mainframe queue manager, which is a shared queue , spanning  across four queue managers.

    QM1: TO.ABC.AUTH (distributed QMGR)

    QM2: ABC.AUTH( Mainframe QMGR) 

    We are sending messages in a batch ( consider 10k messages) from QM1 TO QM2, and they will be picked up by CICS program.

    the mainframe queue is like a bridge queue and is used by different programs. (one is validating the authorization of card payments) .

    Question here is, is there anyway we can set the low priority to the message that are coming through the batch, such a way it allows the other Authorization transactions to go through (without getting timed out which is now set at 10 secs).

    Note:  On our testing , when we sent a batch of 10k records to that mainframe queue(ABC.AUTH) and concurrently we ran the authorization messages, at first we could see the success, but eventually all got timed out, since the messages are all set to priority and are going out on  FIFO basis.

    Thanks, 

    vishal 



    ------------------------------
    vishal kumar kalava
    ------------------------------


  • 2.  RE: How to set Message priority for a message on a queue

    Posted Thu October 17, 2024 05:03 AM

    Vishal,

    You can set the priority different ways.  A default priority can be set on the remote queue used on distributed. An explicit priority can be set on the destination (JMS) or an explicit priority can be set on the message (MQMD) or JMS message properties (msg.setPriority(int i) ).

    Enjoy



    ------------------------------
    Francois Brandelik
    ------------------------------

    Original Message


  • 3.  RE: How to set Message priority for a message on a queue

    Posted Thu October 17, 2024 05:44 AM

    You also need to make sure that the MSGDLVSQ (message delivery sequence) for the queue ABC.AUTH is set to PRIORITY, otherwise the priority in the MQMD is ignored.



    ------------------------------
    Rich Harran
    ------------------------------

    Original Message


  • 4.  RE: How to set Message priority for a message on a queue

    Posted Thu October 17, 2024 10:24 AM
    Edited by vishal kumar kalava Thu October 17, 2024 10:26 AM

    Thanks @Rich Harran & @Francois Brandelik for your valuable inputs.

    I just checked the existing settings for the queue, I see the ABC.AUTH queue on mainframe is already set to PRIORITY, and possible changes I need to make is to change the priority of the messages using the explicit priority can be set on the message MQMD.

    Does that explicit priority set on the messages (MQMD) overwrites the default priority (0) set on the Distributed Remote queue?

    Best Regards,

    Vishal



    ------------------------------
    vishal kumar kalava
    ------------------------------

    Original Message


  • 5.  RE: How to set Message priority for a message on a queue

    Posted Thu October 17, 2024 10:30 AM

    Vishal:

    Q: Does that explicit priority set on the messages (MQMD) overwrites the default priority (0) set on the Distributed Remote queue?

    A: In my experience with the MQ product, an explicit value provided when messages are PUT to a queue will take precedence over a default queue attribute value.

    Bob



    ------------------------------
    Bob
    ------------------------------

    Original Message


  • 6.  RE: How to set Message priority for a message on a queue

    Posted Thu October 17, 2024 10:55 AM

    Thanks @Bob Gibson, that's all I wanted to know.

    You guys are awesome :-) 



    ------------------------------
    vishal kumar kalava
    ------------------------------

    Original Message


  • 7.  RE: How to set Message priority for a message on a queue

    Posted Thu October 24, 2024 05:04 PM
    Edited by vishal kumar kalava Thu October 24, 2024 05:05 PM

    Have one more question!!
    We are trying to change the priority on the remote queue properties to suppose say 5, when the message landing on the remote queue manager which is on the mainframe is not carrying the priority.

    Is there a get that priority forwarded to the Mainframe Queue manager.

    I tested the scenario between two distributed queue managers and its working; however this is not working between distributed QMGR and the Mainframe QMGR.



    ------------------------------
    vishal kumar kalava
    ------------------------------

    Original Message


  • 8.  RE: How to set Message priority for a message on a queue

    Posted Fri October 25, 2024 01:56 AM

    Please be precise. 

    "We are trying to change the priority on the remote queue properties"

    Do you mean change MESSAGE priority?

     "... to suppose say 5, when the message landing on the remote queue manager which is on the mainframe is not carrying the priority." 

    By "it" do you mean the MESSAGE?  If so, what exactly is the message priority of the message that arrives on the z/OS MQ queue?

    "Is there a get that priority forwarded to the Mainframe Queue manager."

    Does the queue on z/OS MQ specify message delivery sequence PRIORITY of FIFO?  If FIFO, priority is ignored at MQGET.

    "I tested the scenario between two distributed queue managers and its working; however this is not working between distributed QMGR and the Mainframe QMGR." 

    See previous question. 

    "... not working ..." is not precise.  What did you expect to see?  What did you see?



    ------------------------------
    bruce2359
    Consultant/trainer
    CTTECH - Computer & Telecommunications Technology
    West Coast, almost
    000-000-0000
    ------------------------------

    Original Message


  • 9.  RE: How to set Message priority for a message on a queue

    Posted Fri October 25, 2024 04:17 AM
    • Changing the properties (object attributes) of a remote queue object doesn't change the attributes of the target object
    • Priority is only set from the default priority of the queue when the priority in the MQMD is set to MQPRI_PRIORITY_AS_Q_DEF (-1), which is the value in the default MQMD. Once it has been set from the default priority of the remote queue definition, it will have a different value, so it won't be changed again when it arrives at the target queue. If it is already set to something other than MQPRI_PRIORITY_AS_Q_DEF when it is put to the remote queue, then it won't be changed from that value.
    • MSGDLVSQ(PRIORITY) needs to be set on the target queue. MSGDLVSQ is not a valid attribute on a remote queue object.
    • Message priority only effects the message delivery sequence of messages that are committed on the queue when the MQGET is done. If you have a lot of low priority messages on the queue, and no high priority ones, then your transactions will start processing those. When a high priority message arrives, the next transaction that does an MQGET will get that instead of one of the low priority ones, but there might be some delay if everything is busy already processing the low priority work.

     



    ------------------------------
    Rich Harran
    ------------------------------

    Original Message


  • 10.  RE: How to set Message priority for a message on a queue

    Posted Fri October 25, 2024 10:53 AM
    A few thoughts: 

    _AS_QDEF does NOT mean default; rather, it means "as defined at the queue."

    IBM MQ comes from the factory with a variety of initial values and near zero defaults.  

    Using the factory supplied MQMD as an example, fields in the MQMD describe attributes of the associated application data. Initial values for persistence, expiry, priority, and others, are NOT production-ready.  IBM doesn't know your business requirements.  It is the responsibility of the developer to set values in the MQMD fields that are appropriate for the business.

    Once a message is MQPUT, the instantiated MQMD does NOT revert to factory initial values.

     



    Original Message


  • 11.  RE: How to set Message priority for a message on a queue

    Posted Fri October 25, 2024 11:50 AM

     I'm not sure if post 10 is a counterpoint to my post 9.

    For clarification, by 'the default priority of the queue ', I mean the value of DEFPRTY on that object, and by 'the value in the default MQMD' I was referring to the value in c language DEFAULT_MQMD.



    ------------------------------
    Rich Harran
    ------------------------------

    Original Message


  • 12.  RE: How to set Message priority for a message on a queue

    Posted Fri October 25, 2024 01:33 PM

    Not a counterpoint.  The OP seemed unclear about a queue, queue properties, and message properties. 

    My strongest recommendation is NOT to use the _AS_QDEF values, as use of these needlessly abstracts and complicates postmortem debugging. 

    My recommendation is that the app first instantiate an IBM-provided MQMD (copybook in COBOL, include-file in C, etc.); then create a working MQMD copy within the app; then explicitly set all of the MQMD fields in the working copy, and set application data payload to appropriate values; then MQPUT the message.  Explicitly here means set MQMD message priority to 5, message persistence to persistent, and not use and rely on the _AS_QDEF.




    Original Message


  • 13.  RE: How to set Message priority for a message on a queue

    Posted Thu October 31, 2024 09:09 AM
    Edited by vishal kumar kalava Thu October 31, 2024 09:11 AM

    Hello @bruce2359 & @Rich Harran

    Thank you for sharing me with all the details!!

    Bruce for you question..

    Q: Do you mean change MESSAGE priority?

    Yes, I am changing the MESSAGE priority of the Remote queue using the MQ explorer

    Q: By "it" do you mean the MESSAGE?  If so, what exactly is the message priority of the message that arrives on the z/OS MQ queue?   

    Ans: I revalidated the message priority of the messages landing on the queue which resides on MF, it is now carrying the priority. I was not seeing the priority as the application team provided me with the wrong queue name to change priority on the QMGR.

    Q:Does the queue on z/OS MQ specify message delivery sequence PRIORITY of FIFO?  If FIFO, priority is ignored at MQGET.

    Ans : Yes, message delivery sequence is set to PRIORITY on MF QMGR.

    Q: "... not working ..." is not precise.  What did you expect to see?  What did you see?

    A: If you see below picture, I see the priorities aligned as expected, if the messages have same priority then they followed FIFO, and if they have different priorities, then they followed the PRIORITY based positions. This is what I expected on the MF queue manager as well.


    Update after changing the queue Priority on the right queues.

    ·       I changed the Default Priority on the Queue (Consider A.REM) to 6, another Queue to 5(Consider B.REM) and C.REM to default 0.

    ·       When I asked the application team to set the priority using MQMD, they are not aware of that and are saying that they are directly putting the messages on the queue.

    ·       I could now see the messages going from Distributed to Mainframe Queue managers are carrying the queue priority for the associated messages. Destination Mainframe queue (MF.QUEUE)

    ·       However, the Authorizations are getting impacted due to their time outs set as 8 seconds.

    o   As per testing they are sending 5tps(Transactions per second)  using A.REM queue to MF.QUEUE, 3tps using B.REM to MF.QUEUE and during that is happening, IIB is sending 3k to 4k messages at once using C.REM to the MF.QUEUE.

    o   So 3 different remote queues, pointing to the same Mainframe Queue.

    §        A.REM used for authorizations. (These are getting impacted)

    §      B.REM used for authorization. (These are getting impacted)

    §       C.REM used for sending 3k messages not authorizations.

    o   We are seeing authorization timeouts, when the newly created C.REM queue is used by IIB application to send 3k to 4k messages at once to the destination queue.

    o   Application which is taking off these messages based on priority is very slow, for an idea to get the messages by the corresponding application on the Mainframe, it is taking 14mins to get the 3k messages. During that time even though the priority is being set correctly on the messages, we are seeing huge impact on the authorization timeouts.

    I understand this is a bad architecture to use the same destination queue for everything, but this is what has been designed and is in use since long time.

    Any thoughts or comments how to effectively improve in getting the messages without impacting the authorizations??





    ------------------------------
    vishal kumar kalava
    ------------------------------

    Original Message


  • 14.  RE: How to set Message priority for a message on a queue

    Posted Fri November 01, 2024 01:30 AM
    Edited by Francois Brandelik Fri November 01, 2024 02:06 AM

    Well all depends on the max time it takes to process a message of the C category.

    So assume all the threads are busy processing messages of the C category.

    A message of the A or B category hits the queue. It will not be processed until a thread that is processing a message of the C category is available for doing the next MQGET.

    So your time out needs to be greater than the time needed to process a message of the C Category...

    Assuming your MF application can keep up processing the authorizations it get at 8 (5 + 3) TPS, it will take 125 ms per message (assuming a single thread)

    For the batch 3K messages in 14 mins it takes 3.572 TPS or 280 ms per message (assuming a single thread)... At 30 threads, you are past your message timeout...

    And you might also want to make sure that the channel XMITQ is set to Priority and not FIFO...

    The alternative of course is to create 2 queues, one for authorizations and one for processing.



    ------------------------------
    Francois Brandelik
    ------------------------------

    Original Message


  • 15.  RE: How to set Message priority for a message on a queue

    Posted Fri November 01, 2024 07:44 PM
    Edited by bruce2359 Fri November 01, 2024 10:33 PM

    "I understand this is a bad architecture to use the same destination queue for everything, but this is what has been designed and is in use since long time.

    Any thoughts or comments how to effectively improve in getting the messages without impacting the authorizations??"

    How many concurrent consumer applications are servicing the z/OS queue?  If each message is a transaction, then look at the possibility of launching multiple consumer applications.  Mainframes are provisioned to support hugely concurrent workloads.



    ------------------------------
    bruce2359
    Consultant/trainer
    CTTECH - Computer & Telecommunications Technology
    West Coast, almost
    000-000-0000
    ------------------------------

    Original Message


  • 16.  RE: How to set Message priority for a message on a queue

    Posted Fri November 01, 2024 09:50 AM
    "When I asked the application team to set the priority using MQMD, they are not aware of that and are saying that they are directly putting the messages on the queue."

    "...directly putting the messages on the queue."  This statement adds little/no clarity.

    When an application MQOPENs a QRemote definition, the resolved queue-name is a transmission queue.  At MQPUT, messages are briefly stored in the transmission queue, and subsequently moved down the Sender-Receiver channel to the destination qmgr.  There the receiver-side Message Channel Agent MQPUTs the messages into destination queues or dead-letter queue.  This is store-and-forward technology, and definitely NOT "directly".

    IMHO, it is the responsibility of the developer to be aware of, and set all MQMD attribute values as appropriate for the business transaction (priority, persistence, etc.), and NOT rely on the _AS_QDEF (as defined at the queue) initial object attribute values, as these values can be ALTERED during application execution.



    Original Message


  • 17.  RE: How to set Message priority for a message on a queue

    Posted Tue November 05, 2024 06:54 AM

    Hi Vishal,

    I think you're now successfully getting the messages to the target queue with the correct priorities, so that's good.

    The behaviour of the getting and processing threads and applications will influence how successfully MQ message priority delivery can help you avoid timeout for your high priority requests.

    One pattern would be to have a number of threads, each of which does a get for a message and completely processes that request before doing another get for the next message. This works well with MQ message priority, because each message stays on the queue until it is really being processed.

    A different pattern would be to have threads each of which does a get, schedules something else to processes the request and moves on to the next get without waiting for the request to be completed. This doesn't automatically work well with MQ message priority, because lots of low priority requests can be scheduled very quickly but not completed when new higher priority requests arrive. It can work with MQ message priority if the scheduling can also be done with differing priorities.

    You also need to consider how the various threads are started and if you start up more threads when there is more work. If you have been using trigger-every to start a new thread to process each message, this would probably not be suitable for a large number of low priority requests arriving at the same time. If you are monitoring queue depth and starting more threads if it is deep, you might also need to consider how many messages are low priority.



    ------------------------------
    Rich Harran
    ------------------------------

    Original Message


  • 18.  RE: How to set Message priority for a message on a queue

    Posted Sat November 09, 2024 07:46 PM

    Thank you all for your valuable inputs.

    For @Francois Brandelik suggestion, "The alternative of course is to create 2 queues, one for authorizations and one for processing."
     
    That was my first suggestion to take off that to a different queue, however Application team says that it is an CICS.BRIDGE and the program on mainframe is written such a way only to get messages from that queue and then sort by some value on the message. 
    @bruce2359 for the questions: How many concurrent consumer applications are servicing the z/OS queue?  If each message is a transaction, then look at the possibility of launching multiple consumer applications.  Mainframes are provisioned to support hugely concurrent workloads.
    Ans: For that particular messages to take off from the queue there are 20 threads running. However, even we increase that thread count that was still not helping us with having timeouts for the authorization transactions.
    Final Update: Due to time crunch they opted for different method, to directly put the messages on to the db2 tables from IIB. Had to create an DSN and IIb dev team decided to send directly to the Db2 tables. 

    Thank you all for your valuable suggestions.



    ------------------------------
    vishal kumar kalava
    ------------------------------

    Original Message