MQ

 View Only
Expand all | Collapse all

Default value for the msgid field on z/OS

  • 1.  Default value for the msgid field on z/OS

    IBM Champion
    Posted Wed November 29, 2023 02:06 PM


    On Unix/Windows platforms, the msgid generated by the Queue Manager consists of a hex string starting with "AMQ", followed by the name of the Queue Manager, and a timestamp.
    Apparently it's a bit different on MQ z/OS, but I don't have a mainframe handy to test.
    Can anyone give me this value?
    Thank you.
    (this is just to check if the msgid received from a z/OS was generated by the Queue Manager or by the application)



    ------------------------------
    Luc-Michel Demey
    DEMEY CONSULTING
    lmd@demey-consulting.fr
    #IBMChampion
    ------------------------------


  • 2.  RE: Default value for the msgid field on z/OS

    IBM Champion
    Posted Wed November 29, 2023 03:50 PM

    Here's an example:

    MsgId : X'C3E2D840C3E2D8F84040404040404040CEFF7FC1C46A1E9B'

    It's 'CSQ ' followed by qmgr name and STCK. In this example, qmgr name is CSQ8.

    See https://www.ibm.com/docs/en/ibm-mq/9.3?topic=descriptor-msgid-mqbyte24-mqmd#fntarg_1

    Cheers,

    Morag



    ------------------------------
    Morag Hughson
    MQ Technical Education Specialist
    MQGem Software Limited
    Website: https://www.mqgem.com
    ------------------------------



  • 3.  RE: Default value for the msgid field on z/OS

    IBM Champion
    Posted Thu November 30, 2023 10:00 AM

    Thanks Morag, I'd missed the page in the KC.



    ------------------------------
    Luc-Michel Demey
    DEMEY CONSULTING
    lmd@demey-consulting.fr
    #IBMChampion
    ------------------------------



  • 4.  RE: Default value for the msgid field on z/OS

    Posted Thu November 30, 2023 04:27 AM

    Just a very minor clarification, the final 8 bytes of an MQ generated MsgId/CorrelId/GroupId on the distributed platforms are not actually a timestamp.

    Timestamps on distributed are not sufficiently unique.

    On the distributed platforms the last 8 bytes are initially based on the timestamp when the queue manager started, the queue manager then allocates ranges of timestamps to different hCon's based on that timestamp and once all ranges are exhausted then a new timestamp is taken. Hence on the distributed platforms you can't deduce that a MsgId with a higher value in the last 8 bytes was put before a message with a lower value, unless it was put by the same hCon (and even that wouldn't be guaranteed in future releases). 



    ------------------------------
    Andrew Hickson
    ------------------------------



  • 5.  RE: Default value for the msgid field on z/OS

    IBM Champion
    Posted Thu November 30, 2023 10:03 AM

    Um... Very interesting!
    I don't remember reading about this in the documentation, but I note with interest this information.



    ------------------------------
    Luc-Michel Demey
    DEMEY CONSULTING
    lmd@demey-consulting.fr
    #IBMChampion
    ------------------------------



  • 6.  RE: Default value for the msgid field on z/OS

    Posted Thu November 30, 2023 11:38 AM

    This is worth noting, too.

    On the distributed side, it looks like Unix epoch time is being used for the msg id timestamp piece. Unix epoch time ends in 2038 and will wrap. Once it wraps, you probably still have the range of 1/1/1970 until the time in the 1990s when IBM MQ became in use on distributed to have unique Unix epoch time values for the product lifetime of IBM MQ distributed. However, at some point in the 2060s you will have the potential of duplicate values for Unix epoch time for message ids. The range of the 4 byte Unix epoch time is about 68 years.



    ------------------------------
    Tim Zielke
    ------------------------------



  • 7.  RE: Default value for the msgid field on z/OS

    Posted Wed December 06, 2023 08:39 PM

    https://en.wikipedia.org/wiki/Year_2038_problem

    I will be long retired by then!  It will be interesting to see the IT industry approach, and similarities to the Y2K bug (I did significant work for that - to fix 2 digit year calculation issues in various apps).



    ------------------------------
    Glenn Baddeley
    Senior Middleware Software Engineer
    Coles Supermarkets Australia Pty Ltd
    ------------------------------



  • 8.  RE: Default value for the msgid field on z/OS

    Posted Thu December 07, 2023 08:34 AM
    Edited by Tim Zielke Thu December 07, 2023 08:34 AM

    Yes, it does feel like Y2K. I also don't know all the internals of how IBM MQ generates unique message ids and if they already have some code in place to handle things like 32 bit time wrapping. It is worth noting though that the JMS specification expects things like long running queue managers to generate unique message ids for their existence, at least how I read it.

    From the Java Message Service 2.0 specification:

    "A JMSMessageID is a String value which should function as a unique key for identifying messages in a historical repository. The exact scope of uniqueness is provider defined. It should at least cover all messages for a specific installation of a provider where an installation is some connected set of message routers."



    ------------------------------
    Tim Zielke
    ------------------------------



  • 9.  RE: Default value for the msgid field on z/OS

    Posted Thu December 07, 2023 09:32 AM

    The key term describing MsgId is "should" - "... should function as a unique key" or "... should be unique."  MsgId is not (intended to be) user data - this is different than Y2K where a complete (explicit) date requires "19".  In a request-reply model, how long must (should) the unique MsgId live? 



    ------------------------------
    bruce2359
    Consultant/trainer
    CTTECH - Computer & Telecommunications Technology
    West Coast, almost
    000-000-0000
    ------------------------------



  • 10.  RE: Default value for the msgid field on z/OS

    Posted Fri December 08, 2023 10:53 AM
    Edited by Tim Zielke Fri December 08, 2023 11:23 AM

    The IBM MQ manual does make comments about the MsgId being unique.

    https://www.ibm.com/docs/en/ibm-mq/9.2?topic=mqmd-msgid-mqbyte24

    "For the MQPUT and MQPUT1 calls, if MQMI_NONE or MQPMO_NEW_MSG_ID is specified by the application, the queue manager generates a unique message identifier 1 when the message is put, and places it in the message descriptor sent with the message."

    "1 A MsgId generated by the queue manager consists of a 4-byte product identifier (AMQ¬ or CSQ¬ in either ASCII or EBCDIC, where ¬ represents a blank character), followed by a product-specific implementation of a unique string. In IBM® MQ this contains the first 12 characters of the queue manager name, and a value derived from the system clock. All queue managers that can intercommunicate must therefore have names that differ in the first 12 characters, in order to ensure that message identifiers are unique. The ability to generate a unique string also depends on the system clock not being changed backward. To eliminate the possibility of a message identifier generated by the queue manager duplicating one generated by the application, the application must avoid generating identifiers with initial characters in the range A through I in ASCII or EBCDIC (X'41' through X'49' and X'C1' through X'C9'). However, the application is not prevented from generating identifiers with initial characters in these ranges."

    There is this qualification:

    "This is a byte string that is used to distinguish one message from another. Generally, no two messages should have the same message identifier, although this is not disallowed by the queue manager."

    But this qualification seems to be implying that the intention is for a MsgId to be unique, and the queue manager has a tolerance for when it is not unique.

    I don't see anything in this document addressing what happens when a queue manager runs longer than the unique time value allotted in a 31 bit integer. Maybe their code already accounts for this. Not sure. 



    ------------------------------
    Tim Zielke
    ------------------------------



  • 11.  RE: Default value for the msgid field on z/OS

    Posted Fri December 08, 2023 11:41 AM

    When requested by the application, the qmgr attempts to create a unique MsgId - only unique for this one qmgr, but not guaranteed to be unique across any other or all qmgrs, across any/all o/s's in the network, and across time.  Unique in this context means "not likely to be replicated,"  but not enforced by MQ internals.



    ------------------------------
    bruce2359
    Consultant/trainer
    CTTECH - Computer & Telecommunications Technology
    West Coast, almost
    000-000-0000
    ------------------------------



  • 12.  RE: Default value for the msgid field on z/OS

    Posted Fri December 08, 2023 02:33 PM

    As usual, my comments on QM internals relate to the distributed code base.

    As long as the first 12 characters of the QMName are kept unique then any MQ generated identifier (MsgId, CorrelId, GroupId) will be globaly unique.

    There is no requirement for a MsgId on a user generated message to be unique, but if you let MQ itself generate the MsgId's they will be unique (and also unique amongst all other MQ generated identifiers (e.g. CorrelId, GroupId)) (there IS a requirement that GroupId's are unique).

    On the distributed platforms this used to depend upon the system clock generating a monatonically increasing sequence, but this was eliminated in V9 by recording the highest used range of MsgId's in the recovery log. A combination of a cold start, and a backwards jump in the system clock could still hypothetically result in a non unique identifier. Cold start is not a supported procedure and the generation of a non unique identifier caused by a cold start overlapping a backwards movement of the system clock is a minor concern compared to the other possible implications of cold start. 

    Before the 32 bit value for seconds since 1970 wraps (still several years away), some internal code change will be required to ensure that no subsequent clashes occur. This could be as simple as allowing the set of identifiers from 1970 to the first ever MQ release to be used, but could equally be done with a different prefix ( currently "AMQ ").



    ------------------------------
    Andrew Hickson
    ------------------------------



  • 13.  RE: Default value for the msgid field on z/OS

    Posted Sat December 09, 2023 01:36 PM

    "(there IS a requirement that GroupId's are unique)."  More precisely ...  https://www.ibm.com/docs/en/ibm-mq/9.3?topic=descriptor-groupid-mqbyte24-mqmd

    This is a requirement IF message groups or message segmentation is involved. 

    If none of these flags is set, GroupId has the special null value MQGI_NONE.
    The application does not need to set this field on the MQPUT or MQGET call if:
    • On the MQPUT call, MQPMO_LOGICAL_ORDER is specified.
    • On the MQGET call, MQMO_MATCH_GROUP_ID is not specified.


    ------------------------------
    bruce2359
    Consultant/trainer
    CTTECH - Computer & Telecommunications Technology
    West Coast, almost
    000-000-0000
    ------------------------------