Original Message:
Sent: Fri December 08, 2023 02:32 PM
From: Andrew Hickson
Subject: Default value for the msgid field on z/OS
As usual, my comments on QM internals relate to the distributed code base.
As long as the first 12 characters of the QMName are kept unique then any MQ generated identifier (MsgId, CorrelId, GroupId) will be globaly unique.
There is no requirement for a MsgId on a user generated message to be unique, but if you let MQ itself generate the MsgId's they will be unique (and also unique amongst all other MQ generated identifiers (e.g. CorrelId, GroupId)) (there IS a requirement that GroupId's are unique).
On the distributed platforms this used to depend upon the system clock generating a monatonically increasing sequence, but this was eliminated in V9 by recording the highest used range of MsgId's in the recovery log. A combination of a cold start, and a backwards jump in the system clock could still hypothetically result in a non unique identifier. Cold start is not a supported procedure and the generation of a non unique identifier caused by a cold start overlapping a backwards movement of the system clock is a minor concern compared to the other possible implications of cold start.
Before the 32 bit value for seconds since 1970 wraps (still several years away), some internal code change will be required to ensure that no subsequent clashes occur. This could be as simple as allowing the set of identifiers from 1970 to the first ever MQ release to be used, but could equally be done with a different prefix ( currently "AMQ ").
------------------------------
Andrew Hickson
Original Message:
Sent: Fri December 08, 2023 11:40 AM
From: bruce2359
Subject: Default value for the msgid field on z/OS
When requested by the application, the qmgr attempts to create a unique MsgId - only unique for this one qmgr, but not guaranteed to be unique across any other or all qmgrs, across any/all o/s's in the network, and across time. Unique in this context means "not likely to be replicated," but not enforced by MQ internals.
------------------------------
bruce2359
Consultant/trainer
CTTECH - Computer & Telecommunications Technology
West Coast, almost
000-000-0000
Original Message:
Sent: Fri December 08, 2023 10:53 AM
From: Tim Zielke
Subject: Default value for the msgid field on z/OS
The IBM MQ manual does make comments about the MsgId being unique.
https://www.ibm.com/docs/en/ibm-mq/9.2?topic=mqmd-msgid-mqbyte24
"For the MQPUT and MQPUT1 calls, if MQMI_NONE or MQPMO_NEW_MSG_ID is specified by the application, the queue manager generates a unique message identifier 1 when the message is put, and places it in the message descriptor sent with the message."
"1 A MsgId generated by the queue manager consists of a 4-byte product identifier (AMQ¬ or CSQ¬ in either ASCII or EBCDIC, where ¬ represents a blank character), followed by a product-specific implementation of a unique string. In IBM® MQ this contains the first 12 characters of the queue manager name, and a value derived from the system clock. All queue managers that can intercommunicate must therefore have names that differ in the first 12 characters, in order to ensure that message identifiers are unique. The ability to generate a unique string also depends on the system clock not being changed backward. To eliminate the possibility of a message identifier generated by the queue manager duplicating one generated by the application, the application must avoid generating identifiers with initial characters in the range A through I in ASCII or EBCDIC (X'41' through X'49' and X'C1' through X'C9'). However, the application is not prevented from generating identifiers with initial characters in these ranges."
There is this qualification:
"This is a byte string that is used to distinguish one message from another. Generally, no two messages should have the same message identifier, although this is not disallowed by the queue manager."
But this qualification seems to be implying that the intention is for a MsgId to be unique, and the queue manager has a tolerance for when it is not unique.
I don't see anything in this document addressing what happens when a queue manager runs longer than the unique time value allotted in a 32 bit integer, or longer than 68 years. Maybe their code already accounts for this. Not sure.
------------------------------
Tim Zielke
Original Message:
Sent: Thu December 07, 2023 09:31 AM
From: bruce2359
Subject: Default value for the msgid field on z/OS
The key term describing MsgId is "should" - "... should function as a unique key" or "... should be unique." MsgId is not (intended to be) user data - this is different than Y2K where a complete (explicit) date requires "19". In a request-reply model, how long must (should) the unique MsgId live?
------------------------------
bruce2359
Consultant/trainer
CTTECH - Computer & Telecommunications Technology
West Coast, almost
000-000-0000
Original Message:
Sent: Thu December 07, 2023 08:33 AM
From: Tim Zielke
Subject: Default value for the msgid field on z/OS
Yes, it does feel like Y2K. I also don't know all the internals of how IBM MQ generates unique message ids and if they already have some code in place to handle things like 32 bit time wrapping. It is worth noting though that the JMS specification expects things like long running queue managers to generate unique message ids for their existence, at least how I read it.
From the Java Message Service 2.0 specification:
"A JMSMessageID is a String value which should function as a unique key for identifying messages in a historical repository. The exact scope of uniqueness is provider defined. It should at least cover all messages for a specific installation of a provider where an installation is some connected set of message routers."
------------------------------
Tim Zielke
Original Message:
Sent: Wed December 06, 2023 08:39 PM
From: Glenn Baddeley
Subject: Default value for the msgid field on z/OS
https://en.wikipedia.org/wiki/Year_2038_problem
I will be long retired by then! It will be interesting to see the IT industry approach, and similarities to the Y2K bug (I did significant work for that - to fix 2 digit year calculation issues in various apps).
------------------------------
Glenn Baddeley
Senior Middleware Software Engineer
Coles Supermarkets Australia Pty Ltd
Original Message:
Sent: Thu November 30, 2023 11:38 AM
From: Tim Zielke
Subject: Default value for the msgid field on z/OS
This is worth noting, too.
On the distributed side, it looks like Unix epoch time is being used for the msg id timestamp piece. Unix epoch time ends in 2038 and will wrap. Once it wraps, you probably still have the range of 1/1/1970 until the time in the 1990s when IBM MQ became in use on distributed to have unique Unix epoch time values for the product lifetime of IBM MQ distributed. However, at some point in the 2060s you will have the potential of duplicate values for Unix epoch time for message ids. The range of the 4 byte Unix epoch time is about 68 years.
------------------------------
Tim Zielke
Original Message:
Sent: Thu November 30, 2023 10:02 AM
From: Luc-Michel Demey
Subject: Default value for the msgid field on z/OS
Um... Very interesting!
I don't remember reading about this in the documentation, but I note with interest this information.
------------------------------
Luc-Michel Demey
DEMEY CONSULTING
lmd@demey-consulting.fr
#IBMChampion
Original Message:
Sent: Thu November 30, 2023 04:27 AM
From: Andrew Hickson
Subject: Default value for the msgid field on z/OS
Just a very minor clarification, the final 8 bytes of an MQ generated MsgId/CorrelId/GroupId on the distributed platforms are not actually a timestamp.
Timestamps on distributed are not sufficiently unique.
On the distributed platforms the last 8 bytes are initially based on the timestamp when the queue manager started, the queue manager then allocates ranges of timestamps to different hCon's based on that timestamp and once all ranges are exhausted then a new timestamp is taken. Hence on the distributed platforms you can't deduce that a MsgId with a higher value in the last 8 bytes was put before a message with a lower value, unless it was put by the same hCon (and even that wouldn't be guaranteed in future releases).
------------------------------
Andrew Hickson
Original Message:
Sent: Wed November 29, 2023 02:05 PM
From: Luc-Michel Demey
Subject: Default value for the msgid field on z/OS
On Unix/Windows platforms, the msgid generated by the Queue Manager consists of a hex string starting with "AMQ", followed by the name of the Queue Manager, and a timestamp.
Apparently it's a bit different on MQ z/OS, but I don't have a mainframe handy to test.
Can anyone give me this value?
Thank you.
(this is just to check if the msgid received from a z/OS was generated by the Queue Manager or by the application)
------------------------------
Luc-Michel Demey
DEMEY CONSULTING
lmd@demey-consulting.fr
#IBMChampion
------------------------------