MQ

 View Only
  • 1.  Administrative subscription not working

    IBM Champion
    Posted Fri April 02, 2021 03:23 PM
    We're trying to set up an administrative subscription on one of our MQ servers.   We have a publisher client that posts messages to a MQ topic and other subscribers listening to the topic.  Internally, on the queue manager we have defined an administrative subscription that listens for messages and copies these to a queue.  Another client is connected to this queue to receive the messages.

    The server is built into a Docker image so it runs in our private cloud.  I am able to build and configure this server locally (running in Docker on my laptop) so that the subscription works as expected.  By this I mean I can publish a message and see a subscriber client display the content as well as another client that is connected to the queue.  However, in the cloud deployment this does not work.  I see the subscriber print the message but not the client that is connected to the queue.

    We have verified that we can directly read/write to the topic and directly read/write to the queue.  The problem seems to be somewhere inside the queue manager in that it does not automatically put messages published to the topic onto the queue.

    The configuration is as follows, where "prospect-messages" is the topicstring.  We use LDAP security, thus the AUTHRECs.  In this particular case the ID we're using is a member of both LDAP groups.

    DEFINE TOPIC('PRI.RECRUITING.V1') TOPICSTR('prospect-messages') REPLACE

    SET AUTHREC GROUP('mq_prospect_messages_recv') OBJTYPE(QMGR) AUTHADD(CONNECT,INQ)
    SET AUTHREC GROUP('mq_prospect_messages_sndr') OBJTYPE(QMGR) AUTHADD(CONNECT,INQ)
    SET AUTHREC PROFILE('PRI.RECRUITING.V1') GROUP('mq_prospect_messages_sndr') OBJTYPE(TOPIC) AUTHADD(PUB)
    SET AUTHREC PROFILE('PRI.RECRUITING.V1') GROUP('mq_prospect_messages_recv') OBJTYPE(TOPIC) AUTHADD(SUB)

    DEFINE QLOCAL('PROSPECT.QUEUE.V1') REPLACE

    SET AUTHREC GROUP('mq_prospect_messages_recv') OBJTYPE(QMGR) AUTHADD(CONNECT,INQ)
    SET AUTHREC GROUP('mq_prospect_messages_sndr') OBJTYPE(QMGR) AUTHADD(CONNECT,INQ)
    SET AUTHREC PROFILE('PROSPECT.QUEUE.V1') GROUP('mq_prospect_messages_sndr') OBJTYPE(QUEUE) AUTHADD(PUT)
    SET AUTHREC PROFILE('PROSPECT.QUEUE.V1') GROUP('mq_prospect_messages_recv') OBJTYPE(QUEUE) AUTHADD(GET,INQ,BROWSE)

    DEFINE SUB('PRI.RECRUITING.V1') TOPICSTR('prospect-messages') DESTCLAS(PROVIDED) DEST('PROSPECT.QUEUE.V1') EXPIRY(3000) PUBPRTY(ASPUB)

    I'm fairly new to MQ and still learning.  Any thoughts or help in debugging this is much appreciated.  Is there some condition that would cause the queue to become disabled or disconnected from its subscription in this scenario?  The MQ version is 9.1.14 running on Ubuntu.  We are using a single queue manager.  

    Does any of the following affect an administrative subscription?
    • DEFINE SUB completes before the DEFINE TOPIC completes and thus no subscription occurs.
    • A SUBUSER parameter is required (This is not used in the local scenario that is working).

    Thanks,
    Jim

    ------------------------------
    Jim Creasman
    ------------------------------


  • 2.  RE: Administrative subscription not working

    IBM Champion
    Posted Sun April 04, 2021 07:40 PM

    When you have a subscription using a provided queue instead of a managed queue, the subscribing user ID must have PUT authority to the queue (because it is causing messages to be put to the queue). You do not have PUT authority on the PROSPECT.QUEUE.V1 queue for your mq_prospect_messages_recv group. However, this won't be your problem just now because you said that the user ID is in both groups and the mq_prospect_messages_sndr group has PUT authority for some reason I don't understand. Perhaps sometimes it puts directly to the subscribers queue in a scenario you haven't described to us?

    Doing a DEFINE SUB before the DEFINE TOPIC may cause a different behaviour for security reasons. You are making the subscription using the TOPICSTR rather than the TOPICOBJ, but the queue manager will look for any defined topic object at that point in the topic tree to see if there are authority records defined at that point. If it is not yet defined then the security might be done based on the SYSTEM.BASE.TOPIC instead. So, I would advise continue using the script where the TOPIC object and it's authority records are put in place first.

    You say that SUBUSER is required - what user ID did you use in that parameter, since clearly that will make a difference. Is it possible that the SUBUSER is not in both groups as you earlier stated? If so, my first paragraph does become important.

    I would also be curious whether any of the commands failed? And I would be curious whether there are any error messages in the queue manager AMQERR01.LOG? And I would be curious whether the putting application saw and non-zero reason codes?

    Cheers,
    Morag



    ------------------------------
    Morag Hughson
    MQ Technical Education Specialist
    MQGem Software Limited
    Website: https://www.mqgem.com
    ------------------------------



  • 3.  RE: Administrative subscription not working

    IBM Champion
    Posted Mon April 05, 2021 09:50 AM
    Morag,

         Thanks for the reply.  When you say "the subscribing user ID must have PUT authority to the queue," are you referring to the client that is connected to the queue?  That is my assumption since a subscriber to the topic does not know (or care) about the queue.  If I understand you correctly you are saying that any client wishing to perform MGET on queue PROSPECT.QUEUE.V1 will also require PUT authority on this queue.  This is because the queue manager will be putting messages to this queue on behalf of the client.  Is that correct?   Does this also mean that the user connecting to such a queue also needs SUB authority to the original source topic?

         Is there an advantage of using TOPICOBJ over TOPICSTR when defining the administrative subscription?  The documentation indicated I could use either or both and it would be resolved.  

         I need to clarify the part about SUBUSER.  This is not required by MQ from what I can tell.  My question was whether I had misread the doc and in fact it should be used in my case.  I'm not currently adding the SUBUSER parameter, but perhaps it is needed to remove some ambiguity?  Would adding this parameter help?  If so, which user would go there?  Our MQ configuration uses LDAP groups and the only user known to the queue manager is the LDAP bind ID that is used to connect with the LDAP server.

         The configuration runs in the order I have listed above and there are no errors at configuration.  I've monitored the MQ server log (/var/mqm/qmgrs/QC1/errors/AMQERR01.LOG) and do not see any errors.  I also do not see any errors coming from the clients.  All the commands seem to work, only the MQGET from the client connected to the queue never sees the message.  It continues to wait until the timeout occurs (currently set at 60 seconds).  Are there some other logs or trace settings I can use for debugging?

    Thanks,
    Jim


    ------------------------------
    Jim Creasman
    ------------------------------



  • 4.  RE: Administrative subscription not working

    IBM Champion
    Posted Mon April 05, 2021 08:14 PM
    It was brought to my attention that I may be misinterpreting the meaning of the EXPIRY parameter on the DEFINE SUB command.  I took this value to be the amount of time a message would persist on the queue before it was removed.  Perhaps instead this time limit applies to the subscription, itself.  

    I set this value to 3000 (5 minutes).  The documentation reads:

    EXPIRY
    The time to expiry of the subscription object from the creation date and time.

    (integer)
    The time to expiry, in tenths of a second, from the creation date and time.

    UNLIMITED
    There is no expiry time. This is the default option supplied with the product.

    Does that mean the administrative subscription goes away after 5 minutes in my case?  Clearly the queue is still there as the client connects to this without issue.  I will do some further testing tomorrow, but would appreciate clarification on the correct interpretation of this vallue.

    Thanks,
    Jim

    ------------------------------
    Jim Creasman
    ------------------------------



  • 5.  RE: Administrative subscription not working

    IBM Champion
    Posted Mon April 05, 2021 08:46 PM
    I can confirm EXPIRY on DEFINE SUB is when the subscription will be removed.

    ------------------------------
    Morag Hughson
    MQ Technical Education Specialist
    MQGem Software Limited
    Website: https://www.mqgem.com
    ------------------------------



  • 6.  RE: Administrative subscription not working

    IBM Champion
    Posted Tue April 06, 2021 06:31 PM
    The use of EXPIRY on the DEFINE SUB was definitely a misinterpretation of the documentation on my part.  Knowing this applied to the subscription itself (and not the messages) allowed me to recreate the problem locally and then verify it was resolved by the removal of the EXPIRY parameter (or rather by setting it to UNLIMITED).

    However, we are still seeing the same behavior on our MQ server deployed in the private cloud.  After restarting the server the administrative subscription works for a time and then messages cease to show up on the queue.  The commands used in defining the queue and subscription now look like this:
     : * Queue definitions:
    17 : DEFINE QLOCAL('PLS.FRAMEWORK.V1') CUSTOM('CAPEXPRY(3000)') REPLACE
    AMQ8006I: IBM MQ queue created.
    18 : DEFINE QLOCAL('PROSPECT.QUEUE.V1') CUSTOM('CAPEXPRY(3000)') REPLACE
    AMQ8006I: IBM MQ queue created.
    : *
    : * Queue AUTHREC statements:
    19 : SET AUTHREC GROUP('mq_pls_messages_recv') OBJTYPE(QMGR) AUTHADD(CONNECT,INQ)
    AMQ8862I: IBM MQ authority record set.
    20 : SET AUTHREC GROUP('mq_pls_messages_sndr') OBJTYPE(QMGR) AUTHADD(CONNECT,INQ)
    AMQ8862I: IBM MQ authority record set.
    21 : SET AUTHREC GROUP('mq_prospect_messages_recv') OBJTYPE(QMGR) AUTHADD(CONNECT,INQ)
    AMQ8862I: IBM MQ authority record set.
    22 : SET AUTHREC GROUP('mq_prospect_messages_sndr') OBJTYPE(QMGR) AUTHADD(CONNECT,INQ)
    AMQ8862I: IBM MQ authority record set.
    23 : SET AUTHREC PROFILE('PROSPECT.QUEUE.V1') GROUP('mq_prospect_messages_sndr') OBJTYPE(QUEUE) AUTHADD(PUT)
    AMQ8862I: IBM MQ authority record set.
    24 : SET AUTHREC PROFILE('PLS.FRAMEWORK.V1') GROUP('mq_pls_messages_sndr') OBJTYPE(QUEUE) AUTHADD(PUT)
    AMQ8862I: IBM MQ authority record set.
    25 : SET AUTHREC PROFILE('PROSPECT.QUEUE.V1') GROUP('mq_prospect_messages_recv') OBJTYPE(QUEUE) AUTHADD(GET,INQ,BROWSE)
    AMQ8862I: IBM MQ authority record set.
    26 : SET AUTHREC PROFILE('PLS.FRAMEWORK.V1') GROUP('mq_pls_messages_recv') OBJTYPE(QUEUE) AUTHADD(GET,INQ,BROWSE)
    AMQ8862I: IBM MQ authority record set.
    : *
    : * Administrative subscriptions (if any):
    27 : DEFINE SUB('PRI.RECRUITING.V1') TOPICSTR('prospect-messages') DESTCLAS(PROVIDED) DEST('PROSPECT.QUEUE.V1') EXPIRY(UNLIMITED) PUBPRTY(ASPUB)
    AMQ8094I: IBM MQ subscription created.
    28 : DEFINE SUB('PRI.PLS.V1') TOPICSTR('pls-cases') DESTCLAS(PROVIDED) DEST('PLS.FRAMEWORK.V1') EXPIRY(UNLIMITED) PUBPRTY(ASPUB)
    AMQ8094I: IBM MQ subscription created.


    Note we did add an expiration on the queues using "CUSTOM('CAPEXPRY(3000)')".  My understanding is that does cause messages to expire (and not the queue).



    ------------------------------
    Jim Creasman
    ------------------------------



  • 7.  RE: Administrative subscription not working

    IBM Champion
    Posted Tue April 06, 2021 09:59 PM

    Issue the following commands after you have set things up, and then again later when it stops working and see if there is any difference.

    DISPLAY SUB(PRI.*) ALL

    DISPLAY SBSTATUS(PRI.*) ALL

    How long is "for a time"? Do you have an approximation of the time period at least?

    P.S. Yes, your understand of CAPEXPRY is correct.

    Cheers,
    Morag



    ------------------------------
    Morag Hughson
    MQ Technical Education Specialist
    MQGem Software Limited
    Website: https://www.mqgem.com
    ------------------------------



  • 8.  RE: Administrative subscription not working

    IBM Champion
    Posted Thu April 08, 2021 11:01 AM

    Just want to close the loop on this and say that the issue is resolved.  The original problem was that I had misinterpreted the meaning of the EXPIRY parameter on the administrative subscription.   Correcting this actually fixed the problem.  However, the issue appeared to still be there as we were testing this change.  We would publish a test message, but not see it on the queue.

    After a day or so of debugging the culprit turned out to be a client at a 3rd party that was connected to the same queue.  This client was polling every few milliseconds and getting the messages from the queue before our test client could get them.  We did occasionally see a message which only made the problem more bizarre.  Once we discovered this and had them shut it down everything started working as expected.

    I really appreciate the help and follow up from Morag and Josh.   You guys rock!

    Thanks,

    Jim



    ------------------------------
    Jim Creasman
    ------------------------------



  • 9.  RE: Administrative subscription not working

    IBM Champion
    Posted Fri April 09, 2021 01:21 AM

    Glad to hear you're all sorted Jim.

    Cheers,
    Morag



    ------------------------------
    Morag Hughson
    MQ Technical Education Specialist
    MQGem Software Limited
    Website: https://www.mqgem.com
    ------------------------------