MQ

  • 1.  MQ subscriptions with multiple client instances

    Posted Mon September 20, 2021 03:22 PM
    I'm trying to understand how to deal with the scenario where I have a subscriber to a topic, and the subscriber is running in a service that may exist as multiple instances (e.g., as in the case of being scaled up for handling extra load).  Normally, each subscriber receives it's own copy of a message.  In this case if I have say 3 instances of the service running I don't want each one to get their own copy.  Instead, I would like to distribute the subscription across the set of instances, similar to how HTTP requests are handled by a load balancer.  

    One way I can see to do this is to define an administrative subscription on the queue manager that receives messages published to the topic, and then puts these on a queue (not topic).  Then, change the clients to connect to the queue.  Each instance of the client would get a message, thus preventing the same message from being handled by another instance.  I'm wondering if there is a better way to do this that keeps with the original pub/sub model.

    From what I've been able to read I believe this is referred to as a "shared durable subscription".  This approach relies on the subscriber ID and client ID of each instance being the same.  Does this require JMS?  I found this approach mentioned in documentation for WAS ND (not MQ, itself).

    With cloud deployment and automatic scaling by Kubernetes it seems this would be a common scenario to cover.   I'm looking for the best pattern(s) for handling client scaling when you have multiple instances of the same client subscribed to a topic -- and the desire is to NOT have each client instance process the same message.

    Thanks,
    Jim

    ------------------------------
    Jim Creasman
    ------------------------------


  • 2.  RE: MQ subscriptions with multiple client instances

    Posted Tue September 21, 2021 06:55 PM
    Hi Jim,

    I don't use programmatic subscriptions a great deal, so I may have missed something, but I can't see a way to implement what you need with the MQSUB call. Your proposed solution of an administrative subscription putting messages onto a queue, and then having multiple applications instances share access to the queue seems to be the right approach.

    Regards,


    ------------------------------
    Neil Casey
    Senior Consultant
    Syntegrity Solutions
    Melbourne, Victoria
    IBM Champion (Cloud) 2019-21
    ------------------------------



  • 3.  RE: MQ subscriptions with multiple client instances

    Posted Thu September 30, 2021 08:43 AM
    With MQ, your only options for sharing a single application-defined subscription between multiple application instances are JMS 2.0 or AMQP. As Neil says, your approach using an admin sub and having applications consume from queues is a perfectly valid alternative if you're not using one of those APIs.

    ------------------------------
    Matthew Whitehead
    ------------------------------



  • 4.  RE: MQ subscriptions with multiple client instances

    Posted Fri October 01, 2021 08:35 AM
    Edited by Francois Brandelik Fri October 01, 2021 08:35 AM
    If you are using a uniform cluster, you could also make an administrative subscription with a cluster alias. That will load balance the subscription in the cluster. You then need enough consumers to consume from each queue. This of course means that there is no message affinity.

    ------------------------------
    Francois Brandelik
    ------------------------------