MQ

 View Only
Expand all | Collapse all

Trigger Monitor task for Z/os

  • 1.  Trigger Monitor task for Z/os

    Posted Tue February 13, 2024 10:33 AM

    In our infrastructure we are using batch trigger monitor for batch jobs which is configured based on MA95 support pac.

    Recently we are getting lot of CPU usage issues for trigger monitor.

    what is the best solution and recommended for MQ trigger monitor task in Z/OS?



    ------------------------------
    Lakshmi Narasimha Rao Veguru
    ------------------------------


  • 2.  RE: Trigger Monitor task for Z/os

    Posted Wed February 14, 2024 11:57 AM

    yes, we have the same issues. With the trigger monitor task as well can anyone suggest on this 



    ------------------------------
    Radha krishna Reddy Gundreddy
    ------------------------------



  • 3.  RE: Trigger Monitor task for Z/os

    IBM Champion
    Posted Thu February 15, 2024 03:20 AM
    A couple of thoughts/questions

    • How much increase - 10% 10 times?
    • Has anything changed, such as new version of MQ,  or new application?
    • The CPU increase could be caused by increase in number of trigger messages- can you check this ( does your trigger monitor display the number of trigger messages it processed, or do you get this  info from the stats)
    • Do you trigger first, trigger every?
      • If you use trigger every and the triggered application is slow to start, you will get more trigger messages which will lead to an increase in CPU.
    Colin






  • 4.  RE: Trigger Monitor task for Z/os

    Posted Fri February 16, 2024 02:25 AM

    Hello Colin,

    Thank you so much for your reply and happy for this.

    The current CPU increased 30% and no changes from infra side.

    MQ is running with 9.3 version.

    We have verified the number of jobs per month and it's around 282297 jobs through trigger monitor.

    The trigger interval defined at queue managers in 1 min and many of the queues are having trigger type as Every.

    The current trigger monitor task using MA95.

    //Lakshmi



    ------------------------------
    Lakshmi Narasimha Rao Veguru
    ------------------------------



  • 5.  RE: Trigger Monitor task for Z/os

    Posted Fri February 16, 2024 08:25 AM
    Edited by bruce2359 Fri February 16, 2024 08:37 AM

    282297 jobs per month = 9431 per day = 392 per hour = 6.6 per minute. 

    How long (in minutes) does an individual job take to complete?

    Has the total number of jobs increased from last month?  The month before?  The month before that?

    Is the exact same job being triggered 292927 times?  Or different jobs?  Are these jobs written in COBOL?  JAVA?  JMS? 

    What tool are you using to see the 30% cpu metric?  

    Are any of these jobs missing established SLAs (Service Level Agreements)? 

    Please post the JCL that is submitted by the trigger monitor?


    ------------------------------
    bruce2359
    Consultant/trainer
    CTTECH - Computer & Telecommunications Technology
    West Coast, almost
    000-000-0000
    ------------------------------



  • 6.  RE: Trigger Monitor task for Z/os

    Posted Thu February 15, 2024 10:03 AM

    First questions - what has changed?  Typically these tasks are lightweight because they do not process the messages from the queue, but only the trigger message to get the task started.  Is the reason for the rise in CPU because it is triggered much more often for some reason?   Was the trigger interval on the queue manager altered recently? Have there been an upgrade to hardware or software recently? 
    Second question: What trigger are the queues using, first, every or depth? 
    Third question: How are you measuring the CPU use?

    Fourth question: Is there a poison message getting onto the INTQ or the triggered queue and getting rolled back a lot?  Browse the initq to see if there is something that cannot be processed.  Do the same for the 'base' queue.  

    Fifth question: Is the triggered queue defined to harden the backout counter?  If not, please turn that on. 
    Hope this helps



    ------------------------------
    Lyn Elkins
    IBM
    Avon, IN
    elkinsc@us.ibm.com
    ------------------------------



  • 7.  RE: Trigger Monitor task for Z/os

    Posted Fri February 16, 2024 02:26 AM

    Hello Elkins,

    Thank you so much for your reply.

    The current CPU increased 30% and no changes from infra side.

    MQ is running with 9.3 version.

    We have verified the number of jobs per month and it's around 282297 jobs through trigger monitor.

    The trigger interval defined at queue managers in 1 min and many of the queues are having trigger type as Every.

    The current trigger monitor task using MA95.

    //Lakshmi



    ------------------------------
    Lakshmi Narasimha Rao Veguru
    ------------------------------



  • 8.  RE: Trigger Monitor task for Z/os

    Posted Fri February 16, 2024 05:10 PM

    "The current CPU increased by 30%" - well then something changed. 
    Where is this CPU number coming from?  Typically we only see jumps of that much under a few scenarios:
    1) The most common  by a very large margin- when customer starts letting poorly behaved client applications connect directly to z/OS queue managers or queue sharing groups and the CPU usage in the channel initiator address space begins to skyrocket. If all MQ processes are getting lumped into one CPU number, split them out and see if it is actually clients connecting.  Review the CHIN JES log for client connect and disconnect messages if they have not been suppressed.  Turn on class(4) accounting to see what channels are active.

    2) When monitoring tools have been changed or turned on and the measurements are no longer being based on the same fields.  Often the allocation of system overhead can be skewed towards some processes, and it may change when the tool is changed. And yes, we have seen incidents of monitoring tools 'taking over' when lots of people are watching and refreshing their view.   In one outstanding case we found that 94% of the tasks executed during the critical window were monitoring tasks.

    3) The application has made significant changes that do not require MQ Administration changes.  
    4) The infrastructure changed in an area that everyone assumed would not impact the z/OS queue managers, but did.  This can include things like hardware moves or replacements (DASD, Coupling Facility, etc.).  We have seen some truly startling increases in CPU when a CF was moved or when someone turned on system managed duplexing - things that the MQ admins were not aware of.  
    5) Has the workload pattern changed?  If it was distributed fairly evenly across the day and is now being sent in batches, the costs of starting up 30 instances in 1 second can be higher than starting 1 instance every second for 30 seconds.  

    6) Has the CPU usage been 'creeping up' over time and has just now been noticed?  

    good luck



    ------------------------------
    Lyn Elkins
    IBM
    Avon, IN
    elkinsc@us.ibm.com
    ------------------------------



  • 9.  RE: Trigger Monitor task for Z/os

    Posted Thu February 15, 2024 09:05 AM
    Edited by bruce2359 Thu February 15, 2024 12:13 PM

    I suspect that the OP and the "me, too" post are AI or spam.  No technical details one would expect from an MVS sysprog.  SupportPac MA95 – A REXX interface to Websphere MQ is long out of support. 



    ------------------------------
    bruce2359
    Consultant/trainer
    CTTECH - Computer & Telecommunications Technology
    West Coast, almost
    000-000-0000
    ------------------------------



  • 10.  RE: Trigger Monitor task for Z/os

    Posted Fri February 16, 2024 02:28 AM

    Hello Bruce,

    The current CPU increased 30% and no changes from infra side.

    MQ is running with 9.3 version.

    We have verified the number of jobs per month and it's around 282297 jobs through trigger monitor.

    The trigger interval defined at queue managers in 1 min and many of the queues are having trigger type as Every.

    The current trigger monitor task using MA95 and it's out of support.

    Please suggest another option in place of MA95.

    //Lakshmi



    ------------------------------
    Lakshmi Narasimha Rao Veguru
    ------------------------------