MQ

I need some assistance.

Our application, particularly the Put operation, experiences intermittent delays. Initially, when the applications are launched, both Put and Get response times range from 1 to 3 seconds, which is normal.

However, after running for 2 or 3 days, there's a noticeable delay upon startup, with responses taking nearly a minute. We're only utilizing a single svrconn channel with SHARECNV(10).

Any suggestions on which areas to investigate? The developers are beginning to attribute the issue to the MQ Server configuration.

------------------------------
Hla Tun
------------------------------

I need some assistance.

Our application, particularly the Put operation, experiences intermittent delays. Initially, when the applications are launched, both Put and Get response times range from 1 to 3 seconds, which is normal.

However, after running for 2 or 3 days, there's a noticeable delay upon startup, with responses taking nearly a minute. We're only utilizing a single svrconn channel with SHARECNV(10).

Any suggestions on which areas to investigate? The developers are beginning to attribute the issue to the MQ Server configuration.

------------------------------
Hla Tun
------------------------------

Hello Hia,

Could you say a little about what "startup" means in this context. You say there is a "noticeable delay upon startup". Is this the application program starting up, or just making a new connection to the queue manager but the program was already running, or are you referring to the queue manager starting up, or something else?

Cheers 

Morag

I need some assistance.

Our application, particularly the Put operation, experiences intermittent delays. Initially, when the applications are launched, both Put and Get response times range from 1 to 3 seconds, which is normal.

However, after running for 2 or 3 days, there's a noticeable delay upon startup, with responses taking nearly a minute. We're only utilizing a single svrconn channel with SHARECNV(10).

Any suggestions on which areas to investigate? The developers are beginning to attribute the issue to the MQ Server configuration.

Hello Morag,

Sorry about the confusion. What I meant is that putting a message into the specific queue is taking longer after the application has been running for 2 or 3 days, just like the developer mentioned. I've already checked the MQ log and monitored hardware resource usage, but nothing abnormal was found. Given that our application scenario is time-sensitive and messages must be put in on time, this delay is concerning. The developer sent the log to me, they have investigated  the time it takes to put a message into the specific queue. I attached it for your check.

When the put application start connected the queue manager and specific queue working well in a day (1 Day)

When the application has been running for 2 or 3 days, you can noticed significant delay as below

Hello Hia,

Could you say a little about what "startup" means in this context. You say there is a "noticeable delay upon startup". Is this the application program starting up, or just making a new connection to the queue manager but the program was already running, or are you referring to the queue manager starting up, or something else?

Cheers 

Morag

I need some assistance.

Our application, particularly the Put operation, experiences intermittent delays. Initially, when the applications are launched, both Put and Get response times range from 1 to 3 seconds, which is normal.

However, after running for 2 or 3 days, there's a noticeable delay upon startup, with responses taking nearly a minute. We're only utilizing a single svrconn channel with SHARECNV(10).

Any suggestions on which areas to investigate? The developers are beginning to attribute the issue to the MQ Server configuration.

Hello Hla,

Thank you for the additional information. This measurement that you have collected, is this JUST the MQPUT call or MQOPEN+MQPUT+MQCLOSE or MQCONN+MQOPEN+MQPUT+MQCLOSE+MQDISC? Does it also include the time waiting for a response to the initial message or is this a send and forget style application?

You mention that the application stays running for a number of days, does it stay connected to the queue manager all the time, or does it only make a connection when it needs to? Does it stay connected to put a number of messages, or does it only connect for the duration of one message?

As you can imagine, answers to the above questions make a big different to response times.

Cheers,
Morag

Hello Morag,

Sorry about the confusion. What I meant is that putting a message into the specific queue is taking longer after the application has been running for 2 or 3 days, just like the developer mentioned. I've already checked the MQ log and monitored hardware resource usage, but nothing abnormal was found. Given that our application scenario is time-sensitive and messages must be put in on time, this delay is concerning. The developer sent the log to me, they have investigated  the time it takes to put a message into the specific queue. I attached it for your check.

When the put application start connected the queue manager and specific queue working well in a day (1 Day)

When the application has been running for 2 or 3 days, you can noticed significant delay as below

Hello Hia,

Could you say a little about what "startup" means in this context. You say there is a "noticeable delay upon startup". Is this the application program starting up, or just making a new connection to the queue manager but the program was already running, or are you referring to the queue manager starting up, or something else?

Cheers 

Morag

I need some assistance.

Our application, particularly the Put operation, experiences intermittent delays. Initially, when the applications are launched, both Put and Get response times range from 1 to 3 seconds, which is normal.

However, after running for 2 or 3 days, there's a noticeable delay upon startup, with responses taking nearly a minute. We're only utilizing a single svrconn channel with SHARECNV(10).

Any suggestions on which areas to investigate? The developers are beginning to attribute the issue to the MQ Server configuration.

Hello Morag,

Sorry about the confusion. What I meant is that putting a message into the specific queue is taking longer after the application has been running for 2 or 3 days, just like the developer mentioned. I've already checked the MQ log and monitored hardware resource usage, but nothing abnormal was found. Given that our application scenario is time-sensitive and messages must be put in on time, this delay is concerning. The developer sent the log to me, they have investigated  the time it takes to put a message into the specific queue. I attached it for your check.

When the put application start connected the queue manager and specific queue working well in a day (1 Day)

When the application has been running for 2 or 3 days, you can noticed significant delay as below

Hello Hia,

Could you say a little about what "startup" means in this context. You say there is a "noticeable delay upon startup". Is this the application program starting up, or just making a new connection to the queue manager but the program was already running, or are you referring to the queue manager starting up, or something else?

Cheers 

Morag

I need some assistance.

Our application, particularly the Put operation, experiences intermittent delays. Initially, when the applications are launched, both Put and Get response times range from 1 to 3 seconds, which is normal.

However, after running for 2 or 3 days, there's a noticeable delay upon startup, with responses taking nearly a minute. We're only utilizing a single svrconn channel with SHARECNV(10).

Any suggestions on which areas to investigate? The developers are beginning to attribute the issue to the MQ Server configuration.

Hello HIa,

While the application is running can you please provide: from the samples either the amqsrua or amqsruac output for

amqsrua -m qmgr -c STATMQI -t CONNDISC

amqsrua -m qmgr -c STATQ -t GET -o qname

amqsrua -m qmgr -c STATQ -t PUT -o qname

amqsrua -m qmgr -c STATQ -t OPENCLOSE -o qname

amqsrua -m qmgr -c STATQ -t GENERAL -o qname

run these at the beginning of the test (good response time) and at the end of the test (bad response times)

Can you also verify that the GET calls are transacted  ?

You can also run a statapp call with application name. That too should give us some insight

append -n x if you only want x results, x should have a minimum of 4 to get some good results at 10 second interval...

Hope it helps 

Hello Morag,

Sorry about the confusion. What I meant is that putting a message into the specific queue is taking longer after the application has been running for 2 or 3 days, just like the developer mentioned. I've already checked the MQ log and monitored hardware resource usage, but nothing abnormal was found. Given that our application scenario is time-sensitive and messages must be put in on time, this delay is concerning. The developer sent the log to me, they have investigated  the time it takes to put a message into the specific queue. I attached it for your check.

When the put application start connected the queue manager and specific queue working well in a day (1 Day)

When the application has been running for 2 or 3 days, you can noticed significant delay as below

Hello Hia,

Could you say a little about what "startup" means in this context. You say there is a "noticeable delay upon startup". Is this the application program starting up, or just making a new connection to the queue manager but the program was already running, or are you referring to the queue manager starting up, or something else?

Cheers 

Morag

I need some assistance.

Our application, particularly the Put operation, experiences intermittent delays. Initially, when the applications are launched, both Put and Get response times range from 1 to 3 seconds, which is normal.

However, after running for 2 or 3 days, there's a noticeable delay upon startup, with responses taking nearly a minute. We're only utilizing a single svrconn channel with SHARECNV(10).

Any suggestions on which areas to investigate? The developers are beginning to attribute the issue to the MQ Server configuration.

Hello Francois,

Thank you for your attention. Bad response time occurs randomly and it's only through reviewing the application log and response records that we've identified the timing issue. I have attached your requested output to review. Kindly give some favor for hiding the queue name.
Thanks.

Hello HIa,

While the application is running can you please provide: from the samples either the amqsrua or amqsruac output for

amqsrua -m qmgr -c STATMQI -t CONNDISC

amqsrua -m qmgr -c STATQ -t GET -o qname

amqsrua -m qmgr -c STATQ -t PUT -o qname

amqsrua -m qmgr -c STATQ -t OPENCLOSE -o qname

amqsrua -m qmgr -c STATQ -t GENERAL -o qname

run these at the beginning of the test (good response time) and at the end of the test (bad response times)

Can you also verify that the GET calls are transacted  ?

You can also run a statapp call with application name. That too should give us some insight

append -n x if you only want x results, x should have a minimum of 4 to get some good results at 10 second interval...

Hope it helps 

Hello Morag,

Sorry about the confusion. What I meant is that putting a message into the specific queue is taking longer after the application has been running for 2 or 3 days, just like the developer mentioned. I've already checked the MQ log and monitored hardware resource usage, but nothing abnormal was found. Given that our application scenario is time-sensitive and messages must be put in on time, this delay is concerning. The developer sent the log to me, they have investigated  the time it takes to put a message into the specific queue. I attached it for your check.

When the put application start connected the queue manager and specific queue working well in a day (1 Day)

When the application has been running for 2 or 3 days, you can noticed significant delay as below

Hello Hia,

Could you say a little about what "startup" means in this context. You say there is a "noticeable delay upon startup". Is this the application program starting up, or just making a new connection to the queue manager but the program was already running, or are you referring to the queue manager starting up, or something else?

Cheers 

Morag

I need some assistance.

Our application, particularly the Put operation, experiences intermittent delays. Initially, when the applications are launched, both Put and Get response times range from 1 to 3 seconds, which is normal.

However, after running for 2 or 3 days, there's a noticeable delay upon startup, with responses taking nearly a minute. We're only utilizing a single svrconn channel with SHARECNV(10).

Any suggestions on which areas to investigate? The developers are beginning to attribute the issue to the MQ Server configuration.

Hi Hla,

Just be aware that the MQConnect is the most expensive call that you can make.

It would behoove the application to cache the connection and reuse it. However, if broken or bad the application will need to recycle the connection and acquire a new one.

This should lead to marked improvement for the throughput / timing.

On the other hand the degradation over such a long period of time makes me suspect you may have a resource problem.
Verify the application's use of resources, and most specific memory and paging. A slow memory leak could explain the behavior.

Hope it helps

Hello Francois,

Thank you for your attention. Bad response time occurs randomly and it's only through reviewing the application log and response records that we've identified the timing issue. I have attached your requested output to review. Kindly give some favor for hiding the queue name.
Thanks.

Hello HIa,

While the application is running can you please provide: from the samples either the amqsrua or amqsruac output for

amqsrua -m qmgr -c STATMQI -t CONNDISC

amqsrua -m qmgr -c STATQ -t GET -o qname

amqsrua -m qmgr -c STATQ -t PUT -o qname

amqsrua -m qmgr -c STATQ -t OPENCLOSE -o qname

amqsrua -m qmgr -c STATQ -t GENERAL -o qname

run these at the beginning of the test (good response time) and at the end of the test (bad response times)

Can you also verify that the GET calls are transacted  ?

You can also run a statapp call with application name. That too should give us some insight

append -n x if you only want x results, x should have a minimum of 4 to get some good results at 10 second interval...

Hope it helps 

Hello Morag,

Sorry about the confusion. What I meant is that putting a message into the specific queue is taking longer after the application has been running for 2 or 3 days, just like the developer mentioned. I've already checked the MQ log and monitored hardware resource usage, but nothing abnormal was found. Given that our application scenario is time-sensitive and messages must be put in on time, this delay is concerning. The developer sent the log to me, they have investigated  the time it takes to put a message into the specific queue. I attached it for your check.

When the put application start connected the queue manager and specific queue working well in a day (1 Day)

When the application has been running for 2 or 3 days, you can noticed significant delay as below

Hello Hia,

Could you say a little about what "startup" means in this context. You say there is a "noticeable delay upon startup". Is this the application program starting up, or just making a new connection to the queue manager but the program was already running, or are you referring to the queue manager starting up, or something else?

Cheers 

Morag

I need some assistance.

Our application, particularly the Put operation, experiences intermittent delays. Initially, when the applications are launched, both Put and Get response times range from 1 to 3 seconds, which is normal.

However, after running for 2 or 3 days, there's a noticeable delay upon startup, with responses taking nearly a minute. We're only utilizing a single svrconn channel with SHARECNV(10).

Any suggestions on which areas to investigate? The developers are beginning to attribute the issue to the MQ Server configuration.

Hello Francois,

Thank you.

I would like to ask, could you provide some ratings for the overall performance of the previously mentioned query report.

I'm finding it a bit difficult to understand.

Hi Hla,

Just be aware that the MQConnect is the most expensive call that you can make.

It would behoove the application to cache the connection and reuse it. However, if broken or bad the application will need to recycle the connection and acquire a new one.

This should lead to marked improvement for the throughput / timing.

On the other hand the degradation over such a long period of time makes me suspect you may have a resource problem.
Verify the application's use of resources, and most specific memory and paging. A slow memory leak could explain the behavior.

Hope it helps

Hello Francois,

Thank you for your attention. Bad response time occurs randomly and it's only through reviewing the application log and response records that we've identified the timing issue. I have attached your requested output to review. Kindly give some favor for hiding the queue name.
Thanks.

Hello HIa,

While the application is running can you please provide: from the samples either the amqsrua or amqsruac output for

amqsrua -m qmgr -c STATMQI -t CONNDISC

amqsrua -m qmgr -c STATQ -t GET -o qname

amqsrua -m qmgr -c STATQ -t PUT -o qname

amqsrua -m qmgr -c STATQ -t OPENCLOSE -o qname

amqsrua -m qmgr -c STATQ -t GENERAL -o qname

run these at the beginning of the test (good response time) and at the end of the test (bad response times)

Can you also verify that the GET calls are transacted  ?

You can also run a statapp call with application name. That too should give us some insight

append -n x if you only want x results, x should have a minimum of 4 to get some good results at 10 second interval...

Hope it helps 

Hello Morag,

Sorry about the confusion. What I meant is that putting a message into the specific queue is taking longer after the application has been running for 2 or 3 days, just like the developer mentioned. I've already checked the MQ log and monitored hardware resource usage, but nothing abnormal was found. Given that our application scenario is time-sensitive and messages must be put in on time, this delay is concerning. The developer sent the log to me, they have investigated  the time it takes to put a message into the specific queue. I attached it for your check.

When the put application start connected the queue manager and specific queue working well in a day (1 Day)

When the application has been running for 2 or 3 days, you can noticed significant delay as below

Hello Hia,

Could you say a little about what "startup" means in this context. You say there is a "noticeable delay upon startup". Is this the application program starting up, or just making a new connection to the queue manager but the program was already running, or are you referring to the queue manager starting up, or something else?

Cheers 

Morag

I need some assistance.

Our application, particularly the Put operation, experiences intermittent delays. Initially, when the applications are launched, both Put and Get response times range from 1 to 3 seconds, which is normal.

However, after running for 2 or 3 days, there's a noticeable delay upon startup, with responses taking nearly a minute. We're only utilizing a single svrconn channel with SHARECNV(10).

Any suggestions on which areas to investigate? The developers are beginning to attribute the issue to the MQ Server configuration.

Hi Hlan,

As the reporting interval is 10 seconds, you learn much more from an application that does 10 calls per second, than from an application that does 1 call a minute. In order to have valid data your time interval needs to span at least 3 API calls from the app.

Hope this helps

Hello Francois,

Thank you.

I would like to ask, could you provide some ratings for the overall performance of the previously mentioned query report.

I'm finding it a bit difficult to understand.

Hi Hla,

Just be aware that the MQConnect is the most expensive call that you can make.

It would behoove the application to cache the connection and reuse it. However, if broken or bad the application will need to recycle the connection and acquire a new one.

This should lead to marked improvement for the throughput / timing.

On the other hand the degradation over such a long period of time makes me suspect you may have a resource problem.
Verify the application's use of resources, and most specific memory and paging. A slow memory leak could explain the behavior.

Hope it helps

Hello Francois,

Thank you for your attention. Bad response time occurs randomly and it's only through reviewing the application log and response records that we've identified the timing issue. I have attached your requested output to review. Kindly give some favor for hiding the queue name.
Thanks.

Hello HIa,

While the application is running can you please provide: from the samples either the amqsrua or amqsruac output for

amqsrua -m qmgr -c STATMQI -t CONNDISC

amqsrua -m qmgr -c STATQ -t GET -o qname

amqsrua -m qmgr -c STATQ -t PUT -o qname

amqsrua -m qmgr -c STATQ -t OPENCLOSE -o qname

amqsrua -m qmgr -c STATQ -t GENERAL -o qname

run these at the beginning of the test (good response time) and at the end of the test (bad response times)

Can you also verify that the GET calls are transacted  ?

You can also run a statapp call with application name. That too should give us some insight

append -n x if you only want x results, x should have a minimum of 4 to get some good results at 10 second interval...

Hope it helps 

Hello Morag,

Sorry about the confusion. What I meant is that putting a message into the specific queue is taking longer after the application has been running for 2 or 3 days, just like the developer mentioned. I've already checked the MQ log and monitored hardware resource usage, but nothing abnormal was found. Given that our application scenario is time-sensitive and messages must be put in on time, this delay is concerning. The developer sent the log to me, they have investigated  the time it takes to put a message into the specific queue. I attached it for your check.

When the put application start connected the queue manager and specific queue working well in a day (1 Day)

When the application has been running for 2 or 3 days, you can noticed significant delay as below

Hello Hia,

Could you say a little about what "startup" means in this context. You say there is a "noticeable delay upon startup". Is this the application program starting up, or just making a new connection to the queue manager but the program was already running, or are you referring to the queue manager starting up, or something else?

Cheers 

Morag

I need some assistance.

Our application, particularly the Put operation, experiences intermittent delays. Initially, when the applications are launched, both Put and Get response times range from 1 to 3 seconds, which is normal.

However, after running for 2 or 3 days, there's a noticeable delay upon startup, with responses taking nearly a minute. We're only utilizing a single svrconn channel with SHARECNV(10).

Any suggestions on which areas to investigate? The developers are beginning to attribute the issue to the MQ Server configuration.

You state 

Typical Messages In/out per hour : 600 msgs
Message Size : 1MB to 4MB

however the stats you've provided show 

QUEUE_NAME                                     MQPUT/MQPUT1 count 131584
QUEUE_NAME                                     MQPUT byte count 545119562 213/sec

suggesting an average message size of a little over 4KB on this queue.

At the time you ran the amqsrua commands there doesn't seem to have been any put/get activity, but around 55 opens/closes per second. 55/second ought to be a trivial workload but it would be sensible to try and understand the apparent imbalance in these numbers.

The commands you were requested to run were serially, however it would have been a bit more interesting if they'd been concurrent, as it is there might have been no activity when you ran the put/get commands but some activity when you ran the open/close commands.

amqsrua isn't really the ideal tool for understanding where latency is occurring, the only latency numbers I recall amqsrua providing are the latency when writing to the recovery log (which might be worth checking, but it's a long shot).

Even 1->3 seconds is a relative eternity for an MQPUT or an MQGET, yet alone a minute. Others have already mentioned the inefficiency of doing a loop of MQCONN; MQOPEN;MQPUT;MQCLOSE;MQDISC (or even MQCONN;MQPUT1;MQDISC), rather than reusing a longer lived connection. 

That being said, it ought to be feasible for IBM L2/L3 to see a delay of this magnitude in an MQ trace. Have you engaged with IBM support on this topic ?

Hi Hlan,

As the reporting interval is 10 seconds, you learn much more from an application that does 10 calls per second, than from an application that does 1 call a minute. In order to have valid data your time interval needs to span at least 3 API calls from the app.

Hope this helps

Hello Francois,

Thank you.

I would like to ask, could you provide some ratings for the overall performance of the previously mentioned query report.

I'm finding it a bit difficult to understand.

Hi Hla,

Just be aware that the MQConnect is the most expensive call that you can make.

It would behoove the application to cache the connection and reuse it. However, if broken or bad the application will need to recycle the connection and acquire a new one.

This should lead to marked improvement for the throughput / timing.

On the other hand the degradation over such a long period of time makes me suspect you may have a resource problem.
Verify the application's use of resources, and most specific memory and paging. A slow memory leak could explain the behavior.

Hope it helps

Hello Francois,

Thank you for your attention. Bad response time occurs randomly and it's only through reviewing the application log and response records that we've identified the timing issue. I have attached your requested output to review. Kindly give some favor for hiding the queue name.
Thanks.