Hi,

We changed from our IBM datapower implementations to a Mulesoft API management platform. Both platforms are JM/JEE based.
On IBM Datapower JMS is implemented by IBM and we need only configure some params like for example whether to use transactional jobs or not. We have done this for a couple years and runs like a charm.

When we have to configure transactional processes in Mulesoft we have to configure also "things".  This question is not about Mulesoft but about JMS implementations on JEE based apps in general.

The above information is just a side note.

Say we have this implementation in Java:
We get a message from a queue ( TEST1.DAT) via a listener program or a simple get from a queue and in the same process we do a http POST and after the post comes back with a http 200 or 201 then the MQ commit must be done. Off course we can do this in the code but we don't want this it must be happening via transactional. To make it a little bit more complicated we want to use a backout queue (TEST1.ERR) and after 3 attempts to reach the http POST end address the message must be put on the backout queue. 
The queues are created on the queuemanager and the TEST1.DAT queue has configured the  TEST1.ERR as backout queue and also the number of backout attempts is configured on the queuemanager.


Question 1:
What is the best practice to solve this in Java JMS, maybe anybody has an example how to do this ?
Maybe this is done by a configuration on the queue manager, and not in the application ? Or maybe we do have to make a choice whether or not solve this on the queuemanager or in the application ?

Question 2:
When this process runs we should like to see in debugging modus what is happening on the queue manager and with the message itself.
Is it possible to debug this anyway,  for example how does the queue manger knows how many times the message is redirected to the backout queue ?  Is this done for example with MQ message properties ?

Hope to get some answers
Thanks





------------------------------
Bernard Pittens
Integration Engeneer
Sligro Foodgroup B.V.
Veghel
------------------------------

JMS is like JDBC - it is a specification that needs to be implemented by the providers of Messaging or Database Systems.
It is "only" an API that Java Developers use if they need something from Messaging or a Database..
Weather the JMS implementation of IBM, MuleSoft or Oracle works better or worse is outside my knowledge.

"Java Transactions" are usually coming from something called XA or JTS.
https://docs.oracle.com/javaee/1.4/api/javax/jms/XAQueueConnectionFactory.html
Again IBM, Oracle, MuleSoft and co. are implementing that in Java.

Weather implicit or explicit the Java Application has to tell the "API" weather to begin, commit or rollback a transaction, message or DB statement.
The "queue manager" cannot know what the use case is.
Often the qmgr rolls back a message if the client started something without committing it.
If your Java is doing it, you know that it gets done.
If your Java does not do it, you depend on some middleware/drivers/providers.
Then from release to release one should test if that code still works to your satisfaction.

JMS best practice
https://docs.oracle.com/cd/A97688_16/generic.903/bp/j2ee.htm#1015154

Uncomitted messages are visible in IBM MQ
https://www.ibm.com/support/pages/how-do-you-tell-if-there-are-uncommitted-messages-ibm-mq-queue


------------------------------
Matthias Jungbauer
------------------------------

Original Message:
Sent: Fri October 02, 2020 02:28 AM
From: Bernard Pittens
Subject: Use transactional processes in Java JMS with IBM MQ.

Hi,

We changed from our IBM datapower implementations to a Mulesoft API management platform. Both platforms are JM/JEE based.
On IBM Datapower JMS is implemented by IBM and we need only configure some params like for example whether to use transactional jobs or not. We have done this for a couple years and runs like a charm.

When we have to configure transactional processes in Mulesoft we have to configure also "things".  This question is not about Mulesoft but about JMS implementations on JEE based apps in general.

The above information is just a side note.

Say we have this implementation in Java:
We get a message from a queue ( TEST1.DAT) via a listener program or a simple get from a queue and in the same process we do a http POST and after the post comes back with a http 200 or 201 then the MQ commit must be done. Off course we can do this in the code but we don't want this it must be happening via transactional. To make it a little bit more complicated we want to use a backout queue (TEST1.ERR) and after 3 attempts to reach the http POST end address the message must be put on the backout queue. 
The queues are created on the queuemanager and the TEST1.DAT queue has configured the  TEST1.ERR as backout queue and also the number of backout attempts is configured on the queuemanager.


Question 1:
What is the best practice to solve this in Java JMS, maybe anybody has an example how to do this ?
Maybe this is done by a configuration on the queue manager, and not in the application ? Or maybe we do have to make a choice whether or not solve this on the queuemanager or in the application ?

Question 2:
When this process runs we should like to see in debugging modus what is happening on the queue manager and with the message itself.
Is it possible to debug this anyway,  for example how does the queue manger knows how many times the message is redirected to the backout queue ?  Is this done for example with MQ message properties ?

Hope to get some answers
Thanks





------------------------------
Bernard Pittens
Integration Engeneer
Sligro Foodgroup B.V.
Veghel
------------------------------

Hi @Bernard Pittens​,

It sounds like your application is running in a JEE container which is managing the transaction i.e., you can't call commit / rollback directly. For IBM MQ / JMS the transaction will complete when the listener / MDB code exits cleanly. Should your listener code execution result in an exception, then the container will rollback the transaction and message. In this case the JMS Messaging Property JMSXDeliveryCount will be incremented. This Stackoverflow question provides a Spring Boot example for checking JMSXDeliveryCount. In the case where the http POST result is not 200/201, you would throw SomeKindOfException designed to cause the container to rollback the message.

Should JMSXDeliveryCount exceed your threshold of 3, the you can queue the message programatically to TEST1.ERR or configure the Queue Manager to do this based on the BOTRESH setting as described here. 

Once rolled back the message will be redelivered immediately, so if there is temporary outage in the http endpoint you are using, it might be that you hit your max retry value quite quickly. One strategy might be to check JMSXDeliveryCount and in the case where this greater than zero, but less than 3, introduce a wait time.


------------------------------
RICHARD COPPEN
------------------------------

Original Message:
Sent: Fri October 02, 2020 02:28 AM
From: Bernard Pittens
Subject: Use transactional processes in Java JMS with IBM MQ.

Hi,

We changed from our IBM datapower implementations to a Mulesoft API management platform. Both platforms are JM/JEE based.
On IBM Datapower JMS is implemented by IBM and we need only configure some params like for example whether to use transactional jobs or not. We have done this for a couple years and runs like a charm.

When we have to configure transactional processes in Mulesoft we have to configure also "things".  This question is not about Mulesoft but about JMS implementations on JEE based apps in general.

The above information is just a side note.

Say we have this implementation in Java:
We get a message from a queue ( TEST1.DAT) via a listener program or a simple get from a queue and in the same process we do a http POST and after the post comes back with a http 200 or 201 then the MQ commit must be done. Off course we can do this in the code but we don't want this it must be happening via transactional. To make it a little bit more complicated we want to use a backout queue (TEST1.ERR) and after 3 attempts to reach the http POST end address the message must be put on the backout queue. 
The queues are created on the queuemanager and the TEST1.DAT queue has configured the  TEST1.ERR as backout queue and also the number of backout attempts is configured on the queuemanager.


Question 1:
What is the best practice to solve this in Java JMS, maybe anybody has an example how to do this ?
Maybe this is done by a configuration on the queue manager, and not in the application ? Or maybe we do have to make a choice whether or not solve this on the queuemanager or in the application ?

Question 2:
When this process runs we should like to see in debugging modus what is happening on the queue manager and with the message itself.
Is it possible to debug this anyway,  for example how does the queue manger knows how many times the message is redirected to the backout queue ?  Is this done for example with MQ message properties ?

Hope to get some answers
Thanks





------------------------------
Bernard Pittens
Integration Engeneer
Sligro Foodgroup B.V.
Veghel
------------------------------

Hi,
Thanks @RICHARD COPPEN  and @Matthias Jungbauer   for your responses.
Kind regards
Bernard
​​

------------------------------
Bernard Pittens
Integration Engeneer
Sligro Foodgroup B.V.
Veghel
------------------------------

Original Message:
Sent: Tue October 06, 2020 06:11 AM
From: RICHARD COPPEN
Subject: Use transactional processes in Java JMS with IBM MQ.

Hi @Bernard Pittens​,

It sounds like your application is running in a JEE container which is managing the transaction i.e., you can't call commit / rollback directly. For IBM MQ / JMS the transaction will complete when the listener / MDB code exits cleanly. Should your listener code execution result in an exception, then the container will rollback the transaction and message. In this case the JMS Messaging Property JMSXDeliveryCount will be incremented. This Stackoverflow question provides a Spring Boot example for checking JMSXDeliveryCount. In the case where the http POST result is not 200/201, you would throw SomeKindOfException designed to cause the container to rollback the message.

Should JMSXDeliveryCount exceed your threshold of 3, the you can queue the message programatically to TEST1.ERR or configure the Queue Manager to do this based on the BOTRESH setting as described here. 

Once rolled back the message will be redivided immediately, so if there is temporary outage in the http endpoint you are using, it might be that you hit your max retry value quite quickly. One strategy might be to check JMSXDeliveryCount and in the case where this greater than zero, but less than 3, introduce a wait time.


------------------------------
RICHARD COPPEN

Original Message:
Sent: Fri October 02, 2020 02:28 AM
From: Bernard Pittens
Subject: Use transactional processes in Java JMS with IBM MQ.

Hi,

We changed from our IBM datapower implementations to a Mulesoft API management platform. Both platforms are JM/JEE based.
On IBM Datapower JMS is implemented by IBM and we need only configure some params like for example whether to use transactional jobs or not. We have done this for a couple years and runs like a charm.

When we have to configure transactional processes in Mulesoft we have to configure also "things".  This question is not about Mulesoft but about JMS implementations on JEE based apps in general.

The above information is just a side note.

Say we have this implementation in Java:
We get a message from a queue ( TEST1.DAT) via a listener program or a simple get from a queue and in the same process we do a http POST and after the post comes back with a http 200 or 201 then the MQ commit must be done. Off course we can do this in the code but we don't want this it must be happening via transactional. To make it a little bit more complicated we want to use a backout queue (TEST1.ERR) and after 3 attempts to reach the http POST end address the message must be put on the backout queue. 
The queues are created on the queuemanager and the TEST1.DAT queue has configured the  TEST1.ERR as backout queue and also the number of backout attempts is configured on the queuemanager.


Question 1:
What is the best practice to solve this in Java JMS, maybe anybody has an example how to do this ?
Maybe this is done by a configuration on the queue manager, and not in the application ? Or maybe we do have to make a choice whether or not solve this on the queuemanager or in the application ?

Question 2:
When this process runs we should like to see in debugging modus what is happening on the queue manager and with the message itself.
Is it possible to debug this anyway,  for example how does the queue manger knows how many times the message is redirected to the backout queue ?  Is this done for example with MQ message properties ?

Hope to get some answers
Thanks





------------------------------
Bernard Pittens
Integration Engeneer
Sligro Foodgroup B.V.
Veghel
------------------------------