1. No-Syncpoint is a (get + commit)
2. Syncpoint means that you need an explicit commit for the message to removed from the system.   You typically do an MQget , DB2 update commit.  The doc says CICS® applications must use the EXEC CICS SYNCPOINT command to commit the unit of work explicitly. Alternatively, ending the transaction results in an implicit commit of the unit of work.  So if the transaction fails to issue exec CICS syncpoint, it will be done for you

• do a CICS ENQ RESOURCE on BET1, BET2, etc. until we find a BETx transaction that is not running. START that one, then end
• If all 8 BETx transactions are already running, end

In short, BETA STARTs the first BETx clone that is not already executing. Wait Y seconds, then end (BETA waits a few seconds for the STARTed transaction to GET the message from the queue).

The BETx clone does an MQGET from the queue, processes it, then continues processing the queue until it is empty.

And here is my visual representation:

1. No-Syncpoint is a (get + commit)
2. Syncpoint means that you need an explicit commit for the message to removed from the system.   You typically do an MQget , DB2 update commit.  The doc says CICS® applications must use the EXEC CICS SYNCPOINT command to commit the unit of work explicitly. Alternatively, ending the transaction results in an implicit commit of the unit of work.  So if the transaction fails to issue exec CICS syncpoint, it will be done for you

Hi Curt,

Thanks for your update.

The doc for triggering says If the application queue is open for input, by any program, and has triggering set for FIRST or DEPTH, no trigger event occurs because the queue is already being served. 

So if no queue has the queue open for input then the first message will cause a trigger message to be produced - and so start a transaction.
If there is at least one application with the queue open for input, then there will not be a trigger message produced.

I would expect from what you said that only one transaction (BET1) runs.  If BET1 closes the queue, and there is a message on the queue (due to bad timing) it will cause another trigger message, and so may possibly start BET2.

Despite this, non of this triggering will cause a message to be processed more than once.

_________________________________________________

"MQ delivers the message twice"...

Consider the scenario

MQGET of message1
do something - for example write to TD queue (not in syncpoint) 'Ive processed message 1'
Rollback
Transaction ends

Triggering starts up a new transaction
transaction starts

MQGET of message1
do something - for example write to  CICS TD queue (not in syncpoint) 'Ive processed message 1'
commit
Transaction ends

So in your CICS TD queue you have 2 instances of 'Ive processed message 1', and you say "Ah Ha - MQ has delivered the message more than once".

If would be worth putting some code in your transactions to report if the application did a roll back.  This is good practice, as you should investigate why it has rolled back. (For example if a DB2 request timed out, you should investigate this)

From the MQ accounting data you will see if there is a rollback or commit. 

I cant remember if the CICS stats report the number of commits and backouts, it would be worth looking there.

Putting a few lines of code in your program  to say a rollback has occurred ( and why)  would be good practice, and be less disruptive to turning on trace or accounting.

Colin

• do a CICS ENQ RESOURCE on BET1, BET2, etc. until we find a BETx transaction that is not running. START that one, then end
• If all 8 BETx transactions are already running, end

In short, BETA STARTs the first BETx clone that is not already executing. Wait Y seconds, then end (BETA waits a few seconds for the STARTed transaction to GET the message from the queue).

The BETx clone does an MQGET from the queue, processes it, then continues processing the queue until it is empty.

And here is my visual representation:

1. No-Syncpoint is a (get + commit)
2. Syncpoint means that you need an explicit commit for the message to removed from the system.   You typically do an MQget , DB2 update commit.  The doc says CICS® applications must use the EXEC CICS SYNCPOINT command to commit the unit of work explicitly. Alternatively, ending the transaction results in an implicit commit of the unit of work.  So if the transaction fails to issue exec CICS syncpoint, it will be done for you

Hi Curt,

Thanks for your update.

The doc for triggering says If the application queue is open for input, by any program, and has triggering set for FIRST or DEPTH, no trigger event occurs because the queue is already being served. 

So if no queue has the queue open for input then the first message will cause a trigger message to be produced - and so start a transaction.
If there is at least one application with the queue open for input, then there will not be a trigger message produced.

I would expect from what you said that only one transaction (BET1) runs.  If BET1 closes the queue, and there is a message on the queue (due to bad timing) it will cause another trigger message, and so may possibly start BET2.

Despite this, non of this triggering will cause a message to be processed more than once.

_________________________________________________

"MQ delivers the message twice"...

Consider the scenario

MQGET of message1
do something - for example write to TD queue (not in syncpoint) 'Ive processed message 1'
Rollback
Transaction ends

Triggering starts up a new transaction
transaction starts

MQGET of message1
do something - for example write to  CICS TD queue (not in syncpoint) 'Ive processed message 1'
commit
Transaction ends

So in your CICS TD queue you have 2 instances of 'Ive processed message 1', and you say "Ah Ha - MQ has delivered the message more than once".

If would be worth putting some code in your transactions to report if the application did a roll back.  This is good practice, as you should investigate why it has rolled back. (For example if a DB2 request timed out, you should investigate this)

From the MQ accounting data you will see if there is a rollback or commit. 

I cant remember if the CICS stats report the number of commits and backouts, it would be worth looking there.

Putting a few lines of code in your program  to say a rollback has occurred ( and why)  would be good practice, and be less disruptive to turning on trace or accounting.

Colin

• do a CICS ENQ RESOURCE on BET1, BET2, etc. until we find a BETx transaction that is not running. START that one, then end
• If all 8 BETx transactions are already running, end

In short, BETA STARTs the first BETx clone that is not already executing. Wait Y seconds, then end (BETA waits a few seconds for the STARTed transaction to GET the message from the queue).

The BETx clone does an MQGET from the queue, processes it, then continues processing the queue until it is empty.

And here is my visual representation:

1. No-Syncpoint is a (get + commit)
2. Syncpoint means that you need an explicit commit for the message to removed from the system.   You typically do an MQget , DB2 update commit.  The doc says CICS® applications must use the EXEC CICS SYNCPOINT command to commit the unit of work explicitly. Alternatively, ending the transaction results in an implicit commit of the unit of work.  So if the transaction fails to issue exec CICS syncpoint, it will be done for you

Hi Curt,

Thanks for your update.

The doc for triggering says If the application queue is open for input, by any program, and has triggering set for FIRST or DEPTH, no trigger event occurs because the queue is already being served. 

So if no queue has the queue open for input then the first message will cause a trigger message to be produced - and so start a transaction.
If there is at least one application with the queue open for input, then there will not be a trigger message produced.

I would expect from what you said that only one transaction (BET1) runs.  If BET1 closes the queue, and there is a message on the queue (due to bad timing) it will cause another trigger message, and so may possibly start BET2.

Despite this, non of this triggering will cause a message to be processed more than once.

_________________________________________________

"MQ delivers the message twice"...

Consider the scenario

MQGET of message1
do something - for example write to TD queue (not in syncpoint) 'Ive processed message 1'
Rollback
Transaction ends

Triggering starts up a new transaction
transaction starts

MQGET of message1
do something - for example write to  CICS TD queue (not in syncpoint) 'Ive processed message 1'
commit
Transaction ends

So in your CICS TD queue you have 2 instances of 'Ive processed message 1', and you say "Ah Ha - MQ has delivered the message more than once".

If would be worth putting some code in your transactions to report if the application did a roll back.  This is good practice, as you should investigate why it has rolled back. (For example if a DB2 request timed out, you should investigate this)

From the MQ accounting data you will see if there is a rollback or commit. 

I cant remember if the CICS stats report the number of commits and backouts, it would be worth looking there.

Putting a few lines of code in your program  to say a rollback has occurred ( and why)  would be good practice, and be less disruptive to turning on trace or accounting.

Colin

• do a CICS ENQ RESOURCE on BET1, BET2, etc. until we find a BETx transaction that is not running. START that one, then end
• If all 8 BETx transactions are already running, end

In short, BETA STARTs the first BETx clone that is not already executing. Wait Y seconds, then end (BETA waits a few seconds for the STARTed transaction to GET the message from the queue).

The BETx clone does an MQGET from the queue, processes it, then continues processing the queue until it is empty.

And here is my visual representation:

1. No-Syncpoint is a (get + commit)
2. Syncpoint means that you need an explicit commit for the message to removed from the system.   You typically do an MQget , DB2 update commit.  The doc says CICS® applications must use the EXEC CICS SYNCPOINT command to commit the unit of work explicitly. Alternatively, ending the transaction results in an implicit commit of the unit of work.  So if the transaction fails to issue exec CICS syncpoint, it will be done for you

Hi Curt,

Thanks for your update.

The doc for triggering says If the application queue is open for input, by any program, and has triggering set for FIRST or DEPTH, no trigger event occurs because the queue is already being served. 

So if no queue has the queue open for input then the first message will cause a trigger message to be produced - and so start a transaction.
If there is at least one application with the queue open for input, then there will not be a trigger message produced.

I would expect from what you said that only one transaction (BET1) runs.  If BET1 closes the queue, and there is a message on the queue (due to bad timing) it will cause another trigger message, and so may possibly start BET2.

Despite this, non of this triggering will cause a message to be processed more than once.

_________________________________________________

"MQ delivers the message twice"...

Consider the scenario

MQGET of message1
do something - for example write to TD queue (not in syncpoint) 'Ive processed message 1'
Rollback
Transaction ends

Triggering starts up a new transaction
transaction starts

MQGET of message1
do something - for example write to  CICS TD queue (not in syncpoint) 'Ive processed message 1'
commit
Transaction ends

So in your CICS TD queue you have 2 instances of 'Ive processed message 1', and you say "Ah Ha - MQ has delivered the message more than once".

If would be worth putting some code in your transactions to report if the application did a roll back.  This is good practice, as you should investigate why it has rolled back. (For example if a DB2 request timed out, you should investigate this)

From the MQ accounting data you will see if there is a rollback or commit. 

I cant remember if the CICS stats report the number of commits and backouts, it would be worth looking there.

Putting a few lines of code in your program  to say a rollback has occurred ( and why)  would be good practice, and be less disruptive to turning on trace or accounting.

Colin

• do a CICS ENQ RESOURCE on BET1, BET2, etc. until we find a BETx transaction that is not running. START that one, then end
• If all 8 BETx transactions are already running, end

In short, BETA STARTs the first BETx clone that is not already executing. Wait Y seconds, then end (BETA waits a few seconds for the STARTed transaction to GET the message from the queue).

The BETx clone does an MQGET from the queue, processes it, then continues processing the queue until it is empty.

And here is my visual representation:

1. No-Syncpoint is a (get + commit)
2. Syncpoint means that you need an explicit commit for the message to removed from the system.   You typically do an MQget , DB2 update commit.  The doc says CICS® applications must use the EXEC CICS SYNCPOINT command to commit the unit of work explicitly. Alternatively, ending the transaction results in an implicit commit of the unit of work.  So if the transaction fails to issue exec CICS syncpoint, it will be done for you

Don't give up yet Curt.  With something like this, the devil is in the detail, but we can find those details.

Thank you for sharing the DISPLAY statements you found with us. I am sure they will help. Could you tell us what field the "BETF-255736737" data came from? For example, was it the MQMD-MSGID field?

The contents of the data you have shown us does not look like a queue manager generated message ID value (those have the queue manager name in them with some well known prefix characters). Do you know who generates that data - the application that put the message for example? The fact that you use that value as a primary key to do a DB2 insert also suggests that it is not a queue manager generated message ID value, but something generated by an application that has more awareness of the "shape" of the data to be used as a key.

One hypothesis that might be worth following up, is, do you have two messages on this queue with the same "BETF-255736737" data in them? That would certainly fit the description of the problem you have provided so far.

Cheers,
Morag

Hi Curt,

Thanks for your update.

The doc for triggering says If the application queue is open for input, by any program, and has triggering set for FIRST or DEPTH, no trigger event occurs because the queue is already being served. 

So if no queue has the queue open for input then the first message will cause a trigger message to be produced - and so start a transaction.
If there is at least one application with the queue open for input, then there will not be a trigger message produced.

I would expect from what you said that only one transaction (BET1) runs.  If BET1 closes the queue, and there is a message on the queue (due to bad timing) it will cause another trigger message, and so may possibly start BET2.

Despite this, non of this triggering will cause a message to be processed more than once.

_________________________________________________

"MQ delivers the message twice"...

Consider the scenario

MQGET of message1
do something - for example write to TD queue (not in syncpoint) 'Ive processed message 1'
Rollback
Transaction ends

Triggering starts up a new transaction
transaction starts

MQGET of message1
do something - for example write to  CICS TD queue (not in syncpoint) 'Ive processed message 1'
commit
Transaction ends

So in your CICS TD queue you have 2 instances of 'Ive processed message 1', and you say "Ah Ha - MQ has delivered the message more than once".

If would be worth putting some code in your transactions to report if the application did a roll back.  This is good practice, as you should investigate why it has rolled back. (For example if a DB2 request timed out, you should investigate this)

From the MQ accounting data you will see if there is a rollback or commit. 

I cant remember if the CICS stats report the number of commits and backouts, it would be worth looking there.

Putting a few lines of code in your program  to say a rollback has occurred ( and why)  would be good practice, and be less disruptive to turning on trace or accounting.

Colin

• do a CICS ENQ RESOURCE on BET1, BET2, etc. until we find a BETx transaction that is not running. START that one, then end
• If all 8 BETx transactions are already running, end

In short, BETA STARTs the first BETx clone that is not already executing. Wait Y seconds, then end (BETA waits a few seconds for the STARTed transaction to GET the message from the queue).

The BETx clone does an MQGET from the queue, processes it, then continues processing the queue until it is empty.

And here is my visual representation:

1. No-Syncpoint is a (get + commit)
2. Syncpoint means that you need an explicit commit for the message to removed from the system.   You typically do an MQget , DB2 update commit.  The doc says CICS® applications must use the EXEC CICS SYNCPOINT command to commit the unit of work explicitly. Alternatively, ending the transaction results in an implicit commit of the unit of work.  So if the transaction fails to issue exec CICS syncpoint, it will be done for you

Don't give up yet Curt.  With something like this, the devil is in the detail, but we can find those details.

Thank you for sharing the DISPLAY statements you found with us. I am sure they will help. Could you tell us what field the "BETF-255736737" data came from? For example, was it the MQMD-MSGID field?

The contents of the data you have shown us does not look like a queue manager generated message ID value (those have the queue manager name in them with some well known prefix characters). Do you know who generates that data - the application that put the message for example? The fact that you use that value as a primary key to do a DB2 insert also suggests that it is not a queue manager generated message ID value, but something generated by an application that has more awareness of the "shape" of the data to be used as a key.

One hypothesis that might be worth following up, is, do you have two messages on this queue with the same "BETF-255736737" data in them? That would certainly fit the description of the problem you have provided so far.

Cheers,
Morag

Hi Curt,

Thanks for your update.

The doc for triggering says If the application queue is open for input, by any program, and has triggering set for FIRST or DEPTH, no trigger event occurs because the queue is already being served. 

So if no queue has the queue open for input then the first message will cause a trigger message to be produced - and so start a transaction.
If there is at least one application with the queue open for input, then there will not be a trigger message produced.

I would expect from what you said that only one transaction (BET1) runs.  If BET1 closes the queue, and there is a message on the queue (due to bad timing) it will cause another trigger message, and so may possibly start BET2.

Despite this, non of this triggering will cause a message to be processed more than once.

_________________________________________________

"MQ delivers the message twice"...

Consider the scenario

MQGET of message1
do something - for example write to TD queue (not in syncpoint) 'Ive processed message 1'
Rollback
Transaction ends

Triggering starts up a new transaction
transaction starts

MQGET of message1
do something - for example write to  CICS TD queue (not in syncpoint) 'Ive processed message 1'
commit
Transaction ends

So in your CICS TD queue you have 2 instances of 'Ive processed message 1', and you say "Ah Ha - MQ has delivered the message more than once".

If would be worth putting some code in your transactions to report if the application did a roll back.  This is good practice, as you should investigate why it has rolled back. (For example if a DB2 request timed out, you should investigate this)

From the MQ accounting data you will see if there is a rollback or commit. 

I cant remember if the CICS stats report the number of commits and backouts, it would be worth looking there.

Putting a few lines of code in your program  to say a rollback has occurred ( and why)  would be good practice, and be less disruptive to turning on trace or accounting.

Colin

• do a CICS ENQ RESOURCE on BET1, BET2, etc. until we find a BETx transaction that is not running. START that one, then end
• If all 8 BETx transactions are already running, end

In short, BETA STARTs the first BETx clone that is not already executing. Wait Y seconds, then end (BETA waits a few seconds for the STARTed transaction to GET the message from the queue).

The BETx clone does an MQGET from the queue, processes it, then continues processing the queue until it is empty.

And here is my visual representation:

1. No-Syncpoint is a (get + commit)
2. Syncpoint means that you need an explicit commit for the message to removed from the system.   You typically do an MQget , DB2 update commit.  The doc says CICS® applications must use the EXEC CICS SYNCPOINT command to commit the unit of work explicitly. Alternatively, ending the transaction results in an implicit commit of the unit of work.  So if the transaction fails to issue exec CICS syncpoint, it will be done for you

I am happy to hear that you don't have that problem. Can you share with us why you have that confidence? Perhaps you are also doing a COBOL DISPLAY of the MQMD-MSGID after each successful MQGET? Or perhaps you have got tracing of the issue by now and can see the message ID from there?

Cheers,
Morag

Don't give up yet Curt.  With something like this, the devil is in the detail, but we can find those details.

Thank you for sharing the DISPLAY statements you found with us. I am sure they will help. Could you tell us what field the "BETF-255736737" data came from? For example, was it the MQMD-MSGID field?

The contents of the data you have shown us does not look like a queue manager generated message ID value (those have the queue manager name in them with some well known prefix characters). Do you know who generates that data - the application that put the message for example? The fact that you use that value as a primary key to do a DB2 insert also suggests that it is not a queue manager generated message ID value, but something generated by an application that has more awareness of the "shape" of the data to be used as a key.

One hypothesis that might be worth following up, is, do you have two messages on this queue with the same "BETF-255736737" data in them? That would certainly fit the description of the problem you have provided so far.

Cheers,
Morag

Hi Curt,

Thanks for your update.

The doc for triggering says If the application queue is open for input, by any program, and has triggering set for FIRST or DEPTH, no trigger event occurs because the queue is already being served. 

So if no queue has the queue open for input then the first message will cause a trigger message to be produced - and so start a transaction.
If there is at least one application with the queue open for input, then there will not be a trigger message produced.

I would expect from what you said that only one transaction (BET1) runs.  If BET1 closes the queue, and there is a message on the queue (due to bad timing) it will cause another trigger message, and so may possibly start BET2.

Despite this, non of this triggering will cause a message to be processed more than once.

_________________________________________________

"MQ delivers the message twice"...

Consider the scenario

MQGET of message1
do something - for example write to TD queue (not in syncpoint) 'Ive processed message 1'
Rollback
Transaction ends

Triggering starts up a new transaction
transaction starts

MQGET of message1
do something - for example write to  CICS TD queue (not in syncpoint) 'Ive processed message 1'
commit
Transaction ends

So in your CICS TD queue you have 2 instances of 'Ive processed message 1', and you say "Ah Ha - MQ has delivered the message more than once".

If would be worth putting some code in your transactions to report if the application did a roll back.  This is good practice, as you should investigate why it has rolled back. (For example if a DB2 request timed out, you should investigate this)

From the MQ accounting data you will see if there is a rollback or commit. 

I cant remember if the CICS stats report the number of commits and backouts, it would be worth looking there.

Putting a few lines of code in your program  to say a rollback has occurred ( and why)  would be good practice, and be less disruptive to turning on trace or accounting.

Colin

• do a CICS ENQ RESOURCE on BET1, BET2, etc. until we find a BETx transaction that is not running. START that one, then end
• If all 8 BETx transactions are already running, end

In short, BETA STARTs the first BETx clone that is not already executing. Wait Y seconds, then end (BETA waits a few seconds for the STARTed transaction to GET the message from the queue).

The BETx clone does an MQGET from the queue, processes it, then continues processing the queue until it is empty.

And here is my visual representation:

1. No-Syncpoint is a (get + commit)
2. Syncpoint means that you need an explicit commit for the message to removed from the system.   You typically do an MQget , DB2 update commit.  The doc says CICS® applications must use the EXEC CICS SYNCPOINT command to commit the unit of work explicitly. Alternatively, ending the transaction results in an implicit commit of the unit of work.  So if the transaction fails to issue exec CICS syncpoint, it will be done for you

I am not doing a COBOL DISPLAY of the MQMD-MSGID after each successful MQGET. I cound do that, as a proof, but the application that PUTs to my input queue does not PUT duplicate messages on it. 

I am happy to hear that you don't have that problem. Can you share with us why you have that confidence? Perhaps you are also doing a COBOL DISPLAY of the MQMD-MSGID after each successful MQGET? Or perhaps you have got tracing of the issue by now and can see the message ID from there?

Cheers,
Morag

Don't give up yet Curt.  With something like this, the devil is in the detail, but we can find those details.

Thank you for sharing the DISPLAY statements you found with us. I am sure they will help. Could you tell us what field the "BETF-255736737" data came from? For example, was it the MQMD-MSGID field?

The contents of the data you have shown us does not look like a queue manager generated message ID value (those have the queue manager name in them with some well known prefix characters). Do you know who generates that data - the application that put the message for example? The fact that you use that value as a primary key to do a DB2 insert also suggests that it is not a queue manager generated message ID value, but something generated by an application that has more awareness of the "shape" of the data to be used as a key.

One hypothesis that might be worth following up, is, do you have two messages on this queue with the same "BETF-255736737" data in them? That would certainly fit the description of the problem you have provided so far.

Cheers,
Morag

Hi Curt,

Thanks for your update.

The doc for triggering says If the application queue is open for input, by any program, and has triggering set for FIRST or DEPTH, no trigger event occurs because the queue is already being served. 

So if no queue has the queue open for input then the first message will cause a trigger message to be produced - and so start a transaction.
If there is at least one application with the queue open for input, then there will not be a trigger message produced.

I would expect from what you said that only one transaction (BET1) runs.  If BET1 closes the queue, and there is a message on the queue (due to bad timing) it will cause another trigger message, and so may possibly start BET2.

Despite this, non of this triggering will cause a message to be processed more than once.

_________________________________________________

"MQ delivers the message twice"...

Consider the scenario

MQGET of message1
do something - for example write to TD queue (not in syncpoint) 'Ive processed message 1'
Rollback
Transaction ends

Triggering starts up a new transaction
transaction starts

MQGET of message1
do something - for example write to  CICS TD queue (not in syncpoint) 'Ive processed message 1'
commit
Transaction ends

So in your CICS TD queue you have 2 instances of 'Ive processed message 1', and you say "Ah Ha - MQ has delivered the message more than once".

If would be worth putting some code in your transactions to report if the application did a roll back.  This is good practice, as you should investigate why it has rolled back. (For example if a DB2 request timed out, you should investigate this)

From the MQ accounting data you will see if there is a rollback or commit. 

I cant remember if the CICS stats report the number of commits and backouts, it would be worth looking there.

Putting a few lines of code in your program  to say a rollback has occurred ( and why)  would be good practice, and be less disruptive to turning on trace or accounting.

Colin

• do a CICS ENQ RESOURCE on BET1, BET2, etc. until we find a BETx transaction that is not running. START that one, then end
• If all 8 BETx transactions are already running, end

In short, BETA STARTs the first BETx clone that is not already executing. Wait Y seconds, then end (BETA waits a few seconds for the STARTed transaction to GET the message from the queue).

The BETx clone does an MQGET from the queue, processes it, then continues processing the queue until it is empty.

And here is my visual representation:

1. No-Syncpoint is a (get + commit)
2. Syncpoint means that you need an explicit commit for the message to removed from the system.   You typically do an MQget , DB2 update commit.  The doc says CICS® applications must use the EXEC CICS SYNCPOINT command to commit the unit of work explicitly. Alternatively, ending the transaction results in an implicit commit of the unit of work.  So if the transaction fails to issue exec CICS syncpoint, it will be done for you

That is very good news Curt. We certainly wouldn't want the application to put duplicate messages. I am happy to hear that you have ruled that out as a problem. Is there just a single putting application? I'm not sure that you described the upstream application workflow to us - we have a detailed description of the getters and the triggering of those transactions. Can you show us how messages get to the Fill Queue?

Cheers,
Morag

I am not doing a COBOL DISPLAY of the MQMD-MSGID after each successful MQGET. I cound do that, as a proof, but the application that PUTs to my input queue does not PUT duplicate messages on it. 

I am happy to hear that you don't have that problem. Can you share with us why you have that confidence? Perhaps you are also doing a COBOL DISPLAY of the MQMD-MSGID after each successful MQGET? Or perhaps you have got tracing of the issue by now and can see the message ID from there?

Cheers,
Morag

Don't give up yet Curt.  With something like this, the devil is in the detail, but we can find those details.

Thank you for sharing the DISPLAY statements you found with us. I am sure they will help. Could you tell us what field the "BETF-255736737" data came from? For example, was it the MQMD-MSGID field?

The contents of the data you have shown us does not look like a queue manager generated message ID value (those have the queue manager name in them with some well known prefix characters). Do you know who generates that data - the application that put the message for example? The fact that you use that value as a primary key to do a DB2 insert also suggests that it is not a queue manager generated message ID value, but something generated by an application that has more awareness of the "shape" of the data to be used as a key.

One hypothesis that might be worth following up, is, do you have two messages on this queue with the same "BETF-255736737" data in them? That would certainly fit the description of the problem you have provided so far.

Cheers,
Morag

Hi Curt,

Thanks for your update.

The doc for triggering says If the application queue is open for input, by any program, and has triggering set for FIRST or DEPTH, no trigger event occurs because the queue is already being served. 

So if no queue has the queue open for input then the first message will cause a trigger message to be produced - and so start a transaction.
If there is at least one application with the queue open for input, then there will not be a trigger message produced.

I would expect from what you said that only one transaction (BET1) runs.  If BET1 closes the queue, and there is a message on the queue (due to bad timing) it will cause another trigger message, and so may possibly start BET2.

Despite this, non of this triggering will cause a message to be processed more than once.

_________________________________________________

"MQ delivers the message twice"...

Consider the scenario

MQGET of message1
do something - for example write to TD queue (not in syncpoint) 'Ive processed message 1'
Rollback
Transaction ends

Triggering starts up a new transaction
transaction starts

MQGET of message1
do something - for example write to  CICS TD queue (not in syncpoint) 'Ive processed message 1'
commit
Transaction ends

So in your CICS TD queue you have 2 instances of 'Ive processed message 1', and you say "Ah Ha - MQ has delivered the message more than once".

If would be worth putting some code in your transactions to report if the application did a roll back.  This is good practice, as you should investigate why it has rolled back. (For example if a DB2 request timed out, you should investigate this)

From the MQ accounting data you will see if there is a rollback or commit. 

I cant remember if the CICS stats report the number of commits and backouts, it would be worth looking there.

Putting a few lines of code in your program  to say a rollback has occurred ( and why)  would be good practice, and be less disruptive to turning on trace or accounting.

Colin

• do a CICS ENQ RESOURCE on BET1, BET2, etc. until we find a BETx transaction that is not running. START that one, then end
• If all 8 BETx transactions are already running, end

In short, BETA STARTs the first BETx clone that is not already executing. Wait Y seconds, then end (BETA waits a few seconds for the STARTed transaction to GET the message from the queue).

The BETx clone does an MQGET from the queue, processes it, then continues processing the queue until it is empty.

And here is my visual representation:

1. No-Syncpoint is a (get + commit)
2. Syncpoint means that you need an explicit commit for the message to removed from the system.   You typically do an MQget , DB2 update commit.  The doc says CICS® applications must use the EXEC CICS SYNCPOINT command to commit the unit of work explicitly. Alternatively, ending the transaction results in an implicit commit of the unit of work.  So if the transaction fails to issue exec CICS syncpoint, it will be done for you

That is very good news Curt. We certainly wouldn't want the application to put duplicate messages. I am happy to hear that you have ruled that out as a problem. Is there just a single putting application? I'm not sure that you described the upstream application workflow to us - we have a detailed description of the getters and the triggering of those transactions. Can you show us how messages get to the Fill Queue?

Cheers,
Morag

I am not doing a COBOL DISPLAY of the MQMD-MSGID after each successful MQGET. I cound do that, as a proof, but the application that PUTs to my input queue does not PUT duplicate messages on it. 

I am happy to hear that you don't have that problem. Can you share with us why you have that confidence? Perhaps you are also doing a COBOL DISPLAY of the MQMD-MSGID after each successful MQGET? Or perhaps you have got tracing of the issue by now and can see the message ID from there?

Cheers,
Morag

Don't give up yet Curt.  With something like this, the devil is in the detail, but we can find those details.

Thank you for sharing the DISPLAY statements you found with us. I am sure they will help. Could you tell us what field the "BETF-255736737" data came from? For example, was it the MQMD-MSGID field?

The contents of the data you have shown us does not look like a queue manager generated message ID value (those have the queue manager name in them with some well known prefix characters). Do you know who generates that data - the application that put the message for example? The fact that you use that value as a primary key to do a DB2 insert also suggests that it is not a queue manager generated message ID value, but something generated by an application that has more awareness of the "shape" of the data to be used as a key.

One hypothesis that might be worth following up, is, do you have two messages on this queue with the same "BETF-255736737" data in them? That would certainly fit the description of the problem you have provided so far.

Cheers,
Morag

Hi Curt,

Thanks for your update.

The doc for triggering says If the application queue is open for input, by any program, and has triggering set for FIRST or DEPTH, no trigger event occurs because the queue is already being served. 

So if no queue has the queue open for input then the first message will cause a trigger message to be produced - and so start a transaction.
If there is at least one application with the queue open for input, then there will not be a trigger message produced.

I would expect from what you said that only one transaction (BET1) runs.  If BET1 closes the queue, and there is a message on the queue (due to bad timing) it will cause another trigger message, and so may possibly start BET2.

Despite this, non of this triggering will cause a message to be processed more than once.

_________________________________________________

"MQ delivers the message twice"...

Consider the scenario

MQGET of message1
do something - for example write to TD queue (not in syncpoint) 'Ive processed message 1'
Rollback
Transaction ends

Triggering starts up a new transaction
transaction starts

MQGET of message1
do something - for example write to  CICS TD queue (not in syncpoint) 'Ive processed message 1'
commit
Transaction ends

So in your CICS TD queue you have 2 instances of 'Ive processed message 1', and you say "Ah Ha - MQ has delivered the message more than once".

If would be worth putting some code in your transactions to report if the application did a roll back.  This is good practice, as you should investigate why it has rolled back. (For example if a DB2 request timed out, you should investigate this)

From the MQ accounting data you will see if there is a rollback or commit. 

I cant remember if the CICS stats report the number of commits and backouts, it would be worth looking there.

Putting a few lines of code in your program  to say a rollback has occurred ( and why)  would be good practice, and be less disruptive to turning on trace or accounting.

Colin

• do a CICS ENQ RESOURCE on BET1, BET2, etc. until we find a BETx transaction that is not running. START that one, then end
• If all 8 BETx transactions are already running, end

In short, BETA STARTs the first BETx clone that is not already executing. Wait Y seconds, then end (BETA waits a few seconds for the STARTed transaction to GET the message from the queue).

The BETx clone does an MQGET from the queue, processes it, then continues processing the queue until it is empty.

And here is my visual representation:

1. No-Syncpoint is a (get + commit)
2. Syncpoint means that you need an explicit commit for the message to removed from the system.   You typically do an MQget , DB2 update commit.  The doc says CICS® applications must use the EXEC CICS SYNCPOINT command to commit the unit of work explicitly. Alternatively, ending the transaction results in an implicit commit of the unit of work.  So if the transaction fails to issue exec CICS syncpoint, it will be done for you

That is very good news Curt. We certainly wouldn't want the application to put duplicate messages. I am happy to hear that you have ruled that out as a problem. Is there just a single putting application? I'm not sure that you described the upstream application workflow to us - we have a detailed description of the getters and the triggering of those transactions. Can you show us how messages get to the Fill Queue?

Cheers,
Morag

I am not doing a COBOL DISPLAY of the MQMD-MSGID after each successful MQGET. I cound do that, as a proof, but the application that PUTs to my input queue does not PUT duplicate messages on it. 

I am happy to hear that you don't have that problem. Can you share with us why you have that confidence? Perhaps you are also doing a COBOL DISPLAY of the MQMD-MSGID after each successful MQGET? Or perhaps you have got tracing of the issue by now and can see the message ID from there?

Cheers,
Morag

Don't give up yet Curt.  With something like this, the devil is in the detail, but we can find those details.

Thank you for sharing the DISPLAY statements you found with us. I am sure they will help. Could you tell us what field the "BETF-255736737" data came from? For example, was it the MQMD-MSGID field?

The contents of the data you have shown us does not look like a queue manager generated message ID value (those have the queue manager name in them with some well known prefix characters). Do you know who generates that data - the application that put the message for example? The fact that you use that value as a primary key to do a DB2 insert also suggests that it is not a queue manager generated message ID value, but something generated by an application that has more awareness of the "shape" of the data to be used as a key.

One hypothesis that might be worth following up, is, do you have two messages on this queue with the same "BETF-255736737" data in them? That would certainly fit the description of the problem you have provided so far.

Cheers,
Morag

Hi Curt,

Thanks for your update.

The doc for triggering says If the application queue is open for input, by any program, and has triggering set for FIRST or DEPTH, no trigger event occurs because the queue is already being served. 

So if no queue has the queue open for input then the first message will cause a trigger message to be produced - and so start a transaction.
If there is at least one application with the queue open for input, then there will not be a trigger message produced.

I would expect from what you said that only one transaction (BET1) runs.  If BET1 closes the queue, and there is a message on the queue (due to bad timing) it will cause another trigger message, and so may possibly start BET2.

Despite this, non of this triggering will cause a message to be processed more than once.

_________________________________________________

"MQ delivers the message twice"...

Consider the scenario

MQGET of message1
do something - for example write to TD queue (not in syncpoint) 'Ive processed message 1'
Rollback
Transaction ends

Triggering starts up a new transaction
transaction starts

MQGET of message1
do something - for example write to  CICS TD queue (not in syncpoint) 'Ive processed message 1'
commit
Transaction ends

So in your CICS TD queue you have 2 instances of 'Ive processed message 1', and you say "Ah Ha - MQ has delivered the message more than once".

If would be worth putting some code in your transactions to report if the application did a roll back.  This is good practice, as you should investigate why it has rolled back. (For example if a DB2 request timed out, you should investigate this)

From the MQ accounting data you will see if there is a rollback or commit. 

I cant remember if the CICS stats report the number of commits and backouts, it would be worth looking there.

Putting a few lines of code in your program  to say a rollback has occurred ( and why)  would be good practice, and be less disruptive to turning on trace or accounting.

Colin

• do a CICS ENQ RESOURCE on BET1, BET2, etc. until we find a BETx transaction that is not running. START that one, then end
• If all 8 BETx transactions are already running, end

In short, BETA STARTs the first BETx clone that is not already executing. Wait Y seconds, then end (BETA waits a few seconds for the STARTed transaction to GET the message from the queue).

The BETx clone does an MQGET from the queue, processes it, then continues processing the queue until it is empty.

And here is my visual representation:

1. No-Syncpoint is a (get + commit)
2. Syncpoint means that you need an explicit commit for the message to removed from the system.   You typically do an MQget , DB2 update commit.  The doc says CICS® applications must use the EXEC CICS SYNCPOINT command to commit the unit of work explicitly. Alternatively, ending the transaction results in an implicit commit of the unit of work.  So if the transaction fails to issue exec CICS syncpoint, it will be done for you

That is very good news Curt. We certainly wouldn't want the application to put duplicate messages. I am happy to hear that you have ruled that out as a problem. Is there just a single putting application? I'm not sure that you described the upstream application workflow to us - we have a detailed description of the getters and the triggering of those transactions. Can you show us how messages get to the Fill Queue?

Cheers,
Morag

I am not doing a COBOL DISPLAY of the MQMD-MSGID after each successful MQGET. I cound do that, as a proof, but the application that PUTs to my input queue does not PUT duplicate messages on it. 

I am happy to hear that you don't have that problem. Can you share with us why you have that confidence? Perhaps you are also doing a COBOL DISPLAY of the MQMD-MSGID after each successful MQGET? Or perhaps you have got tracing of the issue by now and can see the message ID from there?

Cheers,
Morag

Don't give up yet Curt.  With something like this, the devil is in the detail, but we can find those details.

Thank you for sharing the DISPLAY statements you found with us. I am sure they will help. Could you tell us what field the "BETF-255736737" data came from? For example, was it the MQMD-MSGID field?

The contents of the data you have shown us does not look like a queue manager generated message ID value (those have the queue manager name in them with some well known prefix characters). Do you know who generates that data - the application that put the message for example? The fact that you use that value as a primary key to do a DB2 insert also suggests that it is not a queue manager generated message ID value, but something generated by an application that has more awareness of the "shape" of the data to be used as a key.

One hypothesis that might be worth following up, is, do you have two messages on this queue with the same "BETF-255736737" data in them? That would certainly fit the description of the problem you have provided so far.

Cheers,
Morag

Hi Curt,

Thanks for your update.

The doc for triggering says If the application queue is open for input, by any program, and has triggering set for FIRST or DEPTH, no trigger event occurs because the queue is already being served. 

So if no queue has the queue open for input then the first message will cause a trigger message to be produced - and so start a transaction.
If there is at least one application with the queue open for input, then there will not be a trigger message produced.

I would expect from what you said that only one transaction (BET1) runs.  If BET1 closes the queue, and there is a message on the queue (due to bad timing) it will cause another trigger message, and so may possibly start BET2.

Despite this, non of this triggering will cause a message to be processed more than once.

_________________________________________________

"MQ delivers the message twice"...

Consider the scenario

MQGET of message1
do something - for example write to TD queue (not in syncpoint) 'Ive processed message 1'
Rollback
Transaction ends

Triggering starts up a new transaction
transaction starts

MQGET of message1
do something - for example write to  CICS TD queue (not in syncpoint) 'Ive processed message 1'
commit
Transaction ends

So in your CICS TD queue you have 2 instances of 'Ive processed message 1', and you say "Ah Ha - MQ has delivered the message more than once".

If would be worth putting some code in your transactions to report if the application did a roll back.  This is good practice, as you should investigate why it has rolled back. (For example if a DB2 request timed out, you should investigate this)

From the MQ accounting data you will see if there is a rollback or commit. 

I cant remember if the CICS stats report the number of commits and backouts, it would be worth looking there.

Putting a few lines of code in your program  to say a rollback has occurred ( and why)  would be good practice, and be less disruptive to turning on trace or accounting.

Colin

• do a CICS ENQ RESOURCE on BET1, BET2, etc. until we find a BETx transaction that is not running. START that one, then end
• If all 8 BETx transactions are already running, end

In short, BETA STARTs the first BETx clone that is not already executing. Wait Y seconds, then end (BETA waits a few seconds for the STARTed transaction to GET the message from the queue).

The BETx clone does an MQGET from the queue, processes it, then continues processing the queue until it is empty.

And here is my visual representation:

1. No-Syncpoint is a (get + commit)
2. Syncpoint means that you need an explicit commit for the message to removed from the system.   You typically do an MQget , DB2 update commit.  The doc says CICS® applications must use the EXEC CICS SYNCPOINT command to commit the unit of work explicitly. Alternatively, ending the transaction results in an implicit commit of the unit of work.  So if the transaction fails to issue exec CICS syncpoint, it will be done for you

That is very good news Curt. We certainly wouldn't want the application to put duplicate messages. I am happy to hear that you have ruled that out as a problem. Is there just a single putting application? I'm not sure that you described the upstream application workflow to us - we have a detailed description of the getters and the triggering of those transactions. Can you show us how messages get to the Fill Queue?

Cheers,
Morag

I am not doing a COBOL DISPLAY of the MQMD-MSGID after each successful MQGET. I cound do that, as a proof, but the application that PUTs to my input queue does not PUT duplicate messages on it. 

I am happy to hear that you don't have that problem. Can you share with us why you have that confidence? Perhaps you are also doing a COBOL DISPLAY of the MQMD-MSGID after each successful MQGET? Or perhaps you have got tracing of the issue by now and can see the message ID from there?

Cheers,
Morag

Don't give up yet Curt.  With something like this, the devil is in the detail, but we can find those details.

Thank you for sharing the DISPLAY statements you found with us. I am sure they will help. Could you tell us what field the "BETF-255736737" data came from? For example, was it the MQMD-MSGID field?

The contents of the data you have shown us does not look like a queue manager generated message ID value (those have the queue manager name in them with some well known prefix characters). Do you know who generates that data - the application that put the message for example? The fact that you use that value as a primary key to do a DB2 insert also suggests that it is not a queue manager generated message ID value, but something generated by an application that has more awareness of the "shape" of the data to be used as a key.

One hypothesis that might be worth following up, is, do you have two messages on this queue with the same "BETF-255736737" data in them? That would certainly fit the description of the problem you have provided so far.

Cheers,
Morag

Hi Curt,

Thanks for your update.

The doc for triggering says If the application queue is open for input, by any program, and has triggering set for FIRST or DEPTH, no trigger event occurs because the queue is already being served. 

So if no queue has the queue open for input then the first message will cause a trigger message to be produced - and so start a transaction.
If there is at least one application with the queue open for input, then there will not be a trigger message produced.

I would expect from what you said that only one transaction (BET1) runs.  If BET1 closes the queue, and there is a message on the queue (due to bad timing) it will cause another trigger message, and so may possibly start BET2.

Despite this, non of this triggering will cause a message to be processed more than once.

_________________________________________________

"MQ delivers the message twice"...

Consider the scenario

MQGET of message1
do something - for example write to TD queue (not in syncpoint) 'Ive processed message 1'
Rollback
Transaction ends

Triggering starts up a new transaction
transaction starts

MQGET of message1
do something - for example write to  CICS TD queue (not in syncpoint) 'Ive processed message 1'
commit
Transaction ends

So in your CICS TD queue you have 2 instances of 'Ive processed message 1', and you say "Ah Ha - MQ has delivered the message more than once".

If would be worth putting some code in your transactions to report if the application did a roll back.  This is good practice, as you should investigate why it has rolled back. (For example if a DB2 request timed out, you should investigate this)

From the MQ accounting data you will see if there is a rollback or commit. 

I cant remember if the CICS stats report the number of commits and backouts, it would be worth looking there.

Putting a few lines of code in your program  to say a rollback has occurred ( and why)  would be good practice, and be less disruptive to turning on trace or accounting.

Colin

• do a CICS ENQ RESOURCE on BET1, BET2, etc. until we find a BETx transaction that is not running. START that one, then end
• If all 8 BETx transactions are already running, end

In short, BETA STARTs the first BETx clone that is not already executing. Wait Y seconds, then end (BETA waits a few seconds for the STARTed transaction to GET the message from the queue).

The BETx clone does an MQGET from the queue, processes it, then continues processing the queue until it is empty.

And here is my visual representation:

1. No-Syncpoint is a (get + commit)
2. Syncpoint means that you need an explicit commit for the message to removed from the system.   You typically do an MQget , DB2 update commit.  The doc says CICS® applications must use the EXEC CICS SYNCPOINT command to commit the unit of work explicitly. Alternatively, ending the transaction results in an implicit commit of the unit of work.  So if the transaction fails to issue exec CICS syncpoint, it will be done for you

That is very good news Curt. We certainly wouldn't want the application to put duplicate messages. I am happy to hear that you have ruled that out as a problem. Is there just a single putting application? I'm not sure that you described the upstream application workflow to us - we have a detailed description of the getters and the triggering of those transactions. Can you show us how messages get to the Fill Queue?

Cheers,
Morag

I am not doing a COBOL DISPLAY of the MQMD-MSGID after each successful MQGET. I cound do that, as a proof, but the application that PUTs to my input queue does not PUT duplicate messages on it. 

I am happy to hear that you don't have that problem. Can you share with us why you have that confidence? Perhaps you are also doing a COBOL DISPLAY of the MQMD-MSGID after each successful MQGET? Or perhaps you have got tracing of the issue by now and can see the message ID from there?

Cheers,
Morag

Don't give up yet Curt.  With something like this, the devil is in the detail, but we can find those details.

Thank you for sharing the DISPLAY statements you found with us. I am sure they will help. Could you tell us what field the "BETF-255736737" data came from? For example, was it the MQMD-MSGID field?

The contents of the data you have shown us does not look like a queue manager generated message ID value (those have the queue manager name in them with some well known prefix characters). Do you know who generates that data - the application that put the message for example? The fact that you use that value as a primary key to do a DB2 insert also suggests that it is not a queue manager generated message ID value, but something generated by an application that has more awareness of the "shape" of the data to be used as a key.

One hypothesis that might be worth following up, is, do you have two messages on this queue with the same "BETF-255736737" data in them? That would certainly fit the description of the problem you have provided so far.

Cheers,
Morag

Hi Curt,

Thanks for your update.

The doc for triggering says If the application queue is open for input, by any program, and has triggering set for FIRST or DEPTH, no trigger event occurs because the queue is already being served. 

So if no queue has the queue open for input then the first message will cause a trigger message to be produced - and so start a transaction.
If there is at least one application with the queue open for input, then there will not be a trigger message produced.

I would expect from what you said that only one transaction (BET1) runs.  If BET1 closes the queue, and there is a message on the queue (due to bad timing) it will cause another trigger message, and so may possibly start BET2.

Despite this, non of this triggering will cause a message to be processed more than once.

_________________________________________________

"MQ delivers the message twice"...

Consider the scenario

MQGET of message1
do something - for example write to TD queue (not in syncpoint) 'Ive processed message 1'
Rollback
Transaction ends

Triggering starts up a new transaction
transaction starts

MQGET of message1
do something - for example write to  CICS TD queue (not in syncpoint) 'Ive processed message 1'
commit
Transaction ends

So in your CICS TD queue you have 2 instances of 'Ive processed message 1', and you say "Ah Ha - MQ has delivered the message more than once".

If would be worth putting some code in your transactions to report if the application did a roll back.  This is good practice, as you should investigate why it has rolled back. (For example if a DB2 request timed out, you should investigate this)

From the MQ accounting data you will see if there is a rollback or commit. 

I cant remember if the CICS stats report the number of commits and backouts, it would be worth looking there.

Putting a few lines of code in your program  to say a rollback has occurred ( and why)  would be good practice, and be less disruptive to turning on trace or accounting.

Colin

• do a CICS ENQ RESOURCE on BET1, BET2, etc. until we find a BETx transaction that is not running. START that one, then end
• If all 8 BETx transactions are already running, end

In short, BETA STARTs the first BETx clone that is not already executing. Wait Y seconds, then end (BETA waits a few seconds for the STARTed transaction to GET the message from the queue).

The BETx clone does an MQGET from the queue, processes it, then continues processing the queue until it is empty.

And here is my visual representation:

1. No-Syncpoint is a (get + commit)
2. Syncpoint means that you need an explicit commit for the message to removed from the system.   You typically do an MQget , DB2 update commit.  The doc says CICS® applications must use the EXEC CICS SYNCPOINT command to commit the unit of work explicitly. Alternatively, ending the transaction results in an implicit commit of the unit of work.  So if the transaction fails to issue exec CICS syncpoint, it will be done for you

• MQMD-MSGID
• date / time 
• MQMD-BACKOUTCOUNT

That is very good news Curt. We certainly wouldn't want the application to put duplicate messages. I am happy to hear that you have ruled that out as a problem. Is there just a single putting application? I'm not sure that you described the upstream application workflow to us - we have a detailed description of the getters and the triggering of those transactions. Can you show us how messages get to the Fill Queue?

Cheers,
Morag

I am not doing a COBOL DISPLAY of the MQMD-MSGID after each successful MQGET. I cound do that, as a proof, but the application that PUTs to my input queue does not PUT duplicate messages on it. 

I am happy to hear that you don't have that problem. Can you share with us why you have that confidence? Perhaps you are also doing a COBOL DISPLAY of the MQMD-MSGID after each successful MQGET? Or perhaps you have got tracing of the issue by now and can see the message ID from there?

Cheers,
Morag

Don't give up yet Curt.  With something like this, the devil is in the detail, but we can find those details.

Thank you for sharing the DISPLAY statements you found with us. I am sure they will help. Could you tell us what field the "BETF-255736737" data came from? For example, was it the MQMD-MSGID field?

The contents of the data you have shown us does not look like a queue manager generated message ID value (those have the queue manager name in them with some well known prefix characters). Do you know who generates that data - the application that put the message for example? The fact that you use that value as a primary key to do a DB2 insert also suggests that it is not a queue manager generated message ID value, but something generated by an application that has more awareness of the "shape" of the data to be used as a key.

One hypothesis that might be worth following up, is, do you have two messages on this queue with the same "BETF-255736737" data in them? That would certainly fit the description of the problem you have provided so far.

Cheers,
Morag

Hi Curt,

Thanks for your update.

The doc for triggering says If the application queue is open for input, by any program, and has triggering set for FIRST or DEPTH, no trigger event occurs because the queue is already being served. 

So if no queue has the queue open for input then the first message will cause a trigger message to be produced - and so start a transaction.
If there is at least one application with the queue open for input, then there will not be a trigger message produced.

I would expect from what you said that only one transaction (BET1) runs.  If BET1 closes the queue, and there is a message on the queue (due to bad timing) it will cause another trigger message, and so may possibly start BET2.

Despite this, non of this triggering will cause a message to be processed more than once.

_________________________________________________

"MQ delivers the message twice"...

Consider the scenario

MQGET of message1
do something - for example write to TD queue (not in syncpoint) 'Ive processed message 1'
Rollback
Transaction ends

Triggering starts up a new transaction
transaction starts

MQGET of message1
do something - for example write to  CICS TD queue (not in syncpoint) 'Ive processed message 1'
commit
Transaction ends

So in your CICS TD queue you have 2 instances of 'Ive processed message 1', and you say "Ah Ha - MQ has delivered the message more than once".

If would be worth putting some code in your transactions to report if the application did a roll back.  This is good practice, as you should investigate why it has rolled back. (For example if a DB2 request timed out, you should investigate this)

From the MQ accounting data you will see if there is a rollback or commit. 

I cant remember if the CICS stats report the number of commits and backouts, it would be worth looking there.

Putting a few lines of code in your program  to say a rollback has occurred ( and why)  would be good practice, and be less disruptive to turning on trace or accounting.

Colin

• do a CICS ENQ RESOURCE on BET1, BET2, etc. until we find a BETx transaction that is not running. START that one, then end
• If all 8 BETx transactions are already running, end

In short, BETA STARTs the first BETx clone that is not already executing. Wait Y seconds, then end (BETA waits a few seconds for the STARTed transaction to GET the message from the queue).

The BETx clone does an MQGET from the queue, processes it, then continues processing the queue until it is empty.

And here is my visual representation:

1. No-Syncpoint is a (get + commit)
2. Syncpoint means that you need an explicit commit for the message to removed from the system.   You typically do an MQget , DB2 update commit.  The doc says CICS® applications must use the EXEC CICS SYNCPOINT command to commit the unit of work explicitly. Alternatively, ending the transaction results in an implicit commit of the unit of work.  So if the transaction fails to issue exec CICS syncpoint, it will be done for you

• MQMD-MSGID
• date / time 
• MQMD-BACKOUTCOUNT

That is very good news Curt. We certainly wouldn't want the application to put duplicate messages. I am happy to hear that you have ruled that out as a problem. Is there just a single putting application? I'm not sure that you described the upstream application workflow to us - we have a detailed description of the getters and the triggering of those transactions. Can you show us how messages get to the Fill Queue?

Cheers,
Morag

I am not doing a COBOL DISPLAY of the MQMD-MSGID after each successful MQGET. I cound do that, as a proof, but the application that PUTs to my input queue does not PUT duplicate messages on it. 

I am happy to hear that you don't have that problem. Can you share with us why you have that confidence? Perhaps you are also doing a COBOL DISPLAY of the MQMD-MSGID after each successful MQGET? Or perhaps you have got tracing of the issue by now and can see the message ID from there?

Cheers,
Morag

Don't give up yet Curt.  With something like this, the devil is in the detail, but we can find those details.

Thank you for sharing the DISPLAY statements you found with us. I am sure they will help. Could you tell us what field the "BETF-255736737" data came from? For example, was it the MQMD-MSGID field?

The contents of the data you have shown us does not look like a queue manager generated message ID value (those have the queue manager name in them with some well known prefix characters). Do you know who generates that data - the application that put the message for example? The fact that you use that value as a primary key to do a DB2 insert also suggests that it is not a queue manager generated message ID value, but something generated by an application that has more awareness of the "shape" of the data to be used as a key.

One hypothesis that might be worth following up, is, do you have two messages on this queue with the same "BETF-255736737" data in them? That would certainly fit the description of the problem you have provided so far.

Cheers,
Morag

Hi Curt,

Thanks for your update.

The doc for triggering says If the application queue is open for input, by any program, and has triggering set for FIRST or DEPTH, no trigger event occurs because the queue is already being served. 

So if no queue has the queue open for input then the first message will cause a trigger message to be produced - and so start a transaction.
If there is at least one application with the queue open for input, then there will not be a trigger message produced.

I would expect from what you said that only one transaction (BET1) runs.  If BET1 closes the queue, and there is a message on the queue (due to bad timing) it will cause another trigger message, and so may possibly start BET2.

Despite this, non of this triggering will cause a message to be processed more than once.

_________________________________________________

"MQ delivers the message twice"...

Consider the scenario

MQGET of message1
do something - for example write to TD queue (not in syncpoint) 'Ive processed message 1'
Rollback
Transaction ends

Triggering starts up a new transaction
transaction starts

MQGET of message1
do something - for example write to  CICS TD queue (not in syncpoint) 'Ive processed message 1'
commit
Transaction ends

So in your CICS TD queue you have 2 instances of 'Ive processed message 1', and you say "Ah Ha - MQ has delivered the message more than once".

If would be worth putting some code in your transactions to report if the application did a roll back.  This is good practice, as you should investigate why it has rolled back. (For example if a DB2 request timed out, you should investigate this)

From the MQ accounting data you will see if there is a rollback or commit. 

I cant remember if the CICS stats report the number of commits and backouts, it would be worth looking there.

Putting a few lines of code in your program  to say a rollback has occurred ( and why)  would be good practice, and be less disruptive to turning on trace or accounting.

Colin

• do a CICS ENQ RESOURCE on BET1, BET2, etc. until we find a BETx transaction that is not running. START that one, then end
• If all 8 BETx transactions are already running, end

In short, BETA STARTs the first BETx clone that is not already executing. Wait Y seconds, then end (BETA waits a few seconds for the STARTed transaction to GET the message from the queue).

The BETx clone does an MQGET from the queue, processes it, then continues processing the queue until it is empty.

And here is my visual representation:

1. No-Syncpoint is a (get + commit)
2. Syncpoint means that you need an explicit commit for the message to removed from the system.   You typically do an MQget , DB2 update commit.  The doc says CICS® applications must use the EXEC CICS SYNCPOINT command to commit the unit of work explicitly. Alternatively, ending the transaction results in an implicit commit of the unit of work.  So if the transaction fails to issue exec CICS syncpoint, it will be done for you