IBM webMethods Hybrid Integration

IBM webMethods Hybrid Integration

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
Expand all | Collapse all

MQ Adapter Backout Rollback problem

  • 1.  MQ Adapter Backout Rollback problem

    Posted Mon November 01, 2004 11:17 PM

    Hi Gurus,

    I am using MQ Adapter ver 6.0. I have listener running to poll REQUEST queue and when the message is available , it passes the message to asynchronous MQ adapter notification. When a message is put on the queue, listener picks it up and passes to notifier. Notifier successfully publishes the document to IS locally. Even trigger is also invokes the processing service successfully. But when my processing service encounters some problem and it throws any exception, the message should go back to the REQUEST queue. But I keep losing the message. I tried to implement explicit transaction in the processing service. for example i used startTransaction, commitTransaction, rollbackTransaction also. Still while any runtime exception, message is lost everytime.

    In detail, my service has TRY/CATCH in the flow service. When any exception occurs ,it gets cought in catch, i call rollbackTransaction and then exit with success. Still the message is not getting rolled back and message is lost at all.

    Can someone tell me how to rollback a message while the message is picked up by an MQ listener ?

    Thanks in advance for your help.

    Regards,
    Nilesh


    #Integration-Server-and-ESB
    #Adapters-and-E-Standards
    #webMethods


  • 2.  RE: MQ Adapter Backout Rollback problem

    Posted Mon November 01, 2004 11:41 PM

    Nilesh,

    I havent used latest MQ adapter but i have few suggesstions,

    In the MQ listener i believe there is an option for “persist” this makes message not get deleted from the Queue after delivered to IS.

    Map the same transactionName object in the RollbackTransaction which you set in the StartTransaction/commitTransaction either explicity set this or use the pipeline

    sample snippet:

    StartTransaction(transactionName=test) this will be carried out in the pipeline.
    Sequence (ON Success)
    —Sequence(ON Failure)
    CommitTransaction(transactionName=test)

    —Sequence(ON Done)
    getLastError
    RollbackTransaction(transactionName=test)
    Exit(On Failure or Success)

    Try this scenario and this is generally followed for any DB(LOCAL_TRANSACTION) interactions,so this might work for MQ too.

    HTH,
    RMG


    #Integration-Server-and-ESB
    #Adapters-and-E-Standards
    #webMethods


  • 3.  RE: MQ Adapter Backout Rollback problem

    Posted Mon November 01, 2004 11:57 PM

    RMG,
    Thanks for your response.

    I am not sure, whether MQ Listener has any setting like ‘persist’. I couldnt find it. If you can guide me in this regards, i will set that to true.

    The example which you gave, it is really explanatory in it. But i tried the same implementing and even after that i kept losing my message.

    I dont know what is the exact criteria while rolling back the MQ message.

    What I believe is When there is any MQ Exception, then only it will get rolled back by MQ Adapter. Im My case, whenever any other exception like flow exception or service exception, i want to roll back.

    i tried using non-transactional connection, too.

    The following paragraph from MQ series adapter user guide is little confusing :


    Behavior of Listeners :
    Listeners exhibit different behaviors when they are associated with either a transactional connection or a non-transactional connection.
    Listeners and Transactional Connections.
    A listener that is associated with a transactional connection exhibits the following behavior:

    The Listener monitors the queue.

    The Listener receives a message. (The WebSphere MQ Adapter removes this message

    from the queue.)

    If the message matches all of the selection criteria for any notification, then the notification processes the message. However, if the message does not match the selection criteria for the notification, then the notification discards the message.

    ------------ AND -----------

    Listeners and Non-Transactional Connections :
    A listener that is associated with a non-transactional connection exhibits the following behavior:

    The Listener monitors the queue.

    The Listener receives a message. (The WebSphere MQ Adapter tags this message as unavailable, but the adapter does not remove the message from the queue.)

    If the message matches all of the selection criteria for any notification, then the notification processes the message.

    If the notification processes the message, then the Listener commits the message. (The WebSphere MQ Adapter removes the message from the queue.) If no notification does not process the message, then the Listener rolls back the message. (The WebSphere MQ Adapter tags this message as available.)


    What would you say , the write up is ok or it should be reverse for in case of transactional & non-transactional connections.

    Regards,
    Nilesh


    #Adapters-and-E-Standards
    #Integration-Server-and-ESB
    #webMethods


  • 4.  RE: MQ Adapter Backout Rollback problem

    Posted Tue November 02, 2004 12:26 AM

    The persist option checkbox will be either in MQ Server admin side or MQ Adapter listen handler configuration page.This option might be removed or moved to different configuration settings page in latest MQ adapter.


    #Integration-Server-and-ESB
    #Adapters-and-E-Standards
    #webMethods


  • 5.  RE: MQ Adapter Backout Rollback problem

    Posted Tue November 02, 2004 03:08 PM

    RMG,

    I had a talk with MQ Admin folks and they told that ‘persist’ option is at queue level. They have already set that option ‘ON’.

    In the documentation of MQ Adapter, it is specified that if there is any error while processing the message, it will get rolled back to the queue. But in my case, that is not happening at all.

    Still I am waiting for help on how to implement transactions with MQ Adapter.

    Thanks for all your help.

    Regards,
    Nilesh


    #webMethods
    #Adapters-and-E-Standards
    #Integration-Server-and-ESB


  • 6.  RE: MQ Adapter Backout Rollback problem

    Posted Tue November 02, 2004 04:51 PM

    Nilesh,

    If MQ Adapter doc says the rollback functionality work then the above Sequence scenario should work.please make sure the transactionName object exists in the pipeline or may be it is missing during runtime.

    sorry for not able to help more on this issue.


    #webMethods
    #Adapters-and-E-Standards
    #Integration-Server-and-ESB


  • 7.  RE: MQ Adapter Backout Rollback problem

    Posted Thu November 04, 2004 12:00 AM

    Hi Nilesh - You’re probably still on that Y2K’esque global TN issue that happened earlier.

    Regarding this post, are you using a transactional MQ Connection? If so, then you may have to work with your MQ engineers to coordinate usage of MQ’s SyncPoint. I don’t think this is related to the “Default Persistence” parameter you provided when you requested the Queue.

    Hope that helps,
    Rajesh


    #Adapters-and-E-Standards
    #webMethods
    #Integration-Server-and-ESB


  • 8.  RE: MQ Adapter Backout Rollback problem

    Posted Tue November 30, 2004 09:34 AM

    Hi, I’m working on IS 4.6 and MQ adapter 3.1. What is the recommended way to get messages of the queue? I am currently using scheduled (get)services. But the scheduler seems to be incredibly unstable. Is there no way to get the messages of the queue in realtime?


    #webMethods
    #Adapters-and-E-Standards
    #Integration-Server-and-ESB


  • 9.  RE: MQ Adapter Backout Rollback problem

    Posted Tue November 30, 2004 10:02 PM

    Can you use a Listener + Listener Notification + trigger + trigger service mechanism with the WebSphere MQ 3.0 adapter? This would be the approach if you were using the MQ 6.0 adapter.


    #Adapters-and-E-Standards
    #Integration-Server-and-ESB
    #webMethods


  • 10.  RE: MQ Adapter Backout Rollback problem

    Posted Tue June 07, 2005 02:56 PM

    Hi All,
    I am in a similar situation and hence feel the need to re invoke this thread. I am also using MQ Adapter 6.0 version. I first implemented a solution using the manual transaction management via, start/rollback/commitTransaction, which worked fairly well, except that I had a scheduler which runs at some x interval of time. I had to move to make my system more real time and picked the listener/notification solution.
    Now I have a
    Non-Transactional MQ Connection <–> Listener <–> Asynchronous Notification <–> BROKER <–> Trigger <–> Flow service.
    I am sure the transition management for MQ Adapter will be limited till the mortification publishes the document. I had a situation , where the connection to the broker was lost, which will lead to the publish failure.
    The messages arrived from MQ series, listener picks up the message, passes on to Notification, which in turn publishes. The publishing fails as broker connection is broken, and the transaction should be rolled back and the messages should not be deleted form the queue. But I lost all my messages. I don’t fine them any where, not even in dead letter queue.
    Is there anything I am missing in my implementation or is this the way MQ behaves?
    Bye for now,
    Sathya KM


    #Integration-Server-and-ESB
    #webMethods
    #Adapters-and-E-Standards


  • 11.  RE: MQ Adapter Backout Rollback problem

    Posted Tue June 07, 2005 10:17 PM

    Hi Sathya,

    I am not aware of your WM environment although May I know about your MQ Queue properties ? Is your queue set for message ‘persistence’ ?

    • Nilesh D

    #Adapters-and-E-Standards
    #Integration-Server-and-ESB
    #webMethods


  • 12.  RE: MQ Adapter Backout Rollback problem

    Posted Wed June 08, 2005 02:34 PM

    Hi Nilesh,
    I think the concept of persistence has to be manipulated only at MQMS level, I do not find any such option in the MQ Adapter settings. I am sure it is switched on at the MQ level.
    Here is my MQ Connection Properties

    Queue Manager Name XXXXXXX
    Host Name XXXXXXXXXXXXXXXX
    TCP/IP Port 1414
    Server Connection Channel CN.CLIENT.EAI
    CCSID CP819
    User Id
    Password
    Queue Name MQ.XXXXXXXXXXXXXX
    Dynamic Queue Prefix
    Send Exit Name
    Send Exit Init Parms
    Receive Exit Name
    Receive Exit Init Parms
    Security Exit Name
    Security Exit Init Parms
    Connection Management Properties
    Enable Connection Pooling true
    Minimum Pool Size 1
    Maximum Pool Size 10
    Pool Increment Size 1
    Block Timeout (msec) 1000
    Expire Timeout (msec) 1000

    And my listener looks like this:
    Listener Properties
    Wait Interval 60000
    Filter on Message ID
    Filter on Correlation ID
    Filter on Group ID
    Filter on Sequence Number
    Filter on Offset
    Dead Letter Queue Name
    Dead Letter Queue Manager Name
    Backout Threshold 1
    Open Queue in Shared Mode? true

    Bye for now,
    Sathya KM


    #Integration-Server-and-ESB
    #webMethods
    #Adapters-and-E-Standards


  • 13.  RE: MQ Adapter Backout Rollback problem

    Posted Tue August 16, 2005 09:00 PM

    Nilesh,
    Have you figured out the transaction problem with the MQ 6.0, even I encountered a similar issue.

    Using transactional processing I want to have control on commit/rollback based on condition.

    here is psuedo code…

    startTransaction
    MQ.Put
    condition -1
    commit
    condition -2
    rollback

    this throws an exception
    “com.wm.pkg.art.error.DetailedServiceException: [ART.114.303] Adapter Runtime (Transaction): Unable to commit transaction.
    null”

    TiA

    Cheers*!
    Uday


    #Adapters-and-E-Standards
    #webMethods
    #Integration-Server-and-ESB


  • 14.  RE: MQ Adapter Backout Rollback problem

    Posted Tue August 16, 2005 09:20 PM

    Hey,

    Sorry to say that …I could not figure out that issue. Well, I have moved out of that project since long time back. So I may not help you more.

    • Nilesh

    #Integration-Server-and-ESB
    #Adapters-and-E-Standards
    #webMethods


  • 15.  RE: MQ Adapter Backout Rollback problem

    Posted Tue August 16, 2005 09:34 PM

    Ok…

    Let me explain in detail…

    the flow service has to PUT some data on MQ Queue, the commit/rollback should happen on some condition. To carry on the transactional processing with adapter I’m using the WmART package services.

    the above error is copied from getLastError service and the errorType is nullPointerException. I’m not sure why it is throwing this error, since I’m assign the variable transaction name manually at start,commit and rollback.

    Cheers,
    Uday


    #Adapters-and-E-Standards
    #Integration-Server-and-ESB
    #webMethods


  • 16.  RE: MQ Adapter Backout Rollback problem

    Posted Tue August 16, 2005 11:53 PM

    Folks,
    found the issue for MQ 6.0 transactions.

    When configuring connections from IS Admin for MQ, we have to choose the option “WebSphere MQ Transactional Connection”.

    I was already using the existing connections which are non transactional which are created by the earlier developer, so it failed when I used in transactional processing.

    Hope this helps…

    Cheers
    Uday


    #Adapters-and-E-Standards
    #webMethods
    #Integration-Server-and-ESB


  • 17.  RE: MQ Adapter Backout Rollback problem

    Posted Thu September 29, 2005 08:54 PM

    This has been an old thread and seems like ev one either found a solution or they just moved on. I encountered a similar problem today and was able to find a solution shortly (reading the parameters of the get service helped). I’m putting it here in detail for someone who may stumble over the same block in future. I was testing rollback so I created 2 similar services:

    1. The first service starts a transaction, gets the message then creates a dummy exception. This causes the flow to go to the catch block where the transaction is rolled back
    2. The second service starts a transaction, gets the message, writes this message to a file (test) and finally commits the transaction.

    Each of these two services had appropriate debugLogs to write to server what was happening. Here’s what the services looked like…

    Service One
    Start Transaction [txn_one]
    —Sequence [Success - MAIN]
    -----Sequence [Failure - TRY]
    -------Write start transaction message to server log
    -------Get message from MQ Queue (using Adapter GET service)
    -------Throw Exception (generated this error on purpose to force rollbk)
    -------Commit Transaction [txn_one] (this step will never get executed)
    -----Sequence [Failure - CATCH]
    -------Write rollback message to server log
    -------Rollback Transaction [txn_one]

    Service Two
    Start Transaction [txn_two]
    —Sequence [Success - MAIN]
    -----Sequence [Failure - TRY]
    -------Write start transaction message to server log
    -------Get message from MQ Queue (using Adapter GET service)
    -------write message to a file
    -------Commit Transaction [txn_two]
    -------Write ‘transaction (txn_two) committed’ to server log
    -----Sequence [Failure - CATCH]
    -------Write ‘txn (txn_two) rolled back’ to server log (won’t execute)
    -------Rollback Transaction [txn_two] (this step won’t execute)

    Finally I created a main service that used a put service of adapter to put the message on the queue, then called service one followed by service two. When I executed this main service, I was expecting to see server log messages that the first txn was rolled back and second was committed, besides seeing the contents of actual message in the file… but to my dismay, that didn’t happen. In the server log I saw though the first transaction was rolled back successfully, however, when the second service tried to get the message from the same queue, it encountered an error. The error message said smth about the message being moved to deadLetterQueue. It took me a while before I read the documentation to find the parameter Backout Threshold. The value of this parameter decides how many “reads” (or “gets”) can be made for that message before it is moved to the deadLetterQueue (queue for messages that didn’t reach their detination). The default value of this parameter is 1 when you create the adapter service… thus when the message was rolled back (after reading it once), it was being sent to the deadLetterQueue and not the original queue. I increased the value of this parameter and voila… I got the thigs working just the way I was expecting them to. It might be tricky to decide what should be the value in this parameter, but well, that will depend on how many retries you want.

    PS: If you are using a listener and not the adapter Get service, you will find this parameter under Listener Properties.

    Hope this will be helpful.

    Rohit


    #Adapters-and-E-Standards
    #Integration-Server-and-ESB
    #webMethods


  • 18.  RE: MQ Adapter Backout Rollback problem

    Posted Thu September 29, 2005 09:48 PM

    Oops! Forgot to mention - the MQ Adapter Connection that was used by the Put and the Get services (or by a listener if that is the case in your scenario) has to be MQ Transactional Connection (but of course!!!).

    Rohit


    #Adapters-and-E-Standards
    #webMethods
    #Integration-Server-and-ESB


  • 19.  RE: MQ Adapter Backout Rollback problem

    Posted Thu December 13, 2007 03:10 PM

    Hi Eveyone,

    Even I am facing similar problems. Only difference is I am trying to use synchronous transactional listener. It works perfectly ok in normal sceanario and does not work as expected in erroneous conditions. In case of notification fails to process the message, original message is moving to a dead letter queue which we dont want. We want message to stay in original queue only.

    Thanks & Regards,
    Kapil


    #Integration-Server-and-ESB
    #Adapters-and-E-Standards
    #webMethods