DataPower

DataPower

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
  • 1.  Datapower not able to process messages from queue

    Posted 3 days ago

    Dears,

    Currently, we are facing a weird issue where Datapower is stops reading messages from the queue though there are more than 200K messages in the queue. All the corresponding objects like MQ QM group, QM and Handler are up. 

    Whenever there is a spike in the traffic , this issue is happening and we have more then 5 nodes and is happening with all the nodes. As soon as the Datapower appliances are restarted, the messages are getting processed. 

    There is no spike in CPU or memory usage while the issue is happening . We could not find any error logs which suggest connection related issues.

    We are currently using Datapower 10.5.0.17 and MQ v9. The system was working without any issue for more than 5 months and this issue started happening randomly since 2 weeks and the last firmware update has happened more than 1 month ago.

    Please let me know what could be the issue.



    ------------------------------
    R K
    ------------------------------


  • 2.  RE: Datapower not able to process messages from queue

    Posted 3 days ago

    SLM or other rate limiting policy kicking in?



    ------------------------------
    Joseph Morgan
    CEO - Independent
    ------------------------------



  • 3.  RE: Datapower not able to process messages from queue

    Posted 2 days ago

    Adding to Joseph's question that anything on the MQ logs or any limitation on the MQ side? 



    ------------------------------
    Hermanni Pernaa
    Solutions Architect
    Digia Plc
    Helsinki
    ------------------------------



  • 4.  RE: Datapower not able to process messages from queue

    Posted 2 days ago
    Edited by ServiceTeam Access _Management 2 days ago

    There is not a lot of information to go on. But you mentioning that there is nothing in the logs (both on datapower and MQ as well?) and mentioning that all instances stop working at the same time. Please have a look at the MQ side if there are still running channels to/from the datapower. If that is the case than my bet would be the firewall. A firewall that thinks that sessions have been runnning for too long. And cuts them off without informing anything (so both Datapower and MQ are thinking the channels are still valid but there is no triggering anymore). We've seen that happening to us with another product (not datapower). Do you have the keep alive setting on the MQ channel set ?
    Hope these suggestions help in knowing where to look for a solution.

    If the channel is not running on the MQ side you may have a look if the queuemanager has been restarted. While the client (Datapower) is still waiting to get a trigger on a channel that is no longer running on the MQ side (but DP is still listening on an orphaned channel)

    ------------------------------
    ServiceTeam Access _Management
    ------------------------------



  • 5.  RE: Datapower not able to process messages from queue

    Posted 2 days ago

    Yes, the Keep Alive setting has been enabled on MQ. Even I was thinking about the firewall. But the issue got fixed as soon as we restarted the server.(Domain Restart did not help).I want to know how it got fixed when datapower has been restarted.

    There has been no SLM or rate Limiting .

    Only Notable change is a new client has been onboarded and the there has been a traffic spike because of that. 



    ------------------------------
    R K
    ------------------------------