BPM, Workflow, and Case

BPM, Workflow, and Case

Come for answers. Stay for best practices. All we’re missing is you.

 View Only
  • 1.  Hung threads

    Posted Mon March 30, 2020 01:42 PM
    Hi All,

    We are seeing hung threads and deadlocks in our bpm production. We do not see any long running services in process admin console -> process/ service monitor .

    How to proceed with this to find out the root cause ?

    Thread "WebContainer : 33" (00000195) has been active for 729801 milliseconds and may be hung.  There is/are 20 thread(s) in total in the server that may be hung.

                  at java.net.SocketInputStream.socketRead0(Native Method)

    Transaction (Process ID 124) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.

    [3/30/20 9:01:02:605 EDT] 00002eeb SystemErr     R        at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:216)



    ------------------------------
    RK
    ------------------------------


  • 2.  RE: Hung threads

    Posted Mon March 30, 2020 02:55 PM
    Rameshkumar -

    You've not provided us much detail to work with, nor have to told us what you have / have not done in trying to research the underlying problem.

    • Are you using any IID sourced parts in your solution, or is this all BPMN models?  
    • Have you checked the code for the existence of Multi-Instance Loops?  
    • Have you checked the Event Manger to see if there is a backlog there (which will be an indicator of the MIL problem)?
    • Does this happen at a regular time each day, or randomly?
    • What else is the system doing before this occurs?  Have you checked for resource leaks etc?


    ------------------------------
    Andrew Paier
    ------------------------------



  • 3.  RE: Hung threads

    Posted Mon March 30, 2020 03:47 PM
    Hi Andrew,

    Please find my response below. 
     
    • Are you using any IID sourced parts in your solution, or is this all BPMN models?   - No IID involved here
    • Have you checked the code for the existence of Multi-Instance Loops?    - No MIL 
    • Have you checked the Event Manger to see if there is a backlog there (which will be an indicator of the MIL problem)? - No backlog in Event Manager
    • Does this happen at a regular time each day, or randomly?  - It happened only today. New thing what I observed is. We have seen 3000 new instances created for one process app and currently 6k + active instances in total. when user tried to open the work tab, it is not loaded properly (we are in 8.5.6)
    • What else is the system doing before this occurs?  Have you checked for resource leaks etc?  - Please let me know, how to check the resource leaks


    ------------------------------
    Rameshkumar Chandrasekar
    ------------------------------



  • 4.  RE: Hung threads

    Posted Thu April 02, 2020 04:13 AM

    Hi Rameshkumar,

    There is a good description of java.net.SocketInputStream.socketRead0 in this article https://dzone.com/articles/threads-stuck-in-javanetsocketinputstreamsocketrea. But basically BPM is waiting for a reply from an external source, mostly likely in my experience a web services call or a database call. Some years ago when I was heavily involved in BPM we did from time to time experience DB locking issues. With help from the DBAs we could get logs and traces from DB2 and diagnose the problem. From the second part of the message you posted it looks like you may have a database issue. My suggestion would be to work with the DBAs to see if they cannot throw some light on the issue. No real experience with SQLServer so cannot be more helpful. There are also some BPM specific traces that can be enabled in WebSphere which may also give insight on the issue. Unfortunately I cannot provide more help at this stage as my focus for the last couple of years has been anything other than BPM.



    ------------------------------
    Darryl Frost
    Middleware Architect
    IBM
    Sandton
    South Africa
    ------------------------------



  • 5.  RE: Hung threads

    Posted Thu April 02, 2020 03:04 PM
    Thank you Darryl. I will look into it

    ------------------------------
    Rameshkumar Chandrasekar
    ------------------------------