WebSphere Application Server & Liberty

WebSphere Application Server & Liberty

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
  • 1.  Hung threads in AppServer

    Posted 10/08/13 12:36 PM
    My application server is processing large chunk of  data everymdinght, during this period  I am seeing hungthreads in the systemout logs.  However   the thread unhungs itself after processing data after about hour and half. Just wanted to know if I can prevent this.

    Here is the stack trace.

    10/8/13 0:52:57:013 CDT] 00000023 ThreadMonitor W   WSVR0605W: Thread "MessageListenerThreadPool : 0" (00000025) has been active for 621883 milliseconds and may be hung.  There is/are 1 thread(s) in total in the server that may be hung.
            at java.lang.Thread.sleep(Native Method)
            at com.norkom.acquisition.business.scheduler.batchbridge.BatchBridgeJob.sleep(BatchBridgeJob.java:573)
            at com.norkom.acquisition.business.scheduler.batchbridge.BatchBridgeJob.batchBridgePending(BatchBridgeJob.java:601)
            at com.norkom.acquisition.business.scheduler.batchbridge.BatchBridgeJob.batchBridge(BatchBridgeJob.java:539)
            at com.norkom.acquisition.business.scheduler.batchbridge.BatchBridgeJob.execute(BatchBridgeJob.java:316)
            at com.norkom.acquisition.business.scheduler.Schedule.executeJob(Schedule.java:328)
            at com.norkom.acquisition.business.scheduler.Schedule.executeJobs(Schedule.java:254)
            at com.norkom.acquisition.business.scheduler.Schedule.chainJobs(Schedule.java:229)
            at com.norkom.acquisition.business.scheduler.Schedule.executeJobs(Schedule.java:260)
            at com.norkom.acquisition.business.scheduler.Schedule.chainJobs(Schedule.java:229)
            at com.norkom.acquisition.business.scheduler.Schedule.executeJobs(Schedule.java:260)
            at com.norkom.acquisition.business.scheduler.Schedule.chainJobs(Schedule.java:229)
            at com.norkom.acquisition.business.scheduler.Schedule.executeJobs(Schedule.java:260)
            at com.norkom.acquisition.business.scheduler.Schedule.fireTrigger(Schedule.java:567)
            at com.norkom.acquisition.business.application.ScheduleMessageConsumer.fireTrigger(ScheduleMessageConsumer.java:229)
            at com.norkom.acquisition.business.application.ScheduleMessageConsumer.trigger(ScheduleMessageConsumer.java:215)
            at com.norkom.acquisition.business.application.ScheduleMessageConsumer.executeMessage(ScheduleMessageConsumer.java:82)
            at com.norkom.acquisition.services.mdb.ScheduleMdb.onMessage(ScheduleMdb.java:78)
            at com.ibm.ejs.jms.listener.MDBWrapper$PriviledgedOnMessage.run(MDBWrapper.java:302)
            at com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.java:63)
            at com.ibm.ejs.jms.listener.MDBWrapper.callOnMessage(MDBWrapper.java:271)
            at com.ibm.ejs.jms.listener.MDBWrapper.onMessage(MDBWrapper.java:240)
            at com.ibm.mq.jms.MQSession$FacadeMessageListener.onMessage(MQSession.java:147)
            at com.ibm.msg.client.jms.internal.JmsSessionImpl.run(JmsSessionImpl.java:2846)
            at com.ibm.mq.jms.MQSession.run(MQSession.java:862)
            at com.ibm.ejs.jms.JMSSessionHandle.run(JMSSessionHandle.java:1055)
            at com.ibm.ejs.jms.listener.ServerSession.connectionConsumerOnMessage(ServerSession.java:1082)
            at com.ibm.ejs.jms.listener.ServerSession.onMessage(ServerSession.java:752)
            at com.ibm.ejs.jms.listener.ServerSession.dispatch(ServerSession.java:718)
            at sun.reflect.GeneratedMethodAccessor293.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            at java.lang.reflect.Method.invoke(Method.java:597)
            at com.ibm.ejs.jms.listener.ServerSessionDispatcher.dispatch(ServerSessionDispatcher.java:47)
            at com.ibm.ejs.container.MDBWrapper.onMessage(MDBWrapper.java:98)
            at com.ibm.ejs.container.MDBWrapper.onMessage(MDBWrapper.java:136)
            at com.ibm.ejs.jms.listener.ServerSession.run(ServerSession.java:574)
            at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1646)
    [10/8/13 2:29:24:352 CDT] 00000025 ThreadMonitor W   WSVR0606W: Thread "MessageListenerThreadPool : 0" (00000025) was previously reported to be hung but has completed.  It was active for approximately 6409225 milliseconds.  There is/are 0 thread(s) in total in the server that still may be hung


  • 2.  Hung threads in AppServer

    Posted 10/08/13 12:55 PM
    message processing is talking long time. Is your application expecting a reply after making that call ? Thread trace is showning thread in sleep state.

    Good thing is that the hung threads are getting released after xx amount of time.


  • 3.  Hung threads in AppServer

    Posted 10/08/13 01:37 PM
    No I do not think if anything is waiting on this.

    Is there  a way to prevent this?


  • 4.  Hung threads in AppServer

    Posted 10/09/13 11:35 AM
    If this is typical behavior, you may wish to adjust the detection parameters for the hung thread state in the JVM.  Additional information can be found here: 
    pic.dhe.ibm.com/infocenter/wasinfo/v8r0/...
     
    As a warning, carefully consider the ramifications of such actions.


  • 5.  Hung threads in AppServer

    Posted 10/10/13 04:59 PM
    Thanks Eric, I Will check into this.