Hung Thread on WebSphere 6.0

--------------------------------

I am receiving the following ThreadMonitor message in my SystemOut.log file:

> [4/2/09 10:17:53:690 CDT] 00000026 ThreadMonitor W
> WSVR0605W: Thread /"WebContainer : 177/" (000000df) has
>been active for 707630 milliseconds and may be hung.
> There is/are 21 thread(s) in total in the server that
>may be hung.

I have seen documentation which suggests to issue /"kill -3 [pid]/" and see from native_stdout.log what /"Thread /"WebContainer : 177/"/" is doing.

My questions are the following:

1. What does Thread /"WebContainer : 177/" (000000df) mean exactly? Is this the Web container which is responsible for thread pool creation at the application server level which ncludes servlet, JSP, EJB, dynamic page creation, and back-end pass-through processing. What is (000000df) mean?

2. Does killing the pid mean I must kill the pid for the WebSphere Server or the WebSphere Application Server?

3. Is it possible this could be related to a subexclusive lock (SX) on a table in the oracle database?

Your help is greatly appreciated.

-Andrew

--------------------------------

Posted By: aplata at Apr 4 2009 12:49AM

kill -3 initiates a thread dump. You should examine the dump with the thread dump analyzer. It is possible that you will see a thread that is /"blocking/" other threads. You will more than likely see on thread doing work, while the other threads are in a /"CW/" connection wait status waiting on the thread monitor.

--------------------------------

Posted By: brewdude at Apr 4 2009 4:59PM

/"WebContainer : 177/" is just the name of the thread from the thread pool.

The message you see if put out by the hung thread detector. If a thread has been active for a certain (configurable) amount of time, this message will be logged. If the thread later clears and completes, another message will be logged indicating this.

The most important piece of diagnostic information is a javacore from the running process. This is done by issuing a 'kill -3' against the PID. If the process is configured properly, you should get a javacore - and the process will not be terminated.

You can then look in the javacore for the thread that has been reported as /"hung/" - in this case, WebContainer : 177. There should be a java stack for the code and you can see what is executing that is not completing. (Sometimes the javacore will not have that java stack for the threads. In this case, you will need to get a system core and process - - not nearly as easy to create, process or analyze)

Not sure exactly what you mean by the lock - - sounds like some type of lock on the db side? If the thread is reading from the database, it could be possible that a db lock (or something else at the db side) is causing the read not to complete. The thread will stay there, waiting for the read to complete and thus become hung.

--------------------------------

Posted By: cathy at Apr 20 2009 10:41PM