Hello All,

I am currently facing issue that the system threads is getting increased at an alarming rate and too many threads are in WAITING state,

“NonStopCache [SoftwareAG.IS.Core.AMSDEV] Executor Thread-54 for ‘HTTP Handler 127.0.0.1’” Id=70855 in WAITING
at java.lang.Object.wait(Native Method)
- waiting on <0x3794b1fa> (a net.sf.ehcache.constructs.nonstop.NonstopThreadPool$Worker)
at java.lang.Object.wait(Object.java:503)
at net.sf.ehcache.constructs.nonstop.NonstopThreadPool$Worker.waitUntilTaskAvailable(NonstopThreadPool.java:276)
at net.sf.ehcache.constructs.nonstop.NonstopThreadPool$Worker.run(NonstopThreadPool.java:231)
at java.lang.Thread.run(Thread.java:724)

“Service Thread Pool” Id=70810 in WAITING
at java.lang.Object.wait(Native Method)
- waiting on <0x6153e7d5> (a com.wm.app.b2b.server.TMPooledThread)
at java.lang.Object.wait(Object.java:503)
at com.wm.util.pool.PooledThread.run(PooledThread.java:110)
at java.lang.Thread.run(Thread.java:724)

Please find the statistics below for the system threads.

Current	Peak	Limit

Total Sessions 4 16 -
Licensed Sessions 0 6 2147483647
Stateful Sessions 0 5 2147483647
Service Instances 5 37 -
Service Threads 3 9 -

System Threads 551 555 -

Below is the JVM memory settings for the IS,

set JAVA_MIN_MEM=2048M
set JAVA_MAX_MEM=2048M
set JAVA_MAX_PERM_SIZE=512M

Please find the server thread pool info also,

Available Threads 97 % (772 Threads)
Maximum Threads 800
Minimum Threads 50
Available Threads Warning Threshold 15 % (120 Threads)
Scheduler Thread Throttle 75 % (600 Threads)
Scheduler Current Threads 0

My question to the team is why the threads are staying in WAITING state instead of getting released.

Also we are seeing the below GC related message in the ehcache logs

ClientID[21]: Lock GC collected 4 garbage locks

Please help

Thanks and Regards
Abir Banerjee

See if there are any service which are running from a long time or if it is hung.

Restart the IS so that the waiting threads will be released.