"On some Docker version (Docker Engine 20.10.x) the memory management is different. You should run this command line: docker run -e LICENSE=accept -p 9060:9060 -p 9443:9443 -e JVM_ARGS='-Xmx3000m' -e SAMPLE=true icr.io/odm-k8s/odm:8.11"
Original Message:
Sent: Fri June 10, 2022 12:10 AM
From: Kian Seng Goh
Subject: Exception in thread "Attach API wait loop" java/lang/OutOfMemoryError: Java heap space
another one,
[INFO ] Start the maintenance task of the ruleset cache on 6/10/22 5:54:30 AM.
[INFO ] Start the maintenance task of the ruleset cache on 6/10/22 5:57:19 AM.
[INFO ] Start the maintenance task of the ruleset cache on 6/10/22 5:59:30 AM.
[INFO ] Start the maintenance task of the ruleset cache on 6/10/22 6:02:19 AM.
JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2022/06/10 04:03:53 - please wait.
JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2022/06/10 04:03:53 - please wait.
JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2022/06/10 04:03:53 - please wait.
JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2022/06/10 04:03:53 - please wait.
JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2022/06/10 04:03:53 - please wait.
JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2022/06/10 04:03:53 - please wait.
JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2022/06/10 04:03:53 - please wait.
JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2022/06/10 04:03:53 - please wait.
JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2022/06/10 04:03:53 - please wait.
JVMDUMP032I JVM requested Heap dump using '/opt/ibm/wlp/output/defaultServer/heapdump.20220610.040353.112.0001.phd' in response to an event
JVMDUMP010I Heap dump written to /opt/ibm/wlp/output/defaultServer/heapdump.20220610.040353.112.0001.phd
JVMDUMP032I JVM requested Heap dump using '/opt/ibm/wlp/output/defaultServer/heapdump.20220610.040353.112.0002.phd' in response to an event
JVMDUMP010I Heap dump written to /opt/ibm/wlp/output/defaultServer/heapdump.20220610.040353.112.0002.phd
JVMDUMP032I JVM requested Heap dump using '/opt/ibm/wlp/output/defaultServer/heapdump.20220610.040353.112.0003.phd' in response to an event
JVMDUMP010I Heap dump written to /opt/ibm/wlp/output/defaultServer/heapdump.20220610.040353.112.0003.phd
JVMDUMP032I JVM requested System dump using '/opt/ibm/wlp/output/defaultServer/core.20220610.040353.112.0004.dmp' in response to an event
JVMDUMP010I System dump written to /opt/ibm/wlp/output/defaultServer/core.20220610.040353.112.0004.dmp
JVMDUMP032I JVM requested Snap dump using '/opt/ibm/wlp/output/defaultServer/Snap.20220610.040353.112.0010.trc' in response to an event
JVMDUMP010I Snap dump written to /opt/ibm/wlp/output/defaultServer/Snap.20220610.040353.112.0010.trc
JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
JVMDUMP032I JVM requested Heap dump using '/opt/ibm/wlp/output/defaultServer/heapdump.20220610.040353.112.0005.phd' in response to an event
JVMDUMP010I Heap dump written to /opt/ibm/wlp/output/defaultServer/heapdump.20220610.040353.112.0005.phd
JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
JVMDUMP032I JVM requested Java dump using '/opt/ibm/wlp/output/defaultServer/javacore.20220610.040353.112.0006.txt' in response to an event
JVMDUMP010I Java dump written to /opt/ibm/wlp/output/defaultServer/javacore.20220610.040353.112.0006.txt
JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
JVMDUMP032I JVM requested Java dump using '/opt/ibm/wlp/output/defaultServer/javacore.20220610.040353.112.0008.txt' in response to an event
JVMDUMP010I Java dump written to /opt/ibm/wlp/output/defaultServer/javacore.20220610.040353.112.0008.txt
JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
JVMDUMP032I JVM requested Java dump using '/opt/ibm/wlp/output/defaultServer/javacore.20220610.040353.112.0007.txt' in response to an event
JVMDUMP010I Java dump written to /opt/ibm/wlp/output/defaultServer/javacore.20220610.040353.112.0007.txt
JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
JVMDUMP032I JVM requested Java dump using '/opt/ibm/wlp/output/defaultServer/javacore.20220610.040353.112.0009.txt' in response to an event
JVMDUMP010I Java dump written to /opt/ibm/wlp/output/defaultServer/javacore.20220610.040353.112.0009.txt
JVMDUMP032I JVM requested Snap dump using '/opt/ibm/wlp/output/defaultServer/Snap.20220610.040353.112.0011.trc' in response to an event
JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
JVMDUMP010I Snap dump written to {nothing to snap}
JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
JVMDUMP032I JVM requested Snap dump using '/opt/ibm/wlp/output/defaultServer/Snap.20220610.040353.112.0013.trc' in response to an event
JVMDUMP010I Snap dump written to {nothing to snap}
JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
JVMDUMP032I JVM requested Snap dump using '/opt/ibm/wlp/output/defaultServer/Snap.20220610.040353.112.0012.trc' in response to an event
JVMDUMP010I Snap dump written to {nothing to snap}
JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
[err] Exception in thread "MVStore background writer nio:/opt/ibm/wlp/usr/servers/defaultServer/dbdata/rtsdb.mv.db"
[err] java.lang.OutOfMemoryError
[err] :
[err] Java heap space
[ERROR ] Unexpected exception in Temporary snapshot cleanup thread
Java heap space
[err] Exception in thread "Default Executor-thread-77"
[err] java.lang.OutOfMemoryError
[err] :
[err] Java heap space
[err] at
[err] java.util.concurrent.ConcurrentLinkedQueue
[err] .
[err] offer
[err] (
[err] ConcurrentLinkedQueue.java
[err] :
[err] 3
[err] 4
[err] 3
[err] )
[err] at
[err] java.util.concurrent.ConcurrentLinkedQueue
[err] .
[err] add
[err] (
[err] ConcurrentLinkedQueue.java
[err] :
[err] 3
[err] 0
[err] 8
[err] )
[err] at
[err] com.ibm.ws.threading.internal.BoundedBuffer
[err] .
[err] waitGet_
[err] (
[err] BoundedBuffer.java
[err] :
[err] 1
[err] 7
[err] 5
[err] Exception in thread "Connection evictor"
[err] Exception in thread "Connection evictor"
[err] Exception in thread "Scheduled Executor-thread-1"
[err] at
[err] java.util.concurrent.ConcurrentHashMap$ValuesView
[err] .
[err] )
[err] at
[err] com.ibm.ws.threading.internal.BoundedBuffer
[err] .
[err] take
[err] (
[err] BoundedBuffer.java
[err] :
[err] 6
[err] 4
[err] 7
[err] )
[err] iterator
[err] (
[err] ConcurrentHashMap.java
[err] at
[err] java.util.concurrent.ThreadPoolExecutor
[err] .
[err] getTask
[err] (
[err] ThreadPoolExecutor.java
[err] :
[err] 1
[err] 0
[err] 8
[err] 5
[err] )
[err] at
[err] java.util.concurrent.ThreadPoolExecutor
[err] .
[err] runWorker
[err] (
[err] ThreadPoolExecutor.java
[err] :
[err] 1
[err] 1
[err] 4
[err] 5
[err] )
[err] at
[err] java.util.concurrent.ThreadPoolExecutor$Worker
[err] :
[err] .
[err] run
[err] (
[err] ThreadPoolExecutor.java
[err] :
[err] 6
[err] 3
[err] 5
[err] )
[err] at
[err] java.lang.Thread
[err] .
[err] run
[err] (
[err] Thread.java
[err] :
[err] 8
[err] 2
[err] 5
[err] )
[err] 4
[err] 6
[err] 9
[err] 4
[err] )
[err] at
[err] org.h2.mvstore.MVStore
[err] .
[err] compactGetOldChunks
[err] (
[err] MVStore.java
[err] :
[err] 1
[err] 8
[err] 2
[err] 7
[err] )
[err] at
[err] org.h2.mvstore.MVStore
[err] .
[err] compact
[err] (
[err] MVStore.java
[err] :
[err] 1
[err] 8
[err] 0
[err] 5
[err] )
[err] at
[err] org.h2.mvstore.MVStore
[err] .
[err] writeInBackground
[err] (
[err] MVStore.java
[err] :
[err] 2
[err] 5
[err] 1
[err] 2
[err] )
[err] at
[err] <unknown class>
[err] .
[err] <unknown method>
[err] (
[err] Unknown Source
[err] )
[err] java.lang.OutOfMemoryError
[err] :
[err] Java heap space
[err] at
[err] org.apache.http.pool.AbstractConnPool
[err] .
[err] closeExpired
[err] (
[err] AbstractConnPool.java
[err] :
[err] 6
[err] 4
[err] 8
[err] )
[err] at
[err] org.apache.http.impl.conn.PoolingHttpClientConnectionManager
[err] .
[err] closeExpiredConnections
[err] (
[err] PoolingHttpClientConnectionManager.java
[err] :
[err] 4
[err] 4
[err] 9
[err] )
[err] at
[err] org.apache.http.impl.client.IdleConnectionEvictor$1
[err] .
[ERROR ] The WADL file was not found.
javax.xml.bind.JAXBException incompatible with org.w3c.dom.Element
[err] run
[err] (
------------------------------
Kian Seng Goh
Original Message:
Sent: Thu June 09, 2022 11:34 PM
From: Kian Seng Goh
Subject: Exception in thread "Attach API wait loop" java/lang/OutOfMemoryError: Java heap space
Hi
i keep seeing this error when I am working with the Loan Server using the docker image, Docker Hub
Any idea why?
[INFO ] Building ruleapp: deploy
[INFO ] {"version":"1.0","id":"4b242aae-227d-4788-89c3-9dd144b8a8c1","author":"odmAdmin","date":1654831602999,"type":"SnapshotCreated","content":[{"id":"b3e63e3a-f2b1-40ee-b4e5-5128bc31c31c","internalId":"dsm.DsDeploymentBsln:36:36","name":"deploy_2022-06-10T03_26_39Z","parentId":"513dc201-b051-45ee-a6a7-5c7e47ae974b","documentation":null,"buildMode":"DecisionEngine","kind":"DeploymentSnapshot"}],"details":[{"targetURL":"http://172.17.0.2:9060/decisioncenter/t/library#overviewsnapshot?id=dsm.DsDeploymentBsln%3A36%3A36&datasource=jdbc%2FilogDataSource&baselineId=dsm.DsDeploymentBsln%3A36%3A36"}],"sourceName":"Decision Center","sourceLink":"http://172.17.0.2:9060/decisioncenter?datasource=jdbc%2FilogDataSource","project":{"id":"c991c2be-bd22-4559-afa3-cd5e2c906daa","internalId":"brm.RuleProject:33:33","name":"KianSengRule","buildMode":"DecisionEngine","advancedProperties":null},"decisionService":{"id":"c991c2be-bd22-4559-afa3-cd5e2c906daa","internalId":"brm.RuleProject:33:33","name":"KianSengRule","buildMode":"DecisionEngine","advancedProperties":null}}
[INFO ] {"version":"1.0","id":"f524e821-3286-4f1b-b5f9-852f8d77c9ae","author":"odmAdmin","date":1654831603045,"type":"SnapshotUpdated","content":[{"id":"b3e63e3a-f2b1-40ee-b4e5-5128bc31c31c","internalId":"dsm.DsDeploymentBsln:36:36","name":"deploy_2022-06-10T03_26_39Z","parentId":"513dc201-b051-45ee-a6a7-5c7e47ae974b","documentation":null,"buildMode":"DecisionEngine","kind":"DeploymentSnapshot"}],"details":[{"targetURL":"http://172.17.0.2:9060/decisioncenter/t/library#overviewsnapshot?id=dsm.DsDeploymentBsln%3A36%3A36&datasource=jdbc%2FilogDataSource&baselineId=dsm.DsDeploymentBsln%3A36%3A36"}],"sourceName":"Decision Center","sourceLink":"http://172.17.0.2:9060/decisioncenter?datasource=jdbc%2FilogDataSource","project":{"id":"c991c2be-bd22-4559-afa3-cd5e2c906daa","internalId":"brm.RuleProject:33:33","name":"KianSengRule","buildMode":"DecisionEngine","advancedProperties":null},"decisionService":{"id":"c991c2be-bd22-4559-afa3-cd5e2c906daa","internalId":"brm.RuleProject:33:33","name":"KianSengRule","buildMode":"DecisionEngine","advancedProperties":null}}
[INFO ] Building ruleset: findMyMRTId using Decision Engine
[INFO ] Extracting rules...
[INFO ] Rules extraction took: 53ms
JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2022/06/10 03:26:57 - please wait.
JVMDUMP032I JVM requested System dump using '/opt/ibm/wlp/output/defaultServer/core.20220610.032657.112.0001.dmp' in response to an event
JVMDUMP010I System dump written to /opt/ibm/wlp/output/defaultServer/core.20220610.032657.112.0001.dmp
JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2022/06/10 03:27:00 - please wait.
JVMDUMP032I JVM requested Heap dump using '/opt/ibm/wlp/output/defaultServer/heapdump.20220610.032700.112.0002.phd' in response to an event
JVMDUMP010I Heap dump written to /opt/ibm/wlp/output/defaultServer/heapdump.20220610.032700.112.0002.phd
JVMDUMP032I JVM requested Heap dump using '/opt/ibm/wlp/output/defaultServer/heapdump.20220610.032657.112.0003.phd' in response to an event
JVMDUMP010I Heap dump written to /opt/ibm/wlp/output/defaultServer/heapdump.20220610.032657.112.0003.phd
JVMDUMP032I JVM requested Java dump using '/opt/ibm/wlp/output/defaultServer/javacore.20220610.032700.112.0004.txt' in response to an event
JVMDUMP010I Java dump written to /opt/ibm/wlp/output/defaultServer/javacore.20220610.032700.112.0004.txt
JVMDUMP032I JVM requested Snap dump using '/opt/ibm/wlp/output/defaultServer/Snap.20220610.032700.112.0006.trc' in response to an event
JVMDUMP010I Snap dump written to /opt/ibm/wlp/output/defaultServer/Snap.20220610.032700.112.0006.trc
JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
JVMDUMP032I JVM requested Java dump using '/opt/ibm/wlp/output/defaultServer/javacore.20220610.032657.112.0005.txt' in response to an event
JVMDUMP010I Java dump written to /opt/ibm/wlp/output/defaultServer/javacore.20220610.032657.112.0005.txt
JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2022/06/10 03:27:07 - please wait.
JVMDUMP032I JVM requested Snap dump using '/opt/ibm/wlp/output/defaultServer/Snap.20220610.032657.112.0007.trc' in response to an event
JVMDUMP010I Snap dump written to /opt/ibm/wlp/output/defaultServer/Snap.20220610.032657.112.0007.trc
JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2022/06/10 03:27:11 - please wait.
JVMDUMP032I JVM requested Heap dump using '/opt/ibm/wlp/output/defaultServer/heapdump.20220610.032707.112.0008.phd' in response to an event
JVMDUMP010I Heap dump written to /opt/ibm/wlp/output/defaultServer/heapdump.20220610.032707.112.0008.phd
JVMDUMP032I JVM requested Heap dump using '/opt/ibm/wlp/output/defaultServer/heapdump.20220610.032711.112.0009.phd' in response to an event
JVMDUMP010I Heap dump written to /opt/ibm/wlp/output/defaultServer/heapdump.20220610.032711.112.0009.phd
JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2022/06/10 03:27:17 - please wait.
JVMDUMP032I JVM requested Snap dump using '/opt/ibm/wlp/output/defaultServer/Snap.20220610.032711.112.0012.trc' in response to an event
JVMDUMP010I Snap dump written to {nothing to snap}
JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
JVMDUMP032I JVM requested Java dump using '/opt/ibm/wlp/output/defaultServer/javacore.20220610.032717.112.0011.txt' in response to an event
JVMDUMP010I Java dump written to /opt/ibm/wlp/output/defaultServer/javacore.20220610.032717.112.0011.txt
JVMDUMP032I JVM requested Java dump using '/opt/ibm/wlp/output/defaultServer/javacore.20220610.032707.112.0010.txt' in response to an event
JVMDUMP010I Java dump written to /opt/ibm/wlp/output/defaultServer/javacore.20220610.032707.112.0010.txt
JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
JVMDUMP032I JVM requested Snap dump using '/opt/ibm/wlp/output/defaultServer/Snap.20220610.032717.112.0013.trc' in response to an event
JVMDUMP010I Snap dump written to {nothing to snap}
JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
Exception in thread "Attach API wait loop" java/lang/OutOfMemoryError: Java heap space
at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:867)
at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:861)
at java/lang/Thread.uncaughtException (Thread.java:1341)
Exception in thread "Executor Service Control Timer" Exception in thread "zip file reaper" java/lang/OutOfMemoryError: Java heap space
at java/lang/Throwable.printStackTrace (Throwable.java:555)
at java/lang/Throwable.printStackTraceHelper (Throwable.java:338)
at java/lang/Throwable.printStackTrace (Throwable.java:307)
at java/lang/Throwable.printStackTrace (Throwable.java:250)
Exception in thread "Temporary snapshot cleanup thread" java/lang/OutOfMemoryError: Java heap space
at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:867)
at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:861)
at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:861)
at java/lang/Thread.uncaughtException (Thread.java:1341)
java/lang/OutOfMemoryError: Java heap space
at java/util/zip/ZipFile.close (NativeMethod:4294967295)
at java/util/zip/ZipFile.close (ZipFile.java:714)
at com/ibm/ws/artifact/zip/cache/internal/ZipFileUtils$2.run (ZipFileUtils.java:65)
at com/ibm/ws/artifact/zip/cache/internal/ZipFileUtils$2.run (ZipFileUtils.java:62)
at java/security/AccessController.doPrivileged (AccessController.java:738)
at com/ibm/ws/artifact/zip/cache/internal/ZipFileUtils.closeZipFile (ZipFileUtils.java:62)
at com/ibm/ws/artifact/zip/cache/internal/ZipFileData.closeZipFile (ZipFileData.java:766)
at com/ibm/ws/artifact/zip/cache/internal/ZipFileReaper.fullyClose (ZipFileReaper.java:1009)
at com/ibm/ws/artifact/zip/cache/internal/ZipFileReaper.reap (ZipFileReaper.java:1419)
at com/ibm/ws/artifact/zip/cache/internal/ZipFileReaper$ReaperRunnable.run (ZipFileReaper.java:516)
at java/lang/Thread.run (Thread.java:825)
Exception in thread "Long SQL Statement Monitoring Thread" Exception in thread "Connection evictor" java/lang/OutOfMemoryError: Java heap space
at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:867)
at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:861)
at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:861)
at java/lang/Thread.uncaughtException (Thread.java:1341)
Exception in thread "Search index updater" Exception in thread "Connection evictor" java/lang/OutOfMemoryError: Java heap space
at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:867)
at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:861)
at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:861)
at java/lang/Thread.uncaughtException (Thread.java:1341)
Exception in thread "Timer-5" java/lang/OutOfMemoryError: Java heap space
at java/lang/Throwable.printStackTrace (Throwable.java:555)
at java/lang/Throwable.printStackTraceHelper (Throwable.java:338)
at java/lang/Throwable.printStackTrace (Throwable.java:307)
at java/lang/Throwable.printStackTrace (Throwable.java:250)
------------------------------
Kian Seng Goh
------------------------------