WebSphere Application Server & Liberty

Encountering issues with WebSphere Application Server process terminates with no footprint.

Environment
- Windows 2016
- WebSphere Application Server Traditional 8.5.5 FP 22 (also occurring on FP21, FP18)
- WAS running as a windows service using a Domain ID

Issue
- WAS process terminates intermittently with no warning, no core dump, no ffdc, no logs indicating cause of termination
- All application logs, system logs, native logs does not show any anomaly
- Can occur at at time, even when there is no use of the hosted application or within an hour of service/server restart and no utilization.

Troubleshooting done so far
- Enabled xtrace with following triggers. Issue occurred but no logs or JVM core dump as well.
-Dcom.ibm.ws.runtime.dumpShutdown=true
-Xtrace:trigger=method{java/lang/System.exit,javadump}
-Xtrace:trigger=method{java/lang/Runtime.exit,javadump}

Seeking for advice on what potentially could be the issue or troubleshooting that can be done.

------------------------------
Kam Tyng Loo
------------------------------

Hello Kam,

Any type of antivirus? at system viewer do you see any type of alert about process killed.

If you have at application level an "SystemExit(0)" during some exception your -Xtrace:trigger=method({java/kang/Sytem.exit,javadump} need to "catch".

Hope this helps. Tell us if you need more support.

Regards

------------------------------
Gabriel Aberasturi
Versia tecnologias emergentes
------------------------------

Original Message:
Sent: Fri September 23, 2022 06:52 AM
From: Kam Tyng Loo
Subject: Troubleshooting Websphere Application Server 8.5.5 process terminates with no logs

Encountering issues with WebSphere Application Server process terminates with no footprint.

Environment
- Windows 2016
- WebSphere Application Server Traditional 8.5.5 FP 22 (also occurring on FP21, FP18)
- WAS running as a windows service using a Domain ID

Issue
- WAS process terminates intermittently with no warning, no core dump, no ffdc, no logs indicating cause of termination
- All application logs, system logs, native logs does not show any anomaly
- Can occur at at time, even when there is no use of the hosted application or within an hour of service/server restart and no utilization.

Troubleshooting done so far
- Enabled xtrace with following triggers. Issue occurred but no logs or JVM core dump as well.
-Dcom.ibm.ws.runtime.dumpShutdown=true
-Xtrace:trigger=method{java/lang/System.exit,javadump}
-Xtrace:trigger=method{java/lang/Runtime.exit,javadump}

Seeking for advice on what potentially could be the issue or troubleshooting that can be done.

------------------------------
Kam Tyng Loo
------------------------------

Hi Gabriel, 

We do have antivirus, have checked the Windows Event viewer and there is no event where it mentions that the websphere process would be killed. 
Do we have a known history that antivirus might be causing this?

With regards to tracing, is the following trigger different from what you have specified? I have the below enabled and it did not catch the issue.
-Xtrace:trigger=method{java/lang/System.exit,javadump}

Thanks.

Regards, 
Justin

------------------------------
Kam Tyng Loo
------------------------------

Original Message:
Sent: Mon September 26, 2022 04:14 AM
From: Gabriel Aberasturi
Subject: Troubleshooting Websphere Application Server 8.5.5 process terminates with no logs

Hello Kam,

Any type of antivirus? at system viewer do you see any type of alert about process killed.

If you have at application level an "SystemExit(0)" during some exception your -Xtrace:trigger=method({java/kang/Sytem.exit,javadump} need to "catch".

Hope this helps. Tell us if you need more support.

Regards

------------------------------
Gabriel Aberasturi
Versia tecnologias emergentes

Original Message:
Sent: Fri September 23, 2022 06:52 AM
From: Kam Tyng Loo
Subject: Troubleshooting Websphere Application Server 8.5.5 process terminates with no logs

Encountering issues with WebSphere Application Server process terminates with no footprint.

Environment
- Windows 2016
- WebSphere Application Server Traditional 8.5.5 FP 22 (also occurring on FP21, FP18)
- WAS running as a windows service using a Domain ID

Issue
- WAS process terminates intermittently with no warning, no core dump, no ffdc, no logs indicating cause of termination
- All application logs, system logs, native logs does not show any anomaly
- Can occur at at time, even when there is no use of the hosted application or within an hour of service/server restart and no utilization.

Troubleshooting done so far
- Enabled xtrace with following triggers. Issue occurred but no logs or JVM core dump as well.
-Dcom.ibm.ws.runtime.dumpShutdown=true
-Xtrace:trigger=method{java/lang/System.exit,javadump}
-Xtrace:trigger=method{java/lang/Runtime.exit,javadump}

Seeking for advice on what potentially could be the issue or troubleshooting that can be done.

------------------------------
Kam Tyng Loo
------------------------------

Hi, try adding some -XDump params to the JVM Generic Options:

-Xdump:system:events=gpf+abort+traceassert+corruptcache,label=/home/user/core.%Y%m%d.%H%M%S.%pid.%seq.dmp
-Xdump:java:events=gpf+abort+traceassert+corruptcache,label=/home/user/javacore.%Y%m%d.%H%M%S.%pid.%seq.txt

Brian

------------------------------
Brian S Paskin
Sr. Cloud Engineer
IBM Cloud Engineering
------------------------------

Original Message:
Sent: Mon September 26, 2022 04:30 AM
From: Kam Tyng Loo
Subject: Troubleshooting Websphere Application Server 8.5.5 process terminates with no logs

Hi Gabriel, 

We do have antivirus, have checked the Windows Event viewer and there is no event where it mentions that the websphere process would be killed. 
Do we have a known history that antivirus might be causing this?

With regards to tracing, is the following trigger different from what you have specified? I have the below enabled and it did not catch the issue.
-Xtrace:trigger=method{java/lang/System.exit,javadump}

Thanks.

Regards, 
Justin

------------------------------
Kam Tyng Loo

Original Message:
Sent: Mon September 26, 2022 04:14 AM
From: Gabriel Aberasturi
Subject: Troubleshooting Websphere Application Server 8.5.5 process terminates with no logs

Hello Kam,

Any type of antivirus? at system viewer do you see any type of alert about process killed.

If you have at application level an "SystemExit(0)" during some exception your -Xtrace:trigger=method({java/kang/Sytem.exit,javadump} need to "catch".

Hope this helps. Tell us if you need more support.

Regards

------------------------------
Gabriel Aberasturi
Versia tecnologias emergentes

Original Message:
Sent: Fri September 23, 2022 06:52 AM
From: Kam Tyng Loo
Subject: Troubleshooting Websphere Application Server 8.5.5 process terminates with no logs

Encountering issues with WebSphere Application Server process terminates with no footprint.

Environment
- Windows 2016
- WebSphere Application Server Traditional 8.5.5 FP 22 (also occurring on FP21, FP18)
- WAS running as a windows service using a Domain ID

Issue
- WAS process terminates intermittently with no warning, no core dump, no ffdc, no logs indicating cause of termination
- All application logs, system logs, native logs does not show any anomaly
- Can occur at at time, even when there is no use of the hosted application or within an hour of service/server restart and no utilization.

Troubleshooting done so far
- Enabled xtrace with following triggers. Issue occurred but no logs or JVM core dump as well.
-Dcom.ibm.ws.runtime.dumpShutdown=true
-Xtrace:trigger=method{java/lang/System.exit,javadump}
-Xtrace:trigger=method{java/lang/Runtime.exit,javadump}

Seeking for advice on what potentially could be the issue or troubleshooting that can be done.

------------------------------
Kam Tyng Loo
------------------------------

Kam,

Are you in ND scenario or standalone?

In addtion to @Brian S Paskin suggestion try enabling com.ibm.ws.runtime.dumpShutdown System property.

Based https://www.ibm.com/support/pages/process-get-thread-dump-or-javacore-during-server-shutdown


Regards, ​

------------------------------
Gabriel Aberasturi
Versia tecnologias emergentes
------------------------------

Original Message:
Sent: Mon September 26, 2022 04:30 AM
From: Kam Tyng Loo
Subject: Troubleshooting Websphere Application Server 8.5.5 process terminates with no logs

Hi Gabriel, 

We do have antivirus, have checked the Windows Event viewer and there is no event where it mentions that the websphere process would be killed. 
Do we have a known history that antivirus might be causing this?

With regards to tracing, is the following trigger different from what you have specified? I have the below enabled and it did not catch the issue.
-Xtrace:trigger=method{java/lang/System.exit,javadump}

Thanks.

Regards, 
Justin

------------------------------
Kam Tyng Loo

Original Message:
Sent: Mon September 26, 2022 04:14 AM
From: Gabriel Aberasturi
Subject: Troubleshooting Websphere Application Server 8.5.5 process terminates with no logs

Hello Kam,

Any type of antivirus? at system viewer do you see any type of alert about process killed.

If you have at application level an "SystemExit(0)" during some exception your -Xtrace:trigger=method({java/kang/Sytem.exit,javadump} need to "catch".

Hope this helps. Tell us if you need more support.

Regards

------------------------------
Gabriel Aberasturi
Versia tecnologias emergentes

Original Message:
Sent: Fri September 23, 2022 06:52 AM
From: Kam Tyng Loo
Subject: Troubleshooting Websphere Application Server 8.5.5 process terminates with no logs

Encountering issues with WebSphere Application Server process terminates with no footprint.

Environment
- Windows 2016
- WebSphere Application Server Traditional 8.5.5 FP 22 (also occurring on FP21, FP18)
- WAS running as a windows service using a Domain ID

Issue
- WAS process terminates intermittently with no warning, no core dump, no ffdc, no logs indicating cause of termination
- All application logs, system logs, native logs does not show any anomaly
- Can occur at at time, even when there is no use of the hosted application or within an hour of service/server restart and no utilization.

Troubleshooting done so far
- Enabled xtrace with following triggers. Issue occurred but no logs or JVM core dump as well.
-Dcom.ibm.ws.runtime.dumpShutdown=true
-Xtrace:trigger=method{java/lang/System.exit,javadump}
-Xtrace:trigger=method{java/lang/Runtime.exit,javadump}

Seeking for advice on what potentially could be the issue or troubleshooting that can be done.

------------------------------
Kam Tyng Loo
------------------------------

Kam,

As you are in Windows environment and the server is registered as a service review the following notes:

https://www.ibm.com/mysupport/s/question/0D50z000062k7XyCAI/why-does-logging-off-of-windows-system-crash-websphere-application-server?language=en_US

the link is taken from

https://www.ibm.com/support/pages/websphere-application-server-process-exits-without-leaving-footprint-log-files-or-no-core-dumps

Regards

------------------------------
Gabriel Aberasturi
Versia tecnologias emergentes
------------------------------

Original Message:
Sent: Mon September 26, 2022 05:51 AM
From: Gabriel Aberasturi
Subject: Troubleshooting Websphere Application Server 8.5.5 process terminates with no logs

Kam,

Are you in ND scenario or standalone?

In addtion to @Brian S Paskin suggestion try enabling com.ibm.ws.runtime.dumpShutdown System property.

Based https://www.ibm.com/support/pages/process-get-thread-dump-or-javacore-during-server-shutdown


Regards, ​

------------------------------
Gabriel Aberasturi
Versia tecnologias emergentes

Original Message:
Sent: Mon September 26, 2022 04:30 AM
From: Kam Tyng Loo
Subject: Troubleshooting Websphere Application Server 8.5.5 process terminates with no logs

Hi Gabriel, 

We do have antivirus, have checked the Windows Event viewer and there is no event where it mentions that the websphere process would be killed. 
Do we have a known history that antivirus might be causing this?

With regards to tracing, is the following trigger different from what you have specified? I have the below enabled and it did not catch the issue.
-Xtrace:trigger=method{java/lang/System.exit,javadump}

Thanks.

Regards, 
Justin

------------------------------
Kam Tyng Loo

Original Message:
Sent: Mon September 26, 2022 04:14 AM
From: Gabriel Aberasturi
Subject: Troubleshooting Websphere Application Server 8.5.5 process terminates with no logs

Hello Kam,

Any type of antivirus? at system viewer do you see any type of alert about process killed.

If you have at application level an "SystemExit(0)" during some exception your -Xtrace:trigger=method({java/kang/Sytem.exit,javadump} need to "catch".

Hope this helps. Tell us if you need more support.

Regards

------------------------------
Gabriel Aberasturi
Versia tecnologias emergentes

Original Message:
Sent: Fri September 23, 2022 06:52 AM
From: Kam Tyng Loo
Subject: Troubleshooting Websphere Application Server 8.5.5 process terminates with no logs

Encountering issues with WebSphere Application Server process terminates with no footprint.

Environment
- Windows 2016
- WebSphere Application Server Traditional 8.5.5 FP 22 (also occurring on FP21, FP18)
- WAS running as a windows service using a Domain ID

Issue
- WAS process terminates intermittently with no warning, no core dump, no ffdc, no logs indicating cause of termination
- All application logs, system logs, native logs does not show any anomaly
- Can occur at at time, even when there is no use of the hosted application or within an hour of service/server restart and no utilization.

Troubleshooting done so far
- Enabled xtrace with following triggers. Issue occurred but no logs or JVM core dump as well.
-Dcom.ibm.ws.runtime.dumpShutdown=true
-Xtrace:trigger=method{java/lang/System.exit,javadump}
-Xtrace:trigger=method{java/lang/Runtime.exit,javadump}

Seeking for advice on what potentially could be the issue or troubleshooting that can be done.

------------------------------
Kam Tyng Loo
------------------------------

Standalone scenario.

Already have the com.ibm.ws.runtime.dumpShutdown trace configured. Didn't catch anything. 
Will try the corrupt cache tracing and monitor. 

Regards, 
Justin

------------------------------
Kam Tyng Loo
------------------------------

Original Message:
Sent: Mon September 26, 2022 06:06 AM
From: Gabriel Aberasturi
Subject: Troubleshooting Websphere Application Server 8.5.5 process terminates with no logs

Kam,

As you are in Windows environment and the server is registered as a service review the following notes:

https://www.ibm.com/mysupport/s/question/0D50z000062k7XyCAI/why-does-logging-off-of-windows-system-crash-websphere-application-server?language=en_US

the link is taken from

https://www.ibm.com/support/pages/websphere-application-server-process-exits-without-leaving-footprint-log-files-or-no-core-dumps

Regards

------------------------------
Gabriel Aberasturi
Versia tecnologias emergentes

Original Message:
Sent: Mon September 26, 2022 05:51 AM
From: Gabriel Aberasturi
Subject: Troubleshooting Websphere Application Server 8.5.5 process terminates with no logs

Kam,

Are you in ND scenario or standalone?

In addtion to @Brian S Paskin suggestion try enabling com.ibm.ws.runtime.dumpShutdown System property.

Based https://www.ibm.com/support/pages/process-get-thread-dump-or-javacore-during-server-shutdown


Regards, ​

------------------------------
Gabriel Aberasturi
Versia tecnologias emergentes

Original Message:
Sent: Mon September 26, 2022 04:30 AM
From: Kam Tyng Loo
Subject: Troubleshooting Websphere Application Server 8.5.5 process terminates with no logs

Hi Gabriel, 

We do have antivirus, have checked the Windows Event viewer and there is no event where it mentions that the websphere process would be killed. 
Do we have a known history that antivirus might be causing this?

With regards to tracing, is the following trigger different from what you have specified? I have the below enabled and it did not catch the issue.
-Xtrace:trigger=method{java/lang/System.exit,javadump}

Thanks.

Regards, 
Justin

------------------------------
Kam Tyng Loo

Original Message:
Sent: Mon September 26, 2022 04:14 AM
From: Gabriel Aberasturi
Subject: Troubleshooting Websphere Application Server 8.5.5 process terminates with no logs

Hello Kam,

Any type of antivirus? at system viewer do you see any type of alert about process killed.

If you have at application level an "SystemExit(0)" during some exception your -Xtrace:trigger=method({java/kang/Sytem.exit,javadump} need to "catch".

Hope this helps. Tell us if you need more support.

Regards

------------------------------
Gabriel Aberasturi
Versia tecnologias emergentes

Original Message:
Sent: Fri September 23, 2022 06:52 AM
From: Kam Tyng Loo
Subject: Troubleshooting Websphere Application Server 8.5.5 process terminates with no logs

Encountering issues with WebSphere Application Server process terminates with no footprint.

Environment
- Windows 2016
- WebSphere Application Server Traditional 8.5.5 FP 22 (also occurring on FP21, FP18)
- WAS running as a windows service using a Domain ID

Issue
- WAS process terminates intermittently with no warning, no core dump, no ffdc, no logs indicating cause of termination
- All application logs, system logs, native logs does not show any anomaly
- Can occur at at time, even when there is no use of the hosted application or within an hour of service/server restart and no utilization.

Troubleshooting done so far
- Enabled xtrace with following triggers. Issue occurred but no logs or JVM core dump as well.
-Dcom.ibm.ws.runtime.dumpShutdown=true
-Xtrace:trigger=method{java/lang/System.exit,javadump}
-Xtrace:trigger=method{java/lang/Runtime.exit,javadump}

Seeking for advice on what potentially could be the issue or troubleshooting that can be done.

------------------------------
Kam Tyng Loo
------------------------------