Hi all,

We have three independent Windows servers, all running the same application on WAS 7, and we've been experiencing apparently random crashes of the app server on all three for the past week.  There are no errors in any of the logs -- SystemOut, SystemErr, native_stdout, native_stderr -- when this occurs; the WAS process simply disappears.  There are also no Java dump files to be found; to all appearances, the app server should still be running, aside from the fact that it's writing nothing to the logs -- and running 'serverStatus -all' returns 'cannot be reached' rather than 'started'.

Has anyone seen anything like this?  We've had multiple people looking at this over the past week, including someone from our WAS support team, but none of us has been able to identify the problem.

At the moment all three app servers are up, and have been running since at least several hours (one was started this morning, one yesterday morning, and the third the day before yesterday).  However, we've also started them, only to see them go right back down within a few minutes.

A few details about the environment:

WAS 7.0.0.29
JRE 1.6.0 IBM J9
Windows Server 2008 Enterprise - 6.0.6002 SP2
8 GB RAM
6 CPUs

Hi sje,

  The java process disappears or still the but you haven't any reponse from them?

  If the process still, I suggest to take manual dumps of the java process to analyze them
 
    MustGather: Generating Javacores and Userdumps Manually For Performance, Hang or High CPU Issues on Windows
    www-01.ibm.com/support/docview.wss?uid=s...
 
  In addtion check the log files of the servers to see the last log entry? review node agents logs too.
 
  If the process disappears, the review the applications looking for System.exit() inside the try catch blocks
 
  Hope this helps.  Tell us if you need more support.
 
regards

Hi Gabriel,

Thanks for the suggestion.  I *think* the Java process is also gone when this happens -- but I'm not absolutely sure -- and of course it hasn't happened again since I posted.

Hi,
May be check the server health when the process goes down. It might be useful to track the memory and cpu patterns when the process goes down.

Thanks

Hi sje

Is this a ND environment or standalone?
  

Hi Tony,

It's standalone -- each of the three servers runs independently (or so we've been assured by our WAS support team).

And to update for everyone, since my initial post two of the servers ran continuously until yesterday (the third was down).  One of them experienced an unrelated glitch yesterday morning, so we called in the WAS team to get them restarted.

They restarted all three, and since then all three have been running normally (although only one is live at a time, so the other two are running but idle).  I'm not sure what to think at this point.  I suppose all we can do is continue to monitor them closely, and try to catch one of them in the act.