Hello everyone,

Can you please help me understand what happened in my WAS7:

I'm trying to deploy a WAR from the DMGR. I select

• WebSphere enterprise applications

 then i select: install
 
then i choose the path to the WAR I want to install.  Up to now everithing is alright,

when i click NEXT. i wait more than 30 min and then the DMGR crash.

Any idea ?

thanks

Hi adilyos,

  Seems the the war is not well packed. Could you put DMGR SystemOut.log? what size have the war? 30 minutes is too long...

  Put the platform and full version 7.0.0.X.

  Thanks.

regards,

Thanks Gabriel.

There is no errors on SystemOut of the DMGR.  We have WAS ND 7.0.0.21 ( the last fixpack) . The size og the WAR is 40 Mo. I have the WARfile  on the same machine where WAS ND is deployed

what problem can be in the WAR ? how to fix this ?

Hi,

  what do you mean with crash? do you have a dump? isn't there ffdc files? if you have a crash sure that in SystemOut of DMGR there is something.

  40 MB war is a big war maybe the problem is an out of memory (OOM) by default DMGR hasn't max heaps size so the max heap size is 256 MB.  But if you are having an OOM you need to have a dump and some traces in SystemOut.
 

regards,

Hi Gabriel,

When I say crash I mean that the graphical console of dmgr
keep printing that the WAR is being loaded, but when I try to
cancel or to log out from dmgr, it does not respond. So I connect
with putty and kill the process of dmgr and execute ./startManager.sh .
This happen everytime I try to deploy this WAR.

I have already deployed a WAR which size is 45 Mo and I haven't
found any problem with it.

Where can I found the ffdc files you're talking about ?

For information my Websphere is on solaris 10 zone.

I have just those messages on SystemOut.log:

00000043 CoordinatorCo W HMGR0152W: CPU Starvation detected.
Current thread scheduling delay is 6 seconds.

ThreadMonitor W WSVR0605W: Thread "WebContainer : 6"
(00000038) has been active for 724189 milliseconds and may
be hung. Ther
e is/are 1 thread(s) in total in the server that may be hung.
at java.util.zip.Inflater.inflateBytes(Native Method)
at java.util.zip.Inflater.inflate(Inflater.java:238)
at java.util.zip.InflaterInputStream.read(Inflater
InputStream.java:135)
at java.io.BufferedInputStream.fill(BufferedInput
Stream.java:218)
at java.io.BufferedInputStream.read1(Buffered
InputStream.java:258)
at java.io.BufferedInputStream.read(Buffered
InputStream.java:317)
at java.io.FilterInputStream.read(FilterInput
Stream.java:116)
at java.io.PushbackInputStream.read(Pushback
InputStream.java:169)
at java.util.zip.InflaterInputStream.fill(Inflater
InputStream.java:221)
at java.util.zip.InflaterInputStream.read(Inflater
InputStream.java:141)
at java.util.zip.ZipInputStream.read(ZipInput
Stream.java:154)
at java.io.FilterInputStream.read(FilterInput
Stream.java:116)
at org.objectweb.asm.ClassReader.a(Unknown
Source)
at org.objectweb.asm.ClassReader.(Unknown
Source)
at com.ibm.ws.amm.scan.util.ClassScanner.scan
InputStream(ClassScanner.java:295)
at com.ibm.ws.amm.scan.util.ClassScanner.scan
InputStream(ClassScanner.java:267)
at com.ibm.ws.amm.AnnotativeMetadataManager
Impl$1.applyTo(AnnotativeMetadataManagerImpl.java:263)
at com.ibm.wsspi.amm.scan.context.ArchiveInput
StreamData.iterate(ArchiveInputStreamData.java:241)
at com.ibm.ws.amm.AnnotativeMetadataManager
Impl.scan(AnnotativeMetadataManagerImpl.java:261)
at com.ibm.ws.amm.AnnotativeMetadataManager
Impl.scan(AnnotativeMetadataManagerImpl.java:222)
at com.ibm.ws.amm.commonarchive.Annotations
ProcessorImpl.merge(AnnotationsProcessorImpl.java:108)
at org.eclipse.jst.j2ee.commonarchivecore.internal.impl.
WARFileImpl.processAnnotations(WARFileImpl.java:303)
at org.eclipse.jst.j2ee.commonarchivecore.internal.impl.
WARFileImpl.getDeploymentDescriptor(WARFileImpl.java:249)
at org.eclipse.jst.j2ee.commonarchivecore.internal.impl.
WARFileImpl.getDeploymentDescriptor(WARFileImpl.java:179)
at com.ibm.ws.management.application.client.AppInstall
Helper.createEarWrapper(AppInstallHelper.java:718)
at com.ibm.ws.management.application.client.AppInstall
Helper.createEarWrapper(AppInstallHelper.java:553)
at com.ibm.ws.management.application.client.AppInstall
Helper.createEarWrapper(AppInstallHelper.java:521)
at com.ibm.ws.console.appmanagement.action.
UploadAction.formUpload(UploadAction.java:478)
at com.ibm.ws.console.appmanagement.action.
UploadAction.execute(UploadAction.java:156)
at org.apache.struts.action.RequestProcessor.
processActionPerform(Unknown Source)
at org.apache.struts.action.RequestProcessor.
process(Unknown Source)
at org.apache.struts.action.ActionServlet.
process(Unknown Source)
at org.apache.struts.action.ActionServlet.
doPost(Unknown Source)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:738)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:831)
at com.ibm.ws.webcontainer.servlet.Servlet
Wrapper.service(ServletWrapper.java:1663)
at com.ibm.ws.webcontainer.servlet.Servlet
Wrapper.service(ServletWrapper.java:1597)
at com.ibm.ws.webcontainer.filter.WebAppFilter
Chain.doFilter(WebAppFilterChain.java:131)
at com.ibm.ws.console.core.servlet.WSCUrlFilter.
setUpCommandAssistence(WSCUrlFilter.java:960)
at com.ibm.ws.console.core.servlet.WSCUrlFilter.
continueStoringTaskState(WSCUrlFilter.java:500)
at com.ibm.ws.console.core.servlet.WSCUrlFilter.
doFilter(WSCUrlFilter.java:321)
at com.ibm.ws.webcontainer.filter.FilterInstance
Wrapper.doFilter(FilterInstanceWrapper.java:188)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.
doFilter(WebAppFilterChain.java:116)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.
_doFilter(WebAppFilterChain.java:77)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.
doFilter(WebAppFilterManager.java:908)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.
handleRequest(ServletWrapper.java:934)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.
handleRequest(ServletWrapper.java:502)
at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.
handleRequest(ServletWrapperImpl.java:179)
at com.ibm.ws.webcontainer.webapp.WebApp.handle
Request(WebApp.java:3935)
at com.ibm.ws.webcontainer.webapp.WebGroup.
handleRequest(WebGroup.java:276)
at com.ibm.ws.webcontainer.WebContainer.
handleRequest(WebContainer.java:931)
at com.ibm.ws.webcontainer.WSWebContainer.
handleRequest(WSWebContainer.java:1592)
at com.ibm.ws.webcontainer.channel.WCChannelLink.
ready(WCChannelLink.java:186)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.
handleDiscrimination(HttpInboundLink.java:452)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.
handleNewRequest(HttpInboundLink.java:511)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.
processRequest(HttpInboundLink.java:305)
at com.ibm.ws.http.channel.inbound.impl.HttpICLRead
Callback.complete(HttpICLReadCallback.java:83)
at com.ibm.ws.ssl.channel.impl.SSLReadServiceContext
$SSLReadCompletedCallback.complete(SSLReadService
Context.java:1784)
at com.ibm.ws.tcp.channel.impl.WorkQueueManager.
requestComplete(WorkQueueManager.java:557)
at com.ibm.ws.tcp.channel.impl.WorkQueueManager.
attemptIO(WorkQueueManager.java:607)
at com.ibm.ws.tcp.channel.impl.WorkQueueManager.
workerRun(WorkQueueManager.java:984)
at com.ibm.ws.tcp.channel.impl.WorkQueueManager
$Worker.run(WorkQueueManager.java:1069)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1604)

Thanks for your help.

Hi,

  You can find ffdc in the \logs\ffdc in the directory up to DMGR logs.

  Could you generate a new war an try installing again again?  i have see more than once that a bad package war fails to install.

  how do you upload the war file to server, binary or ascii? sometimes this is the problem the cpu starvation is ocurred when is inflating the zip an maybe tha war an their deployment descriptors are not good.

  can you do test with another ear for example default.ear application from \installableApps.

regards,

Hi, I have uploaded the WAR file with Binary mode. When I try with another EAR file of another application it works. So I think the problem is with the WAR file. How could I fix this in the WAR file ?

Hi adilyos,

  Well!!! seems that the problem is with the war.  I suggest export again from RAD or generate again with your packing method .   Who gives you the war file? ask him/her.

regards,

Hi I'm still facing the problem with this WAR. When I deploy an EAR there is no problem. Do you think it's because of the WAR ? Should I deploy an EAR file rather then a WAR file ? What does WAS do in this step where I blok ?

Thanks 

Hi Adilyos,

  Ok.  do you pack again the war file? i think that the problem isn't because is a war file.

  Can you check the ffdc logs?

  regards,

I Have tried with another WAR file and I'm still facing this problem.

I have tried to install it from local file system and from Remote file system. I have transmitted the WAR file in binary mode.

When I click NEXT I can't do anything on the console even to log out so I'm obliged to kill his process. There is no log in ffdc at the time of the problem.

Can you please explain what does the WAS do when we click NEXT ?

What are the main differences between WAR file and EAR file ?

Thanks,

(Click on the image below to for a full-sized view)

Hi,


  I suggest to do the next mustgather to take traces.

  MustGather: Problems during deployment of EAR/JAR/WAR files in WebSphere Application Server
  www-01.ibm.com/support/docview.wss?uid=s...

  Follow the section Collecting data manually.  The trace enable in runtime so you don't need to stop the server.

  With traces enable try again deploying the war file.

  with this traces we have all the information to see what is happenning in war deplyoment.

  If you have some problem taking traces tell us.

  In what version of solaris and JVM are you?

regards,

Hi,

What is the
1. Version of WAS
2. OS?
3. Total Memory installed?
4. Free Memory?
5. Minumum and Maximum Heap Size configured.
6. Type of  GC configured.

Regards

The logs state:

00000043 CoordinatorCo W HMGR0152W: CPU Starvation detected. 
Current thread scheduling delay is 6 seconds.

Can you prstat and check CPU usage when you are deploying? Seems to me like it's starving the CPU when unpacking the WAR file. 

Can you run prtdiag and give us an output of what the Zone configuration is? 

Hey everyone sorry for being late. Thanks for your help.

We have deployed the war file by changing the Generic JVM arguments of the DMGR. Here are the arghuments that we have added:

-XX:MaxPermSize=512m -Xms256m -Xmx1024m

When we have done this we have succefuly deployed the war and the apoplication is running.

Can you exaplain please exactly what does this 3 arguments do ?

We have other application deployed in the same cell so I think we could face problem's while deploying other WAR files of other application if we keep this arguments.


We have : * WAS ND 7.0.0.21 running on Solaris 10 zone.

                 * Capped memory of this zone is: 11G

@Ashraf Ansar: Where can I find the version of GC configured ?

Thanks for your help

the detailed explanation about xms and xmx parameters can be found here: docs.oracle.com/javase/1.3/docs/tooldocs...

MaxPermSize specifies the the maximum size for the permanent generation heap, a heap that holds objects such as classes and methods. in your case, I suspect MaxPermsize is running out of memory and not necessarily the main memory.

Aren't Xms/Xmx configured via admin console (Initial/ Maximum HEAP Size)?

Perhaps you should enable garbage collection logging (I think you are using Hotspot JVM for WAS on Solaris, just add the following JVM generic arguments: -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -Xloggc:${SERVER_LOG_ROOT}/loggc.log ) - it creates loggc.log in the server's log directory. With this logfile, you can see the memory usage in the different Generations including PermGen. You can also visualize this logfile / create reports using IBM Support Assistant.

Hi Adilyos,

  You are putting the memory parameters in the DMGR and not in the servers so the other application isn't impacted.

 DMGR is for cell management and the servers has the applications.

 You can rest assured.

regards,