AIOps & Management

 View Only

Debugging DASH Installation in JazzSM: a Sample Case

By Kristen Meren posted Mon October 02, 2017 06:06 AM

  

by Aldo Bucossi

Most of the time installation failures that occur during the installation of JazzSM components depends on very common root causes like file system space shortage, missing authorizations, or missing pre-requisites. Despite this, sometimes it may not be trivial to figure out the root cause, due to the meaningful amount of steps and elements involved in the installation process.

A first hint is provided by the error messages issued by the Installation Manager GUI, that show an excerpt from the log file. From the GUI you have the chance to open the log file generated during installation process by IIM, but you can open it manually using your favourite text editor too.

The file is an xml file having the following naming convention:

For example : 20151230_1715.xml

The Installation Manager log files are located under paths:

 

UNIX:

 

WINDOWS:

C:\ProgramData\IBM\Installation Manager\logs

or

C:\Documents and Settings\All Users\Application Data\IBM\Installation Manager

 

Into the logs folder you can also find a file called index.xml, that lists the log files generated by recent activity. You can refer to the most recent one in case you have just executed an installation attempt, or look for the one with the timestamp closer to the time you performed the installation. In the log file, you can find a message that clearly explains why the step in error was not able to complete.

For example: 

ERROR: The following error occurred while executing this line:
/opt/IBM/JazzSM/install/tip/tipWrapperInstall.xml:113: The following error occurred while executing this line:
/opt/IBM/JazzSM/install/tip/tipInstall.xml:298: The following error occurred while executing this line:
/opt/IBM/JazzSM/install/tip/tipInstall.xml:170: CTGWA0124E Fail to deploy isc.ear. [/opt/IBM/JazzSM/install/tip/tipWrapperInstall.xml:36]
A package has an issue that cannot be resolved by Installation Manager.

 

This sequence of messages helps understanding that the installer failed while trying to deploy the isc.ear, but does not provide the cause for the error. We know that the problem occurred while dealing with WAS, so we may have a further look at the logs generated by WAS itself, but also other files can help us in troubleshooting activity. Generally speaking, it is always very useful to gather all the problem determination logs and document by running the PD script as suggested in the following technote: http://www-01.ibm.com/support/docview.wss?uid=swg21627669

 

The script collects the necessary logs from all the involved components: IIM, DB2,Launchpad, WAS logs and JazzSM service, and creates a tar file that can be then reviewed by the administrator or by the Support engineers.

In the sample scenario I mentioned above, I said we may need to review WAS logs to better understand what went wrong, but we can also verify log files generated by the JazzSM installation steps first.

Into the package created by Jazz_SM_PD_Script, you can find logs created for each JazzSM component being installed, each one stored in its own folder. In our case we were trying to install DASH, so we can find log files under:

/logs

 

You could check the .out files for additional meaningful information. We already know the installation failed while trying to deploy isc.ear, and looking at the list of the log files we can find one of them having name:

consoleDeployISCEar.out

 

This file shows at the end an error messages like:

WASX7017E: Exception received while running file "/opt/IBM/JazzSM/ui/bin/deplyISCear.py"; exception information: org.eclipse.jst.j2ee.commonarchivecore.internal.exception.SaveFailureException: org.eclipse.jst.j2ee.commonarchivecore.internal.exception.SaveFailureException: secure/isclite/scripts/tipdojo/ibm/tivoli/tip/navwidget/table/dijit/nls/TableViewAll_zh-tw.js

It confirms what we saw into the Installation Manager log file but added some more detail. For each .out file you have also an err file (the stderr). The file consoleDeployISCEar.err shows instead:

Wrapper encontered error executing command. Return code was 105 from command /opt/IBM/WebSphere/AppServer/bin/wsadmin.sh.

 

This actually does not help so much, but points us definitely to WAS. We need to review the logs related to WAS profile hoping to find more meaningful messages concerning the failure. The tar file generated by Jazz_SM_PD_Script also contains WAS logs, so we can immediately have a look there. The WAS logs are saved into path:

WAS_logs/was_profile_logs/logs/server1

 

For example, the native_stderr.out file may show messages that are useful for our purpose. In this sample scenario, it contains the following message:

JVMSHRC561E Failed to initialize the shared classes cache, there is not enough space in the file system. Available free disk space bytes =
58007552, requested bytes = 62914560.
JVMSHRC686I Failed to startup shared class cache. Continue without using it as -Xshareclasses:nonfatal is specified

 

This indicates a disk space shortage, and despite it is not a fatal error for this specific task (shared classes initialization), it clearly point us to possible problems with disk space, that may instead have severe impact for other tasks. Having a look at the current space allocation for involved filesystems reveals that the available space is not enough to complete JazzSM installation. The suggestion in this specific case is to expand file systems where JazzSM and WAS are installed (or remove useless files to save space) and run installation again. It would be even better to run Prerequisite Checker to be sure that all the expected pre-requisites are respected.

Of course this is just a sample case, root cause for installation failures may vary, but following the suggested process will help gathering useful hints, understand and correct the root cause in most of the error scenarios, by leveraging on the verbosity of the logs generated by Installation Manager, JazzSM and WAS.

0 comments
4 views