Hi experts.. 

We have an issue here at AIX booting time with EMC (Dell) SAN Storage System due path lost.
After seeing Kernel Starting message, suddenly stops at 554 LED code because booting process can't find the path from it had been accessing the IPL disk.

EMC told us we should reduce to just one path only from Storage to Server to avoid  use other path which the Storage System is not offer the active path to these boot disk. 
But doing that, is a problem and represent a task which takes SysAdmin time and could incur in risks by human errors every time an LPAR needs to be booted; like unmap virtual FibreChannel adapters or unconfigure SAN Switches zoning or delete WWPN from the host attachment config at Storage System.

Does somebody know how to avoid this problem happen? 
We thought that maybe a vscsi disk from VIOS to LPARs could help, but, I can't believe every customer with EMC Systems should live with that problem. 

Regards 

Hi experts.. 

We have an issue here at AIX booting time with EMC (Dell) SAN Storage System due path lost.
After seeing Kernel Starting message, suddenly stops at 554 LED code because booting process can't find the path from it had been accessing the IPL disk.

EMC told us we should reduce to just one path only from Storage to Server to avoid  use other path which the Storage System is not offer the active path to these boot disk. 
But doing that, is a problem and represent a task which takes SysAdmin time and could incur in risks by human errors every time an LPAR needs to be booted; like unmap virtual FibreChannel adapters or unconfigure SAN Switches zoning or delete WWPN from the host attachment config at Storage System.

Does somebody know how to avoid this problem happen? 
We thought that maybe a vscsi disk from VIOS to LPARs could help, but, I can't believe every customer with EMC Systems should live with that problem. 

Regards 

Hi Luis

It may be best to open an AIX support case for this one if you have not already tried that avenue.  The solution to intermittent LED 554 issues can involve tweaks on switches or storage but will likely need in-depth investigation starting with AIX support. (I used to work in AIX support handling boot and storage device driver issues).

Hi experts.. 

We have an issue here at AIX booting time with EMC (Dell) SAN Storage System due path lost.
After seeing Kernel Starting message, suddenly stops at 554 LED code because booting process can't find the path from it had been accessing the IPL disk.

EMC told us we should reduce to just one path only from Storage to Server to avoid  use other path which the Storage System is not offer the active path to these boot disk. 
But doing that, is a problem and represent a task which takes SysAdmin time and could incur in risks by human errors every time an LPAR needs to be booted; like unmap virtual FibreChannel adapters or unconfigure SAN Switches zoning or delete WWPN from the host attachment config at Storage System.

Does somebody know how to avoid this problem happen? 
We thought that maybe a vscsi disk from VIOS to LPARs could help, but, I can't believe every customer with EMC Systems should live with that problem. 

Regards 

Hi experts.. 

We have an issue here at AIX booting time with EMC (Dell) SAN Storage System due path lost.
After seeing Kernel Starting message, suddenly stops at 554 LED code because booting process can't find the path from it had been accessing the IPL disk.

EMC told us we should reduce to just one path only from Storage to Server to avoid  use other path which the Storage System is not offer the active path to these boot disk. 
But doing that, is a problem and represent a task which takes SysAdmin time and could incur in risks by human errors every time an LPAR needs to be booted; like unmap virtual FibreChannel adapters or unconfigure SAN Switches zoning or delete WWPN from the host attachment config at Storage System.

Does somebody know how to avoid this problem happen? 
We thought that maybe a vscsi disk from VIOS to LPARs could help, but, I can't believe every customer with EMC Systems should live with that problem. 

Regards 

------------------------------
Luis Alberto Rojas Kramer
------------------------------

Hi experts.. 

We have an issue here at AIX booting time with EMC (Dell) SAN Storage System due path lost.
After seeing Kernel Starting message, suddenly stops at 554 LED code because booting process can't find the path from it had been accessing the IPL disk.

EMC told us we should reduce to just one path only from Storage to Server to avoid  use other path which the Storage System is not offer the active path to these boot disk. 
But doing that, is a problem and represent a task which takes SysAdmin time and could incur in risks by human errors every time an LPAR needs to be booted; like unmap virtual FibreChannel adapters or unconfigure SAN Switches zoning or delete WWPN from the host attachment config at Storage System.

Does somebody know how to avoid this problem happen? 
We thought that maybe a vscsi disk from VIOS to LPARs could help, but, I can't believe every customer with EMC Systems should live with that problem. 

Regards 

Hi experts.. 

We have an issue here at AIX booting time with EMC (Dell) SAN Storage System due path lost.
After seeing Kernel Starting message, suddenly stops at 554 LED code because booting process can't find the path from it had been accessing the IPL disk.

EMC told us we should reduce to just one path only from Storage to Server to avoid  use other path which the Storage System is not offer the active path to these boot disk. 
But doing that, is a problem and represent a task which takes SysAdmin time and could incur in risks by human errors every time an LPAR needs to be booted; like unmap virtual FibreChannel adapters or unconfigure SAN Switches zoning or delete WWPN from the host attachment config at Storage System.

Does somebody know how to avoid this problem happen? 
We thought that maybe a vscsi disk from VIOS to LPARs could help, but, I can't believe every customer with EMC Systems should live with that problem. 

Regards 

Hello,

I don't know which EMC storage we are talking about.
I had a problem with EMC storage, it did not see LUNs on AIX via mpio, so I had to install EMC PowerPath.
Best to check with EMC.
It might be best to create an SSP.

Regards,

Hi experts.. 

We have an issue here at AIX booting time with EMC (Dell) SAN Storage System due path lost.
After seeing Kernel Starting message, suddenly stops at 554 LED code because booting process can't find the path from it had been accessing the IPL disk.

EMC told us we should reduce to just one path only from Storage to Server to avoid  use other path which the Storage System is not offer the active path to these boot disk. 
But doing that, is a problem and represent a task which takes SysAdmin time and could incur in risks by human errors every time an LPAR needs to be booted; like unmap virtual FibreChannel adapters or unconfigure SAN Switches zoning or delete WWPN from the host attachment config at Storage System.

Does somebody know how to avoid this problem happen? 
We thought that maybe a vscsi disk from VIOS to LPARs could help, but, I can't believe every customer with EMC Systems should live with that problem. 

Regards 

Hi experts.. 

We have an issue here at AIX booting time with EMC (Dell) SAN Storage System due path lost.
After seeing Kernel Starting message, suddenly stops at 554 LED code because booting process can't find the path from it had been accessing the IPL disk.

EMC told us we should reduce to just one path only from Storage to Server to avoid  use other path which the Storage System is not offer the active path to these boot disk. 
But doing that, is a problem and represent a task which takes SysAdmin time and could incur in risks by human errors every time an LPAR needs to be booted; like unmap virtual FibreChannel adapters or unconfigure SAN Switches zoning or delete WWPN from the host attachment config at Storage System.

Does somebody know how to avoid this problem happen? 
We thought that maybe a vscsi disk from VIOS to LPARs could help, but, I can't believe every customer with EMC Systems should live with that problem. 

Regards 

Hi experts.. 

We have an issue here at AIX booting time with EMC (Dell) SAN Storage System due path lost.
After seeing Kernel Starting message, suddenly stops at 554 LED code because booting process can't find the path from it had been accessing the IPL disk.

EMC told us we should reduce to just one path only from Storage to Server to avoid  use other path which the Storage System is not offer the active path to these boot disk. 
But doing that, is a problem and represent a task which takes SysAdmin time and could incur in risks by human errors every time an LPAR needs to be booted; like unmap virtual FibreChannel adapters or unconfigure SAN Switches zoning or delete WWPN from the host attachment config at Storage System.

Does somebody know how to avoid this problem happen? 
We thought that maybe a vscsi disk from VIOS to LPARs could help, but, I can't believe every customer with EMC Systems should live with that problem. 

Regards 

Forgot to mention, booting (after hotfix) is more reliable if you set the boot paths to be the preferred paths from lsmpio.

So 'lsmpio -l hdisk4':

hdisk4   0        Enabled  Sel,Opt      fscsi0  58ccf0984d201f37,0
hdisk4   1        Enabled  Non          fscsi0  58ccf0904d201f37,0
hdisk4   2        Enabled  Sel,Opt      fscsi1  58ccf0984d211f37,0
hdisk4   3        Enabled  Non          fscsi1  58ccf0904d211f37,0

Then set the boot list:

# bootlist -m normal hdisk4 pathid=0,2,1,3

Regards,
Henrik Morsing

Hi experts.. 

We have an issue here at AIX booting time with EMC (Dell) SAN Storage System due path lost.
After seeing Kernel Starting message, suddenly stops at 554 LED code because booting process can't find the path from it had been accessing the IPL disk.

EMC told us we should reduce to just one path only from Storage to Server to avoid  use other path which the Storage System is not offer the active path to these boot disk. 
But doing that, is a problem and represent a task which takes SysAdmin time and could incur in risks by human errors every time an LPAR needs to be booted; like unmap virtual FibreChannel adapters or unconfigure SAN Switches zoning or delete WWPN from the host attachment config at Storage System.

Does somebody know how to avoid this problem happen? 
We thought that maybe a vscsi disk from VIOS to LPARs could help, but, I can't believe every customer with EMC Systems should live with that problem. 

Regards 

Hi experts.. 

We have an issue here at AIX booting time with EMC (Dell) SAN Storage System due path lost.
After seeing Kernel Starting message, suddenly stops at 554 LED code because booting process can't find the path from it had been accessing the IPL disk.

EMC told us we should reduce to just one path only from Storage to Server to avoid  use other path which the Storage System is not offer the active path to these boot disk. 
But doing that, is a problem and represent a task which takes SysAdmin time and could incur in risks by human errors every time an LPAR needs to be booted; like unmap virtual FibreChannel adapters or unconfigure SAN Switches zoning or delete WWPN from the host attachment config at Storage System.

Does somebody know how to avoid this problem happen? 
We thought that maybe a vscsi disk from VIOS to LPARs could help, but, I can't believe every customer with EMC Systems should live with that problem. 

Regards 

Hi experts.. 

We have an issue here at AIX booting time with EMC (Dell) SAN Storage System due path lost.
After seeing Kernel Starting message, suddenly stops at 554 LED code because booting process can't find the path from it had been accessing the IPL disk.

EMC told us we should reduce to just one path only from Storage to Server to avoid  use other path which the Storage System is not offer the active path to these boot disk. 
But doing that, is a problem and represent a task which takes SysAdmin time and could incur in risks by human errors every time an LPAR needs to be booted; like unmap virtual FibreChannel adapters or unconfigure SAN Switches zoning or delete WWPN from the host attachment config at Storage System.

Does somebody know how to avoid this problem happen? 
We thought that maybe a vscsi disk from VIOS to LPARs could help, but, I can't believe every customer with EMC Systems should live with that problem. 

Regards 

Hi,

LED 554 at "Kernel Starting" indicates a failure during the varyon of rootvg, and in environments using EMC/Dell SAN this is commonly related to multipath behavior during early boot rather than a requirement to operate with a single path.

In most cases, the issue occurs because AIX may select a path that is visible but not active/optimized from the storage side .

During the early boot phase, MPIO failover is limited, which can lead to rootvg not being accessible and the system stopping at LED 554.

Reducing to a single path is not considered a best practice, as it removes redundancy and introduces operational risk.

Recommended approach:

• Ensure all paths to the boot LUN are consistently presented and correctly optimized on the storage side configuration.
• Verify AIX MPIO configuration:
• reserve_policy=no_reserve
• Appropriate path selection algorithm (e.g. round_robin or fail_over depending on array type)
• Confirm all paths are in Enabled state (lspath) with no missing or failed paths.
• Rebuild boot image after any changes (bosboot, bootlist).
• If applicable, consider using EMC PowerPath or the correct AIX PCM for the array.

As a practical and commonly used, some environments choose to present rootvg via vSCSI from VIOS, allowing VIOS to handle multipathing while keeping NPIV for data disks. This avoids early boot path selection issues.

The correct resolution is to align SAN presentation + AIX MPIO configuration or use vSCSI for rootvg if needed for stability.

Hi experts.. 

We have an issue here at AIX booting time with EMC (Dell) SAN Storage System due path lost.
After seeing Kernel Starting message, suddenly stops at 554 LED code because booting process can't find the path from it had been accessing the IPL disk.

EMC told us we should reduce to just one path only from Storage to Server to avoid  use other path which the Storage System is not offer the active path to these boot disk. 
But doing that, is a problem and represent a task which takes SysAdmin time and could incur in risks by human errors every time an LPAR needs to be booted; like unmap virtual FibreChannel adapters or unconfigure SAN Switches zoning or delete WWPN from the host attachment config at Storage System.

Does somebody know how to avoid this problem happen? 
We thought that maybe a vscsi disk from VIOS to LPARs could help, but, I can't believe every customer with EMC Systems should live with that problem. 

Regards 

Hi experts.. 

We have an issue here at AIX booting time with EMC (Dell) SAN Storage System due path lost.
After seeing Kernel Starting message, suddenly stops at 554 LED code because booting process can't find the path from it had been accessing the IPL disk.

EMC told us we should reduce to just one path only from Storage to Server to avoid  use other path which the Storage System is not offer the active path to these boot disk. 
But doing that, is a problem and represent a task which takes SysAdmin time and could incur in risks by human errors every time an LPAR needs to be booted; like unmap virtual FibreChannel adapters or unconfigure SAN Switches zoning or delete WWPN from the host attachment config at Storage System.

Does somebody know how to avoid this problem happen? 
We thought that maybe a vscsi disk from VIOS to LPARs could help, but, I can't believe every customer with EMC Systems should live with that problem. 

Regards 

Hello Luis,

I have been working with EMC storage and IBM AIX for some time in the past and I remember that led 554 on boot at one time was very likely. We got a fix (I will have to check my e-mail and see whether we fixed it ourselves, got a fix from IBM or from EMC). We also noticed that we often could not create an mksysb (since AIX couldn't determine the disk it booted from) - we fixed that by replacing 'pprootdev' (a Powerpath script from EMC) with our own version.

I will check my e-mail and get back here with some recommendations. Also, some of the suggestions here, sound pretty good to me.

Hi experts.. 

We have an issue here at AIX booting time with EMC (Dell) SAN Storage System due path lost.
After seeing Kernel Starting message, suddenly stops at 554 LED code because booting process can't find the path from it had been accessing the IPL disk.

EMC told us we should reduce to just one path only from Storage to Server to avoid  use other path which the Storage System is not offer the active path to these boot disk. 
But doing that, is a problem and represent a task which takes SysAdmin time and could incur in risks by human errors every time an LPAR needs to be booted; like unmap virtual FibreChannel adapters or unconfigure SAN Switches zoning or delete WWPN from the host attachment config at Storage System.

Does somebody know how to avoid this problem happen? 
We thought that maybe a vscsi disk from VIOS to LPARs could help, but, I can't believe every customer with EMC Systems should live with that problem. 

Regards