PowerVM

1. Introduction

viosupgrade tool has been widely adopted by our customers for upgrading VIOS between major versions 2.2.6.x to 3.1 and this guide will help in seamless upgrade from versions 3.1.2.x or later to 4.1. viosupgrade interface automates the entire VIOS upgrade process. It allows the upgrade of VIOS from command line and restore the complete configuration. Many enhancements have been done in the VIOS variant of the tool since the first release and this blog talks about the various new features that users can leverage for an effortless upgrade.

New features that has been introduced are

• Preserving device name and VIOS configurations.
• New options to install additional software during pre and post restoring of the meta data configuration.

We have detailed information about the new features in the below sections.

For more details on syntax and usage refer viosupgrade command.

2.     What’s New

VIOS versions  3.1.2.40 and later have the below features.

2.1    -F option to force the installation

A new option -F in viosupgrade lets the users to force the VIOS installation which overrides certain default parameters. The supported parameters for -F are skiprestore, forcecopy , devname.

• With skiprestore, the restore of the meta data is skipped.  However users can utilise the backup file to manually restore the VIOS metadata configuration after booting to new rootvg using viosbr -restore command. Details of the backup file is displayed in the output of the viosupgrade -q -l command

viosupgrade -l -i /home/padmin/ios_mksys.img -a hdisk40:hdisk41 -F skiprestore

Example use case: The default mksysb image is not customised with the third party software hence users can use -F skiprestore to skip the restore process and can trigger restore manually after installing the third party software.

• With forcecopy, it copies the list of specified files to the respective paths in the newly installed rootvg.  The forcecopy option is applicable only with the -g flag. The list of files that should be copied to the new rootvg should be passed to -g option.

viosupgrade -l -i /home/padmin/ios_mksys.img -a hdisk40:hdisk41 -F forcecopy -g /home/padmin/copy_files.out

copy_files.out contains the list of files with absolute path names. Each line must contain a single filename.

$cat copy_files.out

/tmp/file.c

/tmp/dir1

Two parameters can be combined and passed to -F as below

viosupgrade -l -i /home/padmin/ios_mksys.img -a hdisk40:hdisk41 -F forcecopy:skiprestore -g /home/padmin/copy_files.out

Note: It is recommended NOT to use -F forcecopy with -g for system specific files like /etc/group.The -g flag without the -F forcecopy option copies the files to the location: /home/padmin/backup_files directory in the new rootvg volume group.

Users will need to merge the contents of such files from /home/padmin/backup_files directory with the corresponding files in the respective paths.

Example : User needs to manually merge the contents present in /home/padmin/backup_files/etc/group to the new roovtg’s file /etc/group.

• With devname option, the  device name and state is preserved in the new rootvg. Default VTD name preserve is only available from version 3.1.4.30 on wards.

viosupgrade -l -i /home/padmin/ios_mksys.img -a hdisk40:hdisk41 -F devname

2.2    Preserve VIOS configuration :

2.2.1      System configuration files

viosupgrade can now migrate the following configuration files to new rootvg by default.

• PROFILE: Merges the content of rootvg /usr/ios/cli/.profile and new rootvg.
• ROOTCRON: The root Cron tab entries of rootvg and new rootvg are merged and copied to new rootvg.
• KSHRC: Overwrites the /home/padmin/.kshrc file for padmin only.
• SSHKEY: Overwrites the SSH keys in the new rootvg.
• PASSWD: Preserves the padmin password in new rootvg.
• ROOTUSRFS: Indicates the user file systems on the rootvg disk. For example, contents of optical VMLibrary are copied to new rootvg.
• TUNABLES: User defined values in /etc/tunables are restored on new rootvg.

A new option -X has been introduced that can be used to exclude one or more of the above files from migration. For example: This will exclude migrating the root crontab entries and padmin passwd to new rootvg.

viosupgrade -l -i /home/padmin/ios_mksys.img -a hdisk40:hdisk41 -X ROOTCRON:PASSWD -F devname

In case of failure to migrate  any of the above system configuration files, users can refer the old vg and new vg file placed in '/home/padmin/viosupg_backup' with extensions '_src.viosupg' and '_dest.viosupg' respectively.

Example:

/home/padmin/viosupg_backup/etc/security/passwd_dest.viosupg
/home/padmin/viosupg_backup/etc/security/passwd_src.viosupg

2.2.2      FILESystem configuration

If any user filesystems are created on rootvg, it will be preserved during viosupgrade. All the files of the user file system will be copied to the new rootvg. If users do not want to preserve user file systems on rootvg by default, then “-X ROOTUSRFS” option can be used to exclude the data.

Also, the file system size for the following rootvg default file systems will be preserved:

• /
• /home
• /usr
• /var
• /tmp
• /admin
• /opt
• /var/adm/ras/livedump

2.2.3      Network interface attributes

If any of the network interface attributes like mtu, tcp_recvspace, tcp_sendspace are tuned to achieve optimal performance by the user, it will also be preserved by viosupgrade in the new rootvg.

2.2.4      General configuration attributes

Timezone, motd and VIOS edition will be saved by the viosupgrade tool on to the new rootvg, by default.  For VIOS 4.1 the default VIOS edition would be enterprise.

In viosupgrade case, the prior version VIOS had the default edition as standard so while upgrading to VIOS 4.1 the upgrade process will set the edition to enterprise.

2.3      Install additional software or customisation

2.3.1      Pre-restore:

Users can install any third party software  or perform any other required customisation before the restore process on the new rootvg, using the -k option.

A pre-restore user script should be passed to -k option which gets copied to the new rootvg by the upgrade process. The script will be executed on the new rootvg before restoring the meta data.

viosupgrade -l -i /home/padmin/ios_mksys.img -a hdisk40:hdisk41 -F devname -k /home/padmin/user_script.sh

viosupgrade process does not validate the user script. If there are any failures encountered with the user script, upon correcting those failures viosupgrade operation can be re-run on the new rootvg using below command.

viosupgrade -o -rerun

However, if there are any dependencies associated with the user script then the additional files or dependencies should be copied to new rootvg by the user utilizing   -g option or the mount path containing additional files should be enabled within the user script.

viosupgrade -l -i /home/padmin/ios_mksys.img -a hdisk40:hdisk41 -F devname -g /home/padmin/user_filelist.out -k /home/padmin/pre_restore_user_script.sh

/home/padmin/user_filelist.out contains list of additional files that are needed for user script’s successful execution.

2.3.2      POST-restore:

Users can install any third party software  or perform any other required customisation after the restore process on the new rootvg, using the -P option.

viosupgrade -l -i /home/padmin/ios_mksys.img -a hdisk40:hdisk41 -F devname -P /home/padmin/post_restore_user_script.sh

Note: Since the third party software is not packaged with the IBM supplied mksysb image, user will either use -k or -P to install the third party software.  No further steps are needed if the user has used -k for installing the third party software. But if the user has installed the software                                                                                                          

• using -P  option OR
• installed manually after the upgrade is complete

viosupgrade -o rerun must be executed by the user for successful restoration of the devices.

2.4    viosupgrade mksysb image creation:

viosupgrade provides -I option to accept the ISO image file which is used for creating the mksysb image. The ISO image can be a DVD iso image or flash iso image. From VIOS 4.1 onwards the VIOS software is distributed in a single DVD.

The mksysb image created will be placed in the user provided directory image_dir as shown below.

For prior versions :

viosupgrade -I  /home/padmin/vios3.1.2.x.dvd.iso1: /home/padmin/vios3.1.dvd.iso2 -w /home/padmin/image_dir

For VIOS 4.1 versions:

viosupgrade -I /home/padmin/vios4.1.dvd.iso -w /home/padmin/image_dir

For flash.iso:

viosupgrade -I /home/padmin/vios3.1.2.x.flash.iso -w /home/padmin/image_dir

Welcome to viosupgrade tool.

Creating an ios_mksysb image from the ISO image(s) that are provided. This might take several minutes ...

Creation of the ios_mksysb image is successful: '/home/padmin/image_dir/viosupg/iosmksysb_13017'

3.     Supported levels

4.     Limitations:

4.1     padmin unable to login- premature password expiration

After an upgrade to VIOS 4.1.0.0 or VIOS 4.1.0.10, the padmin user may be unable to login due to premature password expiration. Before running viosupgrade user should apply the ifix to avoid the issue. Please refer the support document for more details on ifix.

https://www.ibm.com/support/pages/node/7145269

4.2    Full cluster restore is not supported.

viosupgrade tool supports backup and restore of VIOS SSP cluster node with “-c” option. However, full cluster restore is not supported through viosupgrade. So in case of SSP cluster, irrespective of number of nodes, user has to upgrade one node at a time keeping cluster up and running on other nodes. Failure to do so, will result in losing the cluster connectivity.

Note: In case of single node cluster, it’s not possible to use viosupgrade tool to upgrade and restore the cluster. User is expected to handle this manually. Alternatively, user can add one more nodes to the SSP cluster before initiating the upgrade process on the first node.

4.3    Active Memory Sharing (AMS) not supported in VIOS 4.1.

If upgrade is performed from versions 3.1.2.x or later to VIOS 4.1.x, the AMS partition configuration needs to be changed to dedicated memory before the upgrade to VIOS 4.1.  Please refer the link for un-configuring the AMS.

If for any reason un-configuring of AMS is missed by the user, there will be stale entries of shared memory pool on HMC. After the upgrade the VBSD (Virtual Block Storage Device) pool and paging devices will not be available on the VIOS. But through HMC we can still see the shared memory pool. It is user’s responsibility to clean up the stale entries. The user should change the memory mode of the LPAR from shared to dedicated and then delete the shared memory pool from HMC.

4.4    FCNVME stale entries present after upgrade to VIOS 4.1

There is change in the unique type of fcnvme devices in VIOS 4.1. Hence if the upgrade is done from 3.1.x to 4.1, after the upgrade the older type fcnvme devices move to defined state, and new type fcnnvme devices also gets configured and are in Available state. As a result the client LPAR will not be logged into NVME domain as shown below through this VIOS. Other redundant VIOS should continue to serve the paths. If there is no redundant VIOS is configured and fcnvmeXX devices are existing, do not specify “-F devname” in the command options.

$ lsdev |grep fcnvm

fcnvme0          Defined     N/A

fcnvme1          Defined     N/A

fcnvme2          Available   FC-NVMe Protocol Device

fcnvme3          Available   FC-NVMe Protocol Device

$ lsmap -all -npiv -proto

Name          Physloc                            ClntID ClntName       ClntOS

------------- ---------------------------------- ------ -------------- -------

vfchost0      U9080.HEX.13575C8-V4-C307              13 2WS_C1_4342    AIX

Status:CONNECTED

FC name:fcs0                    FC loc code:U78CD.001.FZHBW06-P2-C5-T0

Flags:0xa2<LOGGED_IN,SCSI_CLIENT>

VFC client name:fcs2            VFC client DRC:U9080.HEX.13575C8-V13-C307

SCSI Ports:1    SCSI Queues:8    SCSI Status:LOGGED_IN

SCSI Flags:0x8<STRIP_MERGE>

NVME Ports:0    NVME Queues:0    NVME Status:NOT_LOGGED_IN        

Workaround:

• Remove stale 'fcnvme' entries

rmdev -Rdl fcnvme0

• Unmap the FC port

vfcmap -vadapter vfchost0 -fcp

• Remap the FC port

vfcmap -vadapter vfchost0 -fcp fcs0

• Reconfigure the client FC adapter

cfgmgr -l fcs2

4.5    viosupgrade fails if MAX LP value is more than default value.

If the MAX LPs value of rootvg’s default LV is more than the default value(512), viosupgrade will fail with the below error message. For the ifix, please contact support.

Unable to update /alt_inst//home size to 73400320...

Migration upgrade failed.

5.     Additional Information

While the viosupgrade is in progress the VIOS will go for multiple reboots. Please refrain from making any changes while the upgrade is in progress. Once the upgrade is completed, “Upgrade Completed Successfully “ will be displayed on the console.

Alternatively, check the status of the upgrade using below command.

Wait until “Getting status of node(s):viosupgrade COMPLETED” is displayed.

$ viosupgrade -q -l
Welcome to viosupgrade tool.
Getting status of node(s):viosupgrade COMPLETED

Please see /home/ios/logs/viosupg_status.log for viosupgrade status details

Please see /home/ios/logs/viosupg_restore.log for viosupgrade restore details

For related information refer :

1. VIOS migration/upgrade methods
2. VIOS 3.1
3. Upgrading to VIOS 3.1

6.     Contacting the PowerVM Team

Have questions for the PowerVM team or want to learn more?  Follow our discussion group on LinkedIn IBM PowerVM or IBM Community Discussions