File and Object Storage

Upgrade with Excluding the node(s) using Install-toolkit

By Archive User posted Sun September 30, 2018 04:46 AM

Install-toolkit can be used to upgrade core GPFS, NFS, SMB, CES, object, callhome, fileauditlogging, management GUI, and performance monitoring from the current version of IBM Spectrum Scale to the next higher version.

There was a limitation with install-toolkit prior to 5.0.2 release, all nodes of the cluster must be reachable and active. If any of the node is not reachable, upgrade precheck detects the issue and toolkit doesn't allow you to proceed for the upgrade. In this situation, excluding node feature is critical from a customer perspective, and that's why this feature is introduced in the 5.0.2 release. Now, Install toolkit is capable of doing upgrade with unreachable nodes if they are added in excluded list.

A more detail about adding a node in excluded list:

A node is considered to be unhealthy and can be added in excluded list, if node is not reachable or user don't want to upgrade that node.

Behavior of install-toolkit on excluded node:

The install-tollkit will not act upon that node. That means, toolkit will not try to connect with excluded node. And also there will be no service check, no package upgrade etc. on that node.

Potential use cases for excluding the nodes:

* Node is not reachable or powered down
* Node can not be upgraded during a particular maintenance window
* Controlling which subset of nodes are upgraded

Following is the CLI command to add a node in excluded list:

root@vm1:/usr/lpp/mmfs/ upgrade config exclude -h

usage: spectrumscale upgrade config exclude [-h] [-N NODE] [--clear]


./spectrumscale upgrade config exclude -N,

Sample output layout of upgrade config list, after adding vm3 into excluded list:

* You can add one or more(all) nodes into the excluded list.

After adding a node in excluded list, you can run upgrade precheck:

Once upgrade precheck is successful, you can run upgrade:

[ WARN ] The cluster does not have quorum to maintain a healthy cluster during the upgrade process. You must bring online some of the offline or excluded nodes to maintain quorum before proceeding.
The upgrade process is divided into two phases. In Phase 1, all non-protocol nodes will be upgraded. In Phase 2, all protocol nodes will be upgraded. The upgrade process might cause a brief outage of Object, SMB, NFS, and Performance Monitoring components. Do you really want to begin upgrading? [y/N]: Y
INFO] Repo Version: 5.0.2-0.el7
[INFO] Version of gpfs.gss.pmsensors installed on 5.0.2-0.el7.
[INFO] With 5.0.0 and later releases, the install toolkit enables callhome by default. To use callhome, you must add the callhome configuration data (./spectrumscale callhome config -h) and then run a ./spectrumscale deploy or ./spectrumscale install to apply the callhome configuration. If you do not want to configure callhome at this time, disable callhome in your cluster (./spectrumscale callhome disable)
[INFO] ---------------------------------------------------------
[INFO] Upgrade Success
[INFO] Successfully upgraded components: "license, gui, object, gpfs, nfs, ces, smb, zimon, callhome, fileauditlogging" on 2 node(s): "," in 38 minute(s) and 19 second(s).
[INFO] ---------------------------------------------------------
[INFO] Post Upgrade recommendations
[INFO] ----------------------------
[WARN] The following nodes are still marked as excluded and were not upgraded:,
[WARN] Run ./spectrumscale upgrade config exclude --clear to remove nodes from the toolkit excluded list once desired.
[WARN] Nodes that are already upgraded should be added into the exclude list so they are not impacted when the nodes that were previously excluded are upgraded
[WARN] To complete the upgrade on previously excluded node run './spectrumscale upgrade run' when ready.
[INFO] Once all nodes in the cluster have been successfully upgraded, consider running 'mmchconfig release=LATEST' to migrate the cluster configuration and enable new functionality throughout the cluster. Consider running 'mmchfs Device -V full' to enable all new functionality requiring different on-disk data structures. These commands commit the newly upgraded code levels to the cluster. Review the following topic in the Knowledge Center for further details 'Completing the upgrade to a new level of IBM Spectrum Scale'.

For more detail visit:Performing offline upgrade or excluding nodes from upgrade using installation toolkit