Introduction

Automatic HMC redundant support is a way to automatically change over the host connection in PowerVC to another HMC without any user intervention.

Till PowerVC 1.4.2, system management was supported via single HMC. If the active HMC failed or put under maintenance, user had to manually switch over connection to another HMC from PowerVC or connect the secondary HMC and edit the host connection in PowerVC to make it work. Until it is done, the host and the VMs remained in error state.

With redundant HMC support feature, a host can be connected to multiple number of HMCs. Host will have one primary HMC and many secondary HMC connections. PowerVC uses only primary HMC to perform management operations. In the case where the primary HMC is down or in error state, PowerVC automatically changes-over the connection to the secondary HMC without any user intervention.

However, regardless of this feature, you can manually change-over the connection to secondary HMC for planned HMC maintenance or downtime, and this will make the selected HMC as the primary HMC till the next failover happens.

Restrictions and Limitations

It is recommended to maintain the HMC versions at the same level to keep the hosts operational after an HMC connection failover.

Supported Features and Functions

• Enable users to connect both redundant HMCs to PowerVC.


• No limit on the number of HMCs registered for a host in PowerVC.


• The primary HMC and secondary HMC connection for a host is determined based on the registration order of the HMC in PowerVC.


• First HMC from the list of registered secondary HMCs is used for connection failover.


• Users can edit primary HMC connection via “Edit Host Connection”.


• Supports “Dual HMC and Redundancy” feature provided by HMC for high availability of hosts.


• Out of band host additions or removal on the HMC side is taken care by setting the secondary HMC attribute for the hosts registered in PowerVC.


• Functionality is enabled by default in PowerVC.

Possible cases where the automatic failover can happen

• The primary HMC has connection issues. This might be because of the inability to access the HMC APIs or the HMC was shut down abruptly.


• Password of primary HMC is updated after registering the HMC in PowerVC.


• The SSL certificate of primary HMC got corrupted or has any issues.


• Out of band host removal on the primary HMC.

Using the automatic HMC redundancy feature

As part of the host registration, the selected HMC will be tagged as a primary HMC for that host and all other registered HMCs in which this host is connected will be set as secondary HMC. In a case where the hosts are already registered in PowerVC and there’s a new HMC that is getting added, this will be set as a secondary HMC for any of the host that is also managed by this HMC.

Primary and Secondary HMCs

Host will have one primary HMC and any number of secondary HMCs.




If the host has more than one secondary HMCs registered, then the first HMC from the list will be used for failover connection and in GUI display. You can see the primary and secondary HMCs for the host on the host details page.

Host Detail Page: PowerVC 1.4.2.0 and earlier versions

Host Detail Page: PowerVC 1.4.2.1 and later versions

To toggle the host's primary HMC connection, go to Hosts, select a host, and click Edit Host Connection option.
Given below is an example.

You can see that the primary and the secondary HMC connections have changed.


Removing the primary HMC of a host automatically sets the first registered secondary HMC as the “New” primary HMC for the host.





NOTE: In a case where a specific host doesn’t have a secondary HMC connection set, then the host will be removed from PowerVC as part of the HMC de-register process.

Automatic Switching of HMC Connection

If the API calls to the primary HMC failed, then PowerVC will automatically failover the host HMC connection to the secondary HMC. The average time taken to failover HMC connection is set to 10-12 minutes.





NOTE: The failover to the secondary HMC succeeds if the secondary HMC is accessible. Otherwise, the failover will be blocked and an error message will be shown in the console.
Let us now see some troubleshooting scenarios you might come across while setting up the primary and secondary HMC connections for a host.

General Troubleshooting

Case 1:

REST API issues or problem with registration or de-registration of a host or an HMC.
Solution:
Check the Nova API logs.

Case 2:
Issue with edit host connection.
Solution:
Validate the HMC connection used for changing host connection. Check the Nova API logs.

Case 3:
If the out of band host additions or removal on the HMC side changes do not reflect.
Solution:
Probably an issue with the periodic task that handles this. Investigate compute logs.

Case 4:
Failover issues.
Solution:
Probably issues with the periodic inventory job. Investigate the Compute logs on the controller.

Case 5:
Recurring issues in the background.
Solution:
Mostly due to the connection validation of the Primary HMC/Secondary HMC, during inventory task. Check the HMC status and network connection. Investigate the Compute logs.

Further Reading

For more details about using PowerVC with automatic HMC Redundant support, refer to the documentation in the Knowledge Center, and look for updates on the HMC support when PowerVC 1.4.3 is released.

• PowerVC KC Documentation


• HMC KC Documentation

As you can see, it’s quick and easy to switch over the host HMC connection without manual intervention using HMC Redundancy feature. If you have any questions, feel free to add them below. We’d love to hear from you! And don’t forget to follow us on Facebook, LinkedIn, and Twitter.

#Compute