Welcome to the world of PowerVC. Usually, everything runs just fine, but if you ran into some problems during or immediately after PowerVC installation or upgrade, you're in the right place! If you want to learn more about installing PowerVC, refer to the Knowledge Center. These links go to 1.4.1, but you can always choose a different release once you're there.
Installing IBM Cloud PowerVC ManagerInstalling IBM PowerVC Standard EditionThis blog has tips and solutions to address these common issues:
Issues seen during PowerVC Install/Upgrade due to:
Repo errors
RHEL update errors
Gpgcheck (rpm signing) errors
iFix not installed
Non-availability of required ports
Non-compatibility of rpms
Issues seen after PowerVC Install/Upgrade:
Authentication failure due to version conflict errors
Configuration issues or PowerVC login issue after install/upgrade of PowerVC due to service not starting up
Some other software from IBM stopped working after PowerVC upgrade
Error in host name/IP configuration of PowerVC after PowerVC install/Upgrade
1. Repo errors
If an install or upgrade fails with errors about some required packages not being available to PowerVC, it might be because the repositories are configured improperly. Generally, repo files appear under
/etc/yum.repos.d.
To check if there is a repo issue, run the command:
yum repolist. This command will list all the repos that are set up. If there is any issue, the repolist command will report that issue. It generally includes issues such as the following:
a. Your repo network is not reachable - Give the correct URLs for your repos and verify that those URLs are reachable.
b. The host name of the repo URL is not resolvable - Verify that the /etc/resolv.conf file is properly set up. This file contains the URL for the DNS server that is used to resolve a host name to an IP address.
c. Authorization errors - Verify that the certificates (or any other authentication mechanism) required for reaching the repo are set up properly.
You will have to check with your repo provider for any of the above issues.
2. RHEL update errors
Updating RHEL OS by using a command such as:
yum update can result in installation of:
- Packages that are at a level above the required level for PowerVC.
- Bad packages (bad packages can include name mismatch of the package).
To avoid this issue, it is recommended to install PowerVC with the correct repo set up (as described in point 1 above) before performing any updates (that is, updating any packages by using
yum update) on the system. This will ensure that there are no conficts in packages needed by PowerVC.
If you have already performed the update and PowerVC complains about certain packages in its prerequisite check, downgrade/remove those bad packages.
Tip: You can (and should) do a prerequisite check before PowerVC installation by using the option 't' with the PowerVC install command, i.e.
./install -t. This checks that the required rpms are available from your system's already set up repo and PowerVC local repo.
Following is an example showing prerequisite check output:
# ./install -t
Checking available memory
At least 10 GB of physical memory is recommended.
...
Current maximum file size : unlimited
...
Required disk space for installation: 6.5 GB
Verifying required ports are available
Required port 5000 is available.
...
All the required ports are available
...
repo id repo name status
powervc-noarch IBM PowerVC noarch 33
powervc-ppc64 IBM PowerVC ppc64 2
pvc-openstack-noarch IBM PowerVC Openstack noarch 249
pvc-openstack-ppc64 IBM PowerVC Openstack ppc64 31
...
######################################################
One or more RPMs are not available for install.
Ensure these packages are available for install through yum configurations or
are installed prior to installing IBM PowerVC.
pexpect >= 2.3
...
expect
/usr/bin/nslookup is required for a system check but is not installed.
Install the bind-utils RPM for the system and run the install again.
Check the /opt/ibm/powervc/log/powervc_install_2018-07-09-052558.log
for details on the errors.
###############################################################################
###############################################################################
3. Authentication failure due to version conflict errors:
If you installed PowerVC successfully, but run into authentication problems when you log in to the PowerVC UI using root user soon after installation, you are most likely running in to this error.
To investigate further - open
/var/log/keystone/httpd-error.log and search for
VersionConflict. The error typically looks something like the below:
raise ArgsAlreadyParsedError("cannot register CLI option")
ArgsAlreadyParsedError: arguments already parsed: cannot register CLI option
raise VersionConflict(dist,req) # XXX put more info here
VersionConflict:(/usr/lib/python2.7/site-packages),Requirement.parse('...')
Packages previously installed from other sources typically cause this. For example, by running the command
pip install package-name. In this case, yum gets confused and gives version conflict errors during PowerVC installation.
You can determine whether duplicate packages are on the system by following these steps:
a. Run rpm -qa | grep package-nameCheck for the package that is giving the
VersionConflict error. If that package is listed, then check below:
b. Run ls /usr/lib/python2.7/site-packages | grep package-nameFrom step a. above, you will obtain the installed version of the package. If the above command lists duplicate packages but with with a different version than the installed version of that package, then remove the redundant packages (Be careful here -> Remove only the package that was giving the
VersionConflict error). To remove these redundant packages, remove the *.egg-info files/directories for the redundant package.
c. Run powervc-services keystone restartRestart the identity service to pick up the above changes.
d. Try to log in to PowerVC again.4. Gpgcheck (rpm signing) errors:
Note: PowerVC 1.4.4.0 has gpg signed rpms and therefore this error is not expected for our shipped rpms in PowerVC 1.4.4.0.
PowerVC Installation may fail in the middle with the following error:
Package python-xxx-3.4.0-1.noarch.rpm is not signed
This happens when repos have gpgcheck enabled. The local packages shipped by PowerVC are not signed and therefore the check for signed packages fails.
Repos for a RHEL system are generally set up under
/etc/yum.repos.d directory with an extension of .repo. These are needed for getting the required packages installed on a Red Hat system. If your repo provider provides you with repos with gpgcheck enabled (set to 1), then yum checks whether the rpms being installed are signed or not. Installation of an rpm fails if it is not signed and gpgcheck is enabled.
To fix this problem, check all your repos under
/etc/yum.repos.d and set the
gpgcheck flag to 0 (disabled) before you start PowerVC installation because PowerVC rpms are currently not signed.
If the installation fails again, check for the
gpgcheck flag in
/etc/yum.conf and set it to 0.
5. Configuration issues or PowerVC login issue after install/upgrade of PowerVC because a service didn't start:
The PowerVC install may fail with configuration errors such as:
a. Configuration error appears after PowerVC install:Following is a sample error that can appear:
The validation of PowerVC Services post install was not successful. Validation log file is available at /opt/ibm/powervc/install/piv_validate.log ************************************************************ There were errors detected with a set of configuration tasks. Check the /opt/ibm/powervc/log/powervc_install_2017-08-08-032324.log for details on the errors.
To verify that this is the issue, check the validate log and see if some service is shown as inactive/failed. Alternatively, use the
/opt/ibm/powervc/bin/powervc-services status command to check if some service is in "inactive/failed" state.
b. The /opt/ibm/powervc/bin/powervc-services status command shows some service as "inactive/failed":This error appears if the configuration for a service is not correct.
c. Login to PowerVC fails with "Internal Server Error":Again some service may be down, causing the issue.
d. After logging in to PowerVC, the messages area in the UI shows errors:The UI may report some Internal Server Errors when logged in to PowerVC. These can be due to an issue with some service.
This happens when there is some configuration issue or rpm installation issue. In this case, check the logs under
/var/log for that service and find the actual error that is causing the service to fail. This can be Traceback/Version Conflict errors. Many general issues are also logged in
/var/log/messages apart from the component specific logs.
Report these errors to the PowerVC support team, who can provide further help to provide a quick fix.
6. Some other software from IBM stopped working after upgrading PowerVC:
During an upgrade, the uninstallation of the previous version of PowerVC may remove an rpm that is needed by other IBM software that is running on the same system. That can happen due to a common naming convention. All rpms shipped by PowerVC contains
.ibm. in its name. During uninstallation, any rpm having name
*.ibm.* will be uninstalled (in PowerVC versions 1.4.0 and 1.4.1).
To determine whether any of your rpms were accidentally uninstalled, check
powervc-uninstall.log under
/var/log and review the rpms that were uninstalled during PowerVC uninstallation. You may have to open a bug in PowerVC to get this resolved.
7. Other error in install/upgrade due to a missing iFix:
An error in the install or upgrade may have been caused because you're missing an iFix. Before installing or upgrading PowerVC, be sure to check for any intermediate fixes (iFixes) that need to be applied. iFixes for each release can be found at
IBM Fix Central. There can be critical fixes that may need to be applied without which PowerVC installation will fail.
8. Error in the host name or IP configuration of PowerVC after a PowerVC install/upgrade:
After PowerVC installation, the host name/IP address with which PowerVC can be accessed on a web browser is displayed. If for some reason that host name or IP address is not properly set, it will give you errors such as:
a. IP address / Host name is not visible at the end of installation:
As seen above, there is no IP address or host name listed after a successful PowerVC installation.
b. After a successful PowerVC Installation, you may not be able to log in to PowerVC:
An error such an "Internal Server Error" may appear.
c. After a successful PowerVC installation, the PowerVC UI does not open on the web browser:
The system host name or IP address may have been changed after PowerVC installation and PowerVC does not recognize the new address.
Verify that your system is publicly accessible within your organization; it should have a public IP address. If the IP address is OK, run
hostname -f and verify that the full host name of your system is displayed. When installing PowerVC, the host name is used as the default way to access PowerVC. But if your host name is not resolvable to an IP address, PowerVC will take the system IP address and configure it to be accessible on that IP address.
To check if your host name is resolvable to an IP address, use one of these methods:
Run
ping hostname, the host name should be pingable to configure PowerVC using host name. When the host name is pingable, the IP address of the system will also be listed along with its host name:
ping hostname:
PING hostname (
This shows that the host name is resolvable to an IP address. Additionally, it should be the IP address that was used to configure PowerVC to use the host name of the system.
Use nslookup to see the host name / IP address mapping.
nslookup hostname
Server: Some_IP
Address: Some_IP
Non-authoritative answer:
Name: hostname
Address: IP
A similar result can be obtained by running
nslookup IP. The host name and IP address should match the system configuration.
Once the system configuration for the host name and IP address is done properly on the system which PowerVC is installed, run
powervc-config general ifconfig --set. This will update all the configuration related files of PowerVC with the updated host name and IP address.
9. Non-availability of required ports:
This is an issue found when a user uninstalls a previous version of PowerVC and tries to install the latest version of PowerVC. During uninstallation of PowerVC, it tries to clean up all processes, including all ports used by PowerVC. But there are sometimes ports that are left behind due to the creation of a "zombie" process.
Example of such an error:
These required ports are not available: 9696
Cleaning yum repository files.
Configuring yum repository files.
......
repo id repo name status
powervc-noarch IBM PowerVC noarch 33
powervc-ppc64 IBM PowerVC ppc64 2
......
......
......
Maybe you want: rm -rf /var/cache/yum, to also free up space taken by orphaned data from disabled or removed repos
###############################################################################
......
The required ports are in use. Make these ports available,
then run the installation again.
There were errors detected during install prerequisite check.
Check the /opt/ibm/powervc/log/powervc_install_201.....log for details on the errors.
To clean up this process, restart the host system on which PowerVC is installed. After rebooting the host system, try the PowerVC installation again.
10. Non-compatibility of rpms:
During installation of PowerVC, there can be issues seen similar to:
AttributeError: 'module' object has no attribute 'something' or package conflicts like:
file /usr/lib64/python2.7/site-packages/ldap/controls/sessiontrack.py from install of python2-ldap-3.1.0-2.ibm.el7.x86_64 conflicts with file from package python2-pyldap-2.4.37-5.ibm.el7.x86_64. This can be due to an incompatibility of installed rpm with one of our shipped rpms. We therefore suggest customers to not install software other than PowerVC on the management node. Please refer the "Installing other software" section of this knowledge-centre
link.
The solution to this is removal of a pre-installed rpm which is causing issue in PowerVC installation. To figure out this incompatible rpm, the PowerVC install logs at
/opt/ibm/powervc/log/powervc_install_201.....log can be analyzed. Otherwise reach out to PowerVC support team for the resolution.
For example: a. An issue seen during PowerVC 143 install
File "/usr/lib/python2.7/site-packages/paramiko/ssh_gss.py", line 55, in
This is because
python-gssapi provided by RHEL was not compatible with our shipped
python-paramiko rpm. The solution is removal of
python-gssapi rpm since it is not installed/used by PowerVC.
b. Another example is as below - where package conflict happened during upgrade from PowerVC 1.4.2.0 to latest PowerVC:
file /usr/lib64/python2.7/site-packages/ldap/controls/sessiontrack.py from install of python2-ldap-3.1.0-2.ibm.el7.x86_64 conflicts with file from package python2-pyldap-2.4.37-5.ibm.el7.x86_64
file /usr/lib64/python2.7/site-packages/ldap/controls/sessiontrack.pyc from install of python2-ldap-3.1.0-2.ibm.el7.x86_64 conflicts with file from package python2-pyldap-2.4.37-5.ibm.el7.x86_64
This is because
python2-pyldap was shipped by us in PowerVC 1.4.2.0. And during upgrade to latest PowerVC, this package ideally gets removed, but it remained on the system and caused the package conflict. PowerVC upgrade fails due to this. The conflict happened because there was an external software installed on the system which required
python2-pyldap and so according to our uninstall logic we did not remove it.
Solution to this is to remove
python2-pyldap manually by using command:
/usr/bin/rpm -e --nodeps python2-pyldap and trying the upgrade again.
Conclusion:
It is because of our clients/partners that we have come to know about several install issues, because every customer environment is different. If you find any issues during PowerVC install or upgrade that are not addressed here, please raise an issue with our support team and we will help you with it. We thank our customers who cooperated with us and had the patience to get these kind of issues resolved!
Keep up with the latest PowerVC information by following us on
Facebook,
LinkedIn, and
Twitter.
#troubleshooting#troubleshoot#install#powervc