PowerVC

Expand all | Collapse all

PowerVC can't connect to HMC after HMC SSL Certificate update with a signed CA

  • 1.  PowerVC can't connect to HMC after HMC SSL Certificate update with a signed CA

    Posted Thu October 07, 2021 10:13 AM

    Hello guys,

    I'm having a weird bug with my PowerVC 1.4.4 that can't connect to our HMC since I update the HMC console SSL certificate with a new one that is signed by my company CA. So far :

    1. I restarted the PowerVC services with the command : "powervd-services restart"
    2. I tried to reconnect the HMC to update the TrustStore without succes. Here the error message for that try :
      1. PD-A000122 Connection failed to the Hardware Management Console (HMC). Verify that the HMC is running and accessible, its firewall allows incoming access on port 443, and the HMC version is V8R8.7.2 or later, V9.1.920 or later.
      2. I did check the and the HMC are all available and accessible by the web console and the SSH
    3. I manually update the Trust Store to add our Root and intermediate certificate
    4. I check the log for nova (/etc/log/nova/nova-compute-myHOST.log) and I have this error message
      1. ERROR oslo_service.service [req-b96c0902-3ca5-4b83-921d-3c4b44d38198 - - - - -] Error starting thread.: K2SSLError: SSLError for PUT https://myHMCHostname:443/rest/api/web/Logon: HTTPSConnectionPool(host='MyHMCHostname', port=443): Max retries exceeded with url: /rest/api/web/Logon (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",),))
    Any idea someone :)

    Thank you and have a nice day :)

    Nicolas

    ------------------------------
    Nicolas Rousseau
    ------------------------------


  • 2.  RE: PowerVC can't connect to HMC after HMC SSL Certificate update with a signed CA

    Posted Fri October 08, 2021 03:20 AM
    Error message to me looks like invalid certificate. One can check and validate if the certificate chain is proper or not.

    The below steps 1 to 4 can be used for the same:
     

    References:
    If the certificate was a CA-signed certificate with server, intermediate CA and root CA certificates. I split it three pieces with cert00 being the server cert, cert01 being the intermediate and cert02 being the root CA.
    1. Shows the CA and intermediate and server is signed correctly
    # openssl verify -verbose -CAfile cert02.pem -untrusted cert01.pem cert00.pem
    cert00.pem: OK
    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    2. Below shows how the subject and issuers of each of the certificates are correctly aligned
    # root CA
    # openssl x509 -in cert02.pem -noout -issuer -subject
    issuer= /C=US/ST=Hogwarts/L=Vienna/O=HArry Potter Credit Union/OU=HArry Potter Private/CN=HArry Potter Credit Union Root CA R3
    subject= /C=US/ST=Hogwarts/L=Vienna/O=HArry Potter Credit Union/OU=HArry Potter Private/CN=HArry Potter Credit Union Root CA R3
    # intermediate CA
    # openssl x509 -in cert01.pem -noout -issuer -subject
    issuer= /C=US/ST=Hogwarts/L=Vienna/O=HArry Potter Credit Union/OU=HArry Potter Private/CN=HArry Potter Credit Union Root CA R3
    subject= /C=US/ST=Hogwarts/L=Vienna/O=HArry Potter Credit Union/OU=HArry Potter Private/CN=HArry Potter Credit Union SSL CA01 I2
    # server cert
    # openssl x509 -in cert00.pem -noout -issuer -subject
    issuer= /C=US/ST=Hogwarts/L=Vienna/O=HArry Potter Credit Union/OU=HArry Potter Private/CN=HArry Potter Credit Union SSL CA01 I2
    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    3. General validation of certificate
    # without root CA or intermediate in trust store
    # openssl verify powervc.crt
    powervc.crt: C = US, ST = Hogwarts, L = Vienna, O = HArry Potter Credit Union, OU = ISD, CN = hermionegranger.test.net, emailAddress = enterprise_unix_engineering@harrypotter.org
    error 20 at 0 depth lookup:unable to get local issuer certificate
    # With only root CA in trust store (no intermediate CA in trust store)
    # cp cert02.pem /etc/pki/ca-trust/source/anchors/
    # update-ca-trust
    # openssl verify powervc.crt
    powervc.crt: C = US, ST = Hogwarts, L = Vienna, O = HArry Potter Credit Union, OU = ISD, CN = hermionegranger.test.net, emailAddress = enterprise_unix_engineering@harrypotter.org
    error 20 at 0 depth lookup:unable to get local issuer certificate
    # With root CA and intermediate in the trust store
    # cp cert01.pem /etc/pki/ca-trust/source/anchors/
    # update-ca-trust
    # openssl verify powervc.crt
    powervc.crt: OK
    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    4. You can use the website https://tools.keycdn.com/ssl to check the certificate chain
    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    ------------------------------
    Vijayendra Radhakrishna
    ------------------------------



  • 3.  RE: PowerVC can't connect to HMC after HMC SSL Certificate update with a signed CA

    Posted Wed October 13, 2021 07:22 AM
    Hello,

    Thank you for your completed answer. Sadly, the management decided to move back to self-signed certificate for the HMC for now. So I will probably never see the end of it.

    The good news is that returning to self-signed did fix the connection issue between HMC and PowerVC.

    Thanks again and have a nice day :)

    ------------------------------
    Nicolas Rousseau
    ------------------------------