IBM Cloud Pak System

 View Only

vCenter STS certificate monitoring tool

By Sanjeev Pradhan posted Tue May 31, 2022 08:10 AM

  

VMware vCenter STS certificate expiry monitoring in
IBM Cloud Pak System

 

What is VMware vCenter self-sign certificate

  • The vCenter Single Sign-On includes a Security Token Service (STS),  which is a web service to issue, validate, and renew security tokens.
  • vCenter Single Sign-On Security Token Service (STS) signing certificate is an internal VMware certificate.
  • It authenticates the user that is based on the primary credentials and constructs a SAML token and signs it with an STS signing certificate.

Problem statement:

When the VMware vCenter services in IBM Cloud Pak System are down due to non-renewal of STS certificate, manageability of virtual machines is lost, though virtual machines may be running.

The VMware vCenter self-signed certificate expires every two years, which affects production instances.

The certificate is extended automatically for two years when IBM Cloud Pak System version is upgraded. But since some customers won’t have updated with the fix pack as a result of which the certificate expires. There is no way for the customers to know this certificate expiry and therefore they cannot renew it in time.

IBM Cloud Pak System development has introduced a feature that validates the expiry of certificate and alerts system users about it after which they can remediate the issue.

Monitoring VMware vCenter STS certificate expiry in version 2.3.3.4:

1) Checks validity of the certificate for every hypervisor in the VMware vCenter and gets number of days for expiry.

2) A warning event will be fired which has the following text “CWZIP1344W VMware STS certificate is going to expire in 90 days, Please engage IBM Cloud Pak System Support team to renew STS Certificate.”  

When the threshold of STS certificate expiry is reached, i.e. 30 days, the critical event for “Call Support” is raised every day. This event has the following text, “CWZIP1345E: Please renew STS certificate immediately. VMware STS certificate is going to expire in 30 days”.
3) The generated Job and Event can be viewed in the IBM Cloud Pak System user interface from the Problem determination > Job Queue  and Problem determination >  Events menu.

See the following figure for reference:




4) The event generates every day until expiry (the system administrator must contact IBM Support to renew the certificate).

To summarize: "STS certificate expiry monitoring feature generates a warning event if expiry date is lesser than 90 days compared to current date. Escalate event to critical once 30 day threshold is reached".

Monitor the events for STS certificate:

The admin users can monitor the Warning and critical events and take appropriate actions i.e renew the STS certificate. At 30 days expiry, the event would be escalated to “Call Support” critical event which will be generated daily until certificates are renewed. No Call Home support ticket is generated.

How to check VMware vCenter certificate validity with command

You can check the certificate validity from a web browser by accessing the vCenter URL.




IBM Cloud Pak System job to monitor certificate expiry

 

  •  An internal job “monitor_VMwareSTSCertificate” is run to monitor the certificate expiry:


    See the logs in job as follows:

    pooljvm.1646992185141.25984 [03-11-22 09:50:53] 0031 virtual_management_instances.VMwareVCenterCLIHelper |         **INVALID** Please renew STS certificate, expires in 3566 days.

     

    pooljvm.1646992185141.25984 [03-11-22 09:50:53] 0031 virtual_management_instances.VMwareVCenterCLIHelper |        [] Certificate C5:88:A9:ED:45:A2:93:46:C3:F7:C6:13:98:4F:15:CA:E7:A8:DA:43 will expire in 3566 days (10 years).

    pooljvm.1646992185141.25984 [03-11-22 09:50:53] 0031 virtual_management_instances.VMwareVCenterCLIHelper |         **INVALID** Please renew STS certificate, expires in 3566 days.

     

    pooljvm.1646992185141.25984 [03-11-22 09:50:54] 0031 common.EventUtils | Raising WARNING/ALERT virtual management instance event CWZIP1344W for virtual management instance [id: cd30f188-74c2-40dd-84c8-c50a1c228ab6]

    pooljvm.1646992185141.25984 [03-11-22 09:50:54] 0031 common.EventUtils | {created_time=1609588010777, updated_time=1646322151300, role=primary, type=virt_mgmt_node, physical_memory=[], vms_id=null, routes=[], name=pureVCenter-W2012R2, physical_cpus=[], options=, locations=[, 2e4b75b8-7331-463e-b238-53c64c6c6e4e], id=cd30f188-74c2-40dd-84c8-c50a1c228ab6, state=available, software_version=20191122.0000, vms_uuid=null, events=[], virtual_management_systems=74839bbe-e2f0-4497-bc0e-e0d2d2cf56c5, compute_nodes=[]}

    pooljvm.1646992185141.25984 [03-11-22 09:50:54] 0031 com.ibm.purescale.event.EventRaiserHelper | raiseEvent Raising event: Parent Type, Virtual Management Instance Parent id, cd30f188-74c2-40dd-84c8-c50a1c228ab6 Detail:CWZIP1344W VMware STS certificate is going to expire in 3566 days, Please engage IBM Cloud Pak System Support team to renew STS Certificate.

    pooljvm.1646992185141.25984 [03-11-22 09:50:54] 0042 com.ibm.purescale.users.LdapService | <clinit> Optional /etc/purescale/ldap.properties not found

    This action generates the certificate monitoring job successfully. See the following figure for reference:



Overall summary

The VMware vCenter STS certificate expiry automated event is implemented to alert the customer on expiry of the certificate in advance so that remedial action can be taken and production operations are not disrupted. The customer must engage with IBM Support to plan the certificate renewal.


Blog author details

Sanjeev Pradhan is a senior staff engineer SVT team, Cloud Pak System.


Reviewers

Mohan Manjappa ( Cloud Pak System development team)
Hina Sharma (Cloud Pak System development team)

Anil Hegde (Information Development and Design team, Cloud Pak System)


0 comments
8 views

Permalink