IBM Security Verify

 View Only

Containerized Verify Access : How to enforce connection Verification when fetching snapshot - CONFIG_SERVICE_TLS_CACERT

By Adarsh Nair posted Thu May 16, 2024 04:24 AM


Co Authored By Tushar Prasad


IBM Security Verify Access container has multiple components namely,

·      Verify Access config container,

·      Verify Access Snapshot Manager,

·      Verify Access Reverse Proxy container,

·      Verify Access Runtime Containers,

·      Verify Access Distributed Cache containers (DSC)

Verify Access Config container allows all configuration changes and other container connects on HTTPS connection to fetch the configuration snapshot.

IBM Security Verify Access introduced on how participating containers can Verify HTTPS connections to ensure the snaphost came from a reliable source.

What configuration setting this document talks about ?

This document talks about CONFIG_SERVICE_TLS_CACERT.

TLS Certificate for Configuration Snapshot Service

The configuration snapshot service was changed to require a new environment property CONFIG_SERVICE_TLS_CACERT. This property defines a X509 certificate bundle, which can be used to verify connections to the configuration snapshot service. If this property is missing, then the configuration snapshot is not retrieved from the configured HTTPS URL and the container does not bootstrap.


The CA certificate bundle that is used to verify connection to the configuration snapshot service. 


The file prefix and the path to a PEM formatted certificate bundle. For example: file:/path/to/ca.pem

This CA bundle will be used to verify server certificate during the TLS communication with the config service to download the latest published snapshot during bootstrap process by other components like wrp, dsc and runtime.

The service name used to access the config service must match with the Subject CN or one of the configured SAN of the LMI Management certificate.

Step 4 and Step 5 is an optional step which customer's do not need to do if they already have matching config service name with the subject CN or one of the configured SAN of the LMI Management certificate.


1. Review the current certificate used by config container.

Access the config container LMI.

Navigate to System->Management SSL Certificate.

The reason we are looking into this certificate because this certificate is presented by the config container to the various containers (wrp,dsc and runtime, etc) during the configuration snapshot download request.Therefore wrp,dsc and runtime etc containers needs to trust this certificate.
This is reason why CONFIG_SERVICE_TLS_CACERT is set in wrp, dsc ,runtime and snapshot manager containers.

2. Get the CA bundle for the LMI Management Certificate.

In this case we will use default self signed certificate of the LMI.Download the signer of the certificate from browser or openssl.

We have a route configured for accessing the config container LMI.

Below openssl command can be used to download the signer certificate of LMI.
openssl s_client -showcerts -host -port "443" < /dev/null | sed -ne '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/p'  > configsvccabundle.crt

In below case we will set CONFIG_SERVICE_TLS_CACERT property to use/tmp/configsvctlscabundle.crt’. We will make this file available to container via a secret.

3. Use the LMI CA cert bundle to create a secret.

oc create secret generic configsvc-ca --from-file=configsvccabundle.crt=configsvccabundle.crt

This secret will be used to mount the file in following path /tmp/configsvctlscabundle.crt.

You can rsh to container and verify the same.

4. Create service to access the config service. (Optional)

The service name must match with the certificate Subject CN of the LMI Management certificate or alternately you could use any SAN configured for the LMI Management certificate.

apiVersion: v1
kind: Service
    app: verifyaccessconfig
    name: verifyaccessconfig-config-569fd7f857-hn9vl
    template: verify-access-configRegistry 8c252c25-f086-4488-bbe6-baaa27e1ddf6
  name: verifyaccessconfig-config-569fd7f857-hn9vl
  namespace: isva-10070
  - name: lmi-configsvc
    port: 9443
    protocol: TCP
    targetPort: 9443
    name: verifyaccessconfig-config
  sessionAffinity: None
  type: ClusterIP

Modify the CONFIG_SERVICE_URL parameter to use certificate CN of the LMI Management certificate or alternately you could use any SAN configured for the LMI Management certificate.

5. Modify secret with updated config service URL "https://verifyaccessconfig-config-569fd7f857-hn9vl:9443/shared_volume".(Optional)

As we are reading the CONFIG_SERVICE_URL from secret named ‘verifyaccessconfig-config’, we set the respective config-service-url parameter to use service configured in above step.

apiVersion: v1
  config-read-password: UGFzc3cwcmQ=
  config-read-username: Y2Znc3Zj
  config-service-url: aHR0cHM6Ly92ZXJpZnlhY2Nlc3Njb25maWctY29uZmlnLTU2OWZkN2Y4NTctaG45dmw6OTQ0My9zaGFyZWRfdm9sdW1l
kind: Secret
    app: verifyaccessconfig
    template: verify-access-configRegistry 8c252c25-f086-4488-bbe6-baaa27e1ddf6
  name: verifyaccessconfig-config
  namespace: isva-10070
type: Opaque

On decoding above config-service-url, we get the below value.

echo aHR0cHM6Ly92ZXJpZnlhY2Nlc3Njb25maWctY29uZmlnLTU2OWZkN2Y4NTctaG45dmw6OTQ0My9zaGFyZWRfdm9sdW1l | base64 -d

How to disable certificate validation:
An option has been given to disable the certificate validation for backward compatibility.

At this point, this article shows how WRP,DSC,Runtime and Snapshot Manager pod can be configured to trust the Config container snapshot service url certificate while trying to get the snapshot. The Article also discusses optional step on Service Name change just to make sure the Certificate Subject CN matches with the hostname of config service URL.(customer's environment don't need to do this step if their LMI Certificate already matches with config-service-url).Customer can change LMI certificate Subject CN to match config-service-url or include a SAN name. 

There is also a way you can configure WRP,DSC,Runtime and Snapshot Manager to not validate the config container service url certificate. For that you can refer to the section 'How to disable certificate validation'. This is a backward compatibility and will ensure the same functionality as it was prior to When you include disabled option then WRP,DSC,Runtime and Snapshot Manager will continue to download snapshot as earlier versions.