Working with TLS certificates can sometimes feel like a daunting task. I hope to provide enough information regarding the usage and best practices of TLS certificates within Cloud Pak for Business Automation that you can make the right decisions for your environment and avoid common mistakes. If you are unfamiliar with TLS certificates, I would recommend that you search the web for that term as there are many good articles out there.
The openssl x509 command can be used to view and work with certificates.
Openssl x509 documentationThis command can be used to view the details of a certificate that might be in one of the product secrets.
openssl x509 -in cert_file -text
If you are supplied certificates in the wrong format then you will have to convert them to the proper format with a tool like openssl.
When supplying certificates to the Openshift environment they must be in an unencrypted PEM format.
Kubernetes secrets documentation - TLS Certificate sectionTo build the required files to provide to Openshift or Cloud Pak for Business Automation you will need to open your certificate files usually in .crt, .cer or .pem extensions.
You should see a section which looks like one of these depending on whether it is the public key or private key. If you do not see this then you likely need to decrypt the file into the proper format using a tool like openssl.
A certificate or public key requires this section:
-----BEGIN CERTIFICATE-----
(encoded set of characters)
-----END CERTIFICATE-----
A private key requires this section:
-----BEGIN PRIVATE KEY-----
(encoded set of characters)
-----END PRIVATE KEY-----
When building the file for a root CA certificate or a private key, you should copy the appropriate section and put it into a file that only contains that section.
Although when you are building the certificate file for an endpoint certificate like the ones associated with OCP routes, more information is needed. You should include the certificate of every CA signer in the certificate chain up to the root CA. So when building a file for this you may copy the needed text from multiple source certificates to add to your file. The resulting file should look like this where the endpoint certificate is the first entry followed by each signer until the root is reached.
-----BEGIN CERTIFICATE-----
(Endpoint/wildcard certificate)
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
(Intermediate CA signer)
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
(Root CA signer)
-----END CERTIFICATE-----
If you need to veiw the details of one of the certificates associated with a Cloud Pak for Business Automation setting then you can copy the begin/end certificate block from the related secret. Openssl x509 can be used to display the detailed text for the certificate.
The Cloud Pak for Business Automation provides 3 main ways to interact with the product's certificates and trust stores:
- You can supply the root CA signer certificate which will generate all the certificates that the product needs.
- You can override the product generated certificates on individual routes.
- You can provide the root CA certificate for any CAs that you want the product to trust.
The main reason to work with the certificate settings is to avoid browser warnings when interacting with certificates that are not trusted or when connecting to external services which are not trusted. I will discuss how each of these can impact these behaviors.
Supplying the Root CA Signer
Providing the root CA discusses how you can provide the root CA signer certificate used to generate the Cloud Pak's certificates. This will require both the public certificate and private key for the root CA signer.
This is generally not an option customers would look at due to security concerns. The root CA signer is in an unencrypted format that would allow anyone with access to generate certificates which are trusted by your ecosystem. The private keys are generally very guarded data. It is unlikely that a security team would want this information stored in an OpenShift secret or even allowing them to be shared with OpenShift admins.
I find there are two main reason to interact with the root CA secret:
1. It has the
public certificate generated by the Cloud Pak. If needed, you can import this into the trust store for your external applications and services so they can connect with the Cloud Pak. This certificate can also be imported in a browser to ignore all the all the self-signed errors although a company security team might not allow this.
2. You want to use a customized CA signer even if you aren't going to use a trusted signer.
Note: It is best to customize this before creating the Cloud Pak deployment.
Overriding the Cloud Pak route with trusted certificates
One way to avoid self signed certificate issues for browsers is to replace the certificates for the public user interfaces used by the end users. You would generate a wildcard certificate or individual certificates for each public route that is being used by the end users. The hostnames associated with the certificates would need to match the associated route.
Through the CR, the route settings are generally exposed by settings called external_tls_secret and external_tls_ca_secret.
external_tls_secret - points to the name of a secret which will be used for the associated route. The secret is a TLS secret which contains the public certificate and private key of the certificate to be used by the route. The same secret can be used by multiple routes if they have a shared hostname suffix and a wildcard certificate is used. The certificate file would generally contain the certificate along with any signer certificates needed to complete the chain to the root CA. These two values in this secret map to the OCP route's cert and key settings. For more info on secure routes see Secured routes.
external_tls_ca_secret - this setting points to the name of a secret that contains any CA certificates needed to complete the certificate chain for a route. This maps to the ca-cert setting of the route. In general this setting is unnecessary if you have properly built your certificate files to include the full certificate chain.
The documentation in this area could use some improvement and it would be nice if a wildcard certificate could be provided so that you do not have to configure every route individually. If this is an area of interest, I would watch for improvements to both the documentation and features in future versions.
Update: New in 21.0.2, you can now easily supply a wild card certificate that will be used for all routes generated by the CP4BA operator. The route hostname generation should also be consistent. See Providing certificates for external routes.
Note: Services internal to the Cloud Pak (not exposed by routes) would use certificates generated by the root CA signer. It is not expected that a customer would override these. They should be trusted by all the components of the Cloud Pak.
Provide the root CA certificate for any CAs that you want the product to trust
There are many scenarios where the product might interact with external services and applications. In these scenarios you may need the Cloud Pak to trust the external service's TLS certificates. To do this you need to ensure that the root CA certificate associated with the external service's certificates gets imported into the product's trust store.
The trusted_certificate_list setting can be used to
import root CA certificates which should be trusted. The operator will propagate these certificate into the trust stores associated with all the individual components of the Cloud Pak for Business Automation.
trusted_certificate_list is an array of secret names. Each secret would contain the public certificate of the root CA that needs to be trusted.
Some trust store settings exist at more granular levels although, it is much easier to avoid using those and just use the shared setting.
#bestpractices#CloudPakforBusinessAutomation