“Technical MQ Engineers involved are Atul Sharma, Nipun Goel and Gaurav Sharma @DNB ASA Norway, and this has been reviewed by callumj@uk.ibm.com, Solution Architect @IBM.”
DNB have deployed IBM MQ in Kubernetes on both AWS EKS and the Red Hat OpenShift Service on AWS (ROSA). Aligning to good practice we enabled TLS communication on the MQ channels, this was straightforward when using self-signed certificates, but more challenging when attempting to use CA signed certificates. This blog will document the steps that we completed, although due to the numerous security tools available in the industry you may need to adapt the instructions based on the tooling used in your organization.
Generating the certificates for TLS
The TLS certificates need to be generated, as we are using a certificate authority this involves the generation of the private key, and the creation of a certificate signing request (CSR). This certificate signing request is then submitted for approval, and a public signed certificate returned.
Creation of the private key
We create a new key database of type PKCS12 using the standard MQ command:
runmqckm -keydb -create -db key.p12 -pw <password> -type pkcs12 -stash
The private key is extracted from the key.p12 file in a format that does not require a passphrase. This is important as Kubernetes does not provide an easy mechanism to import private keys that are password protected (see the following GitHub issue for further information).
openssl pkcs12 -nocerts -in key.p12 -out tls.key -nodes
The tls.key will be imported into Kubernetes at a later stage.
Create a certificate signing request for your CA
Create the request by running the following command:
runmqckm -certreq -create -db key.p12 -pw <password> -label ibmwebspheremqqmgr -dn 'distinguished name values' -size 2048 -file qmgr.arm
Take the output file qmgr.arm and request that your certificate authority approves. This will be a slightly different process based on your organization and their processes. In our case we received the following back from our certificate authority:
- Cert.cer – the signed public certificate
- CA1.cer – the root public certificate
- CA2.cer – the issuing public certificate
- CA3.cer – the intermediate public certificate
We then converted these into a pem format using the following commands:
openssl x509 -inform pem -in <certificate.cer>certificate.cer -out tls.crt
openssl x509 -inform pem -in <CA1.cer>root.cer -out ca1.crt
openssl x509 -inform pem -in <CA2.cer>Issuing.cer -out ca2.crt
openssl x509 -inform pem -in <CA3.cer>inter.cer -out ca3.crt
Importing the certificates into Kubernetes
We decided to manually import the certificates into Kubernetes by creating the secret yaml ourselves, allowing us additional flexibility. This involved two steps:
- Base64 encoding the pem files
- Creation of the secret.yaml file
Convert all certificates to base64
cat tls.crt | base64 -w 0
cat tls.key | base64 -w 0
cat ca1.crt | base64 -w 0
cat ca2.crt | base64 -w 0
cat ca3.crt | base64 -w 0
Create a qmgr_secret.yaml file
apiVersion: v1
type: kubernetes.io/tls
kind: Secret
metadata:
name: qmgrsecret
namespace: <namespace>
data:
tls.crt: <cat tls.crt | base64 -w 0>
tls.key: <cat tls.key | base64 -w 0>
ca1.crt: <cat ca1.crt | base64 -w 0>
ca2.crt: <cat ca2.crt | base64 -w 0>
ca3.crt: <cat ca3.crt | base64 -w 0>
Create secret using the qmgr_secret.yaml file
kubectl apply -f qmgr_secret.yaml -n <namespace>
Associating the certificates with an MQ deployment
As mentioned previously we deployed to both EKS and ROSA. When using EKS we deployed using the sample MQ Helm chart, while in ROSA we used the MQ Operator. In both cases a Kubernetes/OpenShift secret is defined and referred to in the deployment. In the case of Helm this is in the values.yaml file, while for an operator deployment it is in the yaml file representing the Queue Manager resource. At runtime the Queue Manager will then be configured to use the TLS certificates.
If you supply invalid configuration then the container will terminate with an appropriate termination message.
In the Helm chart values file or Queue Manager Resource file provide the PKI information:
pki:
keys:
- name: ibmwebspheremqqmgr
secret:
secretName: qmgrsecret
items:
- tls.key
- tls.crt
- ca1.crt
- ca2.crt
- ca3.crt
Note: pki.trust in values.yaml should not be updated in this case since all of the certificates are associated with the key store.
We have decided to use the non-default certificate label (in our case we have used ibmwebspheremqqmgr), and therefore we need to define this within a ConfigMap that is also associated with the deployment. If you are unsure how to associate this, please review the IBM MQ documentation here.
Updating the Certificate Label using MQSC within a configMap
kind: ConfigMap
apiVersion: v1
metadata:
name: <configmap name>
namespace: <namespace>
data:
mq.mqsc: |-
ALTER QMGR CERTLABL('ibmwebspheremqqmgr')
Verifying the MQ environment
After deploying the queue manager, connect to the pod where the MQ is running and execute the below commands:
cd /run/runmqserver/tls
runmqckm -cert -list -db key.kdb -stashed
runmqckm -cert -validate -db key.kdb -stashed -label ibmwebspheremqqmgr
Validation successful.
#Kubernetes#MQ#tls