Both IBM Spectrum Symphony and IBM Spectrum Conductor support self-signed certificates and external certificates for SSL connections. Both also provide self-signed certificates for testing purposes. If these certificates expire or if you want to generate your own self-signed certificate files, use the guidance from this blog to create them.
Context and background
For context and background, understand that IBM Spectrum Symphony and IBM Spectrum Conductor support SSL connections between all key external and internal daemons:
- For web server communications:
- Cluster management console
- RESTful APIs
- Elastic Stack
- Web services gateway
- Instance groups (IBM Spectrum Conductor only)
- For system daemon communications:
- VEMKD and IBM Spectrum Symphony or IBM Spectrum Conductor client
- VEMKD and PEM
- RS and RS client
- SSM and SIM (IBM Spectrum Symphony only)
- SD and SDK client, SD and soap client (IBM Spectrum Symphony only)
- RSA and the RSA client for grid synchronization (IBM Spectrum Symphony only)
- Host factory communication
- RESTful API workload for SYMREST (IBM Spectrum Symphony only)
First enable and configure SSL connections for these daemons; refer to IBM Spectrum Symphony and IBM Spectrum Conductor in IBM Documentation for details:
Next, to prepare your client and service side certificate files for these SSL connections, follow the guidance in this blog.
There are two ways to create these certificate files: mainly using OpenSSL commands or mainly using the keytool key and certification management utility.
Creating certificate files by mainly using OpenSSL
Before running any OpenSSL commands, ensure that you have OpenSSL installed on an RHEL host (version 1.1.1k or later), then complete each of the following high-level steps.
Step 1 – Generate a self-signed root certificate (certificate authority) by using OpenSSL
- Make a new directory (called CA-Gen)for your OpenSSL certificate generation operations:
$ mkdir -p /opt/CA-Gen
$ cd /opt/CA-Gen
- Generate a private key for root CA, called cacert.key:
$ openssl genpkey -algorithm RSA -out cacert.key -pkeyopt rsa_keygen_bits:2048
- Generate a self-signed root CA called cacert.pem:
$ openssl req -new -x509 -days 365 -key cacert.key -out cacert.pem -subj "/C=CN/ST=SX/L=XA/O=IBM/OU=SYM/CN=CACERT/emailAddress=mail-address"
where, in this example syntax, the /C=CN/ST=SX/L=XA/O=IBM/OU=SYM/CN=CACERT/emailAddress=mail-address value for the -subj parameter is as follows:
- C is your two-character country name; in this example, the values is CN.
- ST is your full state or province name; in this example, SX.
- L is your city name; in this example, XA.
- O is the organization or company name; in this example, IBM.
- OU is your organizational unit name; in this example, SYM.
- CN is your common name (for example, your name or your server's hostname, such as CACERT.
- emailAddress is mail-address.
Tip: If you do not use the subj parameter to specify this information set, then you will be prompted to type in each of these values, interactively.
Step 2 – Generate a service certificate to connect to key back-end daemon by using OpenSSL
Use the root CA created in step 1 to sign service side certificate files which will be used for IBM Spectrum Symphony or IBM Spectrum Conductor.
- Generate a private key, called user.key, for your server-side certificate:
$ openssl genpkey -algorithm RSA -out user.key -pkeyopt rsa_keygen_bits:2048
- Prepare a file called endpoint.cnf to contain OpenSSL configurations:
$ cat endpoint.cnf
basicConstraints = critical, CA:FALSE
subjectKeyIdentifier = hash
subjectAltName = @alt_names
[ alt_names ]
DNS.1 = host1.test.com
DNS.2 = host2.test.com
DNS.3 = host3.test.com
...
Note that IBM Spectrum Symphony 7.3.2 with Fix 601711 or IBM Spectrum Conductor 2.5.1 with Fix 601712 supports using SSL server authentication to verify a peer hostname against the server certificate’s DNS (defined in the Subject Alternative Name or SAN) or common name (CN). If you plan to use SSL server authentication in this way, you can put all your DNS names in the SAN, and the host verification behavior is as follows:
- The client will verify server certificate and check the server’s hostname against the server certificate’s DNS (defined in the SAN) or its CN. The hostname checking will succeed if the DNS name defined in certificate’s SAN or CN matches the server’s host name (for the WEBGUI service, the hostname checking requires the DNS name defined in certificate’s SAN to match the server’s host name).
- The certificate’s DNS name defined in the SAN can be one or multiple DNS names.
- The hostname checking is done using OpenSSSL X 509_check_host API during certificate verification, and the hostname checking behavior follows the API description. The hostname checking disables wildcard expansions using the X509_CHECK_FLAG_NO_WILDCARDS flag. For example, verification will fail when the server’s hostname is serverhost.test.com and the DNS name is *.test.com.
For VEMKD-PEM SSL connections, when the SERVER_AUTH parameter is enabled and its value is not NONE, the server side will also require client to send the certificate (configured by the CERTIFICATE of EGO_KD_PEM_TS_PARAMS and EGO_PEM_TS_PARAMS sub parameters) and verify the client’s certificate. In this case, ensure that the certificate’s purpose is not a limited server.
For example, when the following extensions are configured, the certificate can only be used by the SSL server:
nsCertType = server
extendedKeyUsage = serverAuth
c. Generate a service side certificate called user.pem. This certificate file will be used for key daemons in IBM Spectrum Symphony or IBM Spectrum Conductor, such as for the VEMKD, SD, EGOSC, RS, RSA, SSM, ExecProxy, or HostFactory services. You can generate separate certificate files for each service; the following syntax shows an example for generating one of them:
$ openssl req -new -key user.key -days 365 -batch -out user.csr -utf8 -subj '/C=CN/ST=SX/L=XA/O=IBM/OU=SYM/CN=USER/emailAddress=mail-address'
$ openssl x509 -req -sha256 -days 365 -in user.csr -CAkey cacert.key -CA cacert.pem -out user.pem -set_serial 1 -extfile endpoint.cnf
Note: Change the CN and emailAddress values to indicate your service. For other parameters, keep them the same as your root CA’s, defined in step 1.
Step 3 – Generate a PEM certificate for VEMKD-PEM connections by using OpenSSL
- Generate a private key, called pem.key, for your server-side certificate:
$ openssl genpkey -algorithm RSA -out pem.key -pkeyopt rsa_keygen_bits:2048
- Follow the guidance in step 2 to update an endpoint.cnf file if you have different SAN definitions.
- Sign a service-side certificate called pem. This certificate file will be used for the PEM daemon in IBM Spectrum Symphony and IBM Spectrum Conductor:
$ openssl req -new -key pem.key -days 365 -batch -out pem.csr -utf8 -subj '/C=CN/ST=SX/L=XA/O=IBM/OU=SYM/CN=PEM/emailAddress=mail-address'
$ openssl x509 -req -sha256 -days 365 -in pem.csr -CAkey cacert.key -CA cacert.pem -out pem.pem -set_serial 2 -extfile endpoint.cnf
Note: Change the CN and emailAddress values into indicate your service. For other parameters, keep them the same as your root CA’s, defined in step 1.
Creating Java keystore (JKS) files
Steps 1 to 3 walked you through creating the cacert.key, cacert.pem, user.key, user.pem, pem.key, and pem.pem,files. These files are used for SSL connections with back-end daemons. Next, you are ready to generate JKS files for GUI, REST, and PERF, usage in the following steps.
Before creating JKS files, ensure that you have JRE (IBM JRE 8 or later) installed and have the keytool utility available to use. You can use the built-in JRE includes with IBM Spectrum Symphony or IBM Spectrum Conductor at $EGO_TOP/jre/$EGOJRE_VERSION/linux-x86_64/bin.
Step 4 – Generate the serverKeyStore.jks Java keystore (JKS) file for WEBGUI, REST, and OpenIDClient services by using OpenSSL and keytool
The serverKeyStore.jks file contains the server certificate, private key, and cacert.pem file used for WEBGUI, REST, and OpenIDClient services.
- Run these commands to generate the serverKeyStore.jks file, replacing <password> with your actual keytool password:
$ openssl pkcs12 -export -in user.pem -inkey user.key -out user.p12 -passout pass:<password>
$ keytool -importkeystore -deststorepass <password> -srcstorepass <password> -destkeystore serverKeyStore.jks -srckeystore user.p12 -srcstoretype PKCS12 -storepass <password>
$ keytool -importcert -noprompt -alias caalias -file cacert.pem -keystore serverKeyStore.jks -storepass <password>.
- Generate an encrypted password string for your password:
$ $EGO_TOP/wlp/$WLP_VERSION/bin/securityUtility encode --encoding=aes <password>
$ {aes}...
Step 5 – Update certificate files and the serverKeyStore's password
- Gather all the certificate and JKS files that you created in the steps up to this point. These files are required for SSL connections in IBM Spectrum Symphony and IBM Spectrum Conductor:
cacert.pem
pem.key
pem.pem
user.key
user.pem
serverKeyStore.jks
- Back up all the files under the $EGO_TOP/wlp/usr/shared/resources/security directory.
- Copy all the generated files to the $EGO_TOP/wlp/usr/shared/resources/security directory.
- Copy the serverKeyStore.jks file to the $EGO_CONFDIR/perf/conf/security directory.
- Modify the server_rest.xml file in the $EGO_CONFDIR directory, to change the default password to the generated encrypted password string from step 4:
<keystore id="defaultKeyStore" password="{aes}..." type="JKS" location="${wlp.user.dir}/shared/resources/security/serverKeyStore.jks”/>
- Modify the server_gui.xml file in the $EGO_CONFDIR/../../gui/conf directory, to change the default password to the generated encrypted password string from step 4:
<keyStore id="defaultKeyStore" password="{aes}..." type="JKS" location="${wlp.user.dir}/shared/resources/security/serverKeyStore.jks”/>
- Modify the OpenIdClientConf.xml file in the $EGO_CONFDIR/../../soam/profiles/conf directory to change the default password to the generated encrypted password string from step 4:
<keyStore id="defaultKeyStore" password="{aes}..." type="JKS" location="${wlp.user.dir}/shared/resources/security/serverKeyStore.jks"/>
Now you have successfully updated all the necessary certificate files for SSL connections for use with IBM Spectrum Symphony or IBM Spectrum Conductor by mainly using OpenSSL
Tip: Use the ssltool security tool to automatically generate and configure SSL certification files
You can also use the IBM Spectrum Symphony and IBM Spectrum Conductor security (ssltool) tool to automatically generate and configure SSL certification files for GUI,REST, and ELK services. Refer to these steps to do so:
You will need to modify ssltool.conf file to allow the tool to use your root CA and its private key:
ca.private.key.path=/opt/CA-Gen/cacert.key
ca.cert.pem.path=/opt/CA-Gen/cacert.pem
As a best practice, use this tool to generate and configure ELK-related SSL certificate files and configuration, since there are many configuration files and using the tool will make this easier. For all IBM Spectrum Conductor web server communications (GUI, REST, ELK, instance groups, notebooks, and so on), use the tool.
Creating certificate files by mainly using keytool
Instead of mainly using OpenSSL commands, you can use the keytool (key and certification management utility) included with IBM Spectrum Symphony and IBM Spectrum Conductor. The following high-level steps are similar to the OpenSSL flow in that they create the same certificate and JKS that you require for SSL connections with IBM Spectrum Symphony or IBM Spectrum Conductor, but they mainly use keytool commands.
Before running any keytool commands, ensure that you have JRE (IBM JRE 8 or later) installed and have the keytool utility available to use. You can use the built-in JRE includes with IBM Spectrum Symphony or IBM Spectrum Conductor at $EGO_TOP/jre/$EGOJRE_VERSION/linux-x86_64/bin. Then, then complete each of the high-level steps.
Step 1 – Generate a self-signed root certificate by using keytool
Run these commands to generate the cacert.pem and caKeyStore.jks files, replacing <password> with your actual keytool password:
$ $EGO_TOP/jre/$EGOJRE_VERSION/linux-x86_64/bin/keytool -genkeypair -v -alias caalias -dname "C=CN,ST=SX,L=XA,O=IBM,OU=SYM,CN=CACERT,emailAddress=mail-address" -keystore caKeyStore.jks -keypass <password> -storepass <password> -keyalg rsa -keysize 2048 -validity 365 -sigalg SHA256withRSA -ext KeyUsage="keyCertSign" -ext BasicConstraints:"critical=ca:true"
$ $EGO_TOP/jre/$EGOJRE_VERSION/linux-x86_64/bin/keytool -export -v -alias caalias -file cacert.pem -keypass <password> -storepass <password> -keystore caKeyStore.jks -rfc
Step 2 – Generate the serverKeyStore.jks JKS file for WEBGUI, REST, and OpenIDClient services by using keytool
The serverKeyStore.jks file contains the server certificate, private key, and cacert.pem file used for WEBGUI, REST, and OpenIDClient services.
Run these commands to generate the serverKeyStore.jks file, replacing <password> with your actual keytool password:
$ $EGO_TOP/jre/$EGOJRE_VERSION/linux-x86_64/bin/keytool -genkeypair -noprompt -alias srvalias -dname "C=CN,ST=SX,L=XA,O=IBM,OU=SYM,CN=USER,emailAddress=mail-address" -keystore serverKeyStore.jks -storepass <password> -keypass <password> -keyalg rsa -validity 365 -keysize 2048 -sigalg SHA256withRSA -ext "san=dns:host1.test.com,dns:host2.test.com,dns:host3.test.com" -ext BasicConstraints:"critical=ca:false"
$ $EGO_TOP/jre/$EGOJRE_VERSION/linux-x86_64/bin/keytool -certreq -alias srvalias -file srvcertreq.csr -storepass <password> -keystore serverKeyStore.jks -ext "san=dns:host1.test.com,dns:host2.test.com,dns:host3.test.com" -ext BasicConstraints:"critical=ca:false"
$ $EGO_TOP/jre/$EGOJRE_VERSION/linux-x86_64/bin/keytool -gencert -infile srvcertreq.csr -outfile servercertcasigned.pem -alias caalias -keystore caKeyStore.jks -storepass <password> -validity 365 -ext "san=dns:host1.test.com,dns:host2.test.com,dns:host3.test.com" -ext BasicConstraints:"critical=ca:false"
$ $EGO_TOP/jre/$EGOJRE_VERSION/linux-x86_64/bin/keytool -importcert -noprompt -alias caalias -file cacert.pem -keystore serverKeyStore.jks -storepass <password>
$ $EGO_TOP/jre/$EGOJRE_VERSION/linux-x86_64/bin/keytool -import -noprompt -alias srvalias -file servercertcasigned.pem -storepass <password> -keystore serverKeyStore.jks
Step 3 – Generate an EGO-level certificate to connect to key back-end daemons by using keytool and OpenSSL
Generate a service side certificate called user.pem ,and a private key called user.key, for your server-side certificate:
$ $EGO_TOP/jre/$EGOJRE_VERSION/linux-x86_64/bin/keytool -importkeystore -srckeystore serverKeyStore.jks -destkeystore user.p12 -srcstoretype JKS -deststoretype PKCS12 -srcstorepass <password> -deststorepass <password> -srcalias srvalias -destalias srvalias -srckeypass <password> -destkeypass <password> -noprompt
$ $EGO_TOP/jre/$EGOJRE_VERSION/linux-x86_64/bin/keytool -exportcert -alias srvalias -keypass <password> -keystore serverKeyStore.jks -storepass <password> -rfc -file user.pem
$ openssl pkcs12 -in user.p12 -nocerts -nodes -out user.key -passin pass:<password>
Step 4 – Generate a PEM certificate for VEMKD-PEM connections by using keytool
Run the following commands:
$ $EGO_TOP/jre/$EGOJRE_VERSION/linux-x86_64/bin/keytool -genkeypair -noprompt -alias pemalias -dname "C=CN,ST=SX,L=XA,O=IBM,OU=SYM,CN=PEM,emailAddress=mail-address" -keystore pemKeyStore.jks -storepass <password> -keypass <password> -keyalg rsa -validity 365 -keysize 2048 -sigalg SHA256withRSA -ext "san=dns:host1.test.com,dns:host2.test.com,dns:host3.test.com" -ext BasicConstraints:"critical=ca:false"
$ $EGO_TOP/jre/$EGOJRE_VERSION/linux-x86_64/bin/keytool -certreq -alias pemalias -file pemcertreq.csr -storepass <password> -keystore pemKeyStore.jks -ext "san=dns:host1.test.com,dns:host2.test.com,dns:host3.test.com" -ext BasicConstraints:"critical=ca:false"
$ $EGO_TOP/jre/$EGOJRE_VERSION/linux-x86_64/bin/keytool -gencert -infile pemcertreq.csr -outfile pemcertcasigned.pem -alias caalias -keystore caKeyStore.jks -storepass <password> -validity 365 -ext "san=dns:host1.test.com,dns:host2.test.com,dns:host3.test.com" -ext BasicConstraints:"critical=ca:false"
$ $EGO_TOP/jre/$EGOJRE_VERSION/linux-x86_64/bin/keytool -importcert -noprompt -alias caalias -file cacert.pem -keystore pemKeyStore.jks -storepass <password>
$ $EGO_TOP/jre/$EGOJRE_VERSION/linux-x86_64/bin/keytool -import -noprompt -alias pemalias -file pemcertcasigned.pem -storepass <password> -keystore pemKeyStore.jks
$ $EGO_TOP/jre/$EGOJRE_VERSION/linux-x86_64/bin/keytool -importkeystore -srckeystore pemKeyStore.jks -destkeystore pem.p12 -srcstoretype JKS -deststoretype PKCS12 -srcstorepass <password> -deststorepass <password> -srcalias pemalias -destalias pemalias -srckeypass <password> -destkeypass <password> -noprompt
$ $EGO_TOP/jre/$EGOJRE_VERSION/linux-x86_64/bin/keytool -exportcert -alias pemalias -keypass <password> -keystore pemKeyStore.jks -storepass <password> -rfc -file pem.pem
$ openssl pkcs12 -in pem.p12 -nocerts -nodes -out pem.key -passin pass:<password>
Step 5 – Update certificate files and the serverKeyStore's password
- Generate an encrypted password string for your password:
$ $EGO_TOP/wlp/$WLP_VERSION/bin/securityUtility encode --encoding=aes <password>
$ {aes}...
- Gather all the certificate and JKS files that you created in the steps up to this point; these files are required for SSL connections in IBM Spectrum Symphony and IBM Spectrum Conductor.
- Complete Step 5 – Update certificate files and the serverKeyStore's password under the Creating certificate files by mainly using OpenSSL section.
Conclusion
Regardless of whether your primary tool create the files by using OpenSSL or the keytool utility, you end up with the necessary certificates and files for configuring SSL connections for use with IBM Spectrum Symphony or IBM Spectrum Conductor. You’re ready to securely use these products!