{Core}Are you confused when trying to use an existing certificate in other Zowe components? In this blog, we will walk you through each step of configuring Zowe’s z/OS components to use an existing certificate. Specifically, we will explore the scenario where we leverage the same certificate used by the z/OSMF server on the same LPAR as Zowe. By following the step-by-step instructions provided here, you will learn how to smoothly configure your Zowe for an existing certificate. We will introduce other scenarios in different blogs soon. Now, let’s dig in.

Before we start

3 steps to configure Zowe to use an existing certificate:

• Identify the existing certificate: the certificate name, certificate authority chain, and owner.
• Create a key ring accessible by Zowe’s functional user ID that connects to the certificate and its authorities.
• Customize zowe.yaml to point to the key ring, the certificate, and its authorities’ names. See an example zowe.yaml file here.

Something you want to know…

Did you know how Zowe uses TLS (Transport Layer Security)? TLS is used in Zowe to support secure communication between The Open Mainframe Project’s Zowe clients and its z/OS components.

Step 1: Identify the existing certificate

If you are using RACF security management, the command to view the keyring details for the z/OSMF user ID IZUSVR is RACDCERT LISTRING(*) ID(IZUSVR). For Top Secret, the command is TSS LIST(IZUSVR) KEYRING(ALL) and for ACF2 the command is F ACF2,LIST IZUSVR RINGNAME(*).

The z/OSMF default user ID is IZUSVR. A good way to see the certificate it’s using is to look at the key rings owned by that ID with the TSO command RACDCERT LISTRING(*) ID(IZUSVR). If your LPAR is using the defaults, it likely will be DefaultZOSMFCert.IZUDFLT.

If you want to be completely sure which key ring z/OSMF is using, you can retrieve the configuration file by looking at the spool for the z/OSMF IZUSVR1 job - FILENAME property and then searching the USS file for KEYRING_NAME. The graphic below shows this being done with the Zowe CLI commands

• Retrieve the job ID for the IZUSVR1 z/OSMF started task (in this case STC09019 using zowe jobs list jobs --owner "IZUSVR"
• Retrieve all spool content and look for the FILENAME property value using zowe jobs view asc <JOBID> | grep FILENAME
• Viewing the configuration file contents and looking for the KEYRING_NAME value using zowe files view us <FILENAME> | grep KEYRING_NAME

Now, we have determined the name of the certificate we want to use and its certificate authority (that we are poaching from z/OSMF). The next step is to get Zowe to use these.

Step 2: Create a key ring for Zowe that connects to the z/OSMF certificate and certificate authority

The z/OSMF key ring IZUKeyring.IZUDFLT is for use by z/OSMF’s user ID IZUSVR, so we’ll need to create one for Zowe to use that connects to the same certificate and its signing authority.

Zowe provides a helper utility to do this through the zwe init certificate command. The input to this, and to all zwe command invocations, is the input file zowe.yaml that specifies everything related to configuring and bringing up Zowe’s started tasks.

The section zowe.setup.certificate contains a number of templates for different scenarios, and Certificate setup scenario 4 is to assist with the scenario that we’re trying to do in this blog, namely to configure Zowe to use an existing JCERACFKS certificate.

Now we can uncomment the scenario 4. By default, the example-zowe.yaml file template that you copied to create zowe.yaml will have the scenario 1 zowe.setup.certificateuncommented (which is for creating a self-signed PKCS12 certificate that we're not doing in this blog). We need to be sure to comment/remove all of this, so only one zowe.setup.certificate scenario is uncommented. We’ll choose a name for our new key ring of ZoweKeyringZOSMF and update the following fields:

• zowe.setup.certificate.type: to be JCERACFKS to indicate this is a key ring certificate.
• zowe.setup.certificate.keyring.name: to be ZoweKeyringZOSMF which is the name of the key ring we want to generate.
• zowe.setup.certificate.keyring.connect.user: to point to the existing certificate’s owner of IZUSVR.
• zowe.setup.certificate.keyring.connect.label: to be the label of the existing certificate that we found earlier to be DefaultzOSMFCert.IZUDFLT.
• zowe.setup.certificate.importCertificateAuthorities: to be the certificate signing authorities (CAs) for the certificate. This will be the CERTAUTH entries from the existing z/OSMF key ring. In our case, just zOSMFCA as there is only one.

If there is more than one CA (perhaps an intermediate zOSMFCAIntermediate and a root CorporateRoot), then these would both be entered on separate lines as below.

We can only specify two CAs in the zowe.yaml file. However, if we do have more than two, then just enter the first pair, we will show you how to add the additional CERTAUTH entries at the end of this blog.

*Note: in the screenshots above, we are using the Zowe Explorer plugin for Visual Studio Code, which does a fantastic job at coloring the content of zowe.yaml, so we can see comments in green, keywords in blue, values in brown, …. Be careful if you are using tools like ISPF or vi in OMVS or iShell to edit your zowe.yaml file, because it’s easy to make mistakes. For example, having an extra space so the columns don’t line up correctly, or a line you think is runtime syntax is commented out or vice versa.

Now that the zowe.yaml file section zowe.setup.certificate is all customized for scenario 4, we can run the command zowe init certificate. This will generate a JCL member containing all the TSO commands to create the new key ring. The JCL will be placed in the PDS as defined in the zowe.setup.dataset.jcllib value in zowe.yaml. By default, the JCL will be submitted immediately, however this can be prevented by adding the parameter value --security-dry-run. This allows the JCL to be inspected before submission, as well as handed off to a security administrator who has privileges to submit the JCL under their user ID.

The graphic below shows running the zwe init certificate -c ./zowe.yaml --security-dry-run command. It will likely take a minute or so to complete, as it does a fair bit of computation. So let’s be patient as it might take some time to compute. The following graphic is sped up.

After running the JCL generated by zwe init certificate, it is always a good idea to inspect its output spool files. Because zowe.setup.security.product was set to RACF, there will be a RUNRACF file that creates the key ring, (for ACF2 or TSS the spool file is named differently). We can look at each of the spools to check that the key ring created, the connection to the certificate and the certificate authorities made, a number of permissions granted, facility classes enabled, and other security configuration steps.

As a check, we usually look at the DefaultzOSMFCert.IZUDFLT to ensure that is in both its original z/OSMF key ring and our newly created ZoweKeyringZOSMF. This is done with the command RACDCERT LIST(LABEL('DefaultzOSMFCert.IZUDFLT')) ID(IZUSVR) which will show both of the key rings that z/OSMF’s certificate is not part of.

If you want to check the details of the certificate DefaultzOSMFCert.IZUDFLT , run the command zowe tso issue command "RACDCERT LIST(LABEL('DefaultzOSMFCert.IZUDFLT')) ID(IZUSVR)" --ssm .

And you will see the details match the ones you will use in the browser when you finish step 3.

To check the CA zOSMFCA is also connected correctly, run the command RACDCERT LIST(LABEL('zOSMFCA')) CERTAUTH which should show it associated.

Now we’re ready for the third and final act, which is to bring up Zowe’s ZWESLSTC started task using the z/OSMF certificate that we’ve connected to the newly created key ring.

Step 3: Customize zowe.yaml and start Zowe with the z/OSMF certificate

The certificate details used by Zowe’s started task ZWESLSTC are defined in the zowe.certificate section of the zowe.yaml configuration file. This is distinct from the details we defined in the zowe.setup.certificate section, which are only used in certificate creation.

When the zwe init certificate command was run earlier to create the JCL member for building the key ring, it echoed to the console message:

>> Update certificate configuration to ./zowe.yaml
Please manually update to these values:

Directly following this is the nicely formatted syntax that should be copied into the clipboard:

… and then passed into the zowe.yaml file by pasting the values to zowe.yaml file, see details as below.

If you are updating your zowe.yaml file for the first time you’ve copied it from the default template inexample-zowe.yaml, you’ll find there is already a zowe.certificate section. That is for a PKCS12 certificate. So overwrite this with our nice new syntax, telling Zowe that we want to use the z/OSMF certificate in our nice new key ring.

• zowe.certificate.keystore.type: to be JCERACFKS.
• zowe.certificate.keystore.file: syntax is safkeyring:////<ID>/<KeyringName, so this becomes safkeyring:////ZWESVUSR/ZoweKeyringZOSMF.
• zowe.certificate.keystore.alias: is the certificate label of DefaultzOSMFCert.IZUDFLT. Spaces are allowed, and the value is case-sensitive.
• zowe.certificate.truststore: these repeat the values for zowe.certificate.keystore:. Technically, it is possible to use a separate key ring for the trust store (which is where certificates with their public keys are held that Zowe’s server connects to) as opposed to the key store (which is where Zowe’s certificate with a private key is held which is used to encrypt TLS traffic), however in every site we’ve come across the same keystore is used for both.
• zowe.certificate.pem.certificateAuthorities: should point to the CERTAUTH labels for the signing chain of the certificate, with the syntax safkeyring:////<ID>/<KeyringName>&<CERTAUTHLABEL>. For our keyring we only have one CERTAUTH whose label is zOSMFCAso the value is safkeyring:////ZWESVUSR/ZoweKeyringZOSMF&zOSMFCA.

*Note: If we did have more than one CERTAUTH in the signing chain, each would be entered on a separate line beginning with - so the syntax might be:

Trust me!

We are all ready to launch Zowe!!! If you are running Zowe with a certificate that is signed by a certificate authority that your browser trusts, you can proceed to open it on the URL https://<host>:7554. An issue that we often encounter at customer sites is that the certificate they’re using is self signed, or cross signed, or perhaps an expired certificate (all of which are good to still use for encryption but are scenarios that the browser will challenge). Zowe likewise will fail to establish a TLS session to a server that has an untrusted certificate. For debug purposes, and for using Zowe in test or staging environments, we do provide a parameter zowe.veritifyCertificates that can be used to allow Zowe to launch with untrusted certificates. We’ll segue into why, when and how to set this value, however ror production use however we always recommend leave the default of zowe.verifyCertificates: STRICT.

Most of us are used to opening websites that present a certificate that the browser doesn’t trust and challenges with an “Untrusted — do you want to proceed ?”. Zowe is the same, and if it sees a certificate that Zowe doesn’t trust. It will reject the TLS connection. To see whether this is likely for Zowe open a browser on z/OSMF at the URL https://<host>:443/zosmf (assuming your z/OSMF is running on port 443). The graphic below shows that Firefox isn’t happy about the certificate for a couple of reasons: (1) the certificate has expired (today is June 28th 2023 and the certificate is invalid after May 18th 2023); (2) the certificate authority whose common name is zOSMF CertAuth for Security Domain (the zOSMFCA CERTAUTH in the key ring) has signed itself, so the Issuer Field is itself which is known as a self-signed certificate. Because no chain of trust to a known root can be established, Firefox asks nicely whether we’re OK to proceed.

Zowe’s server will do likewise twice.

First time when it tries to connect to z/OSMF (the endpoint details of which need to be specified in zowe.yaml) as it’s using the z/OSMF self signed expired certificate.

Second time is because Zowe’s ZWESLSTC started task begins a number of separate components each of which is a USS process (one for API Gateway, one for Zowe Desktop, one for API Catalog, …) and these communicate with each other using TLS. Each of these uses the certificate defined in zowe.certificate which is the same expired and self signed one that z/OSMF is using. So Zowe will internally reject its own certificate.

To remedy this, set the value zowe.verifyCertificates: to be DISABLED. As the comment says this should be set to STRICT in a production environment which means the certificate must be 100% valid, however in the setup I’m using for this blog (and also many test environments I come across), the certificate will be self signed and/or expired so DISABLED needs to be used. The third enum value of NONSTRICT is a useful one if the certificate being used is valid (signed by a trusted root, not expired, …) but has the wrong network address in its SAN. We will write a separate blog about that as it’s pretty handy to get out of some tricky corners.

*Note: If you happened to set the value to STRICT and run zwe init certificate -c ./zowe.yaml --security-dry-run to connect the certificate, you might come across the error message as shown below. What you can do to fix this error is to set the value zowe.verifyCertificates: to be DISABLED and re-run the command zwe init certificate -c ./zowe.yaml --security-dry-run.

Again, the use of any value other than verifyCertificates: STRICT is not recommended for anything other than launching Zowe in test or staging environments. For production usage, please use certificates that are pass strict checks, just to make sure everyone’s data is safe and secure.

Bring it up

The ZWESLSTC started task can now be started. This can be done through /S ZWESLSTC in SDSF to run the proclib JCL as a started task, or else using zwe start with the -c parameter pointing to the zowe.yaml file.

Once Zowe has finished starting open a browser onto the API Gateway (by default is running on 7554), for this blog we used 27564 which is defined in zowe.components.gateway.port in zowe.yaml, you can click on the padlock to the left of https:// to view the certificate being used. As shown below, we can see that this is indeed the IZUDFLT certificate (the same one z/OSMF is using on 443 that we connected to the keyring that Zowe is using).

All done!

A glimpse of next

In the upcoming blogs, we will show you how to use zwe init to work with PKCS12 certificate held in USS .p12 keystores, as well as how to create your own self-signed certificate for use by Zowe which is handy if you don’t have a valid certificate to hand.

And you won’t miss the upcoming redesign in Zowe certificate configuration on Zowe doc site.

This blog is originally published on Medium.com/zowe.

Zowe™, the Zowe™ logo, and the Open Mainframe Project™ are trademarks of The Linux® Foundation. Linux is a registered trademark of Linus Torvalds.