IBM Cloud Pak System

Introduction

IBM Cloud Pak System can help accelerate your implementation of on-premises Kubernetes platforms. It comes with support for automated deployment and configuration of Red Hat OpenShift Container Platform (OCP). This makes it the perfect platform for on-premises deployment of IBM Cloud Paks and Red Hat OpenShift clusters!

This tutorial focuses on the deployment of Red Hat OpenShift Container Platform 4.2. For more details on 3.11, please refer to the IBM Developer article Accelerate your Red Hat OpenShift Container Platform deployment with IBM Cloud Pak System and the blog post Best Practices deploying OpenShift 3.11 on IBM Cloud Pak System.

For Red Hat OpenShift, it's important to know that there are several different offerings available:

• OpenShift Online

  A fully managed public cloud offering for quickly deploying applications.

• OpenShift Hosted Services

  OpenShift clusters hosted on IBM Cloud, Amazon Web Services (AWS) and Azure.

• OpenShift Container Platform (OCP)

  An enterprise OpenShift cluster deployed on your own on-premises infrastructure (OpenShift Container Platform was previously called OpenShift Enterprise, but the name was changed with the release of version 3.3.).

A more detailed comparison of these offerings can be found on the OpenShift website. Since IBM Cloud Pak System is an on-premises appliance, it only provides support for the OCP offering. In this tutorial, you'll learn how to deploy OCP on a Cloud Pak System. We wrote the steps by assuming that the IBM Cloud Pak System is at 2.3.2.0 firmware, and does not have direct access to the internet.

Prerequisites

Before you can deploy your first OpenShift 4.2 cluster on IBM Cloud Pak System, a number of prerequisites need to be in place. IBM Knowledge Center provides a good starting point for those prerequisites:

• IBM Cloud Pak System 2.3.2.0 or higher

  Intel based IBM Cloud Pak System models W2500, W3500 and W3550 are supported. Unfortunately there is currently no support for the Power based IBM Cloud Pak System model W3700.

• IBM OS image for Red Hat Linux Systems (RHEL 7.7 X64) Version 7.7

  Scenarios using a custom OS image are also supported, as long as it is Red Hat Enterprise Linux (RHEL) 7.7 or higher.

• Red Hat Satellite Server 6.4 shared service deployed

  The shared service should be connected to an existing Red Hat Satellite Server (RHSS), or to RHSS deployed on IBM Cloud Pak System. Note that IBM Cloud Pak System comes with Red Hat subscriptions for RHEL and RHSS.

• Active subscription with Red Hat for the OpenShift Container Platform

  Unlike the Red Hat subscription for RHEL and RHSS, the OpenShift Container Platform (OCP) subscription is not included with IBM Cloud Pak System.*

• Sufficient compute, memory and storage resources on IBM Cloud Pak System

  A single OCP cluster requires at least 28 virtual CPUs, 112 GB of RAM and 1624 GB of storage.

We will take you through the details of what is exactly needed for Red Hat Satellite Server in Step 5 - Prepare Red Hat Satellite Server. And since we are making the assumption that you are working on an environment without direct internet access, Step 7 - Deploy a Private Docker Registry on IBM Cloud Pak System describes how to deploy your own private Docker registry to support the offline installation of OpenShift.

Prepare to load and validate content on IBM Cloud Pak System 2.3.2.0

An IBM Cloud Pak System 2.3.2.0 should have most of the content you need loaded onto the system by default. For sake of completeness, we provide a list of everything you need to have in place though.

We will now show how to verify and/or import the Virtual Images, BYOL Binaries and Pattern Types.

Verifying and importing Virtual Images

By logging onto the IBM Cloud Pak System, you can easily determine whether the Virtual Images you need have been loaded and enabled. Go to Catalog > Virtual Images and filter for the name of the pattern, for example "IBM OS Image for Red Hat Linux". Make sure that the correct version of the Virtual Image is present in the catalog as shown in Figure 1.

Figure 1: IBM OS Image for Red Hat Linux V3.0.15.0 loaded in the catalog

Load the Virtual Image if it is not present. This is best done through the IBM Cloud Pak System Command Line Interface (CLI). See below for the commands to import the Virtual Image and accept the associated licenses.

-bash-4.2# ./pure -h 9.46.123.24 -u admin
Password:
Welcome to the IBM PureApplication Software CLI. Enter 'help' if you
need help getting started.
>>> deployer.virtualimages.create('/content/MAESTRO_RHEL7_X64.ova').waitFor()
>>> deployer.virtualimages.list({'name': 'IBM OS Image for Red Hat Linux Systems', 'version': '3.0.15.0'})[0].acceptLicense()

Verifying and importing BYOL binaries

The "BYOL binaries for Red Hat Openshift V1.0.0 on IBM Cloud Pak System Images" are not installed by default. You can confirm this by logging onto the IBM Cloud Pak System and go to System > Storehouse Browser. If you do not see an entry for /admin/files/RedHatOpenShift as shown in Figure 2, the binaries have not been installed yet.

Figure 2: BYOL binaries for Red Hat Openshift V1.0.0 on IBM Cloud Pak System Images have not been loaded

The process for loading the binaries has been documented here in the IBM Cloud Pak System Knowledge Center.

-bash-4.2# ./cloudpakupload.sh -h 9.46.123.24 -u admin
Password for admin:

IBM Cloud Pak System : Cloud Pak upload utility
*********************************************
 Cloud Pak: IBM Cloud Pak for Applications
*********************************************

Checking cloud pak binaries
Verifying files
config_sum=f2471e758e3016b4381f1ea273dfefbf
cloudpak_storehouse_path=https://9.46.123.24/storehouse/admin/files//RedHatOpenShift/ocpv311-161dockerimages.tar?meta
config_sum=ec6a1e6f5af03dbed276cc120e3948d0
cloudpak_storehouse_path=https://9.46.123.24/storehouse/admin/files//RedHatOpenShift/IBM_OS_Image_for_Red_Hat_Enterprise_Linux_CoreOS_-_16G.4.2.0.ova?meta
config_sum=1cd0140d7892e204a7f714df66a516ca
cloudpak_storehouse_path=https://9.46.123.24/storehouse/admin/files//RedHatOpenShift/ocp4.2.18-x86_64.tgz?meta
config_sum=76881c9d203fb7025ac9867af66f8711
cloudpak_storehouse_path=https://9.46.123.24/storehouse/admin/files//RedHatOpenShift/IBM_OS_Image_for_Red_Hat_Enterprise_Linux_CoreOS_-_250G.ova?meta
config_sum=3e851b8e846a4670a96ded1e747eeeca
cloudpak_storehouse_path=https://9.46.123.24/storehouse/admin/files//RedHatOpenShift/ocp4.2.18-x86_64-extra.tar?meta
config_sum=1fffde9f3c7944f063265e9a5e67ae4f
cloudpak_storehouse_path=https://9.46.123.24/storehouse/admin/files//RedHatOpenShift/jq-linux64?meta
config_sum=7f10d5dbd0b479037bf692fd88453ecd
cloudpak_storehouse_path=https://9.46.123.24/storehouse/admin/files//RedHatOpenShift/openshift-client-linux-4.2.18.tar.gz?meta
config_sum=9d44639d48d631901dde98847ddb64f3
cloudpak_storehouse_path=https://9.46.123.24/storehouse/admin/files//RedHatOpenShift/openshift-install-linux-4.2.18.tar.gz?meta
Verifying Cloud Pak System pattern binaries
1) openshift-client-linux-4.2.18.tar.gz: found locally but not on server
2) IBM_OS_Image_for_Red_Hat_Enterprise_Linux_CoreOS_-_16G.4.2.0.ova: found locally but not on server
3) ocpv311-161dockerimages.tar: found locally but not on server
4) openshift-install-linux-4.2.18.tar.gz: found locally but not on server
5) ocp4.2.18-x86_64.tgz: found locally but not on server
6) ocp4.2.18-x86_64-extra.tar: found locally but not on server
7) IBM_OS_Image_for_Red_Hat_Enterprise_Linux_CoreOS_-_250G.ova: found locally but not on server
8) jq-linux64: found locally but not on server


Uploading needed files...
starting upload of openshift-client-linux-4.2.18.tar.gz
= - openshift-client-linux-4.2.18.tar.gz uploaded
starting upload of IBM_OS_Image_for_Red_Hat_Enterprise_Linux_CoreOS_-_16G.4.2.0.ova
= = = - IBM_OS_Image_for_Red_Hat_Enterprise_Linux_CoreOS_-_16G.4.2.0.ova uploaded
starting upload of ocpv311-161dockerimages.tar
= = = - ocpv311-161dockerimages.tar uploaded
starting upload of openshift-install-linux-4.2.18.tar.gz
= = = = = = - openshift-install-linux-4.2.18.tar.gz uploaded
starting upload of ocp4.2.18-x86_64.tgz
= = = = = = = = = - ocp4.2.18-x86_64.tgz uploaded
starting upload of ocp4.2.18-x86_64-extra.tar
= = = = = = = = = = - ocp4.2.18-x86_64-extra.tar uploaded
starting upload of IBM_OS_Image_for_Red_Hat_Enterprise_Linux_CoreOS_-_250G.ova
= = = = = = - IBM_OS_Image_for_Red_Hat_Enterprise_Linux_CoreOS_-_250G.ova uploaded
starting upload of jq-linux64
= = - jq-linux64 uploaded

Verifying files on remote Cloud Pak System
Verifying files
 = = = = = = = = = = = = = = = = = = = = = = config_sum=f2471e758e3016b4381f1ea273dfefbf
cloudpak_storehouse_path=https://9.46.123.24/storehouse/admin/files//RedHatOpenShift/ocpv311-161dockerimages.tar?meta
= response_md5sum=f2471e758e3016b4381f1ea273dfefbf
file_sum=f2471e758e3016b4381f1ea273dfefbf
config_sum=f2471e758e3016b4381f1ea273dfefbf
 - MD5 match: ocpv311-161dockerimages.tar
= = = = = = = = config_sum=ec6a1e6f5af03dbed276cc120e3948d0
cloudpak_storehouse_path=https://9.46.123.24/storehouse/admin/files//RedHatOpenShift/IBM_OS_Image_for_Red_Hat_Enterprise_Linux_CoreOS_-_16G.4.2.0.ova?meta
response_md5sum=ec6a1e6f5af03dbed276cc120e3948d0
file_sum=ec6a1e6f5af03dbed276cc120e3948d0
config_sum=ec6a1e6f5af03dbed276cc120e3948d0
 - MD5 match: IBM_OS_Image_for_Red_Hat_Enterprise_Linux_CoreOS_-_16G.4.2.0.ova
= = = = = = = = = = = = = = = = = = = = = = config_sum=1cd0140d7892e204a7f714df66a516ca
cloudpak_storehouse_path=https://9.46.123.24/storehouse/admin/files//RedHatOpenShift/ocp4.2.18-x86_64.tgz?meta
= = = response_md5sum=1cd0140d7892e204a7f714df66a516ca
file_sum=1cd0140d7892e204a7f714df66a516ca
config_sum=1cd0140d7892e204a7f714df66a516ca
 - MD5 match: ocp4.2.18-x86_64.tgz
= = = = = = = = = = = = = = = = = = config_sum=76881c9d203fb7025ac9867af66f8711
cloudpak_storehouse_path=https://9.46.123.24/storehouse/admin/files//RedHatOpenShift/IBM_OS_Image_for_Red_Hat_Enterprise_Linux_CoreOS_-_250G.ova?meta
= = response_md5sum=76881c9d203fb7025ac9867af66f8711
file_sum=76881c9d203fb7025ac9867af66f8711
config_sum=76881c9d203fb7025ac9867af66f8711
 - MD5 match: IBM_OS_Image_for_Red_Hat_Enterprise_Linux_CoreOS_-_250G.ova
= config_sum=3e851b8e846a4670a96ded1e747eeeca
cloudpak_storehouse_path=https://9.46.123.24/storehouse/admin/files//RedHatOpenShift/ocp4.2.18-x86_64-extra.tar?meta
= = = = response_md5sum=3e851b8e846a4670a96ded1e747eeeca
file_sum=3e851b8e846a4670a96ded1e747eeeca
config_sum=3e851b8e846a4670a96ded1e747eeeca
 - MD5 match: ocp4.2.18-x86_64-extra.tar
config_sum=1fffde9f3c7944f063265e9a5e67ae4f
cloudpak_storehouse_path=https://9.46.123.24/storehouse/admin/files//RedHatOpenShift/jq-linux64?meta
response_md5sum=1fffde9f3c7944f063265e9a5e67ae4f
file_sum=1fffde9f3c7944f063265e9a5e67ae4f
config_sum=1fffde9f3c7944f063265e9a5e67ae4f
 - MD5 match: jq-linux64
= = = config_sum=7f10d5dbd0b479037bf692fd88453ecd
cloudpak_storehouse_path=https://9.46.123.24/storehouse/admin/files//RedHatOpenShift/openshift-client-linux-4.2.18.tar.gz?meta
= response_md5sum=7f10d5dbd0b479037bf692fd88453ecd
file_sum=7f10d5dbd0b479037bf692fd88453ecd
config_sum=7f10d5dbd0b479037bf692fd88453ecd
 - MD5 match: openshift-client-linux-4.2.18.tar.gz
= = = = config_sum=9d44639d48d631901dde98847ddb64f3
cloudpak_storehouse_path=https://9.46.123.24/storehouse/admin/files//RedHatOpenShift/openshift-install-linux-4.2.18.tar.gz?meta
response_md5sum=9d44639d48d631901dde98847ddb64f3
file_sum=9d44639d48d631901dde98847ddb64f3
config_sum=9d44639d48d631901dde98847ddb64f3
 - MD5 match: openshift-install-linux-4.2.18.tar.gz
Verifying Cloud Pak System pattern binaries
1) openshift-client-linux-4.2.18.tar.gz: verified successfully
2) IBM_OS_Image_for_Red_Hat_Enterprise_Linux_CoreOS_-_16G.4.2.0.ova: verified successfully
3) ocpv311-161dockerimages.tar: verified successfully
4) openshift-install-linux-4.2.18.tar.gz: verified successfully
5) ocp4.2.18-x86_64.tgz: verified successfully
6) ocp4.2.18-x86_64-extra.tar: verified successfully
7) IBM_OS_Image_for_Red_Hat_Enterprise_Linux_CoreOS_-_250G.ova: verified successfully
8) jq-linux64: verified successfully


All binaries verified successfully.

Confirm that the following files are now visible from the IBM Cloud Pak System. Go to System > Storehouse browser and confirm that you see what is shown in Figure 3.

Figure 3: BYOL binaries for Red Hat Openshift V1.0.0 on IBM Cloud Pak System Images have been loaded

Now import and clone the Red Hat Enterprise Linux CoreOS OVA to IBM Cloud Pak System Virtual Images. This can be done from the CPS UI as documented here in the IBM Cloud Pak System Knowledge Center. Below we show how to perform this using the CPS CLI and import both Red Hat Enterprise Linux CoreOS Virtual Images:

-bash-4.2# ./pure -h 9.46.123.24 -u admin
Password:
Welcome to the IBM PureApplication Software CLI. Enter 'help' if you
need help getting started.
>>> deployer.virtualimages.create({'url': 'https://9.46.123.24/storehouse/admin/files/RedHatOpenShift/IBM_OS_Image_for_Red_Hat_Enterprise_Linux_CoreOS_-_16G.4.2.0.ova', 'userid': 'admin', 'password': 'cps@demo'})
{
 "acl": (nested object),
 "advancedoptionsaccepted": "F",
 "build": "",
 "created": Apr 17, 2020 10:09:27 AM,
 "currentmessage": None,
 "currentmessage_text": None,
 "currentstatus": "RM01036",
 "currentstatus_text": "Queued",
 "description": "https://9.46.123.24/storehouse/admin/files/RedHatOpenShift/IBM_OS_Image_for_Red_Hat_Enterprise_Linux_CoreOS_-_16G.4.2.0.ova",
 "hardware": None,
 "id": 3,
 "license": (nested object),
 "licenseaccepted": "F",
 "metaSignature": "1763E62B959A3BE0DC1BBEFC49D432B2F838499F466F944BB9D78B27BE699343D729B97589BE27FD7C446EE8D0BB1EA06ACDCC51F88183093306423162A90C1F",
 "name": "New virtual image 1587118167595-105",
 "operatingsystemdescription": None,
 "operatingsystemid": 0,
 "operatingsystemversion": None,
 "owner": (nested object),
 "parts": (nested object),
 "pmtype": "Unknown",
 "productids": (nested object),
 "servicelevel": "0",
 "signature": "174A9DFA6E126048AA6B0BFEC11CDD986557C4DC6A3318358854E52CF402A5527BD0373B9972DF3EE8B7C3D232F6F949F270EBCD7B70A576DADAA4B65699D1B6",
 "updated": Apr 17, 2020 10:09:27 AM,
 "version": "0"
}
>>> deployer.virtualimages.create({'url': 'https://9.46.123.24/storehouse/admin/files/RedHatOpenShift/IBM_OS_Image_for_Red_Hat_Enterprise_Linux_CoreOS_-_250G.ova', 'userid': 'admin', 'password': 'cps@demo'})
{
 "acl": (nested object),
 "advancedoptionsaccepted": "F",
 "build": "",
 "created": Apr 17, 2020 10:20:04 AM,
 "currentmessage": None,
 "currentmessage_text": None,
 "currentstatus": "RM01036",
 "currentstatus_text": "Queued",
 "description": "https://9.46.123.24/storehouse/admin/files/RedHatOpenShift/IBM_OS_Image_for_Red_Hat_Enterprise_Linux_CoreOS_-_250G.ova",
 "hardware": None,
 "id": 4,
 "license": (nested object),
 "licenseaccepted": "F",
 "metaSignature": "1763E62B959A3BE0DC1BBEFC49D432B2F838499F466F944BB9D78B27BE699343D729B97589BE27FD7C446EE8D0BB1EA06ACDCC51F88183093306423162A90C1F",
 "name": "New virtual image 1587118804539-109",
 "operatingsystemdescription": None,
 "operatingsystemid": 0,
 "operatingsystemversion": None,
 "owner": (nested object),
 "parts": (nested object),
 "pmtype": "Unknown",
 "productids": (nested object),
 "servicelevel": "0",
 "signature": "0B7BB9099870A809014BCD8194E513423B22FE049EEF60835924495DD891F6A9AF56DD1206186DB7401C5768BE4C2C46B07F10C9566DE7E13D11F934419FB4B3",
 "updated": Apr 17, 2020 10:20:04 AM,
 "version": "0"
}

We found that in some cases, the import of the larged Virtual Image (IBM_OS_Image_for_Red_Hat_Enterprise_Linux_CoreOS_-_250G.ova) fails with the following exception as shown in Figure 4.

Failed CWZIP1944E The command execution failed with error: Failed to open virtual disk '/vmfs/volumes/5e9364ef-5e2d9d1a-0cac-0090fac2c712/purescale_cache/b38f0443-2284-4c44-8a44-3c08ef7627b7/disk.vmdk': The file specified is not a virtual disk (15)

Figure 4: Import of Core OS 250 GB Virtual Image fails

As a workaround, you can simply clone the Virtual Image "IBM OS Image for Red Hat Enterprise Linux CoreOS - 16G". From the IBM Cloud Pak System UI, go to Cloud > Virtual images, select this image and click Clone as shown in Figure 5.

Figure 5: Clone Virtual Image "IBM OS Image for Red Hat Enterprise Linux CoreOS - 16G"

Now specify the following as shown in Figure 6 and click OK.

• General information:
  • Name: IBM OS Image for Red Hat Enterprise Linux CoreOS - 250G
  • Description: OpenShift 4 42.80.20190828.2
  • Version: 4.2.0
• Hardware configuration:
  • disk.vmdk (GB): 25

Figure 6: Cloning Core OS 16 GB Virtual Image

The cloning should complete in a minute or so. Once done, under Catalog > Virtual images you should see the Virtual Images shown in Figure 7.

Figure 7: Both Core OS Virtual Images are now available from the catalog

Verifying and importing Pattern Types

By logging onto the IBM Cloud Pak System, you can easily determine whether the Pattern Types you need have been loaded and enabled. Go to Catalog > Pattern Types and filter for the name of the pattern, for example "foundation". Make sure that the correct version of the Pattern Type is present in the catalog and that the status has been set to "Available" as shown in Figure 8.

Figure 8: Foundation 2.1.16.0 Pattern Type loaded in the catalog with status available

Typically the Foundation 2.1.16.0 Pattern Type will have already been loaded, as shown in Figure 8. Confirm which of the other Pattern Types are still missing and load them in the order of the above table.

Loading Patterns Types is best done through the IBM Cloud Pak System Command Line Interface (CLI). The commands below demonstrate how to import and enable the OpenShift 1.0.3.0 Pattern Type.

# pure.sh -h <> <> <> >>> deployer.patterntypes.create('/tmp/openshift-1.0.3.0.tgz') [{ "build_id": "20200317-0331-101", "description": "This pattern deploys OpenShift Container Platform in IBM Cloud Pak System", "license": (nested object), "name": "OpenShift Container Platform Pattern Type", "required": (nested object), "shortname": "openshift", "status": "accepted", "status_text": "Unavailable", "version": "1.0.3.0" }] >>> deployer.patterntypes.get('openshift', '1.0.3.0').enable() {'status': 'avail'} 

Once loaded, you should see the OpenShift 1.0.3.0 Pattern Type in the catalog with status available as show in Figure 9.

Figure 9: Foundation 2.1.16.0 Pattern Type loaded in the catalog with status available

Prepare Red Hat Satellite Server

Most companies choose to integrate their (Intel-based) IBM Cloud Pak System client virtual machines (VMs) with Red Hat Satellite Server (RHSS) 6. When deploying VMs using Red Hat Enterprise Linux (RHEL) 6 or 7, it provides a straightforward process for performing RHEL OS maintenance. (For example, installing security patches on a regular basis.) Of course, it also greatly simplifies the installation of new RPM packages and their dependencies. (For example, you can simply perform a yum install <package-name> from a shell.) You can either deploy RHSS 6 on Cloud Pak System itself, or integrate with an existing RHSS that is already in place. IBM recommends using RHSS 6.4 or higher with IBM Cloud Pak System and IBM Support details how to set it up.

Assuming you have RHSS 6.4 or higher in place, some additional steps are required to deploy RHOCP on Cloud Pak System.

A. Enable required repositories

Make sure that the following Red Hat repositories in Red Hat Satellite Server have been enabled. The RHOCP 4.2 pattern each requires all these Red Hat repositories, most likely you need to only enable "Red Hat Enterprise Linux 7 Server Extra RPMs".

Go to Content > Red Hat Repositories and search for the repository "Red Hat Enterprise Linux 7 Server Extra RPMs" listed under Available Repositories. When the repository shows up, expand it and enable the repository by clicking the plus sign next to it (Figure 10 shows how to do this). By enabling an additional repository, RHSS will download the RPMs for it.

Figure 10: Enabling an additional Red Hat repository in RHSS

Once enabled, you should see the repository "Red Hat Enterprise Linux 7 Server Extra RPMs" listed under Enabled Repositories, as shown in Figure 11.

Figure 11: Viewing enabled Red Hat repository in RHSS

B. Synchronize required repositories

Go to Content > Sync Status, as shown in Figure 12, to confirm that the Result column shows a status of Syncing Complete for each repository or that they are downloaded. You may need to trigger the synchronization process or create a schedule to automatically perform the synchronization at regular intervals.

Figure 12: Confirming synchronization status of repositories in RHSS

C. Ensure that the required repositories are associated with the Content View

Go to Content > Content Views and find the Content View default_contentview. A Content View is associated with any VM that gets deployed and registered with RHSS. For example, it determines what RPMs they can "see." This Content View needs to be updated to include the newly added repository.

Select the Yum Content tab. Under the Repository Selection section, select Add. You should see the Red Hat repository you enabled in Step 2. Select it and click Add Repositories, as shown in Figure 13. When done, click Publish New Version.

Figure 13: Adding new Red Hat repositories to Content View default_contentview in RHSS

D. Ensure that the Content View is associated with the Actication Key

Before you proceed, make sure that the default_contentview page you just updated is associated with the activation key you uses on IBM Cloud Pak System.

Confirm the activation key is associated with your content view by navigating to Content > Activation Keys, as shown in Figure 14.

Figure 14: The osh activation key is associated with Content View default_contentview in RHSS

Within Cloud Pak System, on the Shared Service Instances page, the deployed Red Hat Satellite Six Service instance shows the same activation key, as shown in Figure 15.

Figure 15: The Red Hat Satellite Six Service instance is associated with the osh activation key

Validate that the RPM packages from the required repositories can be installed

From a deployed VM that has registered with RHSS, confirm that you have an RPM package from each of the repositories at your disposal from the yum command line tool. The table below lists the repositories and an RPM for each of those.

By default, only the repositories rhel-7-server-rh-common-rpms and rhel-7-server-rpms are automatically enabled when a VM is deployed on Cloud Pak System. When deploying the RHOCP 4 patterns, the repository rhel-7-server-extras-rpms gets automatically enabled by the Script Package of the Helper VMs in the Virtual System Pattern. However, on the deployed VM that we use for test purposes, we have to manually enable this repository before we can prove that we can install an RPM from each.

1. Run the command yum info <RPM> to demonstrate that the RPM is at your disposal as shown below. Repeat the command for each RPM you need:

-bash-4.2# subscription-manager repos --enable rhel-7-server-extras-rpms Repository 'rhel-7-server-extras-rpms' is enabled for this system.

2.While logged on as root, run the command below to enable the rhel-7-server-extras-rpms repository:

• syslinux
• ansible

-bash-4.2# yum info syslinuxFailed to set locale, defaulting to CLoaded plugins: package_upload, product-id, search-disabled-repos, subscription-managerAvailable PackagesName        : syslinuxArch        : x86_64Version     : 4.05Release     : 15.el7Size        : 991 kRepo        : rhel-7-server-rpms/7Server/x86_64Summary     : Simple kernel loader which boots from a FAT filesystemURL         : http://syslinux.zytor.com/wiki/index.php/The_Syslinux_ProjectLicense     : GPLv2+Description : SYSLINUX is a suite of bootloaders, currently supporting DOS FAT            : filesystems, Linux ext2/ext3 filesystems (EXTLINUX), PXE network boots            : (PXELINUX), or ISO 9660 CD-ROMs (ISOLINUX).

3. Finally, run the command yum install <RPM> to install the RPM. This will validate that your RHSS has a local copy of the actual RPM and any dependencies. Again, repeat the command for each of the RPMs:

• syslinux
• ansible

-bash-4.2# yum -y install syslinuxFailed to set locale, defaulting to CLoaded plugins: package_upload, product-id, search-disabled-repos, subscription-              : managerrhel-7-server-extras-rpms                                | 2.0 kB     00:00rhel-7-server-rh-common-rpms                             | 2.1 kB     00:00rhel-7-server-rpms                                       | 2.0 kB     00:00Resolving Dependencies--> Running transaction check---> Package syslinux.x86_64 0:4.05-15.el7 will be installed--> Processing Dependency: mtools for package: syslinux-4.05-15.el7.x86_64--> Running transaction check---> Package mtools.x86_64 0:4.0.18-5.el7 will be installed--> Finished Dependency ResolutionDependencies Resolved================================================================================ Package        Arch         Version             Repository                Size================================================================================Installing: syslinux       x86_64       4.05-15.el7         rhel-7-server-rpms       991 kInstalling for dependencies: mtools         x86_64       4.0.18-5.el7        rhel-7-server-rpms       203 kTransaction Summary================================================================================Install  1 Package (+1 Dependent package)Total download size: 1.2 MInstalled size: 2.6 MDownloading packages:(1/2): mtools-4.0.18-5.el7.x86_64.rpm                      | 203 kB   00:00(2/2): syslinux-4.05-15.el7.x86_64.rpm                     | 991 kB   00:01--------------------------------------------------------------------------------Total                                              1.0 MB/s | 1.2 MB  00:01Running transaction checkRunning transaction testTransaction test succeededRunning transaction  Installing : mtools-4.0.18-5.el7.x86_64                                   1/2  Installing : syslinux-4.05-15.el7.x86_64                                  2/2Uploading Package ProfileLoaded plugins: product-id, subscription-managerLoaded plugins: product-id, subscription-managerLoaded plugins: product-id, subscription-manager  Verifying  : mtools-4.0.18-5.el7.x86_64                                   1/2  Verifying  : syslinux-4.05-15.el7.x86_64                                  2/2Installed:  syslinux.x86_64 0:4.05-15.el7Dependency Installed:  mtools.x86_64 0:4.0.18-5.el7Complete!

You have now completed the preparation of RHSS for deployment of OCP.

Deploy a Private Docker Registry on IBM Cloud Pak System

The deployment of OCP requires access to a Docker Registry containing the required Docker images. Red Hat provides access to those through registry.redhat.io, however most IBM Cloud Pak System clients do not allow the deployment of VMs with direct internet access. So, the use of a Private Docker Registry is required instead and populated with the Docker images for the OCP.

IBM Cloud Pak System provides the Cloud Paks Docker Private Registry Virtual System Pattern to simplify the deployment of your own Private Docker Registry.

Note: If you already have a Private Docker Registry in place, you can skip this step and go to Populate the Private Docker Registry with OpenShift Docker images section.

While logged on to IBM Cloud Pak System, go to Patterns > Virtual System Patterns and look for the virtual system pattern called "Cloud Paks Docker Private Registry", as shown in Figure 16. By default, it will deploy a VM with RHEL 7.7 and Docker version 18.06.1-ce. But what is new in IBM Cloud Pak System 2.3.2.0 is that this Virtual System Pattern now also automatically pushes and tags the Docker images for OpenShift 4.2.18 into the Docker Private Registry. Those Docker images were loaded into the Storehouse when we imported the BYOL binaries earlier.

Figure 16: Cloud Paks Docker Private Registry virtual system pattern in IBM Cloud Pak System.

Click on the deploy icon highlighted in Figure 17. Provide a Name for your Cloud Paks Docker Registry and ensure that you are targeting your Environment Profile, Cloud Group and IP Group. Make sure to enter your own Password for root and virtuser, and finally expand import_ocp_docker_image to make sure it has ocpversion set to 4.2.18. This determines for which version of OpenShift the Docker images are automatically pushed and tagged into the new Docker Private Registry. Now proceed to deployment by clicking Quick Deploy or Prepare to Deploy.

Figure 17: Deploying Cloud Paks Docker Private Registry virtual system pattern.

Validate that the Cloud Paks Docker Registry is ready

Once deployed, log on to VM to confirm the version of RHEL and Docker.

-bash-4.2# cat /etc/redhat-releaseRed Hat Enterprise Linux Server release 7.7 (Maipo)-bash-4.2# docker --versionDocker version 18.06.1-ce, build e68fc7a

Now verify that the OpenShift 4.2.18 Docker Images are present in the Docker Private Registry:

-bash-4.2# wget --no-check-certificate -cq -O - https://cps-r81-9-46-123-224.rtp.raleigh.ibm.com/v2/ocp4/openshift4/tags/list | python -mjson.tool{    "name": "ocp4/openshift4",    "tags": [        "4.2.18-tests",        "4.2.18-cluster-kube-scheduler-operator",        "4.2.18-openstack-machine-controllers",        "4.2.18-kube-proxy",        ...        ...        "4.2.18-cli",        "4.2.18-operator-lifecycle-manager",        "4.2.18-deployer",        "4.2.18-node"    ]}

Finally confirm that you can pull an OpenShift 4.2.18 Docker Image from the Docker Private Registry:

-bash-4.2# docker pull cps-r81-9-46-123-224.rtp.raleigh.ibm.com/ocp4/openshift4:4.2.18-kube-proxy4.2.18-kube-proxy: Pulling from ocp4/openshift4964d57e311cc: Pull complete6d1814359c74: Pull complete104cc276e39f: Pull complete61783a737c53: Pull complete6fdfe9403559: Pull completeDigest: sha256:c709125d0bbcca1bb2fe6fc3614ea2e5411d689bdcad5bb326afc26263f3582bStatus: Downloaded newer image for cps-r81-9-46-123-224.rtp.raleigh.ibm.com/ocp4/openshift4:4.2.18-kube-proxy-bash-4.2# docker imagesREPOSITORY                                                                         TAG                 IMAGE ID            CREATED             SIZEcps-r81-9-46-123-224.rtp.raleigh.ibm.com/ocp4/openshift4                           4.2.18-kube-proxy   18f6d15cd1f4        2 months ago        289MB

Deploy Cloud Pak System Registry Service

Multiple deployments of OCP can pull images from the same Cloud Paks Docker Private Registry configured in the previous step. A Shared Service is used to automatically pass the information about how to access the Cloud Paks Docker Private Registry to each of the OCP deployments. Go to Patterns > Shared Services to deploy the Cloud Pak System Registry Service as shown in Figure 18.

Figure 18: The Cloud Pak System Registry Service shared service.

Ensure that you are targeting your Environment Profile and Cloud Group for the shared service. Enter the parameters as described below and shown in Figure 19. Finally click Quick Deploy to deploy the shared service.

• Docker Registry Server (Host name should be FQDN): Enter the FQDN of the VM of the Docker Private Registry you deployed on IBM Cloud Pak System
• Helm Repository Server (Host name should be FQDN): (Leave blank as we are now using Helm here)
• Docker Registry User Name: (Leave blank when using the Cloud Paks Docker Private Registry virtual system pattern as this deploys an Docker Private Registry that does not use authentication)
• Docker Registry User Password: (Leave blank when using the Cloud Paks Docker Private Registry virtual system pattern as this deploys an Docker Private Registry that does not use authentication)
• Helm Repository User Name: (Leave blank as we are now using Helm here)
• Helm Repository User Password: (Leave blank as we are now using Helm here)

Figure 19: Deploying the Cloud Pak System Registry Service shared service.

Deployment should complete very quickly, as there are no VMs associated with this shared service. Go to Patterns > Shared Services Instances. There you should see the "Cloud Pak System Registry Service" alongside the "Red Hat Satellite Six Service" as shown in Figure 20.

Figure 20: Deployed Cloud Pak System Registry Service shared service instance.

Examing the OpenShift Container Platform 4 - HA Virtual System Pattern

Go to Patterns > Virtual System Patterns and look for the pattern called OpenShift Container Platform 4 - HA version 1.0.0.0 as shown in Figure 21.

Figure 21: The OpenShift Container Platform 4 - HA Virtual System Pattern.

Open the pattern in the IBM Pattern Builder by clicking the Edit icon. This shows the topology of the OpenShift cluster VMs as shown in Figure 22:

• PrimaryHelper and SecondaryHelper

  There are two Helper nodes running RHEL 7.7 that support the OpenShift Cluster deployed on VMs running Red Hat Core OS. The Helper node provides services for the OpenShift Cluster as documented here in the OpenShift 4 documentation: Helper Git Repository and Helper Blog. IBM implemented two Helper VMs and uses a floating IP address to provide high availability for these services.

• Bootstrap

  There is one Bootstrap node that is used to install the OCP control plane on the Master nodes. It is only used during installation of OCP.

• Master

  There are three Master nodes deployed on VMs running Red Hat Core OS. OpenShift 4 requires three Master nodes, ensuring high availability and quorum of essential Kubernetes services like etcd.

• Worker

  By default there are two Worker nodes deployed on VMs running Red Hat Core OS. This ensures high availability of containers running on those Worker nodes. Depending on the needs for your OpenShift cluster you could opt for a higher number of Worker nodes, or Worker nodes with more cpu and memory. Note that in IBM Cloud Pak System 2.3.2.0 it is possible to add more cpu and memory to Worker nodes after deployment (vertical scaling) but it is not possible to add additional Worker nodes to your OpenShift cluster after deployment (horizontal scaling). Horizontal scaling of OCP clusters is currently targeted for 2.3.3.0.

Figure 22: Examining the OpenShift Container Platform 4 - HA pattern in the IBM Pattern Builder.

As you can see from the table below, a single OCP cluster by default requires 28 virtual CPUs, 112 GB of RAM and 1874 GB of storage. Depending on the number and sizing of the worker nodes, the amount of resources required could be higher of course.

Deployment of your Red Hat OpenShift 4 cluster

With all the previous steps completed, you are now ready to deploy your first Red Hat OpenShift 4 cluster!

Close the IBM Pattern Builder and click the Deploy icon on the OpenShift Container Platform 4 - HA version 1.0.0.0 Virtual System Pattern. Enter or override the following pattern attributes as detailed below and shown in Figure 23.

Helper Node Password (root): password for root user

Helper Node Password (virtuser): password for virtuser user

OpenShift version: 4.2.18 - note that this matches the version of the OCP Docker Images in the Docker Private Registry we deployed

OpenShift Cluster Name: name of your OpenShift cluster - leave empty unless you want to assign a specific cluster name and you have already configured your DNS server accordingly (see Step 12 Post deployment actions

OpenShift Base Domain: subdomain for your OpenShift cluster - leave empty unless you want to assign a specific cluster name and you have already configured your DNS server accordingly

OpenShift Insecure Registry: leave empty

Alternate NFS Server for the OpenShift Image Registry: leave empty as we are using the Docker Private Registry we deployed and reference in the shared service

Alternate NFS Path for the OpenShift Image Registry: leave empty as we are using the Docker Private Registry we deployed and reference in the shared service

Figure 23: Deploying the OpenShift Container Platform 4 - HA Virtual System Pattern.

Make sure you are happy with the values on the panel on the left-hand side for the Name, Environment Profile, Cloud Group, IP Group, and Priority fields. Click Quick Deploy ore Prepare to Deploy to start the deployment.

Typically deployment takes about 30-40 minutes. Upon completion, your Virtual System Instance page should look as shown in Figure 24. Note the link to OpenShift console link under the Primary Helper, we need to complete a number of post deployment actions before we can access it.

Figure 24: "My OpenShift 4 Cluster" Virtual System Instance with OpenShift console link

Post deployment steps for your OpenShift 4 cluster

Before you can use the OpenShift 4 Cluster, a few more steps are required as documented in step 5 of Getting started with OpenShift Container Platform 4.x pattern. When you expand the History section of your Virtual System Instance as shown in Figure 25, you will note that it tells you what needs to be done here.

Figure 25: Post deployment steps shown under History of "My OpenShift 4 Cluster" Virtual System Instance

Retrieve the password for kubeadmin

The kubeadmin password gets generated during installation of OpenShift 4, it cannot be set as a parameter up front like with OpenShift 3.x. Retrieve the kubeadmin password from the command line by using the generated command in the deployment history. Note that the IP address here is the floating IP address that automatically gets assigned to one of the two Helper node VMs. The password used in this command is the Helper node virtuser password that was entered at deploy time.

-bash-4.2# ssh virtuser@9.46.123.228 echo "kubeadmin password is: \$(cat /kubeadmin-password)"virtuser@9.46.123.228's password:kubeadmin password is: PbSXq-WuzII-ahRS6-jTShpkubeadmin password is: ***********************

Configure your DNS server

Set up the followint two DNS wildcard entries for the floating IP address and fully-qualified domain name of your OpenShift 4 Virtual System Instance. This is required in order to be able to access the OpenShift web-console, applications and APIs.

*.<fqdn> IN A <ip>*.apps.<fqdn> IN A <ip>

In the case of our OpenShift 4 cluster here, the floating IP address is 9.46.123.228 with corresponding fully-qualified domain name cps-r81-9-46-123-228.rtp.raleigh.ibm.com. So we would need the following DNS wildcard entries configured:

*.cps-r81-9-46-123-228.rtp.raleigh.ibm.com IN A 9.46.123.228*.apps.cps-r81-9-46-123-228.rtp.raleigh.ibm.com IN A 9.46.123.228

If you are unable to easily make changes to your DNS server, you can add the following entries to your local /etc/hosts file (or equivalent on Windows) for testing purposes. This will allow you to logon to the OpenShift console, but note that you would need additional entries for any applications you would deploy later.

9.46.123.228 console-openshift-console.apps.cps-r81-9-46-123-228.rtp.raleigh.ibm.com oauth-openshift.apps.cps-r81-9-46-123-228.rtp.raleigh.ibm.com

You can find more information about OpenShift external DNS requirements here. The DNS records listed as "This record must be resolvable by both clients external to the cluster ..." are required. DNS is also provided on the Helper Nodes to cover the resolution inside the cluster.

If you are able to configure DNS records up front then the cluster console link will be immediately accessible. Configuring DNS ahead of time is the recommended approach for deploying OCP clusters on CPS. You would need to create the following records in your DNS server for each IP in the IP group you are using to deploy (so that any IP that is selected from the IP group to be the floating IP for the cluster will already have wildcard entries associated with it in DNS):

*.sub.domain IN A <ip>*.mycluster.sub.domain IN A <ip>

Access and Register your OpenShift 4 Cluster with Red Hat

You can now access your OpenShift 4 cluster using the OpenShift console link shown in Figure 26. Logon with the username kubeadmin and the password you retrieved earlier.

Figure 26: Logging in to OpenShift console of "My OpenShift 4 Cluster" Virtual System Instance

Once logged on, go to Compute > Nodes as shown in Figure 27. You will notice that there are 3 Master nodes and 2 Worker nodes, this confirms that the OpenShift 4 cluster topology was deployed as expected.

Figure 27: Confirming topology of "My OpenShift 4 Cluster" in the OpenShift console

Finally do not forget to register your OpenShift cluster with Red Hat. This manual step is required if your OpenShift cluster does not have internet access to reach Red Hat. You can follow step 4 here to register your cluster on the "Cluster registration" page.

Conclusion

IBM Cloud Pak System 2.3.2.0 enables clients to quickly roll out one or more Red Hat OpenShift 4.2 clusters. The automation greatly simplifies the process, ensures consistency and avoids human error. This is also used as the foundation for the deployment of IBM Cloud Paks on the platform.

I would like to thank the following IBMers for their help creating this tutorial: Hugh Hockett, Rahul Nema and Prasag Ganiga.