Deploying IBM RedHat OpenShift Cluster on AWS using IBM Cloud Satellite
Using IBM Cloud Satellite, you can use your own compute infrastructure that is in your on-premises data center, other cloud providers, or edge networks to create a Satellite location.
By setting up an IBM Cloud™ Satellite location you can start running IBM Cloud services on your own infrastructure resources. Visit for more details on Satellite
Satellite can be setup on your own private data center / on-premise, Amazon Web Services (AWS), Google Cloud, Microsoft Azure or using IBM Cloud Infrastructure services.
Mostly setting up the satellite involves below steps which generally performed manually.
- make sure to have the host machine meet the prerequisites
- create a location using IBM Cloud Console
- download the script from IBM Cloud console
- execute the script on the hosts which need to attach to the satellite
- Add the hosts to control plane
However, IBM provides an automated template using IBM Schematics , currently only available for AWS. This automated will perform all the above steps with click of a button.
In this document we will look into the complete steps, starting with pre-reqs, setting up satellite on AWS, install ROKS cluster and enabling worker node host public IPs to cluster subdomain to access the OCP console.
Prerequisites
Check IBM Cloud permissions
The below are the minimum required permissions for the user to create the satellite location, attach the hosts and deploy the ROKS cluster on the satellite.
|
IAM Service name
|
Platform roles
|
Service roles
|
Scope
|
|
Satellite location
|
Administrator
|
|
|
|
Satellite link
|
|
Satellite Link Administrator
|
|
|
IBM Cloud Object Storage
|
|
Manager
|
|
|
IBM Cloud Schematics
|
Administrator
|
|
|
|
Kubernetes service
|
Administrator
|
Writer / Manager
|
All resource group
|
|
Container-registry
|
Administrator
|
|
Account
|
|
Certificate Manager
|
Administrator
|
Manager
|
All resource group
|
|
VPC Infrastructure
|
Administrator
|
|
|
You can look at IBM document for details on the use-cases and relevant roles required for achieving the same.
To assign these roles navigate to IBM Cloud -> Manage > Access (IAM).
Click Users -> Select User-name
Click on Access policies tab
Click Assign access.
For Assigning IAM services roles , select IAM Services tile, and from the Services drop-down, select a service from the above table and assign corresponding platform / service role(s).
Repeat the step for all the IAM services.
Instead of assigning roles to specific user, you can also create Access Groups, add user to the access group and then assign Access polices to the access group.
Visit for more details.
Check AWS Permissions And Get Access Keys
Please follow the instructions to create Access & Secret key for your AWS account.
On AWS Console
Navigate to IAM -> Access Management -> Users -> Select <your-user-name> -> Security Credentials tab
Click on Create Access Key
Note the values of
Access Key ID and Secret Key which is used in later sections.
Install Satellite CLI
You may need to use the IBM Cloud Satellite CLI to create / view some of the configurations. So follow the below steps to install the cli on your environment.
ibmcloud login
ibmcloud plugin install container-service
IBM Cloud plug-in for Kubernetes Service includes ibmcloud sat commands to manage Satellite resources and ibmcloud oc to manage OpenShift cluster resources.
If you don’t have IBM Cloud CLI itself, first install the ibmcloud cli following the instructions from here
Start With Satellite Deployment
Create location
The following steps will help you to create the satellite location on AWS. Please note that you should have followed the pre-reqs section for the making sure that you have appropriate permission for both IBM Cloud and AWS.
In IBM Cloud Console , navigate to
Satellite -> Location -> Click on Create location
On the “Create a Satellite location” page , select Amazon Web Services tile
In the “
AWS Credentials” section, provide
AWS access key ID and
AWS secret access key which you should have obtained for your AWS Account.
Click on Fetch options from AWS
Now you would see the AWS EC2 instance details being fetched automatically and set to the defaults such as region, instance-type and number of instances as shown in below screen-shot.
AWS EC2 instances:
Click on
Edit (pencil icon) if you would want to chose the different region or instance type, instead of default settings. However please remember the number of nodes , instance type should meet the Satellite pre-reqs.
Satellite Location:
Click on Edit to provide the appropriate values for the
Name : <Name of the location >
Resource Group: <Select Resource group in which the location is provisioned>
Managed From: Chose IBM Cloud region that is closest to where your host machines physically reside. This ensures low network latency.
IBM API Key (optional) :
Object Storage:
By Default a new Object storage bucket is created for you.
If you want to use existing instance, click on edit and provide details, otherwise skip to next action.
Review the details provided on Summary page and click on Create location
Now you can view that a location is being created and it may take around 30mins to provision.
You can verify the progress of the satellite location creation by viewing logs from the the schematic workspace which is also created.
To view schematic workspace navigate to
In
IBM Cloud Console ->
Schematics -> Workspaces -> Click on workspace name same your satellite location -> Activity -> View logs
Verify components setup for the Satellite location
Once the satellite location is successfully created, you would see various resources provisioned in both IBM Cloud as well as AWS .
For reference, provided the resources provisioned on both in the following screenshots.
In AWS cloud account :
Resources provisioned:
1 virtual private cloud (VPC).
1 subnet per zone.
1 security group to meet the host networking requirements for Satellite.
6 EC2 instances spread evenly across zones, or the number of hosts that you specified.
You can look into AWS account to verify details
Instances created:
Security Group:
Placement group:
VPC in AWS:
In IBM Cloud account:
Resources provisioned:
1 Satellite location.
3 Satellite hosts that represent the EC2 instances in AWS, attached to the location and assigned to the Satellite location control plane.
3 Satellite hosts that represent the EC2 instances in AWS hosts, attached to the location, unassigned, and available to use for services like an OpenShift cluster. If you added more than 6 hosts, the number of hosts equals the number that you specified minus the 3 that are assigned to the control plane.
Small note: This screenshot has taken later after creating OCP cluster on Satellite and hence showing 3 nodes already assigned to ocp cluster. However you should see these 3 nodes as unassigned which we would assign later for OCP cluster.
Apart from the above there are Satellite link endpoints which are created as shown below
Deploy ROKS Cluster on Satellite Location
Once Satellite location is successfully created, the next step is to deploy the ROKS cluster on the Satellite. As you have noted that, we have attached 6 nodes to Satellite location and out of which 3 had assigned to Control plane. So the remaining three nodes will be attached as worker nodes for the ROKS cluster which is running on the Satellite location.
In IBM Cloud console, navigate to OpenShift -> Clusters -> Create Cluster
Orchestration service
Chose OpenShift version and select Satellite under Infrastructure
Important note: If the OpenShift page is showing up to upload the OCP Entitlement , you need to provide your OCP entitlement key either by uploading the file or paste in the below text box.
In the above screen, you can click on the “pull secret” link which will redirect you to RedHat page. Assuming you already have a RedHat subscription, you can provide your RHN credentials and can download / copy the entitlement key.
Location section:
Chose the Resource group
Select Satellite location
Once satellite location is selected, you would see the Worker Pools section below. Leave the detaults as-is and proceed.
Small note: When the satellite location is selected, you would see 3/6 under Host Usage in the above screen instead of 6/6. This screenshot has taken later after creating OCP cluster on Satellite and hence showing all 6 nodes are used.
Satellite Config
Chose Enable cluster admin access for Satellite config
Resource Details
Provide Cluster Name
Review the details on Summary page and Click on Create
Provisioning of the cluster may take in between 30-90 mins. You may verify the progress on the cluster from OpenShift -> Clusters page
Click on the Cluster name and you should see that all the worker nodes should show in Normal state in Overview page
Accessing cluster from Public network
Now that we have deployed the IBM ROKS cluster on AWS thorugh Satellite, lets try accessing the OpenShift Cluster Console
To access OpenShift WebConsole, Click on OpenShift -> Clusters -> <Cluser_name> -> Overview -> click on OpenShift Web Console
You may not be able to access the OCP console because when deployed on AWS through Satellite, the cluster hosts are only available through private network. Please look at the important note below.
Important note:
If your location hosts have private network connectivity only, or if you use Amazon Web Services, Google Cloud Platform, and Microsoft Azure hosts, you must be connected to your hosts' private network, such as through VPN access, to connect to your cluster and access the OpenShift web console. Alternatively, if your hosts have public network connectivity, you can test access to your cluster by changing your cluster's and location's DNS records to use your hosts' public IP addresses.
So as mentioned in the note, we have 2 options.
- Set up VPN access to the Cloud Provider
- Update cluster's subdomain and location's DNS record to use the public IP addresses of the hosts.
Now, we will take the option b) to update the cluster’s subdomain, as this is for the PoC purpose.
Update the cluster’s subdomain:
Note:
Making your location and cluster subdomains available outside of your hosts' private network to your authorized cluster users is not recommended for production-level workloads
The following set of instructions to be run through the ibmcloud cli. Refer the instructions on pre-reqs section to install the ibmcloud cli and satellite plugins in your environment.
Open command line terminal and login to the ibmcloud using your ibmcloud credentials ( ibmcloud login --sso )
Once logged in, follow the below instructions to udpate the cluster's domain with public IP addresses of hosts.
- Get the Satellite location using the below command. Please provide the name of your satellite location <location-name>
ibmcloud sat location get --location <location-name>
- Review the location subdomains and check the Records for the Private IP addresses of the hosts that are registered in the DNS for the subdomain.
ibmcloud sat location dns ls --location <location-name>
The following steps update the public IP addresses in the subdomain DNS records to access from Public network.
- Retrieve the matching public IP addresses of your hosts from your cloud provider, which is AWS here
Go to AWS Console and retrieve the Public IPs of all 6 EC2 instances created for the Satellite location.
If not visible by default, Enable Public IPV4 address in Preferences -> Attribute column and please note the public IP address of all instances
- Update the location subdomain DNS records with the public IP addresses of each host in the control plane.
ibmcloud sat location dns register --location <location_name_or_ID> --ip <host_IP> --ip <host_IP> --ip <host_IP>
- Verify that the public IP addresses of the control plane hosts are registered with your location DNS records.
ibmcloud sat location dns ls --location <location_name_or_ID>
- Get the Hostname for your cluster in the format <service_name>-<project>.<cluster_name>-<random_hash>-0000.upi.containers.appdomain.cloud and note the private IP(s) that were automatically registered
ibmcloud oc nlb-dns ls --cluster <cluster_name_or_ID>
You can see that private IP addresses of the worker nodes are registered here.
- Add the public IP addresses of the hosts that are assigned as worker nodes to this cluster to your cluster's subdomain.
ibmcloud oc nlb-dns add --ip <public_IP> --cluster <cluster_name_or_ID> --nlb-host <hostname>
<public_IP> : Public worker node and host
<cluster_name_or_ID>: OCP cluster name
<hostname>: the one retrieved in the above step, example : gp-sate…..-0000.upi.containers.appdomain.cloud
Repeat the command for each Worker node host's public IP address.
- Verify the Public IP addresses of the worker nodes are added. You should see all 3 worker node public IP addresses under IP(s) along with private IP addresses.
ibmcloud oc nlb-dns ls --cluster <cluster_name_or_ID>
- Now remove the Private IP addresses of the worker nodes from the subdomain
ibmcloud oc nlb-dns rm classic --ip <private_IP> --cluster <cluster_name_or_ID> --nlb-host <hostname>
Repeat this step for each worker node with its private ip.
- Again verify that only the Public IP addresses of the worker nodes are added. You should see all 3 worker node public IP addresses under IP(s) along with private IP addresses.
ibmcloud oc nlb-dns ls --cluster <cluster_name_or_ID>
Now updating the Cluster subdoamin with Public IPs of hosts is completed, so lets access the RedHat OpenShift Cluster console .
To access OpenShift WebConsole,Click on OpenShift -> Clusters -> <Cluser_name> -> Overview -> click on OpenShift Web Console
Acknowledgements:
I would like to Thank Sivasailam Vellaisamy for all the valuable guidance and support for documenting the steps.
