View Only

Securely access IBM Cloud services from Red Hat OpenShift Container Platform deployed on IBM Power Virtual Server

By Prajyot Parab posted 4 days ago

Like

This blog shows how to access IBM® Cloud® services using IBM Cloud Direct Link from your applications deployed on Red Hat® OpenShift® in IBM® Power Virtual Server.

The IBM Cloud Direct Link service allows access to IBM Cloud resources over a private network from the Power Virtual Server instance. The Power Virtual Server offering includes a highly available 5 Gbps connection to IBM Cloud services at no cost for each customer per data center.

You can read more details about the service in the following link: https://cloud.ibm.com/docs/power-iaas?topic=power-iaas-ordering-direct-link-connect

The following figure describes the reference deployment:

Prerequisites

You need to make sure that the following prerequisites are fulfilled before performing the steps in this blog:

A running OpenShift Container Platform 4.x cluster deployed (refer to the Installing Red Hat OpenShift Container Platform 4.x on IBM Power Systems Virtual Servers tutorial for instructions)
IBM Cloud Direct Link configured
An IBM Cloud Classic Linux instance deployed
IBM Cloud Object Storage bucket created

Estimated time

The approximate time to deploy the sample applications is 5-10 min.

Steps

First we'll configure Squid proxy and routes on the IBM Cloud Classic Linux instance. This acts as the gateway for OpenShift applications deployed on IBM Power Systems Virtual Server to access IBM Cloud services.

For this blog, we have used a CentOS 7 Linux instance in IBM Cloud Classic. All network traffic is allowed on the private interface of the instance by configuring the right security groups. Public interface only allows Secure Shell (SSH), Hypertext Transfer Protocol Secure (HTTPS), and outbound connectivity.

The following screen capture shows the security group settings for the IBM Cloud Linux instance used for this blog.

Install and configure Squid proxy.
1. Run the following command to install Squid proxy:
```
sudo yum install squid –y
```
2. Run the following command to start the firewall service:
```
sudo service firewalld start
```
3. Run the following command to add the Squid proxy service to firewall:
```
sudo firewall-cmd --zone=public --add-service=squid –permanent
```
4. Run the following command to enable the Squid proxy service:
```
sudo systemctl enable squid
```
5. Run the following command to restart the firewall service:
```
sudo systemctl restart firewalld
```
  You will need the Squid proxy URL in later steps. The URL is typically of the format, https://<Private-IP>:3128. For example, if the instance private IP is 10.85.142.218, then the Squid proxy URL is: http://10.85.142.218:3128
  
  If you are configuring High Availability (HA) for Squid proxy using keepalived, then ensure you enable Virtual Router Redundancy Protocol (VRRP) in the security group and the Linux instances.
```
firewall-cmd --add-rich-rule='rule protocol value="vrrp" accept' --permanent
firewall-cmd --reload
```

Configure routes on the IBM Cloud instance.

Run the following command to add network routes to the Power Systems Virtual Server private network used by OpenShift:

sudo route add -net <powervs_private_nw_subnet> gw <ibm_cloud_instance_gw>

Example:

sudo route add -net 192.168.25.0/24 gw 10.85.142.193

Run the following command to list the route entries in the kernel routing table and verify:

$ ip r

default via 158.177.75.81 dev eth1
10.0.0.0/8 via 10.85.142.193 dev eth0
10.85.142.192/26 dev eth0 proto kernel scope link src 10.85.142.218
158.177.75.80/28 dev eth1 proto kernel scope link src 158.177.75.86
161.26.0.0/16 via 10.85.142.193 dev eth0
166.8.0.0/14 via 10.85.142.193 dev eth0
169.254.0.0/16 dev eth0 scope link metric 1002
169.254.0.0/16 dev eth1 scope link metric 1003
192.168.25.0/24 via 10.85.142.193 dev eth0

Configure OpenShift to use Squid proxy running in the IBM Cloud Linux instance.

Navigate to the directory in your system where you had run the openshift-install-powervs helper script.
Add the following Terraform variables in var.tfvars present in current working directory. Refer to the description of these variables for more details. Set the IBM Cloud Direct Link endpoint network CIDR variable in the var.tfvars file. This is the private network subnet of the IBM Cloud instance.
```
ibm_cloud_dl_endpoint_net_cidr = ""
```
Example:
```
ibm_cloud_dl_endpoint_net_cidr = "10.0.0.0/8"
```
Set the IBM Cloud http/squid proxy URL variable in the var.tfvars file. This is the IP address of the IBM Cloud Linux instance.
```
ibm_cloud_http_proxy = ""
```
Example:
```
ibm_cloud_http_proxy = "http://10.85.142.218:3128"
```

Apply the changes.

 openshift-install-powervs create

Verify the configuration details after successfully applying the changes.

Run the following command on bastion or any of the OpenShift nodes to list the route entries in the kernel routing table:

  $ ip route
  default via 192.168.135.105 dev env2 proto static metric 100
  **10.0.0.0/8 via 192.168.25.1 dev env3 proto static metric 101**
  192.168.25.0/24 dev env3 proto kernel scope link src 192.168.25.172 metric 101
  192.168.135.104/29 dev env2 proto kernel scope link src 192.168.135.106 metric 100

You will notice the IBM Cloud Direct Link endpoint CIDR listed in the output.

Run the following commands to fetch the proxy URLs used by your OpenShift cluster:

 $ oc get proxy/cluster -o template --template {{.spec.httpProxy}}
 http://10.85.142.218:3128
 $ oc get proxy/cluster -o template --template {{.spec.httpsProxy}}
 http://10.85.142.218:3128

You will notice both URLs point to the Squid proxy URL that is running on the IBM Cloud Linux instance.

Verify if you have access to the IBM Cloud service.

We'll use a simple curl application to verify access to IBM Cloud Object Storage from the OpenShift cluster.

Log in to the IBM Cloud dashboard using the URL: https://cloud.ibm.com
In the Resource summary section, click Storage to open the Resource list page. This page lists the storage instances deployed under your account. Click storage instance that you want to use. For this blog, we have created the cos-validation-team Cloud Object Storage instance.
Click Buckets and make a note of the bucket name.

We'll be using bucket-validation-team.
Click Endpoints and make a note of the private endpoint.

For our blog, we have used the private endpoint (s3.private.eu-de.cloud-object-storage.appdomain.cloud) from the eu-de region where we have our remaining setup.

Use SSH to connect to the bastion node or go to the system where you have configured the oc client to access the OpenShift cluster to perform the next set of steps.

Create a YAML file named cos-creds.yaml describing the Secret object.

Update <IBM_CLOUD_API_KEY> with your API key.

<cos-creds.yaml>
```
apiVersion: v1
kind: Secret
metadata:
   name: cos-creds
type: Opaque
data:
   apikey: <IBM_CLOUD_API_KEY>
```

Create a script file named upload-text-file.sh with the following content.

<upload-text-file.sh>

#!/bin/sh
# Create dummy file
FILENAME="/tmp/dummy-file"
echo "This is a dummy file to test connectivity" > $FILENAME

TOKEN=$(curl -X "POST" "https://iam.cloud.ibm.com/oidc/token" -H "Accept: application/json" -H "Content-Type: application/x-www-form-urlencoded" --data-urlencode "apikey=$APIKEY" --data-urlencode "response_type=cloud_iam" --data-urlencode "grant_type=urn:ibm:params:oauth:grant-type:apikey" | jq .access_token)
ACCESSTOKEN=$(eval echo $TOKEN)

OBJKEY="dummy-object"
curl -X "PUT" "https://$ENDPOINT/$BUCKET/$OBJKEY"  -H "Authorization: bearer $ACCESSTOKEN" -H "Content-Type: file"  -F "file=@$FILENAME"
echo "uploaded $FILENAME to cloud object storage"

Create a YAML file named upload-job.yaml describing the job object.

Make a note of the ENDPOINT, HTTP_PROXY, and HTTPS_PROXY environment variable values.

<upload-job.yaml>

apiVersion: batch/v1
kind: Job
metadata:
name: upload
spec:
template:
    spec:
    containers:
    - name: upload
      image: quay.io/bpradipt/curl-jq
      args: ["-c", "/upload-script/upload-text-file.sh"]
      env:
      - name: APIKEY
        valueFrom:
            secretKeyRef:
              name: cos-creds
              key: apikey
      - name: ENDPOINT
        value: s3.private.eu-de.cloud-object-storage.appdomain.cloud
      - name: BUCKET
        value: bucket-validation-team
      - name: HTTPS_PROXY
        value: http://10.85.142.218:3128
      - name: HTTP_PROXY
        value: http://10.85.142.218:3128
      volumeMounts:
        - mountPath: /upload-script
          name: upload-script
    restartPolicy: Never
    volumes:
    - name: upload-script
      configMap:
        name: upload-text-file
        defaultMode: 0777
backoffLimit: 2

Create the objects in the OpenShift cluster.

oc create -f  cos-creds.yaml
oc create configmap upload-text-file --from-file=./upload-text-file.sh
oc create -f upload-job.yaml

$ oc get pods
NAME                        READY   STATUS    RESTARTS   AGE
upload-cjqlf                1/1     Running   0          5s 
$  oc logs upload-cjqlf
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100  2679  100  2544  100   135   7416    393 --:--:-- --:--:-- --:--:--  7810
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100   245    0     0  100   245      0    356 --:--:-- --:--:-- --:--:--   355
uploaded /tmp/dummy-file to cloud object storage

Verify if the file has been uploaded to the bucket from the dashboard.

Summary

This blog described how to access IBM Cloud services from an OpenShift cluster on IBM Power Systems Virtual Server with IBM Cloud Direct Link enabled. You can use the same approach for accessing other IBM Cloud services.

Power Developer eXchange

A place for IBM Power open source developers to learn,
collaborate, contribute, and create

Containers, Kubernetes, OpenShift on Power

Securely access IBM Cloud services from Red Hat OpenShift Container Platform deployed on IBM Power Virtual Server

By Prajyot Parab posted 4 days ago

Prerequisites

Estimated time

Steps

Summary

Permalink

Power Developer eXchange

A place for IBM Power open source developers to learn, collaborate, contribute, and create

Containers, Kubernetes, OpenShift on Power

Securely access IBM Cloud services from Red Hat OpenShift Container Platform deployed on IBM Power Virtual Server

By Prajyot Parab posted 4 days ago

Prerequisites

Estimated time

Steps

Summary

Permalink

A place for IBM Power open source developers to learn,
collaborate, contribute, and create