Multi-Arch Compute for Red Hat OpenShift Container Platform on IBM Power systems lets you use a pair of compute architectures such as, ppc64le and amd64, within a single cluster. This exciting feature opens new possibilities for versatility and optimization for composite solutions that span multiple architectures.
As part of the Red Hat OpenShift Multi-Arch Compute effort, I enabled the following Power and Intel compute architecture pairs:
In certain use cases, you need to manage the Intel and Power compute (nodes) independent from each other. This article provides a recipe to create a `power` worker pool with an example of updating for multi-path not supported on Intel.
- Assign a role label called power in addition to worker to existing running power worker nodes.
-
Get all power worker nodes.
# oc get nodes -l kubernetes.io/arch=ppc64le,node-role.kubernetes.io/worker=
NAME STATUS ROLES AGE VERSION
worker0-p Ready worker 27d v1.27.9+e36e183
worker1-p Ready worker 27d v1.27.9+e36e183
-
Add the power node-role to the power nodes.
# oc label node/worker0-p node-role.kubernetes.io/power=
node/worker0-p
-
Verify the nodes have two roles listed: worker and power.
# oc get nodes -l kubernetes.io/arch=ppc64le,node-role.kubernetes.io/worker=
NAME STATUS ROLES AGE VERSION
worker0-p Ready worker,power 27d v1.27.9+e36e183
worker1-p Ready worker 27d v1.27.9+e36e183
-
Create a new MachineConfigPool with the name power. The matchExpression applies the worker and power MachineConfig objects to the MachineConfigPool members. The members are determined based on the matchLabels.
cat <<EOF | oc apply -f -
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfigPool
metadata:
name: power
spec:
machineConfigSelector:
matchExpressions:
- {key: machineconfiguration.openshift.io/role, operator: In, values: [worker,power]}
nodeSelector:
matchLabels:
node-role.kubernetes.io/power: ""
EOF
-
Verify the power MachineConfigPool is created.
# oc get mcp
NAME CONFIG UPDATED UPDATING DEGRADED
master rendered-master-7053d8fc3619388accc12c7759f8241a True False False
worker rendered-worker-6db67f47c0b205c26561b1c5ab74d79b True False False
power rendered-power-6db67f47c0b205c26561b1c5ab74d79b True False False
-
Now that the power MachineConfigPool is created. Special kernel arguments for multipath are created for the Power nodes, and associated based on the matching power label.
cat <<EOF | oc apply -f -
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: power
name: preinstall-power-kargs
spec:
kernelArguments:
- rd.multipath=default
- root=/dev/disk/by-label/dm-mpath-root
EOF
-
Wait for the MachineConfigPool called power to restart. This task can take up to 5 minutes, as it's going to reboot each power instance.
# oc wait mcp/power --for condition=updated --timeout=5m
-
You can now delete the MachineConfig preinstall-worker-kargs belonging to worker mcp as it only applies to Power.
# oc delete mc preinstall-worker-kargs
-
Wait for the worker mcp to get updated This task can take up to 5 minutes, as it's going to reboot each worker instance.
# oc wait mcp/worker --for condition=updated --timeout=5m
-
Verify pods are running fine for openshift-machine-config-operator.
# oc get pods -n openshift-machine-config-operator -l k8s-app=machine-config-daemon -l kubernetes.io/arch=ppc64le
NAME READY STATUS RESTARTS AGE
machine-config-daemon-vxb4c 1/1 Running 2 43m
machine-config-daemon-aab4c 1/1 Running 2 43m
-
Check the Cluster Operator status to ensure that all components are running fine.
# oc get co
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
authentication 4.15.0 True False False 4d9h
baremetal 4.15.0 True False False 47d
cloud-controller-manager 4.15.0 True False False 47d
cloud-credential 4.15.0 True False False 47d
cluster-autoscaler 4.15.0 True False False 47d
config-operator 4.15.0 True False False 47d
console 4.15.0 True False False 37s
control-plane-machine-set 4.15.0 True False False 47d
csi-snapshot-controller 4.15.0 True False False 47d
dns 4.15.0 True False False 47d
etcd 4.15.0 True False False 47d
image-registry 4.15.0 True False False 47d
ingress 4.15.0 True False False 47d
insights 4.15.0 True False False 17d
kube-apiserver 4.15.0 True False False 47d
kube-controller-manager 4.15.0 True False False 47d
kube-scheduler 4.15.0 True False False 47d
kube-storage-version-migrator 4.15.0 True False False 47d
machine-api 4.15.0 True False False 47d
machine-approver 4.15.0 True False False 47d
machine-config 4.15.0 True False False 47d
marketplace 4.15.0 True False False 47d
monitoring 4.15.0 True False False 47d
network 4.15.0 True False False 47d
node-tuning 4.15.0 True False False 25d
openshift-apiserver 4.15.0 True False False 47d
openshift-controller-manager 4.15.0 True False False 47d
openshift-samples 4.15.0 True False False 47d
operator-lifecycle-manager 4.15.0 True False False 47d
operator-lifecycle-manager-catalog 4.15.0 True False False 47d
operator-lifecycle-manager-packageserver 4.15.0 True False False 47d
service-ca 4.15.0 True False False 47d
storage 4.15.0 True False False 47d
These steps confirm that you are done with MCP related changes for Power based OCP cluster supporting `multipath`.
You have seen how to manage the Intel and Power compute (Nodes) independent from each other and use this recipe to setup create a `power` worker pool.