Introduction:
In the previous blog ( https://community.ibm.com/community/user/ibmz-and-linuxone/blogs/deepti-naphade/2023/01/12/adding-persistent-local-storage-to-zcx-for-openshi ) we discussed how the Red Hat OpenShift Cluster built using zCX for OpenShift product allows adding local disks (backed by linear VSAM data sets) either using the ocp_provision.xml workflow or using the ocp_add_local_storage_disks.xml workflow after the OpenShift cluster is up and running.
Now for local disks added post applying the APAR OA64240 and OA64246, you can setup PV’s using the /dev/disk/by-id/virtio-DATAxx as the device path; either creating PV manually or creating PVs using the Local Storage Opertor's LocalVolume resource.
Step-by-step guide:
The disks added to your zCX for OCP instances (Cluster nodes) can be verified by looking at the start.json file in the zCX instance directory.
To view the added local storage disks from the OpenShift cluster node, login to the terminal of the ocp node and first issue the command
sh-4.4# chroot /host
The figure below shows all added disks post applying above APARs. They will appear under /dev/disk/by-id as virtio-DATAxx, see below:
Fig 2: Disks show up under /dev/disk/by-id
The following example shows a PV using by-id devicePath: (Note that the node affinity needs to be specified.)
apiVersion: v1
kind: PersistentVolume
metadata:
name: local-pv-8
spec:
capacity:
storage: 2Gi
volumeMode: Filesystem
fsType: xfs
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Delete
storageClassName: sc8
local:
path: /dev/disk/by-id/virtio-DATA4
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- worker7
Fig 3: pv.yaml
Configuring the Local Storage Operator:
Follow instructions on this page to install the Local Storage Operator:
https://docs.openshift.com/container-platform/4.12/storage/persistent_storage/persistent_storage_local/persistent-storage-local.html#local-storage-install_persistent-storage-local
The following example shows a LocalVolume resource object definition using the /dev/disk/by-id as the devicePath.
apiVersion: local.storage.openshift.io/v1
kind: LocalVolume
metadata:
name: local-disks-local-sc
namespace: openshift-local-storage
spec:
logLevel: Normal
managementState: Managed
nodeSelector:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- worker7
storageClassDevices:
- devicePaths:
- /dev/disk/by-id/virtio-DATA6
- /dev/disk/by-id/virtio-DATA7
fsType: xfs
storageClassName: local-sc
volumeMode: Filesystem
Fig 4. local-volume.yaml
Summary:
Once the APAR OA64240 and OA64246 are applied, we can use the /dev/disk/by-id/virtio-DATAxx as the devicePaths for setting up PV and/or setting up PVs using LSO's LocalVolume resource object.
Existing PV’s defined using by-path ids are not impacted. If we try to create a new LSO LocalVolume resource using devicePath of by-path id or devicePath as mount location such as /dev/vdc, we still get PV’s created using the by-id. Hence these APARs will guard against the user configuration error of using mount path as the device path.
The above two APARs will be required for the LSO LocalVolumeSet and LocalVolumeDiscovery features to work in zCX environment. As of Red Hat Openshift Container Platform version 4.12, automatic discovery and provisioning of local storage devices on OpenShift cluster nodes is technology preview feature only and therefore, not recommended for production use. Please refer to Red Hat OpenShift version 4.12 documentation for more details.
References:
-
Adding persistent local storage to zCX for OpenShift Cluster using Local Storage Operator (LSO): https://community.ibm.com/community/user/ibmz-and-linuxone/blogs/deepti-naphade/2023/01/12/adding-persistent-local-storage-to-zcx-for-openshi
- IBM zCX Foundation for Red Hat Foundation. https://www.ibm.com/products/zcx-openshift
- Red Hat OpenShift. https://www.Red Hat.com/en/technologies/cloud-computing/openshift
- zCX Foundation for Red Hat OpenShift content solution. https://www.ibm.com/support/z-content-solutions/zcx-openshift