The purpose of this blog post is to describe an overview of a real migration involving petabyte-scale data volumes and billions of files. Including high-level architecture.

As Storage Scale continues to evolve, the mmvdisk command has emerged as a foundational tool for managing filesystem transforming complex storage operations into an easy one.

Deployment Scenario

• One storage cluster managing data services
• One remote client cluster accessing the filesystem

► Original Infrastructure:

• 4x ESS 3000 ('meta' and 'nvme' storage pools)
• 2x ESS 5000 (host storage 'hdd' pool)

► New Infrastructure:

• 4x SSS 3500 systems (Hybrid configuration - HDD+nvme)

The system hosted billions of files and several petabytes of data.

The objective was clear: migrate all data to the new SSS 3500 nodes with zero downtime.

Command references:

The mmvdisk command is an integrated command suite and management methodology for IBM Storage Scale RAID. It provides a unified conceptual framework that simplifies many of the more complicated aspects of IBM Storage Scale RAID administration.

Going deeper, we used it for managing vdisk-based file systems using the mmvdisk filesystemsubcommand

The objective of mmchdisk command is to change the state or the parameters of one or more disks in a GPFS file system.

Going deeper, we used mmchdisk suspend to instruct GPFS to stop allocating space on the specified disk. Put a disk in this state when you are preparing to remove the file system data from the disk or if you want to prevent new data from being put on the disk.

When a disk is suspended, the mmrestripefs command migrates the data off that disk.

Migration approach using mmvdisk

The approach was to trigger a restripe of a subset of the “old” vdisks onto the new ones.

Phase 0 – Setup the environment: Add new nodes to the global namespace of the existing cluster.

Phase 1 – Controlled Suspension: Suspend old vdisks mmchdisk <filesystem1> suspend -d <"RG01;..">  to stop writes while preserving read access. This allow the remote compute cluster to continue reading from the existing systems (ESS3000 and ESS5000) while directing all new writes to the new ESS3500 nodes. Now the mmdf output shows all suspended vdisks with the star sign (*).

Phase 2 – trigger data moving with restripe: Trigger target migration using: mmvdisk filesystem delete --file-system <filesystem1> --vdisk-set <vdiskset1, ..>. Before vdisk deletion, the system automatically migrates data to the available vdisks within the new nodes

High level architecture showcase:

Note: During this phase, a restripe was unexpectedly triggered for ALL vdisks that had been in suspended mode. The original plan was to handle them gradually, one step at a time. While this wasn’t anticipated, in our case it actually worked out and matched the desired outcome. That said, this is a critical operation and should be approached with care, depending on the use case.

The documentation have since been updated to include a disclaimer about this behavior. ->  here 

Consideration

The entire migration spanned 8 consecutive days, during which hundreds of terabytes were transparently rebalanced.

Throughout the migration, the system remained fully operational. The HPC cluster continued to run at full capacity, requiring high throughput.

This real-world example is a clear demonstration of the robustness of the Storage Scale code and the exceptional performance of the ESS hardware platform.

The mmvdisk functionality transforms complex storage operations into automated process.

In conclusion, the combination of Storage Scale, Storage Scale System 3500 and IBM deep knowledge forms a high-performance trio that empowers clients to operate with agility and confidence.

#IBMStorageScale #unstructureddata #IBMSoftwareDefinedStorage