Introduction

In this blog, you'll find a step-by-step guide on how to create a protocol virtual machine (VM) on an IBM Storage Scale System 3500 I/O node and how to deploy it using the IBM Storage Scale Toolkit. This approach is supported from IBM Storage Scale System version 6.1.3.1 and reduces the amount of hardware required while still enabling key protocol services such as NFS and SMB within a Storage Scale System cluster.

Host Hardware Requirements

• 8 CPU cores
• 64 GB of RAM
• 64 GB of /essvm filesystem

Features and Advantage 

• Optimized for entry configuration
• Hardware simplification and cost reduction
• Eliminate dedicated protocol node

Restrictions

• Only NFS and SMB (no Object)
• Management via bridge network from host management network. No BMC network
• Fixed VM Configuration
• One VM per canister
• 8 cores (out of 48)(1 socket, 8 threads)
• 64 GB memory (out of 512)
• Max 8 building blocks
• Number of client connections:
  • NFS: 1000
  • SMB: 512
• 195710 MiB Boot drive mapped to dedicated mirrored partition on the host
• 1 or 2 Network Adapters via PCIe-Passthrough – adapters are not shared with host (own IOMMU_group)

Protocol VM implementation

Prerequisites:

• IBM Storage Scale Systems 3500 nodes are completely deployed
• A CES shared root filesystem is created
• Ensure that the /etc/hosts have the entries for VM management network IP address and hostname mapping along with VM high-speed GPFS admin or daemon. All possible CES IP address must be in /etc/hosts.
• An IBM Storage Scale System container is running in the EMS node
• Review and complete network requirements showed in IBM Storage Scale System Deployment Guide

Procedure:

• From container in EMS run: 

ESS UNIFIED v6.2.x.x CONTAINER root@containerHostname:/ # essrun -N ess3500a cesvm --create --number-of-mellanox-device-passthru 1

• After VM is successfully created, follow the next steps to deploy a protocol node
• Log in the 3500 node

ESS UNIFIED v6.2.x.x CONTAINER root@containerHostname:/ # ssh ess3500a 

• Check if the vm is running

[root@ess3500a ~]# virsh list
 Id   Name              State
---------------------------------
 2    ess3500a-essvm   running

• Log in to the VM console

[root@ess3500a ~]# virsh console ess3500a-essvm
Connected to domain 'ess3500a4-essvm'
Escape character is ^] (Ctrl + ])

ess3500a-essvm login: root
Password:

• Run ess_ssr_setup tool to initialize the management network by attaching it to the VM console

[root@ess3500a-essvm ~]# ess_ssr_setup

• • Type “y” to continue

2025-03-31 19:56:41,275 WARNING:	 There is already successful CODE 20 already run in this node

Continuing invalidates the previous CODE 20 run. Do you want to continue (y/n): y

• • Select the Storage Scale System edition you are running (dme or dae)

Please type Storage Scale System edition? (dme/dae): dme

• • Type the vm protocol management ip and netmask

Please type the management IP of this node (i.e. 192.168.10.10): 
Please type the management netmask of this node (i.e. 255.255.255.0): 

• • Type the vm protocol node password

Be aware that the password will not be prompted into the screen and you will not see it on it
Password: 
Please type the same password again
Password:

• • Select if this node is a new deployment or you are adding to a cluster

Is this a new deployment or adding a block to a running cluster? (new/add): new

• • Type if there is a node that you can test ping connectivity

Is there already any node in this environment that we can test ping connectivity with? (y/n): n

• • Type “y” to perform all the previous selected changes

Do you want to continue and perform changes and tests in this node? (y/n): y

• • If everything goes fine you will see the following message

CODE 20 completed.


Exception happened:

0

• Update the VM OS from EMS container

ESS UNIFIED v6.2.x.x CONTAINER root@containerHostname:/ # essrun -N ess3500a-essvm config load -p
ESS UNIFIED v6.2.x.x CONTAINER root@containerHostname:/ # essrun -N ess3500a-essvm update --precheck
ESS UNIFIED v6.2.x.x CONTAINER root@containerHostname:/ # essrun -N ess3500a-essvm update –offline

Protocol VM deployment using Storage Scale Toolkit 

Prerequisites:

• The protocol VM was successfully created and configured in 3500 nodes
• Storage Scale Toolkit installation package is downloaded and copied in EMS node

Procedure: 

• Deploy protocol using Storage Scale Toolkit from EMS
  • Install Storage Scale Toolkit in the EMS node:

[root@ems ~]# /home/deploy/Storage_Scale_Data_Management-5.2.2.0-ppc64LE-Linux-install --text-only --silent

• • Set up cluster

[root@ems ~]# cd /usr/lpp/mmfs/5.2.2.0/ansible-toolkit/
[root@ems ~]# ./spectrumscale setup -s EMS_HS_IP -st ess

• • Populate cluster configuration 

[root@ems ~]# ./spectrumscale config populate -N EMS_HS_IP

• • Add protocol node

[root@ems ~]# ./spectrumscale node add PROTOCOL_VM_HS_IP -p -n 

• • Define a shared file system for protocol

[root@ems ~]# ./spectrumscale config protocols -f cesSharedRoot -m /gpfs/cesSharedRoot

• • Enable the desired protocols (remember that for this protocol vm is only supported SMB and NFS)

[root@ems ~]# ./spectrumscale enable nfs smb

• • Configure protocol IPs

[root@ems ~]# ./spectrumscale config protocols -e PROTOCOL_IPs

• • Install protocols 

[root@ems ~]# ./spectrumscale install –pr
[root@ems ~]# ./spectrumscale install 

• • Deploy protocols

[root@ems ~]# ./spectrumscale deploy –-precheck
[root@ems ~]# ./spectrumscale deploy

Conclusion 

Deploying a protocol VM on IBM Storage Scale System 3500 nodes offers a cost-effective and efficient way to enable NFS and SMB services within your cluster. By leveraging existing hardware and the Storage Scale Toolkit, you can simplify your infrastructure without compromising performance. While there are some limitations (such as the lack of S3 and object protocol support) this method offers a practical solution for organizations seeking scalable and cost-effective storage integration. With the detailed steps and guidance provided in this blog, you should be well-equipped to successfully implement this deployment in your own setup. For more advanced configurations or troubleshooting, always refer to the official IBM documentation.