View Only

Building a software defined infrastructure with iSCSI storage

By Shaik Nisaruddin posted Tue July 03, 2018 07:15 AM

A software-defined infrastructure allows you to virtualize nearly every aspect of your environment: compute resources, networking, and storage. We hear a lot about the benefits of software defined infrastructure, but how exactly do I implement it? In this blog, I will show you how to build a fully software defined infrastructure on IBM® Power Systems with the iSCSI storage configured on them.

First, here's what you need to get started:


  1. Any IBM POWER8 server running PowerVM or KVM:

    • If PowerVM servers are used for the SDI environment, the NovaLink partition must be installed with the software defined environment (SDE) option. The supported operating system for the NovaLink partition is Ubuntu 16.04.

    • If KVM on Power is used for SDI environment, KVM must be installed on Open Power systems or Power systems in OPAL mode. The supported operating systems for KVM on Power are Ubuntu 16.04 and RHEL 7.5.

  2. PowerVC 1.4.1 - This is the cloud virtualization front end built on OpenStack. PowerVC provisions workloads to your cloud compute nodes. It provides a self-service portal for end users to deploy workloads and has advanced features for system administrators to monitor, service, and manage environments.
    The PowerVC server must run on top of a RedHat operating system. RHEL 7.x has been used for this blog.

  3. IBM Spectrum Scale - Provides the software defined storage. It has a robust feature set, is highly scalable, and is very efficient. PowerVC will install and manage IBM Spectrum Scale across the cloud compute nodes, so an end user does not interact with IBM Spectrum Scale directly.

  4. Storage -
    • One boot disk

    • Four or more disks for the Spectrum Scale storage pool. For this example, these disks are provided via iSCSI.

  5. Networking - Software-defined networking (SDN) virtualizes the network in a similar way that compute resources are virtualized. With SDN, PowerVC can deploy networks and change network layout without making any physical changes to your network environment.

For more information on how to set up a Software Defined Infrastructure environment for PowerVC, including the physical wiring requirements, see the following Red Paper.Building a SAN-less Private
Cloud with IBM PowerVM and IBM PowerVC

To configure the Software Defined Infrastructure environment with the iSCSI storage, perform the following steps:

  1. Install the iSCSI initiator on the hosts:

    • For Ubuntu 16.04, run

    • # sudo apt-get -y install open-iscsi

    • For RHEL 7.5, run

    • # sudo yum install -y iscsi-initiator-utils

  2. Edit the configuration file and change startup settings to automatic in /etc/iscsi/iscsid.conf. This makes the host to start the iSCSI services and connect to target automatically even after host restarts.

    # vi /etc/iscsi/iscsid.conf

    node. startup = automatic

  3. Start the iSCSI initiator services on the hosts:

    • For Ubuntu 16.04, run:

    • # sudo /etc/init.d/open-iscsi start

    • For RHEL 7.5, run:

    • # systemctl start iscsi

  4. Get the iSCSI initiator name from the hosts:

    • For Ubuntu 16.04, run:

    • # sudo cat /etc/iscsi/initiatorname.iscsi

    • For RHEL 7.5, run:u

    • # sudo cat /etc/iscsi/initiatorname.iscsi

  5. Create and add a new host in the storage where iSCSI ports have been configured. In this blog, we are using IBM V7000.

  6. Create volumes and map them to the iSCSI initiator host which was created in step 1.

  7. Reboot or restart the iSCSI service on the hosts.

    If you are restarting the iSCSI services -

    • For Ubuntu 16.04, run:

    • # sudo /etc/init.d/open-iscsi restart

    • For RHEL 7.5, run:

    • # systemctl restart iscsi

  8. Discover the disks on the hosts mapped to the storage. The disks will continue to show even when the host restarts if the option node.startup is set to "automatic" in /etc/iscsi/iscsid.conf.

    • For Ubuntu 16.04:

    • # sudo iscsiadm -m discovery -t sendtargets -p
      # sudo fdisk -l
      # lsblk

    • For RHEL 7.5:

    • # sudo iscsiadm -m discovery -t sendtargets -p
      # sudo fdisk -l
      # lsblk

    Steps to configure SDI environment

    1. Install PowerVC on RHEL 7.x on the management Host.

    2. Copy the Spectrum_Scale_5.0.1.1 Spectrum Scale packages to /opt/ibm/powervc/images/spectrum-scale/ .

      Note: Don’t run install script available at /opt/ibm/powervc/images/spectrum-scale/

    Add hosts to PowerVC

    1. Login to PowerVC and on the Hosts page, add the hosts to PowerVC.

    2. Select NovaLink / KVM and specify the IP address or FQDN of the host and credentials of the server.

    3. Once you click Add Host, the Spectrum Scale software is installed on the PowerVC server and on the compute node; a cluster is built and a storage provider is created.

      Note: Adding a node may take around 15 to 20 minutes to register, but PowerVC allows you to register multiple hosts simultaneously.
      When the registration is complete, you will notice that the IBM Spectrum Scale storage pool is created.


    Reboot the hosts to check if the iSCSI disks are rediscovered. The disks must be discovered as a result of step 8 in section "To configure Software Defined Infrastructure environment with the iSCSI storage". If the disks are not discovered, perform the following steps to make the iSCSI disks persistent even on reboot:

    1. Check if the iSCSI session still exists on the hosts.

      #sudo iscsiadm -m session -o show

    2. If the iSCSI session doesn’t exist, then execute the following command for the iSCSI intiator to log in:

      #sudo iscsiadm -m node –login

    3. Discover the disks:

      # sudo iscsiadm -m discovery -t sendtargets -p

    4. To make the disks persistent even on reboot, change node.startup = automatic in the following configuration files:

      • For RHEL 7.5: /etc/iscsi/iscsid.conf

      • For Ubuntu 16.04: /etc/iscsi/nodes/

    5. Discover the disks on the hosts. When the disks are discovered you are ready to configure the GPFS cluster.

    6. The PowerVC server is part of the Spectrum Scale clusters as a manager node. It does not host any of the cluster disks. The spectrum Scale file system is mounted at /gpfs/powervc_gpfs and /gpfs/powervcgpfs/infrastructure/ That is also the directory where volumes will be created.

      Now you can set up your own software defined infrasture and start reaping the benefits of a virtualized environment.

      If you have any questions, feel free to add them below. We’d love to hear from you! And don’t forget to follow us on Facebook, LinkedIn, and Twitter.