File and Object Storage

File and Object Storage

Software-defined storage for building a global AI, HPC and analytics data platform 

 View Only

Deploying IBM Spectrum Scale on AWS Quick Start

By Archive User posted Mon September 18, 2017 12:22 PM

  

    The AWS Quick Starts help you to deploy popular solutions on the AWS cloud. Each Quick Start is designed by AWS solutions architects or partners, and makes use of AWS best practices for security and high availability. You can use them to create test or work environments, and start using them immediately.




Advantages of using Quick Start



    With a single click, you can deploy reference architecture of a highly available and secure IBM Spectrum Scale 4.2.3.1 clusters in an existing or new VPC with bastion host on Amazon public cloud.


    This Quick Start automates the deployment of IBM Spectrum Scale on AWS for users
    who require highly available access to a shared namespace across multiple instances. The entire deployment process completes within 25-55 minutes, depending on the cluster size, and it does not require in-depth knowledge of IBM Spectrum Scale.



IBM Spectrum Scale deployment design on AWS




    The following figure illustrates the high-level design of IBM Spectrum Scale deployment on AWS.




Figure-1: IBM Spectrum Scale cluster on AWS quick start high level design


    The following figure illustrates the low-level design of the IBM Spectrum Scale deployment on AWS.



Figure-2: IBM Spectrum Scale cluster on AWS quick start low level design


Before you start



    Ensure that you are aware of the following details before you start with the deployment process:



  • The Quick Start builds the IBM Spectrum Scale 4.2.3.1 version environment by using a pre-built Amazon Machine Image (AMI) with IBM Spectrum Scale installed on the Red Hat Enterprise Linux (RHEL) version 7.2 operating system.



  • The use of IBM Spectrum Scale on AWS, including all packages provided through the Quick Start offering, and packages derived from these, is only intended to be used for a maximum of 90 days, and is not intended for production use.



  • The IBM Spectrum Scale evaluations page will be updated with details on how to proceed with acquiring an IBM Spectrum Scale license after the 90-day trial expires.


  • When estimating costs using the pricing pages for AWS services, note that reading and writing data to an IBM Spectrum Scale file system causes data to be sent between instances in different Availability Zones, which will result in per-GiB data transfer charges.


  • The Quick Start supports only a subset of the overall functionality available with IBM Spectrum Scale, as described in Restrictions section in deployment guide


  • Before launching the Quick Start, review the terms of the license agreement that is available at http://spectrumscale-license.s3-website-us-west-2.amazonaws.com. You will be asked to accept the license terms during deployment.



    • How to deploy through AWS QuickStart



    It is very easy, just click IBM Spectrum Scale on AWS landing page and follow the steps 1,2 and 3 to complete the deployment.





    Step 1 : Preparation before deployment



    i) If you do not already have an AWS account, create one at https://aws.amazon.com by following the on-screen instructions.


    ii) Use the region selector in the navigation bar to choose the AWS Region where you want to deploy IBM Spectrum Scale on AWS.


    iii) Create a key pair in your preferred region.


    iv) Create an S3 bucket or identify an existing one, for keeping Secure Shell (SSH) keys. Specify the bucket name in the Spectrum S3 Bucket parameter when you launch the Quick Start.


    v) If necessary, request a service limit increase for the Amazon Elastic Compute Cloud (EC2) instance types that you intend to deploy. To do this, perform the following steps:


    - EC2 : 20
    - IOPS : 40000
    - EIPs : 5
    - VPCs : 5
    - EBS : 20TB



      Notes:
    1. The Quick Start needs n+1 EC2 instances, where n is the total number of nodes required in the cluster.


    2. The limits are applicable on default account, which also includes AWS free account. The limit varies based on the size of the instances. For example, instance types like t2.micro, t2.small, t2.nano, t2.medium, and m4.large have limit as 20 but other instance types might have lesser limit. Go to Services > EC2 > Limits to see more details.


    3. Consider these limits while customizing the parameters. The deployment might fail if you try to deploy twice in the same region with default limits.




    Step 2: Launch the Quick Start



      Perform the following steps to launch the Quick Start:

    i) Select one of the following options to launch the AWS CloudFormation template in your AWS account.

    1. Deploy IBM Spectrum Scale into a new VPC (end-to-end deployment): This option builds a new AWS environment consisting of the VPC, subnets,security groups, bastion hosts, and other infrastructure components, and then deploys IBM Spectrum Scale into this new VPC.

    2. Deploy IBM Spectrum Scale into an existing VPC on AWS: This option deploys IBM Spectrum Scale in your existing AWS infrastructure.



    ii) Check the region that is displayed in the upper-right corner of the navigation bar. This is where the network infrastructure for IBM Spectrum Scale will be built. The template is launched in the US East (Ohio) Region by default. You can change the region.



    iii) On the Select Template page, keep the default setting for the template URL, and then choose Next.



    iv) On the Specify Details page, change the stack name if needed. Review the parameters for the template. Provide values for the parameters that require input. For all other parameters, review the default settings and customize them as necessary. When you finish reviewing and customizing the parameters, choose Next.






    Step 3 : How to use Spectrum Scale cluster created on AWS:


    The only way to access the IBM Spectrum Scale cluster that is hosted within private subnets is through a Bastion host instance from the public subnet.
    When the AWS CloudFormation template has successfully created the stack, all instances such as compute and NSD servers launched by the Quick Start will be up and running with the IBM Spectrum Scale file system mounted.


      Bastion host can be accessed through your system IP or set of your CIDR mentioned in “Allowed External Access CIDR” in parameter page while customizing the deployment. The following figure illustrates the IBM Spectrum Scale architecture to connect from the host.






    Figure-3: High-level IBM Spectrum Scale cluster architecture to connect from host

    Steps to connect to the IBM Spectrum Scale cluster



    Perform the following steps to connect to the IBM Spectrum Scale cluster:



    1.  Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.



    2.  In the navigation pane, select Instances. In the list of instances, check for the public DNS (IPv4) value for the instance named LinuxBastion.



    3.  Use your AWS private key, which you specify in the Key Name parameter of the AWS CloudFormation template during deployment,  to connect to the bastion host using SSH.



    From the bastion host,use SSH agent forwarding to log in to any of the compute instances or NSD server instances that were launched by the AWS CloudFormation templates. For more information about using an SSH agent to forward your private key on connection, see the details provided in the GitHub documentation (modify as appropriate for AWS access).



    Important: Do not copy your private key to the bastion host instance.




    After you are log in to the cluster node, try out the following IBM Spectrum Scale mm commands to verify your cluster:


      [ec2-user@ip- ~]$/usr/lpp/mmfs/bin/mmlscluster
      [ec2-user@ip- ~]$/usr/lpp/mmfs/bin/mmlsnsd -L
      [ec2-user@ip- ~]$/usr/lpp/mmfs/bin/mmlsfs fs1
      [ec2-user@ip- ~]$/usr/lpp/mmfs/bin/mmlsdisk fs1 -L
      [ec2-user@ip- ~]$/usr/lpp/mmfs/bin/mmdf fs1
      [ec2-user@ip- ~]$sudo /usr/lpp/mmfs/bin/mmgetstate -a




  • IBM Spectrum Scale on AWS Demo :



    • [embed]https://youtu.be/6j5Xj_d0bh4[/embed]


    Reference links




  • The Quick Starts include comprehensive deployment guide can be downloaded from https://s3.amazonaws.com/quickstart-reference/ibm/spectrum/scale/latest/doc/ibm-spectrum-scale-on-the-aws-cloud.pdf



  • IBM Spectrum Scale evaluations: https://developer.ibm.com/storage/products/ibm-spectrum-scale/#evaluate



  • IBM Spectrum Scale introduction: https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.0/com.ibm.spectrum.scale.v4r2.ins.doc/bl1in_IntroducingIBMSpectrumScale.htm



  • IBM Spectrum Scale architecture: https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.0/com.ibm.spectrum.scale.v4r2.ins.doc/bl1ins_architr.htm



  • How to Create IAM user to deploy IBM Spectrum Scale on AWS : IAM user to deploy IBM Spectrum Scale on AWS


    • #Cloudstorage
    #Softwaredefinedstorage
    0 comments
    0 views

    Permalink

    https://community.ibm.com/community/user/blogs/archive-user/2017/09/18/deploying-ibm-spectrum-scale-on-aws-quick-start