File and Object Storage

 View Only

IBM Spectrum Scale Tuning Overview for Hadoop Workload

By WEI GONG posted Mon August 20, 2018 01:05 AM

IBM Spectrum Scale is flexible and scalable software defined file storage for analytics workload. IBM Spectrum Scale provides unified file and object storage for high performance, large scale workloads that can be deployed on-premises or in the cloud. IBM Spectrum Scale provides NFS, SMB, Object and HDFS services and meets the performance that is required by many industry workloads, such as technical computing, big data, analytics and content management.

IBM Spectrum Scale and Hortonworks Data Platform (HDP) were first certified with IBM Spectrum Scale V4.2.3 and HDP 2.6. Since the first certification, Hortonworks and IBM have signed an agreement to certify both the products on an ongoing basis for their new releases and keep the certification current.

IBM Spectrum Scale provide much more benefits to service your Hadoop workload.
> Extreme scalability with parallel file system architecture
> A global namespace that can span multiple Hadoop clusters and geographical areas
> A reduced data center footprint with the industry's best in-place analytics
> True software-defined storage that is deployed as software or as a pre-integrated system
> IBM Elastic Storage Server advantages

IBM Spectrum Scale can be deployed in a Shared Storage model behind an HDP cluster by using IBM Elastic Storage Server. IBM Elastic Storage Server is a pre-integrated file storage solution that is powered by IBM Spectrum Scale software on IBM Power Systems and disk enclosures (JBOD). Additionally, IBM Spectrum Scale can be deployed with any SAN-attached Shared Storage and is a supported configuration for HDP environments.
Spectrum Scale BD&A Solution ESS Deployment Model

IBM Spectrum Scale can be deployed in a Shared Nothing Storage model directly on storage-rich servers running an HDP stack. This type of IBM Spectrum Scale deployment is called FPO deployment. This deployment is similar to native HDFS deployment.
Spectrum Scale BD&A Solution FPO Deployment Model

To make Spectrum Scale work better on those two deployment models, operation system, network, disk and also Spectrum Scale need to be tuned properly. Now you can check this wiki for more detail on how to do that.

You can check the solution brief to know more about Spectrum Scale Big Data and Analytics solutions.