IBM Spectrum Scale Version 5.0 has been announced at Super Computing 2017 and is scheduled for general availability by the end of December 2017. So what should you expect from this release of the solution that IDC has rated the world’s No 1 Software Defined Storage – a one stop-shop for unstructured data workloads that require the very best performance and scalability? This latest release has been developed to meet the needs of the US Department of Energy CORAL supercomputer project. It has significant improvements in performance, usability and security, among other things.
Here are some highlights from the development team:Core filesystem - We have worked on added performance acceleration via RDMA enhancement (over RDMA transport), enhanced metadata performance, enhanced performance for small files and improved space efficiency.
- Compression feature has been optimized with support for LZ4
Security - Introducing File Audit Logging logs filesystem events to a retention-enabled fileset to track user access to the file system
- Enhanced usability for secure data at rest (encryption)
Protocols - Dynamic modification of NFS exports
- Improved upgrade support for Object
- Ubuntu support for protocol nodes (NFS/SMB/Object)
AFM - Support for File Compression for AFM and AFM DR filesets
- Load balancing enhancements
- ILM support for snapshots for AFM and AFM DR filesets
Management GUI - Enhancement to manage/configure AFM & TCT
- Network monitoring for both IP and RDMA transports
- Upload diagnostic data to a PMR automatically, etc.
REST API - Expanded REST API for Performance data collection, threshold management, snap creation, addition/removal of nodes from cluster.
Big Data and Analytics – Certification with HortonWorks Data Platorm 2.6
- Support short circuit write for better performance
- Support 2+ Spectrum Scale file system support in one HDFS Transparency cluster
- Support local read/write bytes statistic for internal disk based Spectrum Scale nodes
Transparent Cloud Tiering - Remote mounted filesystem support, tier different fileset to different cloud containers, enhanced support for multiple cloud accounts and containers.
** Special Mention**Spectrum Scale Best Practices for Genomics - Release of end to end development tested best practices for genomic solution paper based on composable building block approach.
Migration to 5.0.0 from previous release As with all releases, the migration path to 5.0.0 from 4.X.X is concurrent and requires no downtime or movement of dataOnce these clusters are at 5.0.0, they will benefit from the performance improvements, new features (such as file audit logging), and various enhancements that are included in 5.0.0.
That being said, there is one enhancement that will not be applied to these clusters, and that is the increased number of sub-blocks per block for small file allocation. On filesystems with large block sizes (typically over 1 MB), this enhancement can improve space efficiency for small files, and can improve the performance in small file creation. Filesystems created at 4.X.X and earlier will retain the current sub-block allocation. Only new filesystem created post migration can leverage the increased number of sub-blocks per block feature.
Note: It is not usually recommended to create a new filesystem in 5.0.0 with new sub-block size feature and then manually migrate data from old filesystem to the new one. While it is feasible and possible for smaller environment but will not be worth the migration effort for many production environment.
Outside of that one particular function, the remainder of the performance improvements, metadata improvements, updated compatibility, new functionality, and all of the other enhancements will be immediately available to you once you complete the upgrade to 5.0.0 -- with no need to reformat, move data, or take your data offline.
Refer to the enablement material at the following links:
https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/General%20Parallel%20File%20System%20(GPFS)/page/White%20Papers%20%26%20Mediahttps://developer.ibm.com/storage/blog/PS: Thx to my colleague Nikhil Khandelwal for co-authoring this blog.
Regards
Sandeep
#GPFS#whatsnew#spectrumscale5.0#IBMSpectrumScale#newrelease#Softwaredefinedinfrastructure#Softwaredefinedstorage