IBM Spectrum Scale 5.1.1 adds Tech Preview support for the beta release of NVIDIA Magnum IO GPUDirect Storage (GDS). This innovative technology simplifies the data path from storage to the GPU to improve the performance of applications on the NVIDIA CUDA platform.
GDS enables the CUDA developer to bring data directly from IBM Spectrum Scale storage to the GPU memory using RDMA (Remote Direct Memory Access). GDS eliminates the buffer copies in system (CPU) memory, bypasses the CPU memory, and places data directly into the GPU application memory. GDS delivers tangible benefits of increased data transfer rate, lower latency, and reduced CPU utilization.
Figure 1: Comparing data paths with and without GPUDirect Storage
Internal testing with IBM Spectrum Scale demonstrates GDS improvement across multiple I/O sizes, different threads, and with multiple GPUs. The reduction in CPU utilization for large data transfers is significant. GDS should be strongly considered for solutions with contention for CPU resources and GPU data transfer.
For some systems, the CPU stands as a bottleneck to overall system performance when significant IO is involved. This happens when the CPU is a weigh point for data transfers, when CPU cycles are burned managing transfers into and out of CPU memory, and CPU DRAM bandwidth is consumed. The addition of GDS enhances IBM Spectrum Scale and all-Flash ESS 3200 in its ability to avoid many of those bottlenecks. Of course, once those bottlenecks are relieved, other bottlenecks may be revealed. To reduce data load time, the network and storage need to also be properly sized.
IBM Spectrum Scale is a highly-scalable data solution recognized for high throughput on a breadth of data patterns. It is a parallel file system that distributes data and access across multiple systems and network connections. This eliminates the storage-side bottlenecks associated with NFS and provides performance scalability that is critical for scaling GPU computing. IBM Spectrum Scale can provide client-side caching, page pools, and run-time tuning to optimize performance.
Figure 2: The NFS bottleneck during a single file operation
IBM Spectrum Scale 5.1.1 supports Magnum IO GPUDirect Storage in configurations that meet NVIDIA requirements, which may be found here https://developer.nvidia.com/gpudirect-storage. To obtain the latest in NVIDIA Beta code to run with IBM Spectrum Scale 5.1.1, you can start with the NVIDIA GDS developer page and/or email me douglasof@us.ibm.com, or the development team scale@us.ibm.com
We encourage GPU developers and IT teams to test and evaluate how this innovative technology can accelerate their CUDA kernels and applications.
----
I am also compiling some FAQs on IBM Spectrum Scale and GDS. Feel free to put your questions below.
#Highlights#Highlights-home