In the ever-evolving landscape of enterprise computing, IBM's AIX operating system continues to push the boundaries of performance and efficiency. This blog post aims to provide a comprehensive overview of the IO performance enhancements introduced in the latest AIX releases, highlighting the significant improvements seen in the storage and network stack.
Physical Storage IOPs Improvement
Every storage I/O operation involves a DMA process, during which the data buffer is mapped for DMA access by the physical adapter. Once the I/O is completed by the I/O adapter (IOA), the buffers are unmapped from the DMA address space. In a virtualized platform like POWER, DMA operations are executed using hypervisor calls, which, while necessary, introduce latency into the IO path. Hence a new approach was needed to manage these latencies which is where AIX made improvements. While each I/O operation requires DMA mapping, the corresponding un-map operation can be deferred until the buffer is scheduled for reuse. These unmapped buffers are batched together and at a later point in time, are unmapped using a single hypervisor call. This batching approach significantly reduces the per buffer overhead associated with the un-map operation.
This simple yet effective optimization has yielded performance gains of approximately 50-60% in IOPs(4K) for FC and NVME physical.
Virtual Ethernet Software Multi Queue
Virtual Ethernet in IBM PowerVM is a high-speed, software-based network that enables communication between LPARs within the same physical server without using physical NICs. It supports VLANs and can be bridged to external networks via the Virtual I/O Server (VIOS) using Shared Ethernet Adapter (SEA). Today the Virtual Ethernet Drivers are single threaded/queue which restricts its throughput and increases latency.
In the latest AIX 7.3 TL 3, the driver has been enhanced to support multiple threads/queues for both receive and transmit side, resulting in significant improvements in the throughput.
Internal lab testing using the netop tool demonstrated significant bandwidth improvement for LPAR-to-LPAR communication. We observed more than 80% increase in bandwidth and PPS (packets per second) between LPARs on the same CEC and between LPAR across CECs via SEA.
References for further details
https://community.ibm.com/community/user/blogs/kokil-deuri/2024/12/12/bulk-dma-unmap
https://community.ibm.com/community/user/blogs/lokesh-kambadur/2024/12/19/virtual-ethernet-software-multi-queue