Prerequisite:

OS version: AIX 7.3 TL3 or above

System Firmware version: FW1060 or above

Introduction:

While doing read/write operations to block IO devices over the PCI bus, AIX uses DMA (Direct Memory Access). As per AIX design, the buffers used in the IO operation are not pre-DMA-mapped, but instead DMA mapped and unmapped on-the-fly . In a typical IO flow, once device driver initiates a IO request, the relevant buffer is DMA mapped using a hypervisor call, the actual IO transfer is done by the device DMA controller and then the buffer DMA mappings are removed using another hypervisor call. That is done for every IO request e.g. if the IOPS is X, then this mapping/unmapping will happen X times per sec. As the relevant hypervisor calls are costly operations, this can cause a performance bottleneck limiting IOPS to a lower value.

Solution:

Starting with AIX 7.3 TL3, this design has been enhanced so that the DMA unmapping hypervisor call at the end of every IO completion can be avoided. This is possible because at any instant, if there are sufficient resources available to create new DMA mappings for the incoming requirements, then there is no need to do costly unmapping operation after every IO. Instead, the mapping info is added to a "pending list" and when sufficient quantities of them have accumulated, they are unmapped in "Bulk" using a new hypervisor call. This new hypervisor call (available in systems with firmware FW1060 and above) allows the OS to  pass a list of disjoint DMA mapped areas to be unmapped in one-shot.

This enhancement provides significant boost to IO throughput due to the significant reduction in the frequency of DMA unmapping hypervisor calls which are very impactful for DMA transfers. 

Performance Test Results:

In our in-house performance testing, depending on the workload, we have observed more than 60% IOPS gain for block IO on supported devices. 

Sample results: