AIX

 View Only



LinkedIn Share on LinkedIn

AIX 7.3 vPMEM device support

By Jim Allen posted Tue December 14, 2021 06:15 PM

  

Overview

Virtual Persistent Memory (vPMEM) is provisioned by the Hardware Management Console (HMC) from DRAM in the CEC (Central Electronics Complex) and assigned to an LPAR.  Unlike normal DRAM, the contents of vPMEM remain valid across LPAR reboots as long as the CEC is not rebooted.  This also assumes that if the LPAR is shutdown, one does not reprovision the vPMEM, which will reinitialize it.  Thus vPMEM has performance characteristics similar to DRAM, but with additional persistence.  AIX 7.3 adds support for vPMEM, where each vPMEM instance/volume is configured as an hdisk.  These vPMEM hdisks can be used by AIX 7.3 for most of the same usages as traditional disks:

    • raw disk access

    • LVM/JFS2

    • paging space

    • flash caching

It should be noted that vPMEM volumes can not be used as bootable disks nor dump devices in AIX.  AIX 7.3 supports up to 10 TB of capacity for vPMEM.  This can be provisioned as one hdisk or as multiple hdisks from the HMC.  Earlier releases of AIX (6.1, 7.1, 7.2) do not support vPMEM.

On AIX 7.3, three new types of devices will configure for a vPMEM volume and be represented in ODM.  The first device is the nvmem0 device, which is the parent to all other vPMEM devices.  The nvmem0 device is only configured if it detects any vPMEM volumes assigned to LPAR.  Under this device are the Storage Class Memory (SCM) devices (i.e. scm0, scm1 etc).  Thus the scm devices are children of the nvmem0 device in ODM.  Each vPMEM volume provisioned for the LPAR will have exactly one scm device.  In addition there is an hdisk device that is a child of each scm devices.  So on AIX, one can see these devices from the lsdev (hdisk1 and hdisk2 are the vPMEM disks):

#lsdev -t nvmem
nvmem0 Available Special Use System Memory

#lsdev -t scm
scm0 Available Storage Class Memory
scm1 Available Storage Class Memory

#lsdev -c disk
hdisk0 Available 00-01-01 MPIO IBM FlashSystem Disk
hdisk1 Available Virtual Persistent Memory Disk
hdisk2 Available Virtual Persistent Memory Disk

Supported Hardware and System Firmware for vPMEM on AIX

AIX 7.3 supports vPMEM on all Power 10 systems running running pHyp 1010 or higher.  AIX 7.3 also supports vPMEM on Power 9 systems that are at or above the following pHyp levels:

    • b0716a_2126.930 (F930.50R)

    • b0811a_2125.940 (F940.41R)

    • b0701a_2126.950 (F950.20R)

Earlier levels of pHyp on Power 9, may lead to phantom nvmem0 devices configuring.  In this case there will be an nvmem0 device, but no scm nor vPMEM hdisks.  This issue can resolved by updating the pHyp to the above levels and then doing an rmdev -dl nvmem0.

There is no support for vPMEM on Power 8.

Provisioning vPMEM for AIX

The Hardware Management Console (HMC) running HMC V9.1.940 or later is used to provision vPMEM volumes to an LPAR.  The IBM white paper for using vPMEM for SAP HANA on Linux provides the details on provisioning vPMEM from an HMC on pages 12 - 15 (in the Managing vPMEM volumes with the Hardware Management Console section).  This white paper PDF can be found here: vPMEM SAP HANA Whitepaper.  One difference from that documentation for AIX use, is that the “Affinity” box shown on page 14, should be unchecked.  If it is checked, then it is possible that the one vPMEM volume may be discontiguous and thus AIX will create more than one hdisk for the one vPMEM volume.

Using vPMEM with AIX's LVM/JFS2

As mentioned in the overview section, vPMEM disks can be used in volume groups just like traditional storage hdisks.  This also includes in volume groups mixing vPMEM hdisks with traditional hdisks.

When creating a volume group on an vPMEM hdisk or extending a volume group to a vPMEM hdisk, the mkvg, and extendvg commands, respectively, will display a message reminding the user these disks can loose their data if the CEC is rebooted.  So for example one would see the following message when creating a a volume group on hdisk2 if it is a vPMEM disk:

0516-2067 mkvg: hdisk2 is a Virtual Persistent Memory disk. The data in
the disks is not persistent across Central Electronic Complex (CEC) reboot.

These messages will be the only reminder about this.

Limitations

The following limitations exist for vPMEM on AIX

    • vPMEM volumes can only be created or deleted when the LPAR is shutdown.

    • Changing the size of vPMEM volumes is not supported

    • vPMEM disks do not support boot nor dump

    • LPM is not supported with vPMEM disks
      Live Kernel update is not supported on AIX LPARs using vPMEM disks.

    • vPMEM devices do not support virtualization by VIOS (i.e. no support for vSCSI etc)

    • AIX will uses a portion of the vPMEM volume provisioned for its metadata, thus the usable vPMEM hdisk sizes will be slightly smaller than size seen on the HMC.

    • AIX does not provide load/store access to vPMEM.




0 comments
61 views

Related Content

Permalink