ROCEv1 and RoCEv2 SRIOV support on AIX
Contributors: Vishwashreya Machineni & Swetha Venkannagari
RoCEv1 and RoCEv2 are supported on the SRIOV VF port. While creating the SRIOV VF make sure that you are creating a RoCE enabled port.
List of adapters that support RoCEv1 and RoCEv2
|
Adapter Device ID
|
Adapter Name
|
|
B315211014101407
|
CX7 - Narwhal - 200G
|
|
B315211014101207
|
CX7 - Shale - 25G/10G/1G
|
|
B3151F101410AA06
|
CX6 - Manatee - 25G/10G/1G
|
|
B3151D101410A606
|
CX6 - Cedar Lake - 100G
|
|
B315191014103506
|
CX5 - Haleakala - 100G
|
|
B31513101410F704
|
CX4 - Glacier Park - 100G
|
|
B315151014101E06
|
CX4 - Everglades 25G
|
|
B315151014101F06
|
CX4 - Everglades 10G
|
How to configure RoCEv1/RoCEv2 SRIOV on AIX?
From Command line interface of HMC:
1.Convert the adapter port from dedicated to SRIOV.
2. Modify the SRIOV port to an option that has non-zero RoCE ports from the Maximum SR-IOV Logical Ports option.
Here the max_logical_ports_option can be 0 or 1 or 2. In order to enable RoCE on physical port, the max_logical_ports_option should be 2.
3. Create a SRIOV RoCE VF
The following step is on lpar:
4. Once the Logical port (VF) is created, follow the same steps for RoCEv1 and RoCEv2 configuration.
5. To validate that VF supports RDMA on ROCE VF port check in the entstat by executing the below command.
From HMC GUI console:
The same steps can be done via HMC GUI console.
1.Open HMC GUI and choose the selected adapter port from the respective CEC.
2. Convert the dedicated mode to SRIOV mode (if the adapter is in dedicated mode).
After converting the mode, it shows like below:
3. In order to enable the RoCE on the physical port, select non-zero RoCE option from the Maximum SR-IOV Logical Ports option. The non-zero RoCE option may differ for different adapters.
3. Once you select the option it shows as below
4. Then create a SRIOV RoCE logical port from the lpar partition-Hardware virtualised I/O.
Select the RoCE port type and the respective adapter VF on which the logical port needs to be created.
5. Once the logical Port is created on the lpar, use cfgmgr to configure the logical VF port and follow the same steps to check RoCEv1 and RoCEv2 functionality.
To configure the RoCE on AIX logical partition, follow the below blogs:
For RoCEv1 - https://community.ibm.com/community/user/power/blogs/vishwashreya-machineni/2024/04/15/rocev1-configuration-on-aix
For RoCEv2 - https://community.ibm.com/community/user/power/blogs/vishwashreya-machineni/2024/04/30/rocev2-enablement-on-aix