ROCEv1 Configuration on AIX
Contributors: Neelima Nandula, Swetha Venkannagari, Vishwashreya Machineni
What is RoCEv1?
RoCEv1 protocol is defined as RDMA over Ethernet header. It uses ether-type 0x8915 and can be used with or without the VLAN tag. The regular Ethernet MTU applies on the RoCE frame.
RoCEv1 is a non-encrypted and non-routable RDMA protocol. RoCEv1 mode is default mode.
The ports of RoCEv1 adapters can be cabled directly together or connected by two cables and a single switch.
List of adapters that support RoCEv1
|
Adapter Device ID
|
Adapter Name
|
|
B315211014101407
|
CX7 - Narwhal - 200G
|
|
B315211014101207
|
CX7 - Shale - 25G/10G/1G
|
|
B3151F101410AA06
|
CX6 - Manatee - 25G/10G/1G
|
|
B3151D101410A606
|
CX6 - Cedar Lake - 100G
|
|
B315191014103506
|
CX5 - Haleakala - 100G
|
|
B31513101410F704
|
CX4 - Glacier Park - 100G
|
|
B315151014101E06
|
CX4 - Everglades 25G
|
|
B315151014101F06
|
CX4 - Everglades 10G
|
Enabling RoCEv1
| Description |
Command Syntax |
| Enable RoCEv1 mode |
no -r -o roce_v2_mode=0 |
How to configure RoCEv1 on AIX?
RoCEv1 mode is default mode in AIX.
1.Check with “no” command option if RoCEv1 is enabled.
Here - “0” validates that RoCEv1 is enabled.
2.We have 3 modes for rdma as part of ODM attribute list
- Desired
- Required
- Disabled
Assign “desired” or “required” to enable RoCE on the physical port.
3.Configure the IP on the port and check ibv_devices list where it lists the RoCE enabled port.
4.Check the rping by running server & client as shown below.
