ROCEv1 Configuration on AIX
Contributors: Neelima Nandula, Swetha Venkannagari, Vishwashreya Machineni
What is RoCEv1?
RoCEv1 protocol is defined as RDMA over Ethernet header. It uses ether-type 0x8915 and can be used with or without the VLAN tag. The regular Ethernet MTU applies on the RoCE frame.
RoCEv1 is a non-encrypted and non-routable RDMA protocol. RoCEv1 mode is default mode.
The ports of RoCEv1 adapters can be cabled directly together or connected by two cables and a single switch.
![RoCE configuration on SUT and PP](https://dw1.s81c.com//IMWUC/MessageImages/8c0ac2d43dfb489b974c30edd96cfc00.png)
List of adapters that support RoCEv1
Adapter Device ID
|
Adapter Name
|
B315211014101407
|
CX7 - Narwhal - 200G
|
B315211014101207
|
CX7 - Shale - 25G/10G/1G
|
B3151F101410AA06
|
CX6 - Manatee - 25G/10G/1G
|
B3151D101410A606
|
CX6 - Cedar Lake - 100G
|
B315191014103506
|
CX5 - Haleakala - 100G
|
B31513101410F704
|
CX4 - Glacier Park - 100G
|
B315151014101E06
|
CX4 - Everglades 25G
|
B315151014101F06
|
CX4 - Everglades 10G
|
Enabling RoCEv1
Description |
Command Syntax |
Enable RoCEv1 mode |
no -r -o roce_v2_mode=0 |
How to configure RoCEv1 on AIX?
RoCEv1 mode is default mode in AIX.
1.Check with “no” command option if RoCEv1 is enabled.
Here - “0” validates that RoCEv1 is enabled.
![](https://dw1.s81c.com//IMWUC/MessageImages/7cf2f19b5da148d582bf8934ee7e6483.png)
2.We have 3 modes for rdma as part of ODM attribute list
- Desired
- Required
- Disabled
Assign “desired” or “required” to enable RoCE on the physical port.
![](https://dw1.s81c.com//IMWUC/MessageImages/ab1e12f2b1874f598e622d97aeee20a1.png)
![](https://dw1.s81c.com//IMWUC/MessageImages/caafdaa9945046f9ad5ae5722ca4b8bb.png)
3.Configure the IP on the port and check ibv_devices list where it lists the RoCE enabled port.
![](https://dw1.s81c.com//IMWUC/MessageImages/e93a981d4b3a4c13a3a67f280d65bcc9.png)
![](https://dw1.s81c.com//IMWUC/MessageImages/0e3825a6665644d3a7955a11f8a391e4.png)
4.Check the rping by running server & client as shown below.
![](https://dw1.s81c.com//IMWUC/MessageImages/55695bace6c44541989d9bbf95da1f44.png)
![](https://dw1.s81c.com//IMWUC/MessageImages/c9c4b77072b048f18bec6760ca331b53.png)