Containers, Kubernetes, OpenShift on Power

 View Only

Routing Via Host with OVNKuberenetes

By MICK TARSEL posted Thu January 26, 2023 02:33 PM

  

OVN-Kuberenetes is the default network provider for OpenShift Container Platform. OVN-Kubernetes is based on Open Virtual Network (OVN) and provides an overlay-based networking implementation. OVN-Kubernetes uses something that's called gateway routers to route network traffic out of the OVN layer. Read more about OVN architecture hereBy default, OVN-Kubernetes is configured in shared gateway mode. This default gateway mode is designed to bypass the host routing plane in order to better enable hardware offloading.


What does this mean for ppc64le?


Shared Gateway Mode is enabled with the following flag in OVN-Kubernetes: routingViaHost=false. Today, this is the default gateway mode in OpenShift. This default gateway mode causes HyperShift deployments to fail on ppc64le. Read more about HyperShift here. More specifically, with HyperShift deployments on ppc64le, this bug was first noticed when a port was properly exposed and available on a node. If a different node in the same cluster would try to access data over said port, the packets would successfully reach that port however the replies would vanish. The connection would hang and it is unknown where the reply packets were going to. This would make it impossible for workloads on different nodes to communicate with each other. 


routingViaHost=true is referred to as Local Gateway Mode(LGW). Local gateway mode is required for ppc64le at this time. 


After deployment, we can run this command and set this field to True:

oc patch network.operator/cluster --type merge -p '{"spec":{"defaultNetwork":{"ovnKubernetesConfig":{"gatewayConfig":{"routingViaHost":true}}}}}'

This command forces OVN-Kubernetes to utilize the host kernel routing table and not use the OVN-Kubernetes routing table. This command continues to be a valid Day-2 operation in OpenShift for the near future. You can switch back and forth between these two modes of routing.  

Trade Off:

The reason to use OVN-Kubernetes routing (Shared Gateway Mode) is to enable Hardware Offloading. This, of course, is a performance benefit since packets can bypass the host kernel’s networking stack. The idea is to keep all packets under OVN/OVS control and to try not to have packets on the host networking plane unless needed. 

This does not mean routingViaHost=true is bad performance, it just means routingViaHost=false (the default) would likely provide better performance with hardware offloading enabled. To my knowledge, there are no performance metrics for this feature that currently exist for ppc64le. Besides this, there should not be other major unintended consequences known at this time. 


More Links:

https://developers.redhat.com/blog/2018/11/08/how-to-create-an-open-virtual-network-distributed-gateway-router


https://access.redhat.com/solutions/6969174

Permalink