When I started working as a network engineer, I learned the problems of asymmetric routing, mainly with firewalls as they tend not to like stateless connections. I thought this was a relative infrequent event, however talking with several customers (most of them on the financial services sector) it looks like this is a common issue on Cisco ACI deployments where traffic between pods can take different routes if not configured carefully.
How we detect the problem?
Good news is SevOne can help all these customers to detect this issue, they just need to leverage the power of IPFIX and the plethora of fields available that will give us the information we need to understand better how the traffic is flowing through the network.
If you want to know more about the fields available, please have a look at this page that contains all the fields in IPFIX https://www.iana.org/assignments/ipfix/ipfix.xhtml
One field that will be key on this process is the "ipNextHopIPv4Address" field (aka next hop) that will analyse the packets and tell us which is the IP address of the next hop for the packets going through the switches, allowing us to understand the direction of the traffic. There are other "next hop" fields in IPFIX such the IPv6 version or the BGP next hop field, but for this use case we will use the "common" IPv4 next hop field.
NOTE: SevOne is compatible with any IPFIX field, meaning that we can report on any field available.
SevOne IPFIX Fields
As there are so many fields available, in SevOne we need to create a custom view that shows the fields we choose in a single graph. For that matter I've created a custom view that displays the source IP, destination IP and the next hop of the conversations.
SevOne Asymmetric Routing NetFlow View
With this custom flow view, I can go to the reporting engine of SevOne and create a report that shows all the conversations including the next hop for a specific section of the network.
Asymmetric Routing Report
This view is showing me a lot of valuable information, including the kind of conversations going through my network, but also which next step is being used. As mentioned before, it is key to know the IP of the next step to understand the path followed by the packets.
Another functionality available in SevOne is the ability to filter the information displayed in the widget to a specific subset of data, in this case to a specific conversation. That will allow us to focus the view onto a conversation that we know that might have potential asymmetric routing problems.
On the screenshot below we have filtered the information to show only traffic between 172.24.147.112 and 172.24.147.115 bidirectionally, and we can clearly see that from 112 to 115 we are using three different next hops (.3 , .5 and .6) whereas from 115 to 112 we are using only the .3 next hop, meaning that this is a clear case of asymmetric routing where some traffic uses a different path to reach the destination.
Asymmetric Routing Example
It is great that we can create a report and display this kind of information to "proof" that there is some asymmetric routing going on, however it would be even better if we could proactively detect this kind of issue and alert when this situation occurs in the future. Good news (again) this is something possible with SevOne alerts where we can configure policies that analyse the flow traffic and alert us when the trigger conditions are met.
SevOne Flow Alert
Another happy customer
With these configurations we have managed to proactively avoid asymmetric routing scenarios (for Cisco ACI as well as for other technologies) in several customers, therefore another happy SevOne customer :)
#GeneralQuery#TechnicalBlog