Originally posted by: PontiacGeronimo

Is such configuration suported?

Let's say I have quad-port HEA adapter and two dual-ports PCI(E) network adapters.

What I should do to be able configure SEA adapter on HEA - and what would be redundant solution in such case for SEA-failover configuration (on HEA)?

I assume that I can't mixe together HEA and PCI ports in SEA configuration.

thank you very much in advance for your reliable posts.

kind regards,
P/G.

Originally posted by: ColombianJoker

Of course!
Just remember SEA needs to put ports in promiscuous mode, then you must configure HEA to your VIOS be able to create the SEA -mark VIOS promiscuous able in the LHEA port-.
An promiscuous port in an LPAR gets all ethernet packets entering the port, then the port can only be assigned to VIOS.

RBL.

Originally posted by: gnoble

It is supported, even creating a LACP config between PCI-E/X ports and HEA ports, however, there is a big gotcha with using a HEA in SEA failover.

During a SEA failover, the VIOS LPAR causes a link down on the target interface (by forcing the interface down, then up) to ensure that the switch clears its ARP cache and forces a gratuitous ARP by the VIOs and clients to update the cache with the new, correct port/MAC locations. This is fine on a PCI card, as the LPAR has control of the card and can take the interface down however with a HEA the LPAR does not have physical control of the HEA port and cannot take the interface down. So the ARP update does not happen. This means unless you have a constant stream of outbound connections from the client (e.g. a regular ping of the gateway) to force an ARP update, you can end up in a situation where incoming traffic does not arrive at the clients as it is being sent to the incorrect switch port. This is until something happens to update the ARP table on the switch.

I'd have a look at using multiple vswitches and NIB on the clients, as per - http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101752

As it works just as well with individual or LACP bonded 1 gbit ports.

Originally posted by: PontiacGeronimo

gnoble wrote:
It is supported, even creating a LACP config between PCI-E/X ports and HEA ports, however, there is a big gotcha with using a HEA in SEA failover.
is the restriction on page 21 linked to what you wrote? I am asking about mixing HEA's adapters with PCI ones in SEA Failover...

http://www.redbooks.ibm.com/redpapers/pdfs/redp4340.pdf

"When you assign a physical port to a Virtual I/O Server by enabling the promiscuous mode,
the physical port is then dedicated for SEA use.
The LHEA (Ethernet interface for operating systems) can be used to create an EtherChannel
device, even as the backup adapter, but it is not supported to mix an LHEA with other kinds of
Ethernet device other than LHEAs (such as dedicated adapter ports). All standard
EtherChannel features are available, such as backup adapter and ping dead gateway
detection, to name a few.
When using LHEA ports in aggregation, all other partitions using the LHEA ports from the
same physical HEA ports must also be in aggregation."
also on page 53:

"You cannot mix other kinds of Ethernet devices into an EtherChannel."

Originally posted by: gnoble

No, what I describe is not listed as a restriction anywhere - I discovered it through painful observation. It is definately not covered in this OLD redpaper. I've not found any documentation that mentions it explicitly.

As for etherchannel support with HEAs, you are referring to a 4 year old redpaper.

http://publib.boulder.ibm.com/infocenter/aix/v6r1/index.jsp?topic=/com.ibm.aix.commadmn/doc/commadmndita/etherchannel_adapters.htm

Is current, and indicates that mixing HEAs with physical cards IS supported in AIX 6.1, and as VIOs is based on AIX 6.1 it is supported in VIOs as well. IBM support have confirmed to me that this is a supported configuration (I've asked for confirmation when logging calls in the past) but YMMV.

Originally posted by: PontiacGeronimo

Thank you for your comments. Please kindly help me summarize:

1) HEA logical ports are supported under EtherChannel if all adapters within the EtherChannel are HEA logical ports.

Logical means not dedicated (not in promisciuous mode), right?

2) +For dedicated HEA port, link aggregation with the PCI/PCI-E adapter is supported. +

Dedicated, means whole hea port dedicated to an lpar (in promiscuous mode), right?
To how many LPARs, ports of quad-port HEA adapter can be dedicated? Max to 4 LPARs (1/LPAR), or to 2 (2/LPAR) or only to 1 LPAR (all 4 ports/LPAR)?

3) In addition, PCI/PCI-E and virtual ethernet adapter as a backup adapter is also supported (when the primary adapter contain HEA).

I am not sure how to get it. Does it reffer to a NIB configuration or to some mixed configuration?

Example etherchannel:
Primary list: enX(PCI/PCI-E), enY(HEA dedicated)
Backup: VEA
???

Are the ports on "Primary list" aggregated (ports places in etherchannel on a switch configuration)? How would such etherchannel operate if Primary adapters are not configured in Etherchannel on switch side?

thx in advance,P.

Originally posted by: gnoble

1) HEA logical ports are supported under EtherChannel if all adapters within the EtherChannel are HEA logical ports.

Logical means not dedicated (not in promisciuous mode), right?

I believe so. It's what we have taken it to mean, however we do not use this configuration, we have only used dedicated ports.

+2) For dedicated HEA port, link aggregation with the PCI/PCI-E adapter is supported. +

Dedicated, means whole hea port dedicated to an lpar (in promiscuous mode), right?
To how many LPARs, ports of quad-port HEA adapter can be dedicated? Max to 4 LPARs (1/LPAR), or to 2 (2/LPAR) or only to 1 LPAR (all 4 ports/LPAR)?

Again, I believe so. We use 2 of the 4 ports on a power 750 as members of a etherchannel configuration. The other two ports are not configured as etherchannel members, but are also configured as dedicated to two different LPARs.

3) In addition, PCI/PCI-E and virtual ethernet adapter as a backup adapter is also supported (when the primary adapter contain HEA).

I am not sure how to get it. Does it reffer to a NIB configuration or to some mixed configuration?

Yes, the "backup adapter" comment implies NIB.

I don't think you can do your example without the the "primary list" ports being aggregated - more than one primary port implies port aggregation, through either Cisco etherchannel or 802.3ad. Apart from that yep, the document seems to indicate that you can use VEA as a backup adapter, although I'm struggling to think of a reason you would do it for a SEA on a VIOs :) I'd certainly be getting IBM support to vaildate it as a supported configuration before implementing with this example, they might be able to locate some more VIO/SEA specific information to indicate if it is supported. I've not seen anything that says the use of a VEA as a backing device for a SEA is/is not supported. I can't imagine it is, at least in a VLAN tagged configurations where the SEA cannot be configured to ping an IP to check for connectivity, as it cannot do link detection.

Originally posted by: PontiacGeronimo

I don't think you can do your example without the the "primary list" ports being aggregated - more than one primary port implies port aggregation, through either Cisco etherchannel or 802.3ad.

The multiple adapter on primary list in NIB configuration I found on and I and wonder how it works (incoming connections?):

By NIB I assume not aggregated.

http://publib.boulder.ibm.com/infocenter/aix/v6r1/index.jsp?topic=/com.ibm.aix.commadmn/doc/commadmndita/etherchannel_config.htm

"Network Interface Backup Mode does not exist as an explicit mode in AIX 5.2 and later. To enable Network Interface Backup Mode in AIX 5.2 and later, you can configure multiple adapters in the primary EtherChannel and a backup adapter."

Originally posted by: gnoble

I'm not sure you are reading it correctly.

My pre-5.2 exposure did not include NIB, however from this I gather NIB was not configured through the etherchannel smit screen. If you enter more than one interface in the "EtherChannel / Link Aggregation Adapters" field it will be aggregated. The "Backup" part of NIB is just that, the "Backup Adapter". This can be either a single interface, or a previously defined single etherchannel. It is only used when the primary adapter fails (either pings fail if defined, or link down).