Power

 View Only

  • 1.  PowerVM virtual Network very slow

    Posted 06/07/26 09:06 AM

    Hello,

    We have this setup:

    • Two power11 S1122 FMW version FW1110.10 each with dual VIOS version 4.1.2.0
    • multiple AIX LPAR 7.3 TL4
    • each VIOS on each power uses an PCIe4 LP 2-Port 25/10/1 GbE RoCE SFP28 Adapter with 10GbE SR SFPs Ethernet Adapter

    dual failover SEA setup for network trafic

    On one power :

    System Configuration
    ---------------------
    Hostname : power1vios1 - LPAR ID: 1
    VIOS VRML : 4.1.2.00
    Model : 9824-22A
    Serial : XXXXXXXX
    Firmware : FW1110.10
    ent_capacity : 0.50
    Online CPUs : 1
    Mode : Uncapped
    Type : Shared-SMT-8
    Memory : 8388608 Kbytes

    Snap Taken : Sun May 17 07:57:08 CDT 2026
    **** Last time system re-boot : Apr 05 07:15 ****

    Network Configuration
    ---------------------
    * SEA Adapter - ent4 - SEA Failover with Control Channel
    - State : PRIMARY
    - Priority : 1
    - SEA Flags : THREAD,LARGESEND,
    - Bridge Mode : All
    - ha_mode : auto
    - pvid : 1
    - fb_delay : 30
    - health_time : 60
    - noauto_failback : disabled
    - adapter_reset : no
    - largesend : 1
    - large_receive : no
    - jumbo_frames : no
    - lldpsvc : no
    - plso_bridge : yes

    Real Adapter : ent0 - Physical Adapter - Link: Up - MSS:10Gbps - MSR: 10Gbps Full Duplex - Type: PCIe4 2-port 25/10/1 GbE RoCE SFP28 Adapter x8 (b3151f101410aa06) - jumbo_frames: no
    Virtual Adapter : ent2 - Trk:True Prio:1 Active:True - Platform LSO: Enabled - SwitchID: ETHERNET0 Switch Mode: VEB - PVID:1 VIDs: 310 320 330 340 350 360 370 380 390 410 420 430 440 450 460 470 480 490 510
    Control Adapter : ent3 - Trk:False PVID:99 VIDs: None SwitchID: ETHERNET0

    * SEA Adapter - ent7 - Simplified SEA Failover: Control Channel Not Required
    - State : PRIMARY
    - Priority : 1
    - SEA Flags : THREAD,
    - Bridge Mode : All
    - ha_mode : auto
    - pvid : 590
    - fb_delay : 30
    - health_time : 60
    - noauto_failback : disabled
    - adapter_reset : no
    - largesend : 0
    - large_receive : no
    - jumbo_frames : no
    - lldpsvc : no
    - plso_bridge : yes

    Real Adapter : ent1 - Physical Adapter - Link: Up - MSS:10Gbps - MSR: 10Gbps Full Duplex - Type: PCIe4 2-port 25/10/1 GbE RoCE SFP28 Adapter x8 (b3151f101410aa06) - jumbo_frames: no
    Virtual Adapter : ent6 - Trk:True Prio:1 Active:True - Platform LSO: Enabled - SwitchID: RAC_PRIVE_SWI Switch Mode: VEB - PVID:590 VIDs: None

    * Virtual LAN Adapter - ent5 - Base Adapter: ent4 - TAG: 510


    Network Interfaces Running
    --------------------------
    en5 1500 DD.DD.XX XX.XX.XX.XX CHECKSUM_OFFLOAD(ACTIVE) LARGESEND

    On second Power:

    Hostname : power1vios2 - LPAR ID: 2
    VIOS VRML : 4.1.2.00
    Model : 9824-22A
    Serial : TTTTTTTTTT
    Firmware : FW1110.10
    ent_capacity : 0.50
    Online CPUs : 1
    Mode : Uncapped
    Type : Shared-SMT-8
    Memory : 8388608 Kbytes


    Snap Taken : Mon May 18 04:33:20 CDT 2026
    **** Last time system re-boot : Apr 05 07:15 ****


    Network Configuration
    ---------------------
    * SEA Adapter - ent4 - SEA Failover with Control Channel
    - State : BACKUP
    - Priority : 2
    - SEA Flags : THREAD,LARGESEND,
    - Bridge Mode : None
    - ha_mode : auto
    - pvid : 1
    - fb_delay : 30
    - health_time : 60
    - noauto_failback : disabled
    - adapter_reset : no
    - largesend : 1
    - large_receive : no
    - jumbo_frames : no
    - lldpsvc : no
    - plso_bridge : yes

    Real Adapter : ent0 - Physical Adapter - Link: Up - MSS:10Gbps - MSR: 10Gbps Full Duplex - Type: PCIe4 2-port 25/10/1 GbE RoCE SFP28 Adapter x8 (b3151f101410aa06) - jumbo_frames: no
    Virtual Adapter : ent2 - Trk:True Prio:2 Active:False - Platform LSO: Enabled - SwitchID: ETHERNET0 Switch Mode: VEB - PVID:1 VIDs: 310 320 330 340 350 360 370 380 390 410 420 430 440 450 460 470 480 490 510
    Control Adapter : ent3 - Trk:False PVID:99 VIDs: None SwitchID: ETHERNET0

    * SEA Adapter - ent7 - Simplified SEA Failover: Control Channel Not Required
    - State : LIMBO
    - Priority : 2
    - SEA Flags : THREAD,
    - Bridge Mode : None
    - ha_mode : auto
    - pvid : 590
    - fb_delay : 30
    - health_time : 60
    - noauto_failback : disabled
    - adapter_reset : no
    - largesend : 0
    - large_receive : no
    - jumbo_frames : no
    - lldpsvc : no
    - plso_bridge : yes

    Real Adapter : ent1 - Physical Adapter - Link: Down - MSS:Unknown - MSR: Unknown - Type: PCIe4 2-port 25/10/1 GbE RoCE SFP28 Adapter x8 (b3151f101410aa06) - jumbo_frames: no
    Virtual Adapter : ent6 - Trk:True Prio:2 Active:False - Platform LSO: Enabled - SwitchID: RAC_PRIVE_SWI Switch Mode: VEB - PVID:590 VIDs: None

    * Virtual LAN Adapter - ent5 - Base Adapter: ent4 - TAG: 510

    Network Interfaces Running
    --------------------------
    en5 1500 XX.XX.XX.XX FF.FF.FF.FF CHECKSUM_OFFLOAD(ACTIVE) LARGESEND

    the problem is we notice that the traffic between LPARs is very slow, there is no network drop but network speed seems very slow.

    we setup virtual network using default configurations without any specific tuning. Are there any specific tuning that should be modified so that 10GbE fibre network can operate at there maximum speed ?

    Thanks in advance



    ------------------------------
    Assane BA
    ------------------------------


  • 2.  RE: PowerVM virtual Network very slow

    Posted 06/07/26 09:22 AM
    Hi
    First off.
    You have too many vlans on the one virtual Ethernet adapter.
    Add more and split them with more veth adapters.
    Make sure you use large mtu of 9000 and the switch port is configured the same.
    Use a shared Sea and split the network load across both vios.
    Tune your streams buffers to much higher values as recommended by IBM.
    Ensure you have enough memory and CPU.
    Get vios advisor running and see what it reports. It will give recommendations.
    If you have the adapters consider multiple SEAS or 4 vio servers.
    This configuration is going to consume a lot of CPU.

    Regards 

    Regards 
    Paul Clayton 





  • 3.  RE: PowerVM virtual Network very slow

    Posted 06/07/26 10:02 AM

    Hello Clayton,

    Thanks for your response. Never used the vios advisor, do you any document that explains how it works ?

    If you have also a document or a link for these tunings it would be great.

    Regards



    ------------------------------
    Assane BA
    ------------------------------

    Original Message


  • 4.  RE: PowerVM virtual Network very slow

    Posted 06/07/26 12:08 PM
    Hi

    Look it up on the IBM support site.

    Regards 
    Paul Clayton 



    Original Message


  • 5.  RE: PowerVM virtual Network very slow

    Posted 06/08/26 05:08 AM
    Hi,

    For the streams tuning, this command works for us.

    chdev -l ent$i -a max_buf_control=128 -a max_buf_huge=128 -a max_buf_huge_nu=1024 -a max_buf_large=256 -a max_buf_medium=1024 -a max_buf_small=4096 \ -a max_buf_tiny=4096 -a min_buf_control=128 -a min_buf_huge=128 -a min_buf_huge_nu=512 -a min_buf_large=128 -a min_buf_medium=256 -a copy_buffs=64 \ -a min_buf_tiny=4096 -a min_buf_small=4096 -P
    Example vios advisor command.

    /usr/ios/utils/part -t 2 -i 10

    This will create a bunch of files under /home/padmin.
    You will need a reader to read the vios_advisor_report.xml file.
    I upload to a website that has PHP, XML and XSLT modules loaded to extract and present a web page.
    IBM has some good examples on how to achieve this on there website


     


    Regards

    ___________________________________________________________________________

    Paul Clayton

    Unix Engineer

    IOCO TEch

    Mobile: +27(0) 660847436

    VOIP: +27(0) 219194152





    Original Message


Related Content