Power Virtualization

Power Virtualization

Learn about the virtualization technologies designed specifically for IBM Power including #PowerVM, #PowerVC, #VM Recovery Manager, #HCM/CMC, and more. #Power #TechXchangeConferenceLab #Servers

 View Only

Zero‑Downtime FC Maintenance Starts Here

By Gayathri Guruprasad posted 04/08/26 01:30 AM

  

Keeping Fibre Channel (FC) environments stable during upgrades or unexpected events has traditionally been challenging. Even routine maintenance could lead to delays, application pauses, or nervous wait times while systems recovered from path failures.

That experience is changing!

What Does This Mean for Customers?

Whether you’re patching adapters, upgrading firmware, replacing hardware, or encountering an unexpected FC event, Quick I/O Failover helps ensure your applications stay online and responsive.

Quick I/O Failover delivers a noticeably smoother operational experience:

• Failover happens in seconds, not minutes

• Applications continue running during FC maintenance

• Firmware upgrades and hardware changes require little to no manual intervention

Maintenance, Reimagined

By eliminating long recovery delays and reducing operational complexity, Quick I/O Failover enables:

• Continuous application availability

• Predictable, low‑risk maintenance windows

• Reduced stress for operations teams

In short, it transforms FC maintenance from a disruptive task into a routine, low‑impact activity.

Availability

Quick I/O Failover is available starting with:

• AIX 7.3 TL04

• VIOS 4.1.2.0

In short:

·      Your systems stay online.

·      Your workloads stay happy.

·      Your maintenance becomes stress‑free.

Notes: Available in AIX 73 TL04 release and VIOS 4.1.2.0

Contacting the PowerVM Team

Have questions or want to learn more?

Join the LinkedIn group: IBM PowerVM https://ibm.biz/powervmgrp

Visit IBM Community discussions: https://community.ibm.com/community/user/power/communities/community-home/digestviewer?communitykey=71e6bb8a-5b34-44da-be8b-277834a183b0&tab=digestviewer

Authors

·      Vijay Kumar Shankarappa 

·      Pramod Manjunath

·      Gayathri Guruprasad

·      Bhavika Aggarwal

·      Rupesh Thota

     Anil Kalavakolanu

5 comments
68 views

Permalink

Comments

06/10/26 05:49 AM

Hi Sasi,

This is not specific to Power11.

It works with Power10 (including EN1N / PCIe4 64Gb 2-port FC adapters) and also Power9, provided the required software levels are in place:

  • AIX 7.3 TL04
  • VIOS 4.1.2.0
  • IBM i 7.6 TR1

EN1N on systems like E1050 was part of our primary validation, but the capability is not adapter-specific.

It applies broadly to all supported FC adapters, including Broadcom (SLI‑4) and Marvell stacks ( 16 GFC and above) , across both:

  • Physical FC
  • NPIV / vFC environments

Hi Carlo,

Great question — this builds on Fast I/O Failure but goes further.

Fast I/O Failure (AIX 7.1):

  • fc_err_recov=fast_fail is primarily focused on link failure detection/recovery timing
  • Primarily handled pending I/Os
  • Waited for active I/Os to complete, which could take time
  • In complex stacks like vFC (~6 layers), delays could exceed 20+ seconds

Quick I/O Failover:

  • Quick I/O Failover and related enhancements are VIOS/AIX FC adapter maintenance-specific behaviors (e.g., firmware update, EEH, dump scenarios), handling a different class of events
  • Handles both pending and active I/Os at all the layers.
  • Actively aborts in-flight commands and returns them immediately to MPIO to re-route.
  • Enables quicker failover (few seconds practically as opposed to 30-40 seconds ) in these events.

    During firmware update / dump / EEH scenarios on FC adapter:

    • Affected paths move to Degraded (Deg) state (visible via lsmpio)
    • No new I/O is sent on those paths (few min / based on health-check cycles and params set )
    • Existing I/O and new IO is rapidly redirected to alternate paths

    👉 The key improvement is not waiting for active I/O completion, which significantly reduces failover time.


    Hi David,

    This capability is enabled by default — no dependency on shutdown/restart commands.

    • Works within the FC (vFC) I/O stack
    • Automatically handles scenarios like adapter firmware updates, EEH, and dumps

    At present, it is limited to the FC/vFC stack. Extending similar behavior to the network stack (e.g., SEA) is a valuable suggestion — we’ll take that feedback forward.

    If needed to disable for some reason, On AIX this can be controlled per VFC adapter using:

    • mt_qk_io_recov attribute on fscsiX associated with VFC adapter

    Hi Chris,

    Thanks!

    Quick I/O Failover is fully automatic once running with the supported AIX and VIOS levels — no manual configuration required.

    • Enabled by default
    • No special tuning needed in normal scenarios

    It automatically handles:

    • FC adapter microcode updates
    • EEH events
    • Firmware dumps / maintenance operations

    From an admin perspective, it’s largely transparent, with visibility available via standard tools like lsmpio on AIX, if you want to observe path state transitions.

    04/09/26 10:06 PM

    Hello,

    Assuming this more related to P11's with the new HBA's. Not for adapter EN1N is a PCIe4 64Gb 2-port Optical Fibre Channel Adapter used in IBM Power 10 servers, such as the E1050 model?

    Cheers

    Sasi

    04/09/26 04:23 AM

    Hi Gayathri,

    This looks cool!  How different is it from Fast I/O Failure from AIX 7.1?

    https://www.ibm.com/docs/en/aix/7.1.0?topic=performance-fast-io-failure-fibre-channel-devices

    Thanks!

    04/09/26 04:16 AM

    Hi,
    Any detailed information on how this functionality works?
    Is it part of the $ shutdown -restart command? 
    It could be good to do the same for the SEA to avoid being manually set into standby mode before VIOS fix pack installation or HW maintenance.
    Thanks for the feedback.

    04/08/26 06:57 PM

    Hi Gayathri,

    Great post! Quick question — does Quick I/O Failover work entirely automatically, or is there anything admins need to configure or monitor? A little more detail on what’s automatic versus what requires admin attention would be really helpful.

    Thanks!