Hello,

We have the following site:

• 2 FS5300 version 9.1.1
• 2 SAN Brocade with 16Gb ports
• IBM Power with dual VIOS, LPARs

we are setting up PBHA for some of our AIX LPARs. Both FS5300 are on same site connected to same brocade switches. PBHA replication is done through FC connection. Storage partition has been setup, IP quorum also. Each SAN switch contains two virtual fabrics; One for PBHA replication other for host+storage connection. Each virtual fabric for PBHA replication contains one zone with one physical WWPN from each FS5300 canister. (architecture is below)

after storage partition creation the test LPAR we are using to validate the configuration (an AIX 7.3 LPAR) is facing I/O latency on the provisionned volumes.

Has anyone already worked with a similar configuration?

Is there anything here that might not be properly set up and could be causing the I/O slowness we're seeing?

At this code level, is there a way to check the synchronization of the PBHA volumes between the two storage arrays?

Thanks in advance

Hi Assane,

Need to know workload, host connectivity etc. You can raise a PMR to the IBM for performance. 

Hi Assane,

I have the same setup and I'm doing the configuration on my zoning with SANNAV.

Can you tell me about your zoning? What did you put in the Public Fabric (which WWPN Physical or Virtual) and which zone into Private.Fabric (which WWPN Physical)? Did you create ONE big zone group with all the physical in the private fabric? I dont find any documentation about zoning two FlashSystem 5300...

Hi David,

Here's the link to the IBM Docs for zoning. The new versions of the PB-R and PBHA Redbooks will be published in a few weeks.

https://www.ibm.com/docs/en/flashsystem-9x00/8.7.0?topic=connectivity-zoning-partnerships

Hello David,

I just saw this post. In case it could be useful, I had created one SAN zone per SAN for the private SAN part with all the connected canisters' physical WWPNs. For the public side, it would be with the virtual WWPNs. Basically, it's standard zoning (one initiator with one target from each canister)

Hello Assane,

we have a similar setup, although we have a vSphere cluster connected.
I have observed up to 15ms read latency displayed in the Dashboard.
However, that seemed to be caused by me having hosts connected through FC-SCSi and FC-NVMe at the same time.
I have since removed NVMe entirely and could observe a 50% drop in latency, however it still looks pretty bad:

This is the current state while there is a vMotion migration going on.

Hi Jan,

If you are observing the workload on the volume level during a Storage vMotion between two volumes on the same FlashSystem array, it may indicate that VAAI offload is not functioning properly. Under normal conditions, data movement primarily occurs at the mdisk/backend level rather than generating heavy host-side volume I/O.

Regarding the latency increase when using FC-SCSI and FC-NVMe simultaneously, it sounds like both protocols may be sharing the same FlashSystem target ports. If so, you may have reached the practical limits of those ports due to mixed protocol traffic, which would explain the higher latency.

Are you also monitoring the target port utilization on the FlashSystem side? It would be interesting to see the actual utilization levels during the workload. Also, what does the SAN connectivity look like? FC link speeds on the HBAs and switches, are you also using same ports for HA etc. Those details could help identify whether this is protocol overhead, port saturation, or SAN-side congestion behavior. Better to open a case to IBM because without all those details performance problems is not possible to solve.

Yes, in fact the same ports were shared between FC and NVMe host connections.

As i mentioned, i have since disabled this and re-done the port setup. I am also on kind of a "preview" firmware that i got through the support because i wanted to test the NVMe functionality.
The actual 9.1.2.1 FW that officially supports NVMe Host connections in a PBHA setup is still blocked from being installed because of some obscure SVAPAR.

We now have double redundancy with 2 ports per node, one per san switch (exactly as in the diagram of the OP). All IBM SAN ports are connected with 32GB SFP+ modules, the hosts themself have 16GB.

We have also identified 2 machines in our dev environment that were bursting heavy IO (up to 30000) and moved those back onto our "old" HPE Primera SAN that can cope with this kind of workload much better.

I can confirm that there is no IO on the external facing ports during a vMotion, it seems to be handled internally:


I will monitor the situation, but for now it seems okay.

Thank you for your input!