Primary Storage

 View Only
Expand all | Collapse all

SVC: Confused with FC Port recommendations

  • 1.  SVC: Confused with FC Port recommendations

    Posted Tue March 19, 2024 07:49 AM
    Edited by Sergio Pardos Tue March 19, 2024 09:34 AM

    Hi everyone

    According to this document: Implementation Guide for IBM Spectrum Virtualize Version 8.5

    this would be the recommended distribution for 12-port nodes:

    I cannot wrap my mind around the fact they are advising to place the two only inter-node ports in the same FC card. From where I see it, if the card #1 fails, the intracluster is gone for that node. Wouldn't it be advisable to place both inter-cluster ports in different cards to avoid SPOFs? 

    Thanks in advance.



    ------------------------------
    Sergio Pardos
    ------------------------------



  • 2.  RE: SVC: Confused with FC Port recommendations

    IBM Champion
    Posted Tue March 19, 2024 01:58 PM

    Hi Sergio,

    Inter-node communication, is used for heartbeat and metadata exchange between all nodes of all I/O groups in the cluster. Are you asking for SVC or FlashSystem?



    ------------------------------
    Nezih Boyacioglu
    ------------------------------



  • 3.  RE: SVC: Confused with FC Port recommendations

    Posted Tue March 19, 2024 04:01 PM

    Hi Nezih, thanks for your reply, I am asking for SVC, the ports used for inter-node communication.



    ------------------------------
    Sergio Pardos
    ------------------------------



  • 4.  RE: SVC: Confused with FC Port recommendations

    Posted Wed March 20, 2024 11:55 AM

    Hi, in my opinion this is just a recommendation that we all do it the same way. I myself have had to deviate from this because there are various environments where it is not possible to implement it in this way. If you search the IBM Docs you will find a slightly different overview where ports 3 and 4 of card 3 are also inter-node ports. Maybe this is just a typo in the PDF.
    https://www.ibm.com/docs/en/sanvolumecontroller/8.4.x?topic=cd-fibre-channel-port-masking



    ------------------------------
    Ralph Glanz
    ------------------------------



  • 5.  RE: SVC: Confused with FC Port recommendations
    Best Answer

    Posted Wed March 20, 2024 03:22 AM

    Sergio,

    Documentation is distributing the intracluster ports over two adapters: https://www.ibm.com/docs/en/sanvolumecontroller/8.6.x?topic=pc-planning-more-than-four-fabric-ports-per-node



    ------------------------------
    TMasteen
    ------------------------------



  • 6.  RE: SVC: Confused with FC Port recommendations

    Posted Wed March 20, 2024 05:19 AM

    Hi, thanks for the input... that's conflicting information, the one you provided make a lot more sense, so I'll use that.



    ------------------------------
    Sergio Pardos
    ------------------------------



  • 7.  RE: SVC: Confused with FC Port recommendations

    IBM Champion
    Posted Wed March 20, 2024 05:29 AM

    This was why am I asking :) 

    We will update this Redbook for Spectrum Virtualize 8.7 and release it in July. It will be better to separate this table as FlashSystem and SVC and publish it as two tables. 



    ------------------------------
    Nezih Boyacioglu
    ------------------------------



  • 8.  RE: SVC: Confused with FC Port recommendations

    Posted Wed March 20, 2024 07:45 AM

    Thanks Nezih.



    ------------------------------
    Sergio Pardos
    ------------------------------



  • 9.  RE: SVC: Confused with FC Port recommendations

    Posted Thu March 21, 2024 03:58 AM

    Hi Sergio, 

    Good point. I vaguely remember I worked on some kind of similar topic for another book, and finally I found it - hope this information can be helpful:

    Please check: Performance and Best Practices Guide for IBM Storage FlashSystem and IBM SAN Volume Controller Updated for IBM Storage Virtualize Version 8.6

    https://www.redbooks.ibm.com/redpieces/pdfs/sg248543.pdf 

    Chapter 2.3.5 - search for figures 2-26, 2-27 Port masking configuration on SVC or IBM FlashSystem

    Hope this will be helpful



    ------------------------------
    Denis Olshanskiy
    ------------------------------



  • 10.  RE: SVC: Confused with FC Port recommendations

    Posted Fri March 22, 2024 05:53 AM

    Hello Sergio, 

    these recommendations are also based on a mapping of FC-Ports to CPU-Cores. And I think it has also in mind that maybe some customers start with one HBA and add one or two HBA's later. So on the first 4 Ports you have all necessary connections. I would recommend to check the mapping of FC-Ports to CPU-Cores, because here it may be important to aviod bottlenecks and in the next step to spread the usage of the connections on at least 2 HBA's.



    ------------------------------
    Dorde Knezevic
    ------------------------------



  • 11.  RE: SVC: Confused with FC Port recommendations

    Posted Fri March 22, 2024 06:12 AM

    Hi Dorde, 

    On the latest code level and hw there is no  need to take into account the correlation between FC ports and CPU-Cores and it is not a bottleneck anymore - as per my latest knowledge.



    ------------------------------
    Denis Olshanskiy
    ------------------------------



  • 12.  RE: SVC: Confused with FC Port recommendations

    Posted Fri March 22, 2024 06:17 AM

    Hi Dorde

    Yours is a valid point, I remember reading somewhere the correlation between HBAs and CPU cores. Still, and I might be wrong, if IBM recommend a given port distribution, I must assume that has already been taken into account in such recommendation.



    ------------------------------
    Sergio Pardos
    ------------------------------