Power

 View Only
  • 1.  Virtual I/O Server - Best Practices

    Posted Thu March 30, 2023 11:00 PM

    Hi Team,
    I am looking for best-practices document for deploying a Virtual I/O server for my business partner. Their question

    "I need to decide where to boot the VIOS from – internal storage or SAN storage" - The Power10 systems sacrifice PCIe slot(s) for the NVMe breakout to use the front panel storage slots.  This makes internal storage 'expensive', as you lose the opportunity to use those PCIe slots.

    Just want to know if there is a best practice that everyone else is using.




    ------------------------------
    Suleman Malik
    ------------------------------


  • 2.  RE: Virtual I/O Server - Best Practices

    IBM Champion
    Posted Fri March 31, 2023 03:46 AM
    Edited by Satid Singkorapoom Fri March 31, 2023 03:48 AM

    Dear Suleman

    There is a redbook for this: IBM PowerVM Best Practices at https://redbooks.ibm.com/abstracts/sg248062.html. But its chapter 5 on Storage topic does not cover the aspect you asked about.  


    To answer your question and to know if what you said is "expensive" would be the case or not, we need to know more about the HW and logical environment and expectation you are dealing with. What more can you tell us on this?


    I would say the answer to your question lies in practical aspect of the matter.  For example, I used to provide support to a customer who wanted to migrate 60 small IBM i and AIX LPARs (serving many branches across the globe) with dual VIOS from two POWER7 servers to two POWER9 servers as well as to migrate from DS8K SAN disk to Storwize and they wanted minimum downtime for the entire migration. We saw that using offline partition mobility (as opposed to backup and restore approach) is a solution to match the customer's requirement and this clearly needed SAN. Internal disk was out of the question. So, we also decided VIOS would boot from SAN.



    ------------------------------
    Education is not the learning of facts but the training of the mind to think. -- Albert Einstein.
    ------------------------------
    Satid S.
    ------------------------------



  • 3.  RE: Virtual I/O Server - Best Practices

    Posted Fri March 31, 2023 07:02 AM
    Edited by Tom McGivern Fri March 31, 2023 07:10 AM

    Satid, VIOS does not participate in LPM, so having VIOS boot from SAN or internal  didn't aid nor hinder your  migration.

    Nigel, I agree with you except his question was specific to booting the VIOS.  KISS in that case points to internal (explained below)

    Suleman,
     In my experience, self contained VIOS (boot internal), avoids any issues with missing/aged device drivers.  Depending on the storage manufacturer, you may need ODM tweaks, and new device drivers. We used Hitachi, they needed these things.  You can't install directly to something you need third-party stuff to support.
    while you may be ok today, what about after their next SAN refresh when they change vendors?

    if the VIO boots from internal disk, then you have the hardware up and running, and you can at least always log into an OS to diagnose / support the system.

    Yes, it's more expensive, but they're paying for the Reliability, Availability, Serviceability of IBM/power for a reason!  Don't jeopardize all that to save a few bucks.

    Tom



    ------------------------------
    Tom McGivern
    ------------------------------



  • 4.  RE: Virtual I/O Server - Best Practices

    IBM Champion
    Posted Fri March 31, 2023 07:34 AM
    Edited by Satid Singkorapoom Fri March 31, 2023 07:52 AM

    Dear Tom

    >>>>  VIOS does not participate in LPM, so having VIOS boot from SAN or internal  didn't aid nor hinder your  migration. <<<<

    I forgot to mention that the reason for a decision for VIOS boot from SAN in the case I mentioned was that it used not much of disk space from SAN. The BP said it would not need to adjust the proposed config of Storwize because of this fact. The customer saw no need to buy additional internal disk just for this reason and therefore they saw they saved their investment.  So, the reason was not technical but a financial one.

    As for the reason of reliability or availability, my experience taught me that we need to look at this as part of a WHOLISTIC picture of the matter so that its value is justified.  A colleague of mine used to share with me a case when VIOS booted from internal disk but client LPARs used NPIV to connect to SAN.  But then one day, someone messed up with the SAN somewhere and caused the LPARs to stop working.  To the customer's perspective, the fact that additional investment for VIOS's boot from internal disk for an independent reliability or availability did not provide value in this particularly unfortunate case. When the customer regards  availability, they care more about availability of their business workload LPAR rather than VIOS. 



    ------------------------------
    Education is not the learning of facts but the training of the mind to think. -- Albert Einstein.
    ------------------------------
    Satid S.
    ------------------------------



  • 5.  RE: Virtual I/O Server - Best Practices

    Posted Fri March 31, 2023 04:25 PM

    "But then one day, someone messed up with the SAN somewhere and caused the LPARs to stop working.  To the customer's perspective, the fact that additional investment for VIOS's boot from internal disk for an independent reliability or availability did not provide value in this particularly unfortunate case"

    Someone needed to point out to the customer... "Yes, you've had an outage, this is unfortunate, but, if the VIOS was booting from SAN as well, the hardware would be a brick... nothing would boot.

    Can you imagine trying to diagnose what the issue was when the SAN is suddenly down "bad zoning change for example", and all you have is SMS to try and diagnose?  They'd have really had a lengthy outage.   

    When the SAN team dorks the SAN config such that client LPARs wouldn't boot.  More than likely the other VGs that the Client Lpars need to run also wouldn't have been available.   So, I don't accept that as a valid reason to boot from SAN.

    But, we all live with multiple variables in our lives... so, you gotta do what ultimately makes the best business sense.  I just don't see many combinations of variables that lead to VIOS on SAN being the best option.



    ------------------------------
    Tom McGivern
    ------------------------------



  • 6.  RE: Virtual I/O Server - Best Practices

    Posted Fri March 31, 2023 04:52 AM

    There is lots of VIOS information in the IBM Redbooks. - and a new one out this months but it is an intro level.

    Best Practice is go to https://www.ibm.com/support/pages/power-vug-technical-webinar-series

    and look for session 106 by Jackie Lynch - Power guru.

    As for the internal or SAN boot question, there is no one right answer as both work very well.

    It really is up to you.

    If you have an expert SAN team doing a good job running the storage, then I would go for SAN boot 

    The reason:

    • SAN is fast.
    • Why implement two disk storage mechanisms when one will do the whole job
    • Allows Live Partition Mobilty

    If your servers are small and your needs are few the Keep It Simple Silly (KISS) go for internal boot.
    Reasons:

    • If the SAN fails you can get the VIOS up to investigate
    • NVME is really fast and simple to implement
    • You can host small LPARs on the NVMe disks using vSCSI



    ------------------------------
    Nigel Griffiths
    ------------------------------



  • 7.  RE: Virtual I/O Server - Best Practices

    IBM Champion
    Posted Fri March 31, 2023 08:05 AM

    I'd like to offer my perspective on this.  I have had this situation happen to me MULTIPLE times.
    VIOS on internal disk.  Mirrored pair of drives.  Lost one drive.  Called IBM.  They ask you what OS is supported by VIOS.  This is a huge red flag that you have a long day ahead of you.  I tell them IBM i.  They send a technician out with the new drive.  You log the technician into VIOS.  The first thing the technician does is try to type in the IBM i command STRSST on the VIOS prompt.
    Every. Single. Time.
    With a SAN I would hope not to have this problem.
    However our BP convinced management that having four internal NVMe drives (two mirrored pairs for each pair of VIOS) was the simpler way to go.

    I do get what they are saying about drivers in VIOS though, having some experience with Inventory Scout (invscout) and Microcode Discover Service (MDS).

    Just to avoid confusion we did not host internal drives to IBM i or AIX from VIOS.  Internal drives for use by IBM i and/or AIX were hosted by an lpar of IBM i.



    ------------------------------
    Robert Berendt IBMChampion
    ------------------------------



  • 8.  RE: Virtual I/O Server - Best Practices

    Posted Fri March 31, 2023 09:44 AM
    Edited by Sylvain Fri March 31, 2023 09:45 AM

    We experienced multiple bugs with boot on SAN VIOS when migrating to VIOS 3.1.x with viosupgrade tool.

    viosupgrade tool often fails with SAN drives. 

    To resolve these problems we finally used internal drives for the migration.




  • 9.  RE: Virtual I/O Server - Best Practices

    IBM Champion
    Posted Fri March 31, 2023 11:21 AM

    Wherever possible I prefer to boot my vio servers from internal disk - this allows me to debug SAN problems and gets around many issues with MPIO drivers that can occur during updates. Additionally, it makes it easier to put on I/O updates to firmware for the SAN cards. I also make extensive use of FBO (file backed optical) so I always include 2 disks (if internal) or 1 disk (if on san) for one of the two vio servers to host the FBO files.   I do not put them in rootvg as it leads to huge mksysb files which means backups take longer and restores even longer. The disk/s for FBO can be internal or on the SAN.

    Regardless of whether the VIO is on the san or internal I always provide 2 disks for rootvg.  One is the boot image and the second is for cloning to when doing maintenance.  If vio is on the san then the disks are not mirrored.  If they are internal they run mirrored most of the time and I break the mirror so I can clone for maintenance.

    So my normal configuration (when on internal disk)
    VIO1
      2 internal disks for rootvg
    VIO2
      2 internal disks for rootvg
      2 internal or 1 SAN disk for FBO

    If on the SAN
    VIO1
      2 internal disks for rootvg (no need to mirror though just use the second one for cloning)
    VIO2
      2 internal disks for rootvg (no need to mirror though just use the second one for cloning)
      1 SAN disk for FBO

    I have had times that due to the number of VIO servers I have had to run SAN disks for the VIO - they work just fine but you can run into the issues I mentioned if there are issues with the SAN or when you are doing updates for I/O firmware.

    Jaqui



    ------------------------------
    Jaqui Lynch
    ------------------------------