PowerVM

 View Only
Expand all | Collapse all

Vios I/O Workload Seapration

  • 1.  Vios I/O Workload Seapration

    Posted Wed March 10, 2021 12:43 PM
    I would be interested to know your views on separating out IBM i and AIX i/o workloads over multiple vios servers (dedicated to each o/s )  vrs HBA segregation of the i/o traffic on a dual vios server configuration ?

    Rgds

    Mick

    ------------------------------
    Mick Sharp
    ------------------------------


  • 2.  RE: Vios I/O Workload Seapration

    IBM Champion
    Posted Thu March 11, 2021 11:10 AM

     

     

    Hi Mick,

     

    We have this setup in our environment.  We have IBMi and AIX LPARs running on the same physical managed server.  We have a dual VIO server setup for IBMi and a dual VIO server setup for AIX.  Each VIO server has a dedicated dual port Fiber Channel adapter assigned to it along with a dual port Network adapter.  When we initially setup these VIO servers there was a setting that we were not able to make on the IBMi VIO that we were able to make on the AIX VIO.  But over time that limitation was corrected.  I think it is a good idea to separate them out like this to ensure that your AIX workload does not affect your IBMi workload.  Our IBMi workload does not have as much flexibility as our AIX workload in terms of expectations regarding performance.  Hope this helps.

     

    Regards,

     

    Norman Owens






  • 3.  RE: Vios I/O Workload Seapration

    Posted Fri March 12, 2021 06:44 AM

    Hi Norman , thanks for your response that's interesting.

    We also have a managed server running both IBMi and AIX LPARS , but using a single dual VIO server setup with HBA port separation on each.This worked fine until we recently upgraded our managed server and migrated our LPARS over, but also adding an IBMi LPAR with a large 'Read' profile to the SAN . Subsequently both IBMi and AIX I/O response's suffered and we had to move the new IBMi 'read' workload off onto another managed server / SAN to resolve it.

     Experts involved with resolving the issue were surprised we had this configuration, and have advised at least HBA card separation , if not separate VIOS setups for both.

    Are there any definitive rules for new VIO configurations relating to I/O workload best practise , particularly with IBMi & AIX ?     

     

    Rgds   



    ------------------------------
    Mick Sharp
    ------------------------------



  • 4.  RE: Vios I/O Workload Seapration

    IBM Champion
    Posted Mon March 15, 2021 11:08 AM

    Hi Mick,

    I would answer your question by saying that your VIO configuration will depend on your workload.  We separated our VIO servers by AIX and IBMi workload in the beginning but the configuration of the VIO servers is essentially the same in my environment.  I recently had to do some tuning for the AIX VIO servers because of increased workload on the AIX VIO clients.  I will say that we have 1 IBMi LPAR that is a heavy hitter so we decided to set it up with its own dedicated adapters and it has been running for 6 years very well.  That is the only one that has dedicated adapters, we have about 5 other IBMi LPARs that are VIO clients and they have been running fine.  If you want to look at tuning your VIO servers then I would highly recommend checking out the webinar by  Jaqui Lynch.  She has an excellent presentation called "Care and Feeding your VIO servers".  She is also an IBM Champion.  Check out the webinar on the IBM Virtual Power User group at this link http://ibm.biz/powersystemsvug  You can send an email to Joe Armstrong to be added to the monthly webinars (jdarmstr@us.ibm.com).  Let me know if you have any other questions.

    Regards,
    -Norman



    ------------------------------
    Norman Owens
    Open Systems Architect
    Mohawk Industries
    ------------------------------



  • 5.  RE: Vios I/O Workload Seapration

    Posted Mon March 15, 2021 12:55 PM
    Thanks for the posts Norman , appreciate you taking the time to respond , will check out those recommended links. 

    Rgds

    ------------------------------
    Mick Sharp
    ------------------------------



  • 6.  RE: Vios I/O Workload Seapration

    IBM Champion
    Posted Tue March 16, 2021 02:28 PM

    Thanks Mick.  Very helpful info.  We are definitely using 4 paths.  I'll let you know if I have any further questions.  I'm on Slack too.

     

    Thanks!

     






  • 7.  RE: Vios I/O Workload Seapration

    Posted Mon March 15, 2021 03:33 PM
    Dear Mr. Sharp

    As an IBM i performance consultant, I see that your problem description appears to me as a disk IO throughput limit issue of the SAN disk rather than VIOS issue. But I need more information before I can be sure.    

    You said you "added an IBMi LPAR with a large 'Read' profile to the SAN . Subsequently both IBMi and AIX I/O response's suffered and we had to move the new IBMi 'read' workload off onto another managed server / SAN to resolve it."   In my view, the key information is that all LPARs suffered from (presumably) degraded SAN disk response time .  

    When this performance issue happened, did you produce a SAN disk performance report (showing disk response time of the disk pool in SAN VS disk IOPS VS disk MB/sec. data rate) and review the disk response time of that SAN box during the period of the performance issue to see whether or not its disk response time degraded for the entire disk pool shared by all those IBM i and AIX LPARs?  If so, this issue has nothing to do with having the dual VIOS for all those mixed-OS LPARs but has to do with the SAN disk not being configured to handle the peak total disk IO workload (be it IOPS or MB/sec. or both).  If this is the case, you need to enhance the HW configuration of that SAN disk box for a better handling of higher disk workload throughput.   This is a case that I encountered a few times in the past few years with my IBM i customers in ASEAN geography who used SAN disk for many LPARs in a Power server.

    Just my 2 cents.

    Satid S.
    IBM Thailand


    ------------------------------
    SATID SINGKORAPOOM
    ------------------------------



  • 8.  RE: Vios I/O Workload Seapration

    Posted Tue March 16, 2021 05:56 AM
    Morning Satid, thanks for your post , i'd take 2 cents of your advise any day of the week !
    indeed it was a SAN issue that was the problem, resolved with SAN configuration and architectural changes. My reasoning behind opening up a discussion around the VIO workload separation piece is because during the course of troubleshooting our performance issues , our VIO configuration was examined and the consensus now seems to be separate IBMi and AIX workloads onto separate VIO servers. I'm interested to know the specific reasoning behind this as most if not all the documents for PowerVM show mixed client workloads using single or dual VIO configurations, and actually as you pointed out our mixed dual VIO configuration is performant anyway.
    Rgds

    ------------------------------
    Mick Sharp
    ------------------------------



  • 9.  RE: Vios I/O Workload Seapration

    Posted Tue March 16, 2021 06:28 AM
    Dear Mr. Sharp

    I now see the point in your clarification. But I see that having more than the dual VIOS can increase your burden when you need to apply PTFs to or upgrade the release of VIOS.  I see that this may be a once-in-while burden but this may not be the case in some customers.  I also see that just separating fiber cards between low and high-disk-IO-workload LPARs  while maintaining just the dual VIOS should be sufficient to address the issue if the issue lies in VIOS (and reduce the burden I mentioned).  

    I notice that when customers with VIOS have performance issue in their IBM i and AIX and Linux LPARs running on VIOS (dual or not), many forget that IBM provides a tool that helps identify and address the resource issue in VIOS that can be the cause of performance issue in client LPARs. The tool is called VIOS Performance Advisor and I encourage you use it in the future.  Pls do a Google search on VIOS Performance Advisor to learn more about this useful tool.

    Thanks.

    Satid S.
    IBM Thailand

    ------------------------------
    SATID SINGKORAPOOM
    ------------------------------



  • 10.  RE: Vios I/O Workload Seapration

    IBM Champion
    Posted Tue March 16, 2021 09:34 AM

    Mick, I'm curious to know what configuration changes you made on the SAN to correct the problem.  I'm having similar issues that you described on an AIX LPAR.

     

    Satid, are you on Slack?  I'd like to connect and talk more about the VIO performance advisor.

     

    Thanks,

     

     






  • 11.  RE: Vios I/O Workload Seapration

    Posted Tue March 16, 2021 09:39 PM
      |   view attached
    Dear Mr. Owen

    Slack may not be useful in our case because our time zones are much different.  As for more information on VIOS Advisor, please refer to page 42-54 on the presentation file I attached herewith.  I also see training  videos on VIOS Advisor on Youtube if you would care to refer to it also :
    https://www.youtube.com/watch?v=5uaz_fK6Pvc&t=5s
    https://www.youtube.com/watch?v=DVMWVWgmRhI

    Hope this help.

    Thanks.

    ------------------------------
    SATID SINGKORAPOOM
    ------------------------------

    Attachment(s)



  • 12.  RE: Vios I/O Workload Seapration

    IBM Champion
    Posted Thu March 18, 2021 08:06 AM
    I agree, VIOS Advisor can give a lot of interesting information

    ------------------------------
    Fernando Plaza
    ------------------------------