PowerHA for AIX

 View Only
Expand all | Collapse all

ASM on PowerHA

  • 1.  ASM on PowerHA

    Posted Mon September 20, 2021 03:43 AM
    Hello All,
    I haven't seen any new threads for a long time, but hope someone is still checking the group.
    I'm struggling to find confirmation that Oracle ASM is supported with PowerHA. Can someone point me to right direction?

    J.

    ------------------------------
    Jakub Pacowski
    ------------------------------


  • 2.  RE: ASM on PowerHA

    Posted Tue September 21, 2021 12:58 PM

    Hello Jakub,

    I do not fully understand what you are looking for. What kind of Oracle are you using? A single Oracle installation or Oracle RAC?
    PowerHA can manage Oracle. However, ASM is a black box to AIX. AIX can see the ASM disk but these disks can not be managed by AIX. And therefore it can not be managed by PowerHA.

    What do you like to do?

      Bernd 



    ------------------------------
    Bernhard Buehler
    ------------------------------



  • 3.  RE: ASM on PowerHA

    Posted Tue September 21, 2021 02:04 PM
    Hello Bernhard,
    my client had single node Oracle RAC on ASM - no comment on that, but they finally moved to 2-node PowerHA cluster on LVM. However LVM configuration seems to cause performance issue e.g. due to lack of balancing through physical volumes. I was wondering if it's possible to move to 2-node PowerHA cluster but on ASM. I'm aware it's black box for AIX, but still PowerHA can handle IP addressing, application controllers and monitoring. The question is if such configuration will work and what's more important if it's supported.
    J.

    ------------------------------
    Jakub Pacowski
    ------------------------------



  • 4.  RE: ASM on PowerHA

    Posted Wed September 22, 2021 02:12 AM
    ASM works fine, but ideally you should run ASM outside of PowerHA control, so ASM is active always on all nodes. This makes failing over database services much faster. This does however require licenses for that.

    You can also include ASM start/stop on the RG scripts as well, and start/stop everything under PoweHA control.

    ------------------------------
    Morten Torstensen
    TietoEVRY
    ------------------------------



  • 5.  RE: ASM on PowerHA

    Posted Wed September 22, 2021 02:15 AM
    LVM can also balance through striping, that is basically how ASM does it too. Application controllers and monitors, you just write yourself.

    ------------------------------
    Morten Torstensen
    TietoEVRY
    ------------------------------



  • 6.  RE: ASM on PowerHA

    Posted Wed September 22, 2021 04:02 AM
    Correct me if I'm wrong but ASM does IO load rebalancing so there is no hot spot on the disk group while LVM just strip LPs over all PVs. It could lead to situation where we have hot spots on physical volume controllerd by LVM.
    For active-active instance you would need paid RAC feature which is the reason why my client wanted to move to PowerHA and active-passive solution.

    In your opinion is ASM start/stop required in PowerHA or we can just focus on the application controller layers?

    What about the configuration when we have raw logical volume for ASM and not hdisk. I worked with such configuration with PowerHA for Informix but I'm not sure if it's possible for ASM and if it make sense. Volume group will be managed by PowerHA and ASM could handle the load rebalacing on the LV level.

    ------------------------------
    Jakub Pacowski
    ------------------------------



  • 7.  RE: ASM on PowerHA

    Posted Wed September 22, 2021 03:08 PM
    Jakub, it has been a VERY long time since I have worked with Oracle RAC on IBM Power. However, my strong impression from my dim recollection is that trying to mix two HA managers, Oracle RAC and PowerHA, in the same cluster is not recommended. IIRC, RAC handles moving the virtual IP (or re-points the Oracle client?) from one node to another (and yea, I'm pretty sure two nodes is a minimum) and monitoring/starting/restarting the app (Oracle). And ASM manages the redundancy of block storage in far greater detail than possible with the PowerHA management of AIX LVM. 
    On a hunch, is this someone's bright idea for avoiding payment for additional Oracle licenses??
    At any rate, my advice would be to reach out to Oracle SMEs with experience with RAC cluster configuration and management (and licensing ;-). And leave off trying to serve two masters ;-).

    ------------------------------
    Mackey Morgan
    ------------------------------



  • 8.  RE: ASM on PowerHA

    Posted Wed September 22, 2021 03:39 PM
    Mackey,
    maybe I wasn't clear enough. It's not about mixing RAC with PowerHA. It's about using ASM with PowerHA instead of JFS2 and LVM. I don't want to use both cluster managers. I'm just not sure if PowerHA with ASM (no RAC) is supported configuration.

    Btw. I've also got one client that had 4-node PowerHA cluster with 2x RAC clusters - one site 2-nodes RAC Cluster, second site  2-node RAC clusters all in 4x node PowerHA cluster. Idea was to use PowerHA for replication management between sites and local RAC cluster for high availability. What's more activation of resource group was set up with dynamic resources (DLPAR on CPU). Idea was to limit Oracle licenses required, but I'm not sure if savings were worth making the configuration much more complex.

    ------------------------------
    Jakub Pacowski
    ------------------------------



  • 9.  RE: ASM on PowerHA

    Posted Thu September 23, 2021 03:05 AM

    Hello Jakub,
    I have seen both types of implementations (Single system Oracle DB with LVM, ASM and PowerHA). So I can say its possible.
    However, if you need a official support statement you must get it from IBM.

    As always, both types of installations have advantages and disadvantages. 
    I personally would vote for LVM. The LVM setup can be tuned to get better performance. Some of them got mentioned already. I'm not a tuning expert for this. Again, you need to contact IBM or a IBM business partner for this performance tuning.    



    ------------------------------
    Bernhard Buehler
    ------------------------------



  • 10.  RE: ASM on PowerHA

    Posted Mon September 27, 2021 10:17 AM
    For whatever my 2 cents are worth, I always like a solution to be as simple as possible and no simpler. While LVM may indeed be tunable, to me that implies an iterative and time-consuming process. ASM, OTOH, is self-tuning IIRC. Gotta like that! And if you stick with the Oracle stack, there's the "one throat to choke" benefit, as well. If I could do this with just the AIX base OS and Oracle SW, that's what I would shoot for. Mixing PowerHA and Oracle, both of which expect to be in control seems trying to "serve two masters".  Cleave to the one and despise the other. ;-)
    --Mackey Morgan

    ------------------------------
    Mackey Morgan
    ------------------------------



  • 11.  RE: ASM on PowerHA

    Posted Thu September 23, 2021 05:02 AM

    If you run ASM, it needs access to raw devices. You can provide either PVs or LVs for that purpose. Now, if you ignore licensing, a fully active/active cluster for ASM is the best approach. ASM is always active and controlling disks and you can manage data distribution with whatever tools ASM provides. PowerHA doesn't have to explicitly support ASM in any way, so I am not sure exactly what you need support for?

    Second option is to use ASM on PVs. AIX, LVM or PowerHA will in that case not have anything to do with the disks. PowerHA will only control start and stop of ASM on the node as required. There will be no disks in the RG. This is a good way too, to manage ASM, and you let ASM take full responsibility for data and distribution. We have a policy of renaming all raw device PVs in AIX, so that operators can easily identify them (we use HDS storage, and we rename them to asm_<LDEV>_<SERIAL> so we can easily identify the disks and where they come from, and since this info is in the ODM it is easy to script it). Use native storage methods to mirror/distribute data, or ASM to do the same as far as it supports it.

    Last option is to use raw LVs and let PowerHA manage the VGs as part of the RG. It works well enough, but you need to map each LV to one PV to get the expected performance based on ASM policy of treating a volume as one IO and availability domain. You gain the possibility to manage storage through AIX LVM, but generally I would not recommend it as it is more operationally demanding. Can be mitigated through automated monitoring of LV mapping and sync status. You also need to take care of how to switch. Concurrent VG, and control where it is read/write, risk of split brain and cluster problems due to infrastructure outages or errors, problems coordinating between DBA and OS operators, depending how you are organised...

    PowerHA as such isn't too picky, unless you want to use PowerHA and Hyperswap or other specific PowerHA to storage integrations. You can easily add DLPAR, Storage Management commands and automation and other features, but I would recommend keeping it simple.



    ------------------------------
    Morten Torstensen
    TietoEVRY
    ------------------------------



  • 12.  RE: ASM on PowerHA

    Posted Thu September 23, 2021 05:29 AM
    Morten, thank you for detailed description.
    Fully agree with KISS principle.
    Off topic: I remember that when rendev command appeared for the first time, someone suggested to mess with other admins/users by changing boot device to something like fan, power supply or other component (don't remember right now what it was). Bootlist gave amazing output :D :D :D

    ------------------------------
    Jakub Pacowski
    ------------------------------



  • 13.  RE: ASM on PowerHA

    Posted Thu September 23, 2021 11:16 AM
    Jakub,
    Way back in the day when I worked with Oracle and ASM, I remember that there was nothing to indicate to the AIX SA which volumes were under ASM control. As such, I saw that situation as a "ticking time bomb". Since that time, I believe lspv or lslv or lsvg may have gotten some new features to better manage those. But at the time, I took a "roll your own" approach and wrote some shell scripts to view my disk volume status. I'll include those scripts here, in case you'd be interested in seeing what additional features they provide. It has been a VERY LONG time since then, so I probably wouldn't even recognize my own code!


    --Mackey Morgan
    UNIX Engineering Services  --  Kaiser Permanente
    Work: (940) 239-9794        |         E-mail: Mackey.Morgan@kyndryl.com
    WebEx: https://kyndryl.webex.com/meet/mackey.morgan





  • 14.  RE: ASM on PowerHA

    Posted Wed September 29, 2021 08:34 AM

    Jakub Pacowski,
    Off topic: do you mean something like this:
    # rendev -l hdisk0 -n hdiskindness0
    # bootlist -m service hdiskindness0
    # bootlist -m service -o
    hdiskindness0 blv=hd5 pathid=0​​



    ------------------------------
    Edward Davignon
    ------------------------------