IBM i Global

IBM i Global

Connect, learn, share, and engage with IBM Power.

 View Only
Expand all | Collapse all

Suggestions on keeping the *userdata synchronized with a PowerHA cluster

  • 1.  Suggestions on keeping the *userdata synchronized with a PowerHA cluster

    Posted Wed October 04, 2023 03:51 PM

    We have 4 IBM i LPARS in a PowerHA cluster and use iASP synchronization which works great.   I wanted to keep the *ALLUSR synchronized so when I have failover the supporting libraries are current.

    On the primary server, I have a BRMS control group that runs daily as below:

                         Edit Backup Control Group Entries               PRDPRI     
                                                                                    
     Group . . . . . . . . . . : DALLUSR                                            
     Default activity  . . . . . *BKUPCY                                            
     Text  . . . . . . . . . . . Daily SYSBAS all user                              
                                                                                    
     Type information, press Enter.                                                 
                                                                                    
                                      Weekly   Retain Save     SWA                  
          Backup     List ASP         Activity Object While    Message    Sync      
     Seq  Items      Type Device      SMTWTFS  Detail Active   Queue      ID        
                                                                                    
       10 *EXIT                       *******                                       
       20 *SAVSECDTA                  FFFFFFF  *YES                                 
       30 *SAVCFG                     FFFFFFF  *YES                                 
       40 *ALLUSR         *SYSBAS     FFFFFFF  *YES   *YES     *LIB       *NONE     
       50 *ALLDLO                     FFFFFFF  *YES   *NO                           
       60 ALLUSRLNK  *LNK *SYSBAS     FFFFFFF  *YES   *ALWCKWR *LIB       *NONE     
       70 *EXIT                       *******                                       
       80 *EXIT                       *******                                       
                                                                             Bottom 
     F3=Exit    F5=Refresh    F10=Change item    F11=Display exits                  
     F12=Cancel    F14=Display client omit status    F24=More keys                  

    On the target LPARS for the group, i have a job which runs daily after the save with the following:

                RSTLIBBRM  SAVLIB(*RSTLST) DEV(TAPMLB01) PRLRSC(10) MBROPT(*ALL) ALWOBJDIF(*ALL) +
                 FROMSYS(&FROMSYS) RSTLST(ALLUSER)                                    

    This restored originally worked fine, but now I am getting the error:

    BRM1676    Diagnostic              40   10/02/23  11:01:01.539177  Q1ACRL       QBRM        *STMT    SYNCSYSBAS  DAPQGPL   
                                         From module . . . . . . . . :   Q1ACRL                                                
                                         From procedure  . . . . . . :   Q1ACRL                                                
                                         Statement . . . . . . . . . :   92500                                                 
                                         To module . . . . . . . . . :   SYNCSYSBAS                                            
                                         To procedure  . . . . . . . :   SYNCSYSBAS                                            
                                         Statement . . . . . . . . . :   2000                                                  
                                         Message . . . . :   Save level *CURRENT of library LIBROUTER not restored.            
                                         Cause . . . . . :   Save level *CURRENT of library LIBROUTER could not be             
                                           restored because the specified save level does not exist.                           

                                                                  

    IBM Support says the only way to correct the issue is to perform a CLRLIB on the libraries?  As there are about 75 libraries or more at any given time, this isn't a consideration when trying to automate the process.

    How do other companies using PowerHA keep Sysbas user libraries sync'd so when failing over, you don't have to wait for restores to occur first.



    ------------------------------
    Michael Garczynski
    Director - SAP Architecture
    DAP Products
    ------------------------------


  • 2.  RE: Suggestions on keeping the *userdata synchronized with a PowerHA cluster

    Posted Wed October 04, 2023 04:41 PM

    What do you get if you do the following on the system you're trying to restore to?

    WRKOBJBRM OBJ(LIBROUTER/myfile) OBJTYPE(*FILE) FROMSYS(otherlpar)

    Do you see a current save?  Is the media accessible from this lpar?



    ------------------------------
    Robert Berendt IBMChampion
    ------------------------------



  • 3.  RE: Suggestions on keeping the *userdata synchronized with a PowerHA cluster

    Posted Wed October 04, 2023 10:04 PM
    Edited by Satid Singkorapoom Wed October 04, 2023 10:05 PM

    Dear Michael

    >>>>> How do other companies using PowerHA keep Sysbas user libraries sync'd so when failing over, you don't have to wait for restores to occur first. <<<<<

    By *ALLUSR, if you mean IBM i objects stored in QGPL and QUSRSYS libraries, there is a function in PowerHA for i named "Cluster Administrative Domain"  that performs object replication for system objects stored in SYSBASE.    More information here:

    https://theibmi.org/2017/08/23/how-to-implement-administrative-domains-on-ibmi-powerha/

    https://www.youtube.com/watch?v=U19NiDzQDHM



    ------------------------------
    Chance favors only the prepared mind.
    -- Louis Pasteur
    ------------------------------
    Satid S.
    ------------------------------



  • 4.  RE: Suggestions on keeping the *userdata synchronized with a PowerHA cluster

    Posted Thu October 05, 2023 12:07 PM

    As usual, Satid Singkorapoom is the first with the right answer.

    Definitely look into setting up the Administrative Domain in PowerHA to maintain synchronization between the *SYSBAS objects that need to be kept in sync.

    I do not know of any PowerHA customers that are using regular save/restore of libraries to maintain synchronization of *SYSBAS objects.  That would concern me as a potential problem for maintaining readiness to do an unplanned role swap.

    Usually there are a fairly small number of *SYSBAS objects that need to be maintained in sync between cluster nodes - user profiles being the highest on that list.

    If you have complete libraries in SYSBAS that need to be refreshed, I would ask why aren't those in the IASP?



    ------------------------------
    Vincent Greene
    IT Consultant
    Technology Expert labs
    IBM
    Vincent.Greene@ibm.com


    The postings on this site are my own and don't necessarily represent IBM's positions, strategies or opinions.
    ------------------------------



  • 5.  RE: Suggestions on keeping the *userdata synchronized with a PowerHA cluster

    Posted Fri October 06, 2023 08:55 AM

    We already use the option suggested by Satid Singkorapoom for keeping specific objects synchronized across the cluster.   These are 3rd party libraries which cannot run in the iASP and remain in sysbas.  

    As IBM Support is telling me BRMS is not meant as a tool for replication, I find it difficult to understand why there isn't a way to perform this function without clearing the libraries on the target server.   This would create a significant increase in the time to complete the task.

    From my perspective, it appears there is a hole in BRMS trying to keep the PowerHA clusters using iASP sync'd to provide the quickest failover solution over the full system cluster.  I didn't know the limitations of BRMS when PowerHA was not an option and using the ALWOBJDIF.

    Are there any other products available which can perform this function?



    ------------------------------
    Michael Garczynski
    Director - SAP Architecture
    DAP Products
    ------------------------------



  • 6.  RE: Suggestions on keeping the *userdata synchronized with a PowerHA cluster

    Posted Sat October 07, 2023 09:08 AM

    Dear Michael

    >>>>> These are 3rd party libraries which cannot run in the iASP and remain in sysbas.  <<<<<

    Do these libraries contain typical application objects (PF, LF, PGM, CMD, etc.) that are not supported by Admin Domain?   As you encounter difficulty trying  to utilize BRMS for your purpose, I would suggest you use data replication solution instead such as MIMIX, iCluster, MAXAVA, etc.  You can choose one that is not expensive and use it to replace Admin Domain.



    ------------------------------
    Chance favors only the prepared mind.
    -- Louis Pasteur
    ------------------------------
    Satid S.
    ------------------------------



  • 7.  RE: Suggestions on keeping the *userdata synchronized with a PowerHA cluster

    Posted Sun October 08, 2023 06:50 AM

    Satid

    The libraries contain the standard objects but as I said, the 3rd party doesn't support them on the iASP.  

    Mimix is not a consideration as we bounced them awhile back for lack of support issues which is why we chose PowerHA as well as it was an IBM product.  Unfortunately, when we were evaluating PowerHA, we touched on keeping the sysbas sync'd and were led to believe there would not be any issues. I have the option Full System option, but for these servers deemed hot swap, we chose the solution which would provide the quickest failover without needing a full IPL.  

    As we previously functioned without the cluster, we always assumed on a restore ALWOBJDIF would ignore any conflicts and restore the objects.  It is a huge revelation and surprise that the parameter had exceptions.  

    Thanks for your suggestions. 



    ------------------------------
    Michael Garczynski
    Director - SAP Architecture
    DAP Products
    ------------------------------



  • 8.  RE: Suggestions on keeping the *userdata synchronized with a PowerHA cluster

    Posted Sun October 08, 2023 10:24 PM

    Dear Michael

    >>>> As we previously functioned without the cluster, we always assumed on a restore ALWOBJDIF would ignore any conflicts and restore the objects.  It is a huge revelation and surprise that the parameter had exceptions.   <<<<

    In your situation, there is another choice which is to use straightforward SAVLIB and RSTLIB because ALWOBJDIF (*ALL) will work outside of BRMS context.  



    ------------------------------
    Chance favors only the prepared mind.
    -- Louis Pasteur
    ------------------------------
    Satid S.
    ------------------------------



  • 9.  RE: Suggestions on keeping the *userdata synchronized with a PowerHA cluster

    Posted Wed October 11, 2023 07:47 AM

    Satid,

    If I understand correctly, the ALWOBJDIF functions differently in BRMS and that is what is causing the issue?  The challenge to using SAVLIB/RSTLIB is we use a tape library with multiple LPARs attached and the media associated with a each save is easily identified in BRMS.  There isn't an easy process to manage tapes in the library outside of BRMS that I'm aware of.  

    Thanks for your suggestions



    ------------------------------
    Michael Garczynski
    Director - SAP Architecture
    DAP Products
    ------------------------------



  • 10.  RE: Suggestions on keeping the *userdata synchronized with a PowerHA cluster

    Posted Wed October 11, 2023 08:14 AM

    Dear Michael

    >>>>> If I understand correctly, the ALWOBJDIF functions differently in BRMS and that is what is causing the issue? <<<<<

    I'm not familiar much with actual use of BRMS SAV/RST commands but based on what you described in your first post, I assume BRMS treats ALWOBJDIF differently from SAVLIB/RSTLIB.  I use SAVLIB/RSTLIB with ALWOBJDIF(*ALL) about 6-7 times in the past 2 months to know that it always works (but you will see BRMS message warning you in the job log as well but it will not prevent what you want from happening). 

    You should still use BRMS with other BRMS SAV/RST operations in which you encounter no issue.    But you would need to use a number of tape cartridges outside the context of BRMS to serve the purpose you ask here and this may mean that you need to use TAPnn device rather than TAPMLBnn and this can make your operating environment a bit more tricky to handle.  But if it turns out well enough and you achieve what you want, at least you can satisfy what you want when you asked your first question of this thread.   If it turns out unsatisfactorily, it's may be time you start considering logical replication that I'm sure will make your operation less complex and help you achieve your goal. 



    ------------------------------
    Chance favors only the prepared mind.
    -- Louis Pasteur
    ------------------------------
    Satid S.
    ------------------------------



  • 11.  RE: Suggestions on keeping the *userdata synchronized with a PowerHA cluster

    Posted Mon October 09, 2023 05:47 AM

    Michael, just catching up with this thread and I see that the Admin Domain has quite rightly been suggested. Outside of this please feel free to take a look at Maxava for PowerHA, specifically designed for this purpose. 

    https://www.maxava.com/maxava-for-powerha

    Feel free to get in touch with me here or via LinkedIn for more.

    Kind Regards, Ash Giddings (Maxava Product Manager).



    ------------------------------
    Ash Giddings
    ------------------------------