View Only
  • 1.  Promoting RSS nodes

    Posted Thu April 30, 2020 12:28 PM
    Environment: IDS 12.10.FC12 running on CentOs 7.6.1810

    Picture this...
                                                                                             PROD              |               DR
                                                                                  ifxdb01          ifxdb02  |  ifxdb03          ifxdb04
                                                                                    Pri                  Sec    |    RSS              RSS
    PROD site is Lost                                                                                    X
    Promote ifxdb03 to Primar                                                                      X
    Promote ifxdb04 to HDR Secondary                                                       X
    PROD site returns                                                                                   |

    So, I lose my PROD site and want to fail over over to my DR site making ifxdb03 the new Primary and ifxdb04 the new HDR Secondary. Can anyone confirm to me the sequence of steps (with commands) that I should be following to achieve this, as well as reverting to the original setup when the PROD site sparks into life again?

    Thanks in advance.

    Kirit Rana
    Informix Engineer


  • 2.  RE: Promoting RSS nodes

    Posted Thu April 30, 2020 12:56 PM
    1. Promote ifxdb03 to HDR secondary.
        onmode -d secondary ifxdb01
    2) Promote  ifxdb03 from HDR secondary to primary
         onmode -d make primary ifxdb03
    3) Promote ifxdb04 from RSS to HDR secondary
         onmode -d secondary ifxdb03

    Nagaraju Inturi

  • 3.  RE: Promoting RSS nodes

    Posted Fri May 01, 2020 10:40 AM
    Thanks @Nagaraju Inturi, appreciated​​

    Kirit Rana

  • 4.  RE: Promoting RSS nodes

    IBM Champion
    Posted Thu April 30, 2020 01:27 PM

    Close. Here are the steps as I see them:

    1. Primary site goes down
    2. Promote ifxdb03 to HDR Secondary (you cannot go directly from RSS to primary)
    3. Promote ifxdb03 to HDR Primary
    4. Promote ifxdb04 to HDR Secondary with ifxcb03 as Primary
    5. Original prime site returns.
    6. Assuming the newest logid that was on ifxdb03 at the time of the crash is still online: 
    7. Start ifxdb01 with: oninit -PHY
    8. Promote ifxdb01 to RSS Secondary with ifxdb03 as Primary and let it catch up
    9. Start ifxdb02 with: oninit -PHY
    10. Reset ifxdb02 as RSS Secondary with ifxcb03 as Primary and let it catch up.
    11. Demote ifxdb04 to RSS Secondary
    12. Shutdown ifxdb03
    13. Promote ifxdb01 to HDR Secondary
    14. Promote ifxdb01 to Primary
    15. Promote ifxdb02 to HDR Secondary
    16. Start ifxdb03 with: oninit -PHY
    17. Set ifxdb03 as RSS Secondary to ifxdb01
    18. Reset ifxdb04 as RSS Secondary to idxdb01
    If the newest log that ifxdb01 knew about is no longer online on ifxdb03 then you will have to restore ifxdb01 and ifxdb02 from an archive taken from ifxdb03 or using ifxclone.


    Art S. Kagel, President and Principal Consultant
    ASK Database Management

    Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference.  Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves.

  • 5.  RE: Promoting RSS nodes

    Posted Fri May 01, 2020 10:47 AM
    Well that's pretty comprehensive @Art Kagel; I'll simulate this on a test rig. Let's say ifxdb01 and ifxdb02 were never down and just that network connectivity was lost to the PROD site. When connectivity is restored, what do how might ifxdb01 react now that ifxdb03 is also a Primary node? 

    Thanks again

    Kirit Rana

  • 6.  RE: Promoting RSS nodes

    IBM Champion
    Posted Fri May 01, 2020 11:00 AM


    Well, then you have the dreaded "split brain" problem. Three scenarios:
    1. Some users are still able to access ifxdb01/2 while others can only connect to ifxdb03/4
      This is the hard one. In this case you have new transactions on two different primary servers. Some of which may clash (hopefully your normal mode of operations makes this unlikely). In that case the best solution would be to link the two "primary" servers via ER and let ER sync the two copies of the database. Once that is done you can convert ifxdb02 to an RSS secondary. If the changes create too many clashes you may have to resolve some of the clashes manually. Very painful. This is one reason to use Connection Manager to manage server promotions with additional out-of-network methods for multiple CM instances to coordinate with each other to prevent split brains.
    2. Even though it is still online, ifxdb01/2 is not reachable at all for any user transactions.
      This one is simple, you treat ifxdb01 & ifxdb02 as if they were offline and recover them appropriately once the network is back.
    3. Even though ifxdb01/2 and ifxdb03/4 are not on speaking terms, users can still get to ifxdb01/2 and no one can get to ifxdb03/4
      This one is simple also, ignore that the backup site is unavailable and recover it when the network is repaired.

    Art Kagel

  • 7.  RE: Promoting RSS nodes

    Posted Mon May 04, 2020 06:27 AM
    Thanks @Art Kagel​​, I elect to go with option 2; It's the least painful.

    Kirit Rana

  • 8.  RE: Promoting RSS nodes

    IBM Champion
    Posted Mon May 04, 2020 10:12 AM
    You HOPE you get #2.  B^)​

    Art Kagel

  • 9.  RE: Promoting RSS nodes

    Posted Tue May 05, 2020 08:52 AM
    LOL. Indeed.

    Kirit Rana