Informix

Informix

Connect with Db2, Informix, Netezza, open source, and other data experts to gain value from your data, share insights, and solve problems.

 View Only
  • 1.  How to verify all the requirements for an onbar restore

    Posted Thu March 06, 2025 06:02 PM

    Hi Family.

    I've been mostly quiet on this group because my gigs of late have been Perl with little to no Informix.  That has now changed. (YAY!)

    I have little to no experience with onbar. I performed some backups by rote and used archecker to get back a table my client had inadvertently dropped before his boss found out about it. But never a restore and certainly not a restore form a similar but not identical server.

    I've dug up some documentation from HCL and extracted the vital things to make sure of, but no clue as to HOW to make sure of these.  This is going to be a long one (as usual for me) but with specific points that the pool of expertise here should be able to answer, though not by one person. BTW We are using Veritas as our storage manager.

    • Make sure your storage manager must support imported restores.
      Well, obviously it does, because the DBA I replaced did imported restores.  But lacking that vital information, just how in blazes would I determine what features my storage manager supports and which it does not?  Or is this only in the ixbar and oncfg.<servername>.<servernum>?  If the latter, anyone point me to documentation as to the meaning of all those columns?
    • Make sure all the files/raw devices exist for the chunks. 
      Again, obviously they do here.  But that begs the question: In an ontape recovery, it spits out  a list of the chunks it will restore, like the chunks portion on "onstat -d" taken from the backup media.  Is there any way to get onbar to produce such a useful list?  This would be VERY helpful if the original server got destroyed.
    • Both the source and target computers must be on the same LAN or WAN..
      This important question is more for the network experts: How do I determine this?
      But also, WHY?  I think I should be able to carry the backup media on a big drive in my pocket and walk it to the target server, LAN/WAN be D****d.
    • Identical hardware (binary compatible should be enough), identical OS release (familiar territory here).
    • The same configuration and ROOTPATH information, although the server names and numbers can differ.
      This is funny: Folks here are telling me I should recover the target server with same server name as the source, though I think that sow confusion in the sqlhosts file.  It would seem easier to just leave the same server name and leave sqlhosts alone. Also, only the root device and logical log count really need to be identical.  NETTYPE? LRU params?  Do those really need to be the same?  I doubt it.
    • Identical storage-manager versions.
      I suppose they are the same on both servers but how do I determine the version of storage manager (remember, Veritas in my case) on either host?  And of the source were destroyed, how would I obtain the version that had been on that host?
    •  Compatible XBSA libraries
      GAAAAAK!!  How do I find that out?  I'll be satisfied for the moment of just locating these libraries!  I'll hazard guess that the library is the shared-object file specified by BAR_BSALIB_PATH.  But how would I determine the release version of the library and if it is compatible?  Again, how would I determine this from the backup media if the original server has gone POOF?

    I did know enough to ftp the saved ixbar file with the desired date from the source server, renaming it ixbar.<servernum> (after saving the existing ixbar.<servernum>).

    No matter what I have done I always get (in the bar_act.log) the message stream:

    +--------------------------

     2025-03-06 16:47:40 21962  21960 Working with veritas-netbackup as generic storage manager.
     2025-03-06 16:47:41 21962  21960 Successfully connected to Storage Manager.
     2025-03-06 16:47:42 21962  21960 XBSA Error: (BSAGetObject) This object has not been backed up.
     2025-03-06 16:47:42 21962  21960 Object information: bar_objdesc
         obj_id 0 obj_name 'rootdbs' obj_type 'R' act_id 155217 act_type 2 act_status 0
         act_start '2025-03-06 16:47:41'  act_end '2025-01-01 03:50:03'
         ins_time 0 rsam_time -383688085 seal_time 1735721403 prev_seal_time 1735721313 level 0 copyid hi:lo 1:1735721408 req_act_id 155217
         logstream 0 est_pages hi:lo 0:0 first_log 2912722 chpt_log 2912722 last_log 0
         partial_flag 0 do_query 0 ins_sm_id 0 ins_sm_name ''
         ins_verify 0 ins_verify_date '' restore order 0:0
         objInfo ''
         retry 0 in_catalog 0 in_bootfile 0 child_pid 0 child_state 0
         bkup_host '' backup_order 12842

    +=============================

    That " XBSA Error: (BSAGetObject) This object has not been backed up." is the most frustrating! Of course it has been backed up previously; that the ixbar tells us. I am so frustrated trying to figure out what has gone wrong.  (The sysdamins assure us that the media has been mounted.)

    Gracious!! Did I just write all that?  Ah! Jake is back in form. (yikes!)

    Thanks much for any help.  And now that I'm back in the pool I might start answering some questions as well

    Be well, all!



    ------------------------------
    Jacob Salomon
    ---
    Nobody goes there anymore, it's too crowded.  --Attr: Yogi Berra
    ------------------------------


  • 2.  RE: How to verify all the requirements for an onbar restore

    Posted Fri March 07, 2025 07:34 AM

    Jacob,

    From the target server, can you query your Veritas server via CLI for files from the source one? Maybe, you have to setup an ACL?



    ------------------------------
    Sincerely,
    Dennis
    ------------------------------



  • 3.  RE: How to verify all the requirements for an onbar restore

    Posted Fri March 07, 2025 08:40 AM

    Hey Jacob,

    It sounds like you are trying to do a redirected restore.

    I'll try and address some of your questions, but once you get a working restore, it will make a little more sense.

    • Make sure your storage manager must support imported restores.
      Well, obviously it does, because the DBA I replaced did imported restores.  But lacking that vital information, just how in blazes would I determine what features my storage manager supports and which it does not?  Or is this only in the ixbar and oncfg.<servername>.<servernum>?  If the latter, anyone point me to documentation as to the meaning of all those columns?

    As you say, it does support redirected restores because it's been done before, and veritas does allow redirected restores.  In terms of features, that's a question for your veritas folks.  You often have to treat it as a bit of a black box unless you have access to the veritas API.

    • Make sure all the files/raw devices exist for the chunks. 
      Again, obviously they do here.  But that begs the question: In an ontape recovery, it spits out  a list of the chunks it will restore, like the chunks portion on "onstat -d" taken from the backup media.  Is there any way to get onbar to produce such a useful list?  This would be VERY helpful if the original server got destroyed.

    The dbspaces that are needed will be those listed in the ixbar file.  Also the $INFORMIXDIR/etc/oncfg_<INFORMIXSERVER>.<SERVERNUM> file may be useful.  But you should be taking a list of the dbspaces and chunks from the prod server and storing them somewhere safe, just like you should be with other critical files, like onconfig, sqlhosts, LV information, etc...and ixbar of course.  This is a recommended practice for any system as part of DR.

    • Both the source and target computers must be on the same LAN or WAN..
      This important question is more for the network experts: How do I determine this?
      But also, WHY?  I think I should be able to carry the backup media on a big drive in my pocket and walk it to the target server, LAN/WAN be D****d.

    I don't know about this - you should be able to restore to any compatible server as long as netbackup is available.

    • Identical hardware (binary compatible should be enough), identical OS release (familiar territory here).
    • The same configuration and ROOTPATH information, although the server names and numbers can differ.
      This is funny: Folks here are telling me I should recover the target server with same server name as the source, though I think that sow confusion in the sqlhosts file.  It would seem easier to just leave the same server name and leave sqlhosts alone. Also, only the root device and logical log count really need to be identical.  NETTYPE? LRU params?  Do those really need to be the same?  I doubt it.

    Some things need to match, some things don't.  For example, I seem to remember LOGSIZE must match.  

    • Identical storage-manager versions.
      I suppose they are the same on both servers but how do I determine the version of storage manager (remember, Veritas in my case) on either host?  And of the source were destroyed, how would I obtain the version that had been on that host?

    Probably a question for the veritas admins.  I bet that there's a handy command to run to display this.  There is a file $INFORMIXDIR/etc/sm_versions that stores the version of the storage manager, but this is something that should already be set up.  Still, make sure that it has the same value on the source and target systems.

    •  Compatible XBSA libraries
      GAAAAAK!!  How do I find that out?  I'll be satisfied for the moment of just locating these libraries!  I'll hazard guess that the library is the shared-object file specified by BAR_BSALIB_PATH.  But how would I determine the release version of the library and if it is compatible?  Again, how would I determine this from the backup media if the original server has gone POOF?

    Yes, this would be the file referenced by BAR_BSALIB_PATH.  The file that this points to should match on the source and target servers.  If not then, who is to say whether they are compatible or not - apart from a veritas admin.

    Now, for your redirected restore, there are a few things to do.  It sounds like you are already connecting to some veritas server, or you wouldn't get the message about the rootdbs object not being backed up.  It may not be the right one though - hard to say at this point.

    You have already copied over the ixbar file.  If you just want to restore the latest backup rather than an older generation, I normally delete everything above the last level 0 backup of the rootdbs.  It makes it faster to process when the restore runs.  Column #6 in the ixbar file shows a 0 for the Level 0.  Rename this file to have the target SERVERNUM, e.g. ixbar.<SERVERNUM on target>

    Copy over the $INFORMIXDIR/etc/oncfg_<INFORMIXSERVER>.<SERVERNUM> file to the target.  Rename for the target INFORMIXSERVER and SERVERNUM

    Look on the SOURCE system and check the environment for anything that says INFXBSA, e.g. env | grep INFXBSA.  This could be a big clue if these environment variables are set.  If not set in the environment, it's possible that they are set in any script that performs a backup.  You may have something called INFXBSA_POLICY.  This is a veritas thing.  When doing a restore, you need to make sure that you are specifying the correct policy, or it won't be able to find the object.  So, if you can find this on the source server, make sure you set it the same on the target, e.g. export INFXBSA_POLICY=XXX.  There may be another policy for logical logs too, e.g. INFXBSA_LOGICAL_POLICY.   If not set in the environment on the source server, then take a look at the script referenced by ALARMPROGRAM in the onconfig to see if there are any INFXBSA env variables set there.

    While you may have limited visibility into veritas, there should be a file on the server called bp.conf.  For one of our customers, it is in /usr/openv/netbackup/bp.conf.  This has some information in it, including the name of the backup server.  When you perform the redirected restore, you want to be able to access that same server, but also the server name is important as it will be used as an identifier when Informix requests the object from Veritas.  So, if you can find that bp.conf file on the source system, you can run: grep "^CLIENT_NAME" bp.conf   Hopefully you can get this info.  Then, on the target system, run: export INFXBSA_CLIENT=<the client_name from bp.conf>

    If the SERVERNUM of the source and target systems is different, then on the target, run: export IFX_SERVERNUM=<servernum of source system>

    The onconfig file on the target system needs to have the ROOTPATH set to the same as the ROOTPATH value on the source system. 

    Once you have made these changes, then try a physical restore (onbar -r -p) and hope for the best.

    Good luck!

    Mike



    ------------------------------
    Mike Walker
    xDB Systems, Inc
    www.xdbsystems.com
    ------------------------------



  • 4.  RE: How to verify all the requirements for an onbar restore

    Posted Tue March 11, 2025 08:59 PM
    Cowabonga Mike!

    Noting the length of your reply: Whaddaya tryin'todo?  Compete with me? <Grin> Emoji

    Kidding aside: I indeed checked out the ALARMPROGRAM and pulled three environment variables from there:
    • INFXBSA_CLIENT= <Server name of source intance>
    • INFXBSA_POLICY=PROD-informix-<a domain>
    • TARGET_SRV=${INFORMIXSERVER} of the target for the restore.
    That, after:
    Copying oncfg_<source server name>.<server number> from the source to the target host and renaming it to reflect the target servername and servernumber
    Copying the ixbar.<servernum> to the target host and renaming it according to the target server number.

    After all that I started onbar -r -p -w.

    \And it took off.  I just checked on it 4 hours after starting.  Still going strong!

    Thanks EXTEREMELY!!

    +----- Jacob Salomon --------------------------------------------------+
    | Praise without end for the go-ahead zeal of whoever it was that      |
    |  invented the wheel;                                                 |
    | But never a word for the poor soul's sake that thought ahead,        |
    |  and invented the brake.                                             |
    +-------------- Howard Nemerov, on the US Bicentennial ----------------+






  • 5.  RE: How to verify all the requirements for an onbar restore

    Posted Tue March 11, 2025 11:50 PM

    Well, that sounds like progress!  You still have the logical recovery to go, but fingers crossed it works.  If you have no visibility into the storage manager itself then the process can be frustrating.

    Note that you can use BAR_MAX_RESTORE to restore multiple spaces in parallel for improved restore times, however I believe that Veritas needs to be configured to allow this simultaneous restores, or the other onbar processes will hang and you'll get the storage manager has stalled message.

    Good luck!

    Mike



    ------------------------------
    Mike Walker
    xDB Systems, Inc
    www.xdbsystems.com
    ------------------------------



  • 6.  RE: How to verify all the requirements for an onbar restore

    Posted Fri March 07, 2025 10:26 AM

    Hi Jacob,

    good to see you back in business!

    On your problem, one thing critical with onbar and probably all commercial storage managers:

    the backup objects within the storage manager are IDed with a couple of elements: instance name, backup type (L 0/1/2, or log), object name (space name or log uid) .... and an identifier for the host system the backup is being taken on, as everything else could exist just the same on a different system in your org.

    When restoring the exact same system the backup originates from, then onbar and the XBSA lib automatically will know how to compose the object ID for the object they're gonna request from the SM.

    If, on the other hand, you're trying to restore such backup on a different host, or if the original host's ID changed, you have to inform the restore about that original host ID, the way for which is different with each SM.

    For Veritas, my notes say, you've got to export INFXBSA_CLIENT=<source_host> in onbar's environment.
    Should this not work or not be sufficient, take it as your entry point for further research.  I guess there should also be some info on Veritas' web site specifically for running Informix backup & restore.

    HTH,
     Andreas



    ------------------------------
    Andreas Legner
    Informix Dev
    HCL Software
    ------------------------------



  • 7.  RE: How to verify all the requirements for an onbar restore

    Posted Wed March 12, 2025 07:33 PM
    • Make sure your storage manager must support imported restores.
      Well, obviously it does, because the DBA I replaced did imported restores.  But lacking that vital information, just how in blazes would I determine what features my storage manager supports and which it does not? 

      Read the storage manager documentation or contact the storage manager vendor.
      Or is this only in the ixbar and oncfg.<servername>.<servernum>?  If the latter, anyone point me to documentation as to the meaning of all those columns?

      Read the FAQ - http://smooth1.co.uk/ifaq/ifaq06e.htm#6.85!!!

    • Make sure all the files/raw devices exist for the chunks. 
      Again, obviously they do here.  But that begs the question: In an ontape recovery, it spits out  a list of the chunks it will restore, like the chunks portion on "onstat -d" taken from the backup media.  Is there any way to get onbar to produce such a useful list?  This would be VERY helpful if the original server got destroyed.

        Chunk names are in the oncfg file.

    • Both the source and target computers must be on the same LAN or WAN..
      This important question is more for the network experts: How do I determine this?
      But also, WHY?  I think I should be able to carry the backup media on a big drive in my pocket and walk it to the target server, LAN/WAN be D****d.

      So the target can access the same storage manager.

    • Identical hardware (binary compatible should be enough), identical OS release (familiar territory here).

       Same as for ontape restores, make sure the Informix/storage vendor binaries will run.

    • The same configuration and ROOTPATH information, although the server names and numbers can differ.
      This is funny: Folks here are telling me I should recover the target server with same server name as the source, though I think that sow confusion in the sqlhosts file.  It would seem easier to just leave the same server name and leave sqlhosts alone. Also, only the root device and logical log count really need to be identical.  NETTYPE? LRU params?  Do those really need to be the same?  I doubt it.

        Chunks pathnames/sizes need to be same. 
        Same SERVERNUM unless you can use something like IFX_SERVERNUM
        Not sure about servername although you can always use a temporary onconfig/sqlhosts to make this work, this also avoids needing to use IFX_SERVERNUM if you keep SERVERNUMS unique across your estate.


    • Identical storage-manager versions.
      I suppose they are the same on both servers but how do I determine the version of storage manager (remember, Veritas in my case) on either host?  And of the source were destroyed, how would I obtain the version that had been on that host?

      Check the storage manager documentation or contact the storage vendor.
      Have the storage manager pre-installed on your recovery site!


    •  Compatible XBSA libraries
      GAAAAAK!!  How do I find that out?  I'll be satisfied for the moment of just locating these libraries!  I'll hazard guess that the library is the shared-object file specified by BAR_BSALIB_PATH.  But how would I determine the release version of the library and if it is compatible?  Again, how would I determine this from the backup media if the original server has gone POOF?

      Check the storage manager documentation or contact the storage vendor.
      If the original site is gone then first you need to recover the storage manager, that is not an Informix issue.


    I did know enough to ftp the saved ixbar file with the desired date from the source server, renaming it ixbar.<servernum> (after saving the existing ixbar.<servernum>).

        Let me add more complexity then...imported PIT restores in a different timezone!
        - Get PIT local time on the target server you wish to restore to.
        - Use the ixbar file which contains 2 columns, local time and I believe Timestamp in UTC seconds from 0101970 to  convert PIT restore time to UTC
       -  Convert PIT Restore Time to Target Local time

      - Use target Local time in onbar restore command

        Yes I had this fully scripted once!!

       Also from experience, backup the critical files including ixbar file!
        - https://www.ibm.com/docs/en/informix-servers/15.0.0?topic=data-administrative-files-back-up
       - https://www.ibm.com/docs/en/informix-servers/15.0.0?topic=bar-onbar-b-syntax-backing-up  use -cf yes on backups to be sure.

    Use storage manager commands to restore the critical files inbar/oncfg (can also be scripted) then the whole thing can be scripted.  I noticce the new option https://www.ibm.com/docs/en/informix-servers/15.0.0?topic=data-recreating-chunk-files-during-restore "Use the -O option to recreate missing chunk files and any necessary directories during a restore."<

    Nice, that was a feature request of mine back in 2017!

    Regards,
    David.
     



    ------------------------------
    David Williams
    Senior Database Platform Engineer
    Flutter
    London
    ------------------------------