Informix

 View Only

  • 1.  onsmsync used to recycle backups : how effective is it?

    Posted Tue June 20, 2023 12:55 PM

    Hi Folks,

     

     

    I have been installing PSM at my customers and deployed SPL controlled by the IFMX scheduler.

     

    I have been testing onsmsync with different options to clear the backup objects.

    The options I use are

    • -t "datetime"
    • -I number of days
    • -g generations number to retain

     

    I am doing fine with the syntax and I can see onsmsync does things, but the results I expect are not achieved

     

    What do I expect when I run onsmsync one of those options?

    • Backup objects complying with the conditions disappear from the list rendered by onpsm -O list AND
    • Backup files complying with the conditions  are physically erased from the file system
    • Ixbar file updated accordingly

     

    I also understood that using onsmsync to recycle backup medias also ensures not to delete objects necessary to perform a restore (like for instance deleting a level 0 backup needed by a level 1 backup, or stuff like this)

     

    What I have noticed with my tests is that, at different opportunities, I have warning messages in the bac_act log file that say for instance backup object complying with conditions but not deleted (for an unknown reason). For me those files should clearly be deleted, and they are not.

     

    At the end, I remain with a lot of backup media/objects that should have been deleted long ago, but are still present in the onpsm -O list and file systems.

     

    Did I misunderstand anything critical here?

    This is IFMX 14.10 FC(5 or 6) I think

     

    Thanks

     

    Eric

     

     

     

     

    Eric Vercelletto
    Data Management Architect and Owner / Begooden IT Consulting
    KandooERP Founder and CTO
    IBM Champion 2013,2014,2015,2016,2017,2018,2019,2020
    ibm-champion-rgb-130px

    Tel:     +33(0) 298 51 3210
    Mob : +33(0)626 52 50 68
    skype: begooden-it
    Google Hangout: eric.vercelletto@begooden-it.com
    Email:     eric.vercelletto@begooden-it.com
    www :     http://www.vercelletto.com
    www  https://kandooerp.org

     

     



  • 2.  RE: onsmsync used to recycle backups : how effective is it?

    Posted Wed June 21, 2023 04:12 AM

    Hi
    it should work like that.
    But only when backup object has still an entry in the ixbar.
    Maybe a support case is needed to figure out why an object is not cleaned even when it should be.  



    ------------------------------
    Hedwig Fuchs
    ------------------------------

    Original Message


  • 3.  RE: onsmsync used to recycle backups : how effective is it?

    Posted Wed June 21, 2023 05:00 AM

    Hi Eric,

    I have seen similar problems with onsmsync when using Netbackup as a storage manager, very few of its basic functions work. (Back up and recovery with onbar and Netbackup do work.) In addition it is terminally slow as it works through each backup object in a serial fashion so on a large system it requires a lot of time and patience. Plus IMHO the utility suffers from having to work with multiple storage managers, lack of use - it is immaterial to ontape users - and being non-essential.

    Additionally with Netbackup the storage manager itself has its own retention policies and onsmsync is meant to keep the ixbar and sysutils database in step with this but it only goes through the motions and does nothing.

    I did spend quite a lot of time on it and had a case open with IBM but ultimately it never went anywhere because IBM didn't have a Netbackup storage manager with which to test. I kind of assumed that if we had Tivoli (IBM solution) all would be well.

    As we can't get things like rebuilding the ixbar from the storage manager to work, we implemented our own HA solution for this file.

    One thing I did make a mental note of but never did was to test everything with PSM because this is a fully integrated Informix solution and it would make it easier to raise cases with well-described steps to reproduce. It would also validate my own understanding of how things should work.

    I guess this is to say you could well be having difficulties and if you were to work with IBM/HCL to get these fixed, you would be doing everyone a service.

    Ben.



    ------------------------------
    Benjamin Thompson
    ------------------------------

    Original Message


  • 4.  RE: onsmsync used to recycle backups : how effective is it?

    Posted Mon September 25, 2023 04:59 AM

    Hi Eric,

    did you ever manage to resolve this with Informix Support? I'm having exactly the same problem with PSM+onsmync (configured via Informix HQ) on 14.10.xC8.

    Thanks



    ------------------------------
    Davorin Kremenjas
    ------------------------------

    Original Message


  • 5.  RE: onsmsync used to recycle backups : how effective is it?

    IBM Champion
    Posted Tue September 26, 2023 08:10 AM

    Davorin:

    I do not know if Eric received a resolution to his issue, but, I have clients with similar issues that are using PSM. In those cases I use onpsm to clean up older archive and logical log files:

    date=<oldest date to keep>

    paws=<seconds to sleep between runs>
    for oid in $( onpsm -O list|awk -v dt=$date '/^[0-9]/{if ($2 < dt) {print $1;} else {exit 0;}}' ); do  
           echo "onpsm -O del -y -o $oid";  
           onpsm -O del -y -o $oid;   
           cnt=$(( cnt + 1 ))
           if [[ $(( cnt % 100 )) -eq 0 ]]; then echo "sleeping($paws)"; sleep $paws; echo continuing; cnt=0; fi
    done

    The awk script returns the object ids for any PSM objects older than the value in the "date" environment variable. The sleep setting in the "paws" variable is to prevent the cleanup from locking the PSM catalog while a logical log backup is trying to kick off.


    Witness:

    $ > onpsm -O list 

    OneDB Primary Storage Manager Objects List

    Object ID            Date Created         Size (MB) Version Name
    1                    2023-08-28T14:44:42       88.2       1 /elendil/rootdbs/0
    2                    2023-08-28T14:44:42       59.4       1 /elendil/cdrspace/0
    3                    2023-08-28T14:44:43     1369.9       1 /elendil/datadbs_2/0
    4                    2023-08-28T14:44:46        0.1       1 /elendil/rubble/0
    5                    2023-08-28T14:44:47        0.1       1 /elendil/flintstone/0
    6                    2023-08-28T14:44:48        0.1       1 /elendil/llogspace/0
    7                    2023-08-28T14:44:49      179.5       1 /elendil/sbspace/0
    8                    2023-08-28T14:44:50       61.0       1 /elendil/indexdbs/0
    9                    2023-08-28T14:44:50    12790.8       1 /elendil/dbs_16k/0
    10                   2023-08-28T14:44:51     5545.7       1 /elendil/datadbs_1/0
    11                   2023-08-28T14:44:51        0.1       1 /elendil/plogspace/0
    12                   2023-08-28T14:44:52      561.3       1 /elendil/indexdbs_1/0
    13                   2023-08-28T14:46:00        6.0       1 /elendil/1/3578
    14                   2023-08-28T14:46:01        0.1       1 /elendil/critical_files/ixbar
    15                   2023-08-28T14:46:01        0.1       1 /elendil/critical_files/oncfg
    16                   2023-08-28T14:46:01        0.1       1 /elendil/critical_files/onconfig
    17                   2023-08-28T14:46:01        0.1       1 /elendil/critical_files/sqlhosts
    18                   2023-09-14T10:37:19       90.0       2 /elendil/rootdbs/0
    19                   2023-09-14T10:37:20    24387.0       2 /elendil/datadbs_1/0
    20                   2023-09-14T10:37:21     1369.9       2 /elendil/datadbs_2/0
    ...

    $ > date=2023-09-13 
    informix@Elezar-II:~$ onpsm -O list|awk -v dt=$date '/^[0-9]/{if ($2 < dt) {print $1;} else {exit 0;}}'  
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17

    $ > 

    Notice that items 18, 19 & 20 (and those that came after, are not listed.

    Art



    ------------------------------
    Art S. Kagel, President and Principal Consultant
    ASK Database Management Corp.
    www.askdbmgt.com
    ------------------------------

    Original Message


  • 6.  RE: onsmsync used to recycle backups : how effective is it?

    Posted Wed September 27, 2023 03:37 AM

    Thanks Art, that looks helpful.

    It turned out to be my negligence in this case: the oldest backup (which I was expecting to be cleaned up by onsmync) was done with dbservername set to shm alias. Later on I swapped dbservername and dbserveralias and did all further backups using the new onconfig which then resulted with onsmsync skipping the oldest backup. All other backups (older than 2 generations) were cleaned up as expected. I didn't read the docs, but it sounds like expected behaviour.



    ------------------------------
    Davorin Kremenjas
    ------------------------------

    Original Message


  • 7.  RE: onsmsync used to recycle backups : how effective is it?

    IBM Champion
    Posted Mon September 25, 2023 05:59 PM

    Hi,

    If the object is not deleted does the message include the copy id?

    If so check in the storage manager logs what the error was.

    If required increase BAR_DEBUG or enable verbose logging in the storage manager.

    One thing I did find was that if onsmsync was denied access to an object by the storage manager it would delete the object reference fromsysutils and the ixbar file, a quick way to delete everything!!! 


    Regards,
    David.



    ------------------------------
    David Williams
    ------------------------------

    Original Message


  • 8.  RE: onsmsync used to recycle backups : how effective is it?

    Posted Tue September 26, 2023 02:14 AM

    Could be intermittent, worked OK last night.

    Sounds like I'll have to keep an eye on it for some time, thanks for the suggestions David.



    ------------------------------
    Davorin Kremenjas
    ------------------------------

    Original Message