Data Integration

 View Only
  • 1.  ts_bookmark table

    Posted Tue February 07, 2023 07:23 AM
    i do not see my entry - maybe delay and duplicate entry
    we are thinking about a disaster recovery DR scenario and export on regular basis, the ts_bookmark table
    but now doing a test and deleted all rows from this table and the subs anyhow continues
    shutting down the instance and deleting rows - also subs continue
    looking at pointbase db : we see a table with source bookmarks
    they are saved in internal db ? what is the purpose of ts_bookmark table ? it is not always keeping an entry for each subs and sometimes empty.
    in case of dr, this table export will not help. we backup the pointbase db, but only once a day
    and after restore, we are not in sync. what would be a good scenario to preserve  to most recent bookmark as possible and avoid full refresh of all tables ?

    the problem is that we replicate multiple src tables to 1 target and full refresh is not easy and we need consolidated one-to-one and diff refresh in case of refresh
    another problem with this full refresh is : with db2 load command we can specify a modifier to change load behavior. can this be specified in system config somewhere, so that cdc full refresh would take this modifier into account ?


    ------------------------------
    Thanks for all answers
    Best Regards,
    Guy Przytula
    ------------------------------

    #DataIntegration
    #DataReplication


  • 2.  RE: ts_bookmark table

    Posted Tue February 07, 2023 09:59 AM
    Guy. What is your target? If it is a database target the TS_BOOKMARK in the target is updated in the database in the same unit of work as the replicated log entries and if the target database is replicated in some way to the standby then the TS_BOOKMARK in the target database will automatically reflect the state of the database. So if the replication to standby is not synchronous then in the event of a failover, the subscriptions can restart from the current bookmark in the target database.

    I can understand extracting the bookmark in some cases. Kafka as a target by default stores the bookmark in the internal Pointbase metadata database, so it is good to have the bookmark as of 5 minutes ago if the CDC server is failed over and the Pointbase metadata is not replicated continuously. Also for DB2 and some other source systems it is not possible to restart mirroring from some arbitrary point in time or log position, as you have to specify a bookmark and not a log position, so having a range of recent bookmarks with the associated bookmarks can be useful.

    I don't understand what you are doing when you delete all the rows in the bookmark table. If you are trying to simulate an unplanned failover, I would not attempt to do this until you had tried a planned (orderly) failover first.

    Regards

    ------------------------------
    Robert Philo
    ------------------------------



  • 3.  RE: ts_bookmark table

    Posted Tue February 07, 2023 11:01 AM
    thks for the update
    it are db2 databases as source or target
    there is no standby involved as cdc is installed on nfs drive mounted in prim and standby and integrated in tsa and cdc process follows the db2 database with failover. only 1 installation and controlled by tsa. this works ok
    we are preparing a dr environment and trying to see what we need to be able to restart cdc after crash/disaster and getting new machine which is a vm restore of a vm backup
    we want to know if in current setup we need to export ts_bookmark table to have a save copy for dr. currently we export this table each 15min
    but now we see that this table does not keep all info about all bookmarks for all subs
    so we presume the bookmark data is in the pointbase db. from which we have also backup, but only  once a day
    so how to get closer to the situation before the crash ?
    should we on regular base, export the bookmarks from the pointbase db ? and keep this as backup
    delete of rows from ts_bookmark was to see the effect on restart subs after a crash







    ------------------------------
    Thanks for all answers
    Best Regards,
    Guy Przytula
    ------------------------------



  • 4.  RE: ts_bookmark table

    Posted Wed February 08, 2023 09:55 AM
    Guy

    The TS_BOOKMARK table in the target database is the definitive version of the subscription bookmarks. It has all the information required for subscription restart. Therefore there is no requirement to save off the bookmark unless you need an arbitrary recover point.

    So when you restart from your restored VM, you should be able to simply restart replication as far as the bookmark is concerned.

    Anything in the metadata for DB2 agents is only of secondary importance

    Thanks

    Robert


    ------------------------------
    Robert Philo
    ------------------------------



  • 5.  RE: ts_bookmark table

    Posted Wed February 08, 2023 10:08 AM
    thks for the update
    so it means : if we restore the vm - the subs will restart at last bookmark, created before the vm backup (in the night)
    and if we need to restart a bookmark closer to the crash, we need to export the bookmark on regular base from pointbase db
    and use the dmsetbookmark after restoring the vm in new machine



    ------------------------------
    Thanks for all answers
    Best Regards,
    Guy Przytula
    ------------------------------



  • 6.  RE: ts_bookmark table

    Posted Wed February 08, 2023 10:25 AM
    Yes, that is correct

    ------------------------------
    Robert Philo
    ------------------------------