Planning Analytics

Planning Analytics

Get AI-infused integrated business planning

 View Only
Expand all | Collapse all

"transaction log" in the v12 - or the behaviour with bulk load

  • 1.  "transaction log" in the v12 - or the behaviour with bulk load

    Posted Thu October 17, 2024 09:58 AM

    Hello, 

    as known, for v12, there is / there wont be any transaction log at all, as every step will be logged. 

    what behaviour is planned for the bulk loads? 
    Normally, you deactivate the logging for loading of such a big amount and after that you reactivate it. 
    So what will happen in future with the bulkload in v12? 

    Thanks in advance. 

    With best regards, 
    Vitalij



    ------------------------------
    Vitalij Rusakovskij
    ------------------------------


  • 2.  RE: "transaction log" in the v12 - or the behaviour with bulk load
    Best Answer

    Posted Fri October 18, 2024 07:26 AM

    Hi Vitalij,

    It's a bit more nuanced than that but you are right about effectively everything ending up in the transaction log, and there most certainly still is a transaction log, and you can, as-of 12.4, if you configure the server as such, query that transaction log through the REST API again as well.

    And before I continue, I presume that when you say 'bulkload' you are talking about simply loading a lot of data and are not referring to the old 'bulk-load-mode' that used to exist in TM1 but has been removed in v12.

    Think about it this way, TM1 v12 has to keep track of all the changes, data and metadata, to be able to keep replicas in sync in a high available setup and to be able to guarantee fail-over. For v12 we also still have version management on our roadmap for which we'll need the details of the individual updates as well (read: TM1 v12 maintains all information for doing so already, we haven't exposed an interface to interact with a database at version level just yet). And whilst we started out doing so by putting all the transactions in MongoDB, and as such effectively put the transaction log in there, we are about to replace that with a new storage that is MUCH more efficient (likely in the next minor version of v12 in early-mid 2025). The great advantage of this pending change is that we will, once we've released this, always have all data on disk, making SaveDataAll completely obsolete once and for all.

    Access to "the transaction log" however is available through a publish-subscribe mechanism as well as, as of 12.4 once again, the good old REST API endpoint. Both are optional, if neither are configured our 'publishing service', a separate micro-service managing this under the covers, won't be there and will not be publishing and/or writing the transaction logs to a file for consumption.

    I realize I'm just clarifying how it (is going to) work(s), you are absolutely right that we store all transactions. But I'm convinced that once that version is out next year it'll have benefits for us all.



    ------------------------------
    Hubert Heijkers
    STSM, Program Director TM1 Functional Database Technology and OData Evangelist
    ------------------------------



  • 3.  RE: "transaction log" in the v12 - or the behaviour with bulk load

    Posted Fri October 18, 2024 09:13 AM

    Hello Hubert, 

    thank you for your answer! 
    and yes, with the "bulk-load", I meant the simple loading of lots of data. 

    Let us look forward to your updates and I also think, we will have benefits for us all!

    With best regards, 
    Vitalij



    ------------------------------
    Vitalij Rusakovskij
    ------------------------------