Hi Vitalij,
It's a bit more nuanced than that but you are right about effectively everything ending up in the transaction log, and there most certainly still is a transaction log, and you can, as-of 12.4, if you configure the server as such, query that transaction log through the REST API again as well.
And before I continue, I presume that when you say 'bulkload' you are talking about simply loading a lot of data and are not referring to the old 'bulk-load-mode' that used to exist in TM1 but has been removed in v12.
Think about it this way, TM1 v12 has to keep track of all the changes, data and metadata, to be able to keep replicas in sync in a high available setup and to be able to guarantee fail-over. For v12 we also still have version management on our roadmap for which we'll need the details of the individual updates as well (read: TM1 v12 maintains all information for doing so already, we haven't exposed an interface to interact with a database at version level just yet). And whilst we started out doing so by putting all the transactions in MongoDB, and as such effectively put the transaction log in there, we are about to replace that with a new storage that is MUCH more efficient (likely in the next minor version of v12 in early-mid 2025). The great advantage of this pending change is that we will, once we've released this, always have all data on disk, making SaveDataAll completely obsolete once and for all.
Access to "the transaction log" however is available through a publish-subscribe mechanism as well as, as of 12.4 once again, the good old REST API endpoint. Both are optional, if neither are configured our 'publishing service', a separate micro-service managing this under the covers, won't be there and will not be publishing and/or writing the transaction logs to a file for consumption.
I realize I'm just clarifying how it (is going to) work(s), you are absolutely right that we store all transactions. But I'm convinced that once that version is out next year it'll have benefits for us all.
------------------------------
Hubert Heijkers
STSM, Program Director TM1 Functional Database Technology and OData Evangelist
------------------------------
Original Message:
Sent: Thu October 17, 2024 09:58 AM
From: Vitalij Rusakovskij
Subject: "transaction log" in the v12 - or the behaviour with bulk load
Hello,
as known, for v12, there is / there wont be any transaction log at all, as every step will be logged.
what behaviour is planned for the bulk loads?
Normally, you deactivate the logging for loading of such a big amount and after that you reactivate it.
So what will happen in future with the bulkload in v12?
Thanks in advance.
With best regards,
Vitalij
------------------------------
Vitalij Rusakovskij
------------------------------