Hello TM1 Team,

we just noticed, that on the latest TM1 release (11.5.00000.23) the TransactionLogEntries EntitySet is not filterbale by user.
It would be very useful though, if it was filterable by user.

Do you guys plan to include this feature in a future release ?

Cheers,

Marius

------------------------------
Marius Wirtz
------------------------------

#PlanningAnalyticswithWatson

Hi Marius, can you elaborate a bit as to what it is that is lacking here?

I'm sensing you might have expected the User property to actually be a NavigationProperty and as such of EntityType User? If that's the case it is indeed not, it is merely a string representing the name of the user at the time the log entry was created.

As such an admin could grab the transaction log entries created while I was logged in as in:

http://tm1server:12345/api/v1/TransactionLogEntries?$filter=User%20eq%20'Hubert'


------------------------------
Hubert Heijkers
------------------------------

Original Message:
Sent: Mon January 06, 2020 05:06 PM
From: Marius Wirtz
Subject: REST - FilterableProperties on TransactionLogEntries

Hello TM1 Team,

we just noticed, that on the latest TM1 release (11.5.00000.23) the TransactionLogEntries EntitySet is not filterbale by user.
It would be very useful though, if it was filterable by user.

Do you guys plan to include this feature in a future release ?

Cheers,

Marius

------------------------------
Marius Wirtz
------------------------------

#PlanningAnalyticswithWatson

Hello Hubert, 

You are absolutely right. In version 11.5 and 11.6 I can indeed filter the logs by user.
I just got irritated when I found this (User is not explicitly listed as a filterable property) in the $metadata document :


Thank you

Marius

------------------------------
Marius Wirtz
------------------------------

Original Message:
Sent: Tue January 07, 2020 04:46 AM
From: Hubert Heijkers
Subject: REST - FilterableProperties on TransactionLogEntries

Hi Marius, can you elaborate a bit as to what it is that is lacking here?

I'm sensing you might have expected the User property to actually be a NavigationProperty and as such of EntityType User? If that's the case it is indeed not, it is merely a string representing the name of the user at the time the log entry was created.

As such an admin could grab the transaction log entries created while I was logged in as in:

http://tm1server:12345/api/v1/TransactionLogEntries?$filter=User%20eq%20'Hubert'


------------------------------
Hubert Heijkers

Original Message:
Sent: Mon January 06, 2020 05:06 PM
From: Marius Wirtz
Subject: REST - FilterableProperties on TransactionLogEntries

Hello TM1 Team,

we just noticed, that on the latest TM1 release (11.5.00000.23) the TransactionLogEntries EntitySet is not filterbale by user.
It would be very useful though, if it was filterable by user.

Do you guys plan to include this feature in a future release ?

Cheers,

Marius

------------------------------
Marius Wirtz
------------------------------

#PlanningAnalyticswithWatson

Ah, that's what you meant, hadn't noticed, that's indeed incorrect, I'll have that added to the metadata shortly.

------------------------------
Hubert Heijkers
------------------------------

Original Message:
Sent: Tue January 07, 2020 04:09 PM
From: Marius Wirtz
Subject: REST - FilterableProperties on TransactionLogEntries

Hello Hubert, 

You are absolutely right. In version 11.5 and 11.6 I can indeed filter the logs by user.
I just got irritated when I found this (User is not explicitly listed as a filterable property) in the $metadata document :

<EntitySet Name="TransactionLogEntries" EntityType="tm1.TransactionLogEntry">

<Annotation Term="Capabilities.ChangeTracking">

<Record>

<PropertyValue Property="FilterableProperties">

<Collection>

<PropertyPath>ChangeSetID</PropertyPath>

<PropertyPath>TimeStamp</PropertyPath>

<PropertyPath>ReplicationTime</PropertyPath>

<PropertyPath>Cube</PropertyPath>

<PropertyPath>Tuple</PropertyPath>

<PropertyPath>OldValue</PropertyPath>

<PropertyPath>NewValue</PropertyPath>

<PropertyPath>StatusMessage</PropertyPath>

</Collection>

</PropertyValue>

Thank you

Marius

------------------------------
Marius Wirtz

Original Message:
Sent: Tue January 07, 2020 04:46 AM
From: Hubert Heijkers
Subject: REST - FilterableProperties on TransactionLogEntries

Hi Marius, can you elaborate a bit as to what it is that is lacking here?

I'm sensing you might have expected the User property to actually be a NavigationProperty and as such of EntityType User? If that's the case it is indeed not, it is merely a string representing the name of the user at the time the log entry was created.

As such an admin could grab the transaction log entries created while I was logged in as in:

http://tm1server:12345/api/v1/TransactionLogEntries?$filter=User%20eq%20'Hubert'


------------------------------
Hubert Heijkers

Original Message:
Sent: Mon January 06, 2020 05:06 PM
From: Marius Wirtz
Subject: REST - FilterableProperties on TransactionLogEntries

Hello TM1 Team,

we just noticed, that on the latest TM1 release (11.5.00000.23) the TransactionLogEntries EntitySet is not filterbale by user.
It would be very useful though, if it was filterable by user.

Do you guys plan to include this feature in a future release ?

Cheers,

Marius

------------------------------
Marius Wirtz
------------------------------

#PlanningAnalyticswithWatson

Is there a way to allow non admin users to query the transaction log? If not could it be added to the user capability controls? We would like to expose the log in a a new UI we are building.

------------------------------
Ryan Clapp
------------------------------

Original Message:
Sent: Wed January 08, 2020 03:55 AM
From: Hubert Heijkers
Subject: REST - FilterableProperties on TransactionLogEntries

Ah, that's what you meant, hadn't noticed, that's indeed incorrect, I'll have that added to the metadata shortly.

------------------------------
Hubert Heijkers

Original Message:
Sent: Tue January 07, 2020 04:09 PM
From: Marius Wirtz
Subject: REST - FilterableProperties on TransactionLogEntries

Hello Hubert, 

You are absolutely right. In version 11.5 and 11.6 I can indeed filter the logs by user.
I just got irritated when I found this (User is not explicitly listed as a filterable property) in the $metadata document :

<EntitySet Name="TransactionLogEntries" EntityType="tm1.TransactionLogEntry">

<Annotation Term="Capabilities.ChangeTracking">

<Record>

<PropertyValue Property="FilterableProperties">

<Collection>

<PropertyPath>ChangeSetID</PropertyPath>

<PropertyPath>TimeStamp</PropertyPath>

<PropertyPath>ReplicationTime</PropertyPath>

<PropertyPath>Cube</PropertyPath>

<PropertyPath>Tuple</PropertyPath>

<PropertyPath>OldValue</PropertyPath>

<PropertyPath>NewValue</PropertyPath>

<PropertyPath>StatusMessage</PropertyPath>

</Collection>

</PropertyValue>

Thank you

Marius

------------------------------
Marius Wirtz

Original Message:
Sent: Tue January 07, 2020 04:46 AM
From: Hubert Heijkers
Subject: REST - FilterableProperties on TransactionLogEntries

Hi Marius, can you elaborate a bit as to what it is that is lacking here?

I'm sensing you might have expected the User property to actually be a NavigationProperty and as such of EntityType User? If that's the case it is indeed not, it is merely a string representing the name of the user at the time the log entry was created.

As such an admin could grab the transaction log entries created while I was logged in as in:

http://tm1server:12345/api/v1/TransactionLogEntries?$filter=User%20eq%20'Hubert'


------------------------------
Hubert Heijkers

Original Message:
Sent: Mon January 06, 2020 05:06 PM
From: Marius Wirtz
Subject: REST - FilterableProperties on TransactionLogEntries

Hello TM1 Team,

we just noticed, that on the latest TM1 release (11.5.00000.23) the TransactionLogEntries EntitySet is not filterbale by user.
It would be very useful though, if it was filterable by user.

Do you guys plan to include this feature in a future release ?

Cheers,

Marius

------------------------------
Marius Wirtz
------------------------------
#PlanningAnalyticswithWatson

Hi Ryan,

Interesting question...Not currently possible and it's not on our roadmap.   Why do users need to see transactions?  Is it transactions for all users or just the data changes made by the user running the query?

I think we do want to look into storing transactions in proper database at some point in the distant (or maybe not so distant) future.  We might look at security for transactions at that point.  We won't plan any changes in security in this area in the current version of TM1 Server.

------------------------------
Stuart King
IBM Planning Analytics Offering Manager
------------------------------

Original Message:
Sent: Wed January 08, 2020 12:09 PM
From: Ryan Clapp
Subject: REST - FilterableProperties on TransactionLogEntries

Is there a way to allow non admin users to query the transaction log? If not could it be added to the user capability controls? We would like to expose the log in a a new UI we are building.

------------------------------
Ryan Clapp

Original Message:
Sent: Wed January 08, 2020 03:55 AM
From: Hubert Heijkers
Subject: REST - FilterableProperties on TransactionLogEntries

Ah, that's what you meant, hadn't noticed, that's indeed incorrect, I'll have that added to the metadata shortly.

------------------------------
Hubert Heijkers

Original Message:
Sent: Tue January 07, 2020 04:09 PM
From: Marius Wirtz
Subject: REST - FilterableProperties on TransactionLogEntries

Hello Hubert, 

You are absolutely right. In version 11.5 and 11.6 I can indeed filter the logs by user.
I just got irritated when I found this (User is not explicitly listed as a filterable property) in the $metadata document :

<EntitySet Name="TransactionLogEntries" EntityType="tm1.TransactionLogEntry">

<Annotation Term="Capabilities.ChangeTracking">

<Record>

<PropertyValue Property="FilterableProperties">

<Collection>

<PropertyPath>ChangeSetID</PropertyPath>

<PropertyPath>TimeStamp</PropertyPath>

<PropertyPath>ReplicationTime</PropertyPath>

<PropertyPath>Cube</PropertyPath>

<PropertyPath>Tuple</PropertyPath>

<PropertyPath>OldValue</PropertyPath>

<PropertyPath>NewValue</PropertyPath>

<PropertyPath>StatusMessage</PropertyPath>

</Collection>

</PropertyValue>

Thank you

Marius

------------------------------
Marius Wirtz

Original Message:
Sent: Tue January 07, 2020 04:46 AM
From: Hubert Heijkers
Subject: REST - FilterableProperties on TransactionLogEntries

Hi Marius, can you elaborate a bit as to what it is that is lacking here?

I'm sensing you might have expected the User property to actually be a NavigationProperty and as such of EntityType User? If that's the case it is indeed not, it is merely a string representing the name of the user at the time the log entry was created.

As such an admin could grab the transaction log entries created while I was logged in as in:

http://tm1server:12345/api/v1/TransactionLogEntries?$filter=User%20eq%20'Hubert'


------------------------------
Hubert Heijkers

Original Message:
Sent: Mon January 06, 2020 05:06 PM
From: Marius Wirtz
Subject: REST - FilterableProperties on TransactionLogEntries

Hello TM1 Team,

we just noticed, that on the latest TM1 release (11.5.00000.23) the TransactionLogEntries EntitySet is not filterbale by user.
It would be very useful though, if it was filterable by user.

Do you guys plan to include this feature in a future release ?

Cheers,

Marius

------------------------------
Marius Wirtz
------------------------------
#PlanningAnalyticswithWatson

Original Message------

Hi Ryan,

Interesting question...Not currently possible and it's not on our roadmap.   Why do users need to see transactions?  Is it transactions for all users or just the data changes made by the user running the query?

I think we do want to look into storing transactions in proper database at some point in the distant (or maybe not so distant) future.  We might look at security for transactions at that point.  We won't plan any changes in security in this area in the current version of TM1 Server.

------------------------------
Stuart King
IBM Planning Analytics Offering Manager
------------------------------

#PlanningAnalyticswithWatson

Ryan, if you hadn't told us that you were putting them already in a database I'd have suggested you should;-)

Things being as they are (TM1Server v11 and before) queries against the transaction log are contentious (effectively block any other commits). The occasional brief query doesn't hurt as (hence: why I did a special implementation for deltas) but longer running queries could, especially with larger transaction logs, seriously impact the system.

We contemplated at some point to allow users to see their own transactions (like we do in the REST API for the threads/sessions) but shed away from doing so for the exact same reason, not wanting to invite clever people to add contention to the most contentious resource we have.

So, for now, I'd indeed put it in a separate database and, if I was writing my own client, I wouldn't even use TM1's drill through as the middle man but directly go to that database to retrieve the transactions I'd be looking for. Any reason why you wouldn't?

Now the 'good' news perhaps is that in the next major version of TM1 server we already have these transactions in a database (part of the high availability work we've been working on for a while). We have not finalized any new APIs for that version, might not need any, but at least we'll have tackled the contention issues plus the performance presumably would be way better than ploughing through old fashion text files;-).


------------------------------
Hubert Heijkers
------------------------------

Original Message:
Sent: Wed January 08, 2020 03:02 PM
From: Ryan Clapp
Subject: REST - FilterableProperties on TransactionLogEntries

Today we extract transactions, push them into a database, and then expose them via drill through. There is an inherent time lag due to the length of the processing. In our UI we want users to be able to right click on a data point, cell in a view, and select an option to "Show all Changes".

Original Message------

Hi Ryan,

Interesting question...Not currently possible and it's not on our roadmap.   Why do users need to see transactions?  Is it transactions for all users or just the data changes made by the user running the query?

I think we do want to look into storing transactions in proper database at some point in the distant (or maybe not so distant) future.  We might look at security for transactions at that point.  We won't plan any changes in security in this area in the current version of TM1 Server.

------------------------------
Stuart King
IBM Planning Analytics Offering Manager
------------------------------

#PlanningAnalyticswithWatson

@Hubert Heijkers  Thanks for the detailed response!

------------------------------
Stuart King
IBM Planning Analytics Offering Manager
------------------------------

Original Message:
Sent: Thu January 09, 2020 02:58 AM
From: Hubert Heijkers
Subject: REST - FilterableProperties on TransactionLogEntries

Ryan, if you hadn't told us that you were putting them already in a database I'd have suggested you should;-)

Things being as they are (TM1Server v11 and before) queries against the transaction log are contentious (effectively block any other commits). The occasional brief query doesn't hurt as (hence: why I did a special implementation for deltas) but longer running queries could, especially with larger transaction logs, seriously impact the system.

We contemplated at some point to allow users to see their own transactions (like we do in the REST API for the threads/sessions) but shed away from doing so for the exact same reason, not wanting to invite clever people to add contention to the most contentious resource we have.

So, for now, I'd indeed put it in a separate database and, if I was writing my own client, I wouldn't even use TM1's drill through as the middle man but directly go to that database to retrieve the transactions I'd be looking for. Any reason why you wouldn't?

Now the 'good' news perhaps is that in the next major version of TM1 server we already have these transactions in a database (part of the high availability work we've been working on for a while). We have not finalized any new APIs for that version, might not need any, but at least we'll have tackled the contention issues plus the performance presumably would be way better than ploughing through old fashion text files;-).


------------------------------
Hubert Heijkers

Original Message:
Sent: Wed January 08, 2020 03:02 PM
From: Ryan Clapp
Subject: REST - FilterableProperties on TransactionLogEntries

Today we extract transactions, push them into a database, and then expose them via drill through. There is an inherent time lag due to the length of the processing. In our UI we want users to be able to right click on a data point, cell in a view, and select an option to "Show all Changes".

Original Message------

Hi Ryan,

Interesting question...Not currently possible and it's not on our roadmap.   Why do users need to see transactions?  Is it transactions for all users or just the data changes made by the user running the query?

I think we do want to look into storing transactions in proper database at some point in the distant (or maybe not so distant) future.  We might look at security for transactions at that point.  We won't plan any changes in security in this area in the current version of TM1 Server.

------------------------------
Stuart King
IBM Planning Analytics Offering Manager
------------------------------

#PlanningAnalyticswithWatson