From some of the discussions, webinars and AMA's there are some major changes coming in terms of MDX in the next generation server e.g. how security will be interpreted when executing MDX.

What I am trying to find out is what other changes are coming our way that we need to consider in terms of migration, best practice, upskilling etc.

Are TM1 functions like TM1FilterByLevel, TM1FilterByPattern etc. going to be deprecated in favour of standard MDX using intrinsic properties like ELEMENT_LEVEL, LEVEL_NUMBER and functions like INSTR()?

@Hubert Heijkers - would be great if you have something you could share with us on this please.

Thank you.​

------------------------------
George Tonkin
------------------------------

#PlanningAnalyticswithWatson

Hi George,

These are very useful functions, I don't think they are to be removed or replaced as they are high performance as well.

Best Regards,

------------------------------
Svetlana Pestsova
IBM Planning Analytics Product Manager
------------------------------

Original Message:
Sent: Thu October 28, 2021 03:46 AM
From: George Tonkin
Subject: TM1 specific MDX function support in the Next Gen version

From some of the discussions, webinars and AMA's there are some major changes coming in terms of MDX in the next generation server e.g. how security will be interpreted when executing MDX.

What I am trying to find out is what other changes are coming our way that we need to consider in terms of migration, best practice, upskilling etc.

Are TM1 functions like TM1FilterByLevel, TM1FilterByPattern etc. going to be deprecated in favour of standard MDX using intrinsic properties like ELEMENT_LEVEL, LEVEL_NUMBER and functions like INSTR()?

@Hubert Heijkers - would be great if you have something you could share with us on this please.

Thank you.​

------------------------------
George Tonkin
------------------------------

#PlanningAnalyticswithWatson

Hi George,

TM1 12's MDX query engine is the same as the one you are familiar with from TM1 v11, albeit that we made some performance improvements, fixed some long overdue, typically security-related, issues, got rid of some potential ambiguous cases (in which case you have to be potentially more explicit) and somewhat more strict (most notably, you can't put just any garbage at the end of an MDX query/set expression and continue to get away with it - this should always have been a syntax error).

There is however work on the way for later versions, later than 12.0, focussing on a vastly improved MDX query and execution engine, as well as a brand new core query engine, one that would no longer depend on feeders. Those however will take a bit longer to deliver and might indeed not even see the light of day until the real 'next-gen' TM1 server (read: TM1 12 at its core is just TM1 as you know it, just architecturally vastly matured to be able to run in a containerized environment and able of providing High Availability when running in a Kubernetes based - in IBM's world that means CloudPak on Redhat Open-Shift - environment).

So whilst I would never ever introduce functions like TM1FilterByLevel and TM1FilterByPattern again if I had to design this again, none of these are being removed in TM1 12 (can't make promises about future, next-gen, versions though;-). Any performance advantage, presuming there would still be some in TM1 12, is purely because we've been very bad in optimizing the execution plan for the query. In de new MDX engine we are working on there will absolutely be no difference between expressing these a filter, as one should IMHO, vs using function. Better yet, using the filter will likely allow us to even more optimize query execution depending on the rest of the MDX and the use of those filtered values.

PS seem to have missed that original notice George, apologize for the delayed response. 

Cheers,

------------------------------
Hubert Heijkers
------------------------------

Original Message:
Sent: Thu October 28, 2021 03:46 AM
From: George Tonkin
Subject: TM1 specific MDX function support in the Next Gen version

From some of the discussions, webinars and AMA's there are some major changes coming in terms of MDX in the next generation server e.g. how security will be interpreted when executing MDX.

What I am trying to find out is what other changes are coming our way that we need to consider in terms of migration, best practice, upskilling etc.

Are TM1 functions like TM1FilterByLevel, TM1FilterByPattern etc. going to be deprecated in favour of standard MDX using intrinsic properties like ELEMENT_LEVEL, LEVEL_NUMBER and functions like INSTR()?

@Hubert Heijkers - would be great if you have something you could share with us on this please.

Thank you.​

------------------------------
George Tonkin
------------------------------

#PlanningAnalyticswithWatson

Thanks for the feedback Hubert. Sounds promising as there will be zero disruption in migrating to Next Gen/V12.

I think personally, I will try and rather use more standard MDX e.g. INSTR rather than relying on TM1FilterByPattern etc.

------------------------------
George Tonkin
------------------------------

Original Message:
Sent: Fri November 12, 2021 03:30 AM
From: Hubert Heijkers
Subject: TM1 specific MDX function support in the Next Gen version

Hi George,

TM1 12's MDX query engine is the same as the one you are familiar with from TM1 v11, albeit that we made some performance improvements, fixed some long overdue, typically security-related, issues, got rid of some potential ambiguous cases (in which case you have to be potentially more explicit) and somewhat more strict (most notably, you can't put just any garbage at the end of an MDX query/set expression and continue to get away with it - this should always have been a syntax error).

There is however work on the way for later versions, later than 12.0, focussing on a vastly improved MDX query and execution engine, as well as a brand new core query engine, one that would no longer depend on feeders. Those however will take a bit longer to deliver and might indeed not even see the light of day until the real 'next-gen' TM1 server (read: TM1 12 at its core is just TM1 as you know it, just architecturally vastly matured to be able to run in a containerized environment and able of providing High Availability when running in a Kubernetes based - in IBM's world that means CloudPak on Redhat Open-Shift - environment).

So whilst I would never ever introduce functions like TM1FilterByLevel and TM1FilterByPattern again if I had to design this again, none of these are being removed in TM1 12 (can't make promises about future, next-gen, versions though;-). Any performance advantage, presuming there would still be some in TM1 12, is purely because we've been very bad in optimizing the execution plan for the query. In de new MDX engine we are working on there will absolutely be no difference between expressing these a filter, as one should IMHO, vs using function. Better yet, using the filter will likely allow us to even more optimize query execution depending on the rest of the MDX and the use of those filtered values.

PS seem to have missed that original notice George, apologize for the delayed response. 

Cheers,

------------------------------
Hubert Heijkers

Original Message:
Sent: Thu October 28, 2021 03:46 AM
From: George Tonkin
Subject: TM1 specific MDX function support in the Next Gen version

From some of the discussions, webinars and AMA's there are some major changes coming in terms of MDX in the next generation server e.g. how security will be interpreted when executing MDX.

What I am trying to find out is what other changes are coming our way that we need to consider in terms of migration, best practice, upskilling etc.

Are TM1 functions like TM1FilterByLevel, TM1FilterByPattern etc. going to be deprecated in favour of standard MDX using intrinsic properties like ELEMENT_LEVEL, LEVEL_NUMBER and functions like INSTR()?

@Hubert Heijkers - would be great if you have something you could share with us on this please.

Thank you.​

------------------------------
George Tonkin
------------------------------

#PlanningAnalyticswithWatson

Moving to TM1 12 will still require a migration, albeit most of those because of the environmental changes made by this move to a more cloud-native architecture. I.o.w. the model remains the same, the environment it runs in not so much. But we are using this opportunity as well to clean house [just] a bit ;-).

As for TM1FilterByPattern, that's one of the few, if not the only filter operation you can't easily do otherwise. The thing I don't like about all these functions TM1 added is that they require a SET parameter and therefore implicitly impose another FILTER as opposed to supplying a function that can be used in the filter expression, but it's good you remind me, we've got pattern-based filtering added to the REST API's $filter expressions, we should also introduce an TM1 style pattern filter operation to be used in filter expressions in MDX as well!

------------------------------
Hubert Heijkers
------------------------------

Original Message:
Sent: Fri November 12, 2021 03:44 AM
From: George Tonkin
Subject: TM1 specific MDX function support in the Next Gen version

Thanks for the feedback Hubert. Sounds promising as there will be zero disruption in migrating to Next Gen/V12.

I think personally, I will try and rather use more standard MDX e.g. INSTR rather than relying on TM1FilterByPattern etc.

------------------------------
George Tonkin

Original Message:
Sent: Fri November 12, 2021 03:30 AM
From: Hubert Heijkers
Subject: TM1 specific MDX function support in the Next Gen version

Hi George,

TM1 12's MDX query engine is the same as the one you are familiar with from TM1 v11, albeit that we made some performance improvements, fixed some long overdue, typically security-related, issues, got rid of some potential ambiguous cases (in which case you have to be potentially more explicit) and somewhat more strict (most notably, you can't put just any garbage at the end of an MDX query/set expression and continue to get away with it - this should always have been a syntax error).

There is however work on the way for later versions, later than 12.0, focussing on a vastly improved MDX query and execution engine, as well as a brand new core query engine, one that would no longer depend on feeders. Those however will take a bit longer to deliver and might indeed not even see the light of day until the real 'next-gen' TM1 server (read: TM1 12 at its core is just TM1 as you know it, just architecturally vastly matured to be able to run in a containerized environment and able of providing High Availability when running in a Kubernetes based - in IBM's world that means CloudPak on Redhat Open-Shift - environment).

So whilst I would never ever introduce functions like TM1FilterByLevel and TM1FilterByPattern again if I had to design this again, none of these are being removed in TM1 12 (can't make promises about future, next-gen, versions though;-). Any performance advantage, presuming there would still be some in TM1 12, is purely because we've been very bad in optimizing the execution plan for the query. In de new MDX engine we are working on there will absolutely be no difference between expressing these a filter, as one should IMHO, vs using function. Better yet, using the filter will likely allow us to even more optimize query execution depending on the rest of the MDX and the use of those filtered values.

PS seem to have missed that original notice George, apologize for the delayed response. 

Cheers,

------------------------------
Hubert Heijkers

Original Message:
Sent: Thu October 28, 2021 03:46 AM
From: George Tonkin
Subject: TM1 specific MDX function support in the Next Gen version

From some of the discussions, webinars and AMA's there are some major changes coming in terms of MDX in the next generation server e.g. how security will be interpreted when executing MDX.

What I am trying to find out is what other changes are coming our way that we need to consider in terms of migration, best practice, upskilling etc.

Are TM1 functions like TM1FilterByLevel, TM1FilterByPattern etc. going to be deprecated in favour of standard MDX using intrinsic properties like ELEMENT_LEVEL, LEVEL_NUMBER and functions like INSTR()?

@Hubert Heijkers - would be great if you have something you could share with us on this please.

Thank you.​

------------------------------
George Tonkin
------------------------------

#PlanningAnalyticswithWatson