Planning Analytics

 View Only
Expand all | Collapse all

View types in TM1 / PAW with inconsistent order results

  • 1.  View types in TM1 / PAW with inconsistent order results

    Posted Fri December 08, 2023 03:57 AM

    After the last upgrade of PAW (Q4 2023), I experienced errors in Ti processes related to the view order when using the TM1 cube view as a data source. 

    We are dealing with different behavior depending on where the view is created.

    1. Created outside PAW, using PM or Architect
    2. Created, using the PAW workbench, or "Save to server" in a PAW book.
    3. Created dynamically as a temporary view in the TI process. 

    Below is a list of some view-related problems that are not necessarily related to the latest versions of PAW. 

    Problem I, wrong order of dimension members
    Views created in PM or Architect do not resolve the data in the expected dimension order. 
    This results in a problem for example if we need the data in a specific order in the result sheet.

    Problem II, wrong order of cube dimensions (parameters) 
    Views created in the PAW Workbench do not show the correct order of dimensions when used as a data source in a TI process.
    When using some PAW view as a base and then defining a new temporary view in the process code the order of the cube dimensions does not match. The only workaround I know of is to create a view with PM and use that view as a base in the TI process. This view will have the same parameter order (and names) as the temporary view. 

    Problem III, Naming confusion
    The term view is used for different objects, and that creates confusion when talking about the PAW environment. 

    ---

    Example of saving from a PAW book. PAW view vs server view

    Example of (server) view types displayed in the PAW workbench. 

    ---

    What is the definition of a view in the TM1/PAW environment?

    What is the difference between TM1 views when saved in PAW or PM?
      I am guessing that the view object created from PAW is an MDX object. 
      In PAW Workbench, if I modify some view that was created in PM, then its behavior changes resulting in problems as listed above.

    Why do interface views use the same term (view) as server views?

    What is the best practice when using views in a TI process? 

    My approach would be using temporary views and temporary subsets because then nothing can be tampered with by any user. 
    I am now confused as I have experienced problems and conflicts as mentioned above. 

    Please advise.

    Thanks, Asgeir



    ------------------------------
    Asgeir Thorgeirsson
    ------------------------------


  • 2.  RE: View types in TM1 / PAW with inconsistent order results

    Posted Fri December 08, 2023 04:06 AM

    HI Asgeir,

    There was a similar discussion here: https://www.tm1forum.com/viewtopic.php?p=82007#p82007

    The main issue is that views have never been consistent - if you create a view via TI, it adheres to the order per dimensions seen in PAW/Architect.

    If you create a view in Architect/Perspectives, save that and try and use it in a process, dimensions would be based on title, rows and columns order.

    MDX is different again as it is also going to be based on what is on axis 0, axis 1 and in your Where clause (titles/context)i.e. similar to user views in architect/perspectives.

    Not sure there is a best practice here but ideal scenario would be that programatically, TI reads any cube view using the dimension order it was created with.



    ------------------------------
    George Tonkin
    Business Partner
    MCI Consultants
    Johannesburg
    ------------------------------



  • 3.  RE: View types in TM1 / PAW with inconsistent order results

    Posted Fri December 08, 2023 08:30 AM

    I keep meaning to investigate further and have wondered if there is scope to create a generic script/ process to convert all "View" to be the same type.

    When using the REST Api to query a view we will see two types:

    • ibm.tm1.api.v1.NativeView
    • ibm.tm1.api.v1.MDXView

    I've been fortunate not to hit your issues as yet so have not delved further but it does raise questions as to what happens when a Cubes dimensions are re-ordered

    There is some information on NativeViews in the documentation: https://www.ibm.com/docs/en/planning-analytics/2.0.0?topic=data-cubes-native-views so some further testing will be required



    ------------------------------
    Edward Stuart
    ------------------------------



  • 4.  RE: View types in TM1 / PAW with inconsistent order results

    Posted Sun August 25, 2024 11:30 PM

    I am having this problem, too. Did you ever find a good solution? I thought I had solved it by creating the source view in Architect, although it's not a great solution since Architect is going away. Now I'm getting the same behavior even with views created in Architect. Makes writing TI processes incredibly frustrating!



    ------------------------------
    Almond Eastland
    ------------------------------



  • 5.  RE: View types in TM1 / PAW with inconsistent order results

    Posted Mon August 26, 2024 01:21 AM

    HI Almond,

    When creating view in Architect for use in TI, do not open a cube view, manipulate this and save as this will likely cause order differences as the order will be based on how you configured the view. 

    Rather create a view from TI as a base or use the Export as text data function to allow you to create a view. This will ensure the order is aligned to what you see in the cube dimensions.

    For MDX views, the order changes based on what you have placed on the axes and what remains on the slicers (the Where clause).  This is likely to change based on your MDX query.



    ------------------------------
    George Tonkin
    Business Partner
    MCI Consultants
    Johannesburg
    ------------------------------



  • 6.  RE: View types in TM1 / PAW with inconsistent order results

    Posted Mon August 26, 2024 09:11 AM

    Thank you, George.  It's pretty frustrating that we have to add another step and can't rely on the TI preview anymore.



    ------------------------------
    Almond Eastland
    ------------------------------



  • 7.  RE: View types in TM1 / PAW with inconsistent order results

    Posted Mon August 26, 2024 09:31 AM

    Hi Almond,

    If you are using an MDX view as a data source and don't want to risk users rearranging the position of hierarchies in the view causing the TI process to break then you should create a temporary view in the prolog section of the TI process and assign the temp view as the data source.  This can be done with the ViewCreateByMDX TI function.  Use '1' as the last parameter so the view is temporary (only exists in the context of the TI process).  Part of the reason we have a feature to expose the MDX statement for a view in PAW is so the MDX statement can be copied from the cube viewer into a TI process.   

    There is a reason why Native and MDX views do not work the same in TI processes.  Native views simply use the (external) order of dimensions of the cube to source the variable values.  MDX views cannot work the same way since it's possible to have more or less hierarchies on the view than the number of dimensions in the cube.  In modern versions of PAW the TI editor is smart enough to know if you are working with a Native view or MDX view and apply the correct preview order.  The preview order will match the order of the variables are sourced when the TI process is executed.  Unless you are using a much older release of PAW this shouldn't be an issue.



    ------------------------------
    Stuart King
    Product Manager
    IBM Planning Analytics
    ------------------------------



  • 8.  RE: View types in TM1 / PAW with inconsistent order results

    Posted Mon August 26, 2024 10:33 AM

    I would like to preface this by saying I am a business person, not an IT person so there may be technical terms/nuances I don't understand. I have been doing all our PA development since our initial implementation in 2019, but I rely heavily on existing processes and Google.

    I am on PAW version 2.0.94, and it's messing up the preview regardless of whether I'm using a Native (assuming Native means the static/default views created in Architect) or MDX view. We don't use hierarchies.

    We already do use the Prolog to create the actual source view; the only thing the "Default" view is used for is to specify the variables in the preview, but the processes no longer work because the actual dimension order doesn't match the preview. Is there some way to specify the variable names without using the Data source tab? The only way I've been able to get the last few processes I've created to work is by using Architect to create both the source view and the process. This worries me since we're about to lose Architect. So far, our old processes seem to be working fine, but I don't know if they're about to break, too. Starting from a copy of an existing process is no longer working in PAW.

    If I'm understanding the solution correctly, the only way to create a view that will show up correctly in the Preview is to move the source view creation to a separate process, which is adding an extra step to every single new TI process. I don't understand how changing my source views to use the ViewCreateByMDX function will make them available in the Preview since it seems they won't have been created yet. If they're created by a separate process and are temporary, they similarly won't be available in the Preview. What am I missing?

    I also saw a discussion about using .csv files as the source, which again, adds steps (now I have to create an export process, too). Plus, csv files have the same problem in PAW. Unlike Architect, they don't show up in the Preview; you just have to know the correct order. Should I just stop relying on the Preview altogether? 



    ------------------------------
    Almond Eastland
    Financial Analyst
    Oklahoma City OK
    ------------------------------



  • 9.  RE: View types in TM1 / PAW with inconsistent order results

    Posted Mon August 26, 2024 11:49 AM

    Hi Almond,

    The preview feature in the Ti editor in PAW was updated to support both native and MDX views long before the 2.0.95 release.

    What version of TM1 are you using (e.g 2.0.9.19)?  There was a defect in TM1 that could cause the order of variables in MDX views to be inconsistent.  Based on your comments I expect this might be the issue.  This could cause the preview to not match the order of variables for MDX views.  The TI editor has already worked correctly for Native views (yes, native views are the views saved from Architect and Perspectives).

    Can you show us an example of how the preview variables feature is not working as you expect?

    Note that as of the 2.0.98 release of PAW we only ever save MDX views back to TM1 (using the save to server option option in the cube viewer toolbar).  However, we are going to add the ability to save both MDX and native views from the PAW cube viewer to TM1.  This will likely happen for the 2.0.99 or 2.0.100 release on October or November this year.  This is an important feature for our customers as native views are requires for drill throughs and typically preferred for performance reasons as data sources in TI. 

    Architect, Perspectives, and TM1Web only display native views.  PAW and PAfE will display both MDX and native views.  In PAW we added different icons in the model tree to help differential between MDX and native views.



    ------------------------------
    Stuart King
    Product Manager
    IBM Planning Analytics
    ------------------------------



  • 10.  RE: View types in TM1 / PAW with inconsistent order results

    Posted Mon August 26, 2024 12:42 PM

    My Dev environment is on TM1 2.0.9.19 and PAW 2.0.94, and the Prod environment is on 2.0.9.15 and PAW 2.0.87. We will be upgrading Prod to match Dev later this week, but I am having the same problem in both environments. In the below example, yDefault is a static view created in Architect. Same problem if I use a view created in PAW.

    In Preview, Version appears to be the first dimension (as it is in the source cube), so I name the variable accordingly. However, when I run the process, I get errors including the following message, clearly indicating the actual underlying dimension order does not match the preview:

    Invalid key: Dimension Name: "Period", Element Name (Key): "Working"



    ------------------------------
    Almond Eastland
    Financial Analyst
    Oklahoma City OK
    ------------------------------



  • 11.  RE: View types in TM1 / PAW with inconsistent order results

    Posted Mon August 26, 2024 02:30 PM

    Hi Almond,

    I see something odd in your screenshot.  When you click the preview button do you continue to see the 'Preview title' show V1 to V9?  The preview button should suggest the 'preview title' as the names of the dimension.  We actually have a bit of work to do in this area so that it shows dimension name and hierarchy name when alternate hierarchies are used.

    I really suspect this problem is happening because you had a native view and overwrote it with an MDX view.  You can tell which is which by examining the icons in the model tree.

    We will look into an enhancement in the TI editor to make it easier to understand which view type was selected.



    ------------------------------
    Stuart King
    Product Manager
    IBM Planning Analytics
    ------------------------------



  • 12.  RE: View types in TM1 / PAW with inconsistent order results

    Posted Mon August 26, 2024 02:49 PM

    It is not an MDX view (see screenshot). Yes, the screenshot is exactly how it looks when I click Preview. This behavior is not new (although the mismatching is). If the dimension name is more than one word, it does not not display in the Preview. This has been the case for me for as long as I've been developing in PAW. Here's another example from one of our original processes. There are no MDX views in this source cube, and the source view has not changed:



    ------------------------------
    Almond Eastland
    Financial Analyst
    Oklahoma City OK
    ------------------------------



  • 13.  RE: View types in TM1 / PAW with inconsistent order results

    Posted Mon August 26, 2024 09:48 PM

    Hi Almond,

    We will prioritize an improvement to display the dimension name (with spaces removed) as the preview title instead of using V1, V2, etc.  If the view is MDX then we will display the dimensionname_hierarchyname when a hierarchy with a different name than the dimension is used. 

    Is it a cellputn or cellputs function that is failing in the TI code?



    ------------------------------
    Stuart King
    Product Manager
    IBM Planning Analytics
    ------------------------------



  • 14.  RE: View types in TM1 / PAW with inconsistent order results

    Posted Wed August 28, 2024 09:21 AM

    In this case, it was a cellputn, but since the dimensions were out of order, I'm guessing a cellputs would have failed as well.



    ------------------------------
    Almond Eastland
    Financial Analyst
    Oklahoma City OK
    ------------------------------



  • 15.  RE: View types in TM1 / PAW with inconsistent order results

    Posted Tue August 27, 2024 08:02 AM

    Hi Stuart,

    We are on PAW 2.0.97 and I have experienced this same behavior in some TI processes.  Mainly when I create the process from scratch in PAW, but also some existing processes.  We create the temp code on the Prolog tab, with most, not all, dimensions using MDX.  But then when running the process, we'll get errors that dimensions are not found because it's trying to put elements into the wrong dimensions.  Only work around I've found is creating a source view in Architect.  

    Are you saying we should create the view on prolog as not temp first, then use that view as the TI source so dimensions will be in the correct order, then go back to temp view on prolog?  Sounds like you're saying creating from scratch in 2.0.97 it should not happen, but I have seen it and had to correct it with an architect source view.  



    ------------------------------
    Terry Hamlet
    ------------------------------



  • 16.  RE: View types in TM1 / PAW with inconsistent order results

    Posted Tue August 27, 2024 08:10 AM

    Hello !

    The solution was already done by on of the community experts....

    you have to create a MDX view  named (for sample Z_Extract), use this view as datasource in your TI process and, in prolog construct your working view as you like (viewcreate, wiewssubsetassign, etc ....)

    your Z_Extract view must respect :

    << All views in vorkspace are per definition MDX views. MDX views assign index order of variables based on the view structure.
     1st column dimensions (MDX axis 0), 
    then row dimensions (MDX axis 1), 
    then slicer dimensions.
     So if you constructed your view with all dimensions on columns in the display order of the cube then this would be the easiest
     way to solve the problem. 
    Assuming the temp view which gets created with TI is a trad view. Still seems rather fiddly to me. >>

    I am sorry, I don't remember the original redactor of this marvellous solution

    Regards,

    Philippe

    PS) be carefuul if you have "sand boxe as dimension parameters actived", it will be more difficult



    ------------------------------
    Philippe CHAMPLEBOUX
    ------------------------------



  • 17.  RE: View types in TM1 / PAW with inconsistent order results

    Posted Tue August 27, 2024 08:59 AM

    Thanks Philippe!

    Just to clarify, the order of variables doesn't change between temporary views and permanent views.  Temporary views are often preferred as they only exist within the scope of the TI process, which means other users cannot update the view and break the TI process that uses the view as the data source.

    The order of variables does change depending on a native or MDX view being used as a data source.  Native views always use the external order of dimensions in the cube (the order you see when you look at the dimensions under the cube in the user interfaces).  The order of variables when an MDX view is the data source is what you described.  This order is hierarchies (not dimensions) on the columns, left to right, then rows, left to right, and finally the slicing axis, left to right. 

    Note that both native and MDX views can be temporary or permanent.  

    I also want to reiterate your point that PAW only saves MDX views back to TM1.  PAW does not support saving native views back to TM1...yet.  We are building this feature now and it will release this year (probably October).

    @Terry, I think you are describing native views that are created with subsets that are defined using MDX.  This is not the same as an MDX view.  The only way to create an MDX views in your TI code is to use the CreateViewByMDX function.  The CreateView function only creates native views, even if MDX based subsets are used in the view. 

     



    ------------------------------
    Stuart King
    Product Manager
    IBM Planning Analytics
    ------------------------------