Planning Analytics

Planning Analytics

Get AI-infused integrated business planning

 View Only
Expand all | Collapse all

Multiple hierarchy and leaves hierarchy combination

  • 1.  Multiple hierarchy and leaves hierarchy combination

    Posted Wed September 30, 2020 05:53 AM
    Hi community,
    I am working with multiple hierarchy in one of my models and I am struggling to understand its behaviour against the Leaves hierarchy. In my case I have a dimension with one of 3 hierarchies called "Segment" there I have multiple summation items for example "SM3110". I create this item in TI process as a "C" type item but when I look at my Leaves hierarchy in the dimension I find this item there. My understanding is that this item should not turn up in the Leaves since it is a C level item. If I delete this item from Leaves hierarchy, then it is deleted from the dimension (as expected). 
    What is also strange is that this item has attributes that are as expected in the Segment hierarchy, but when I look at this item in the Leaves hierarchy there are no values on the attributes.
    I am a bit lost here where to look for explanations behind this behaviour, any thoughts/advice will be appreciated.

    Thank you for a great community, and thanks in advance for your assistance.
    Habbi

    ------------------------------
    Hallbjorn Bjornsson
    ------------------------------

    #PlanningAnalyticswithWatson


  • 2.  RE: Multiple hierarchy and leaves hierarchy combination

    Posted Thu October 01, 2020 03:29 AM
    You might be onto something here, may I ask if you actually associated any child elements (a.k.a. components) with that element that you gave the "C" type?

    I'll explain me asking; albeit not intended that way, it has always been possible to declare an element as being a "C" from the get-go and then never associate any children/components with it. IMHO it is then still a leaf (and if that's true that's apparently how it gets treated as well now that we have multi hierarchy support and make sure that all leaf elements are represented in the leaves hierarchy).

    ------------------------------
    Hubert Heijkers
    ------------------------------



  • 3.  RE: Multiple hierarchy and leaves hierarchy combination

    Posted Thu October 01, 2020 04:28 AM
    That is a good question and I have noticed this behaviour. It does not seem to matter if you create the element as C or N ... in the beginning it is a leaf and when you put some child elements under that element it becomes C in both cases. And to answer your question I have child elements in all cases for the elements that I see in my Leaves hierarchy.

    I have been looking for answers and one thought about this. Does it matter when or how you create the top level item in each hierarchy? ... could that somehow come into play here?

    regards,
    Habbi


    ------------------------------
    Hallbjorn Bjornsson
    ------------------------------



  • 4.  RE: Multiple hierarchy and leaves hierarchy combination

    Posted Thu October 01, 2020 05:12 AM
    It shouldn't matter IMHO, but if you are saying that in all your cases you have child elements/components underneath those elements you created as "C" initially, which isn't a crazy thing to do, then I'd say something must be wrong.

    Does this only happen only in particular cases or are you implying this happens consistently? Happen to have a simple example? Mind opening up a support case and let me know (privately) what case you created?

    ------------------------------
    Hubert Heijkers
    ------------------------------



  • 5.  RE: Multiple hierarchy and leaves hierarchy combination

    Posted Thu October 01, 2020 10:33 AM
    Hi,

    When you say you add it as a type "C" using a TI function, which one did you use?   If you use: DimensionElementInsert(DimName, InsertionPoint, ElName,ElType);  Then the element will exist in your primary hierarchy even if you later use HierarchyElementComponentAdd(DimName, HierName, ConsolidatedElName, ElName, ElWeight); to put children under it in a different hierarchy.   So if it exists in your primary hierarchy with no children, then it will be considered a leaf.   A "Leaf" just means you cannot travel farther down the rollup to more children, if you check the "type" of the element it still may be set to "C".

    If you want a consolidation to only exist in a specific hierarchy then you should use.  HierarchyElementInsert(DimName, HierName, InsertionPoint, ElName, ElType);

    Does this help, or am I off base on what you are asking about?

    ------------------------------
    Scott Brown
    ------------------------------



  • 6.  RE: Multiple hierarchy and leaves hierarchy combination

    Posted Fri October 02, 2020 03:57 PM
    Hi Hubert

    I am pretty sure that you said in the past you expected an element inserted as a 'C' to stay as a 'C' but I proved that a 'C' with no children becomes an 'N'. It is probably not the desirable behaviour but that is what currently happens, at least in 2.0.5.

    However, if someone has managed to insert an element as both a leaf in the Leaves hierarchy and as a consolidation in a Named Hierarchy then that looks like a bug.  I would say that is a bug regardless of whether the 'C' has any children below it, although in this case it apparently does. It should not be possible for the same element to exist with two types.

    Today I have got errors because I had an element that was a consolidation in a Named Hierarchy but because it did not exist in the Classic Hierarchy aka Default Hierarchy aka the one called the same as the dimension, the routine was trying to insert it as an 'N' and I got an error saying that it could not be inserted as a leaf because it existed as a consolidation. The only way to program around this was to loop through all Named Hierarchies to check if it was a 'C' and if so then insert it into the Classic Hierarchy as a 'C' instead of as an 'N'. Looping around all Named Hierarchies is probably slow and I have had to build my own cubes to do this. Is there some reason why all the functions that deal with Hierarchies in TI can only be used in Rules? Pretty much every other Rule information statement can be used in Rules and TI, bar the obvious like Skipcheck. That is something that I hope IBM fix soon. 

    Regards

    Paul Simon

    ------------------------------
    Paul Simon
    ------------------------------



  • 7.  RE: Multiple hierarchy and leaves hierarchy combination

    Posted Sat October 03, 2020 08:00 AM
    Hi Scott & Paul,

    I'll be the first to admit that we've had (and probably still have some) challenges in this area (this is me saying that I'm absolutely agreeing with you and we need to get any remaining issues fixed). At the same time, we've been making improvements over the various versions so it wouldn't be a bad idea to pick up the latest 11.8.x version (PA 2.0.9.1+) of TM1 if you have the chance. Also, I wouldn't expect users that are starting to use alternate hierarchies to mix the old, deprecated, Dimension functions with the new Hierarchy based functions. Albeit that the old Dimension once should only simply provide backwards compatibility and as such act on the same-named hierarchy but other than that work in a multi-hierarchy dimension as well.

    Paul, you seem to suggest then besides the old issue of one being to have a 'C' typed leaf element there being another issue with consolidated elements in multi-hierarchy cases. The problem with TI functions, vs using the REST API in which somebody presents the new state of a dimension, or just one hierarchy of a dimension, is that one would like to know if an individual function succeeds or not (as opposed to getting errors at commit time when we really know the user is done editing/building the dimension/hierarchy definition). This requires us to allow for intermediate 'states' to exist that, if they were the end-state, it would be invalid. This is an issue that only exists in TI, so happy to learn about patterns that are causing issues and we should look into.

    Paul you ask:  "Is there some reason why all the functions that deal with Hierarchies in TI can only be used in Rules?", not sure I understand that question or what functions you are referring to here. Are there functions you are missing that tell you about things specific to hierarchies or some that you would want to use in rules that you can't today? Can you elaborate?

     


    ------------------------------
    Hubert Heijkers
    ------------------------------



  • 8.  RE: Multiple hierarchy and leaves hierarchy combination

    Posted Sat October 03, 2020 08:50 AM

    Hi Hubert

     

    Thanks for the reply.

     

    We are currently evaluating 2.0.9 but other work has delayed progress on that. The Finance Dept decided they would rather have a Non Trial Balance collection system which a colleague has built, and a chart of accounts change request system which I have built. This uses a JavaScript Framework (Vue & Node) front end, a SQL Server backend, and various TI processes to update the PA 2.0.5 dimensions.

     

    Finance Depts are risk adverse. At present we are using 2.0.5 in pretty much the same way that we used TM1. We are still using TM1 Web in Production. TM1 Web is important for us as many users are in partner organisations outside of our network.

     

    We are not using PAW. Until it is possible to synchronise selections with TI buttons, we can't use PAW.

     

    We are not using PAX, although we are getting close to resolving some of the SSL certificate issues, and hope to be using it soon. At present only developers can use it.

     

    I have built the Chart of Accounts system to support Named Hierarchies. However, we cannot simply change several years of development or retrain 300 users who are used to TM1 Web. At present, until issues with PAW are fixed there is not, in any case, a viable alternative. Therefore all the TI that I have written to build dimensions from the SQL database that holds the Chart of Accounts information, all replicate the structure of each Named Hierarchy, as an alternate hierarchy in the default hierarchy. I have stuck to the standard that alternate hierarchies only exist in the default hierarchy, and any Named Hierarchy only has a single hierarchy. However, we do need to generate some alternate hierarchies to support existing development and systems.

     

    When you say

    "I wouldn't expect users that are starting to use alternate hierarchies to mix the old, deprecated, Dimension functions with the new Hierarchy based functions"

    I think that the reality is that there will be continued use of both functions, because a lot of people will need to update existing dimensions that are accessed by tools that don't support named hierarchies.

     

    There are some areas where there is not an equivalent hierarchy function. When writing in attribute values if I want to check whether the cell is updateable or if a rule applies to it, I need to use CellIsUpdateable on the Element Attributes Cube, and specify Hierarchy colon Element , then the Attribute, as there is no equivalent of this function with a hierarchy parameter.

     

    Reading from attribute cubes with named hierarchies is a problem, as you can only do this using an MDX View, and TI does not read text from MDX Views correctly. I reported this bug a year or so ago and as far as I know it has still not been resolved.

     

    I may be wrong but I cannot see that many people converting from TI to using the Rest API. I would guess that the majority of all TM1 developers come from an Accountancy rather than IT background. I would guess that they will struggle to get to grips with the Rest API. I am from an IT background and the design of the Rest API is very computer science oriented. It is probably more of a tool for partners than end-users. I also suspect that there will also be cases where TI runs faster than eg Python or JavaScript calling the RestAPI simply because it runs in process.

     

    If you look at the TM1 Reference guide on the IBM Knowledge centre, which should reflect the latest version, and then look at the Hierarchy Rules Functions, which contain things that would be useful in TI to allow a process to loop around all hierarchies in a dimension, all of them say "cannot be used in TurboIntegrator processes. The presence of this function in a process will prevent the process from compiling. ".

    Pretty much any other Rules Function can be used in a TI Process.

     

    Regards

     

     

    Paul

    Paul Simon

    Success Cubed Ltd

    Tel: +44 7941 506 197

    www.successcubed.co.uk

    Skype Name: paulsimongb

    Skype Tel: +44 20 3287 4616

     

     

     






  • 9.  RE: Multiple hierarchy and leaves hierarchy combination

    Posted Tue October 06, 2020 05:32 AM

    Hi Paul,

    I realized this 'dualistic' world is a tricky one and, as you say, the modern components in the box that do support hierarchies aren't 'there' yet and you are using hierarchies while still building for an audience that essentially is stuck in a pre-alternate-hierarchies world that's indeed a bit challenging.
    I can only promise you that on the TM1 server-side we are ironing out the quirks as best we can, hoping that you'll be able to go to a hierarchy enabled 'world' soon. 

    And yes, there are TI functions for which no hierarchy aware version necessarily exists. For most this is true because of the positional nature of the parameters to those functions for which we introduced alternative syntax that allows you to intersect elements from multiple hierarchies of a dimension (most notably the DB function). CellsIsUpdateable is such a positional function, one where, intersected cells not being updateable, you'd only have to specify one element. If that is a consolidated element and you do have multiple hierarchies that you'll have to specify from which hierarchy it came from (unless it only exists in one hierarchy???).

    You write: "TI does not read text from MDX Views correctly", that should be fixed in the latest (11.8.x) version of TM1 already.

    As for "converting from TI to using the Rest API", that wasn't what I was advocating (albeit there are some fantastic cases where that would make sense;-).  I was trying to say that in modern TM1, if you forget the old APIs/clients, TI is the only 'environment' in which some of these 'tricky' situations still exist where the intent is not clear as things happen but only can be concluded ultimately as things unfold vs. the REST API where the 'end-state' is being presented as the new state to be set. Here too we are looking, not in the last place due to a case raised by Hallbjorn in the meantime, in improving this further (where some part of the improvement might imply better documentation). So don't worry, not going to convert finance users into programmers;-).

    Hierarchy Rules Functions, got it, yeah, those 4 functions tell you something about the cell that is being evaluated by the rule. I.o.w. those indeed do not have a context in a TI process and therefore make no sense to be used as such. Is there functionality you are missing that you thought you should be able to use those functions for?

    Cheers,




    ------------------------------
    Hubert Heijkers
    ------------------------------



  • 10.  RE: Multiple hierarchy and leaves hierarchy combination

    Posted Sun October 11, 2020 11:31 AM

    Hi Hubert

     

    Many thanks for your reply.

     

    One thing that I find very confusing about IBM with Planning Analytics is its version numbering. We have 2.0.5 in production but you are talking about 11.8. To a certain extent 11.8 makes more sense in terms of the logical progression of TM1 version numbers. I have been using TM1 since version 6. However, for some reason when IBM changed the name to Planning Analytics they decided to revert to version 2.0.0.

     

    Actually identifying what version number you are on, is actually very difficult, particularly when IBM have two version numbering systems for the core server, and another version numbering scheme for PAW and PAX.

     

    Can I make a plea that IBM sticks to one version numbering system.

     

    How do I actually find out what version number in the 11 series I am actually on? Is it by looking at the version number on the tm1sd.exe reliable?

     

    On a server with version number 2.0.8 installed this is showing 11.6. Therefore it is presumably 2.0.9.3 or something like that which is version 11.8? If it was just a matter of swapping the 2 for an 11 it would be easier but the minor version numbering is not even in step.

     

    If I go to download PA the version numbering still shows 2.0.9.3, you have to click a couple of times into the detail to find that it contains TM1 Server version 11.8.2. However, with PAW and PAX now being on different release schedules to the server, what actually is in a Planning Analytics version apart the server? Can IBM please revert to and standardise on 11 series numbering for the server. If PAW and PAX are numbered differently that is fine, since they potentially have a different update cycle to the server.

     

    Anyway good to know that the issue with TI handling Text from MDX Views is fixed as at present the work around of looping round attributes is considerably slowing down dimension updates.

     

    I don't understand what "not in the last place due to a case raised by Hallbjorn" means? Is this a code word for a new version or another customer?

     

    You asked for a use case for Hierarchy Functions in TI as opposed to rules. There are numerous cases where I have functions that loop over all hierarchies in a dimension. If I could use these hierarchy functions in TI then this would be possible, but without them, the only way of doing this is to maintain my own cubes of the hierarchies in each dimension. I have built a dimension where there are consolidations for each dimension expanding to leaf level elements of the dimension:hierarchy combinations in the dimension to allow me to do this. I use this in a cube to hold information on the hierarchy such as its top level element, the name of its base level, etc. However, not everyone wants to build their own cubes for this.

     

    One example is the difficulty of deleting an element. In general, it must be deleted from the classic default hierarchy, one or more Named Hierarchies, and possibly also the leaf hierarchy. Unless you delete it from all cases, it is likely to work its way back in at the next dimension update. If you can write a TI that loops around all named hierarchies in a dimension, then this is relatively easy, but this ability is not currently supported by native TI functions.

     

    If the hierarchy functions where available in TI instead of just rules I could write a generic TI process that takes a parameter of the Dim and Element to be deleted from all hierarchies.

     

    IF( Dimix( vDim , vElem ) > 0 ) ;

      DimensionElementDelete( vDim , vElem ) ;

    ENDIF ;

    # Also delete from all Named Hierarchies, if the dimension has Named Hierarchies

    IF( Hierarchy( vDim ) = 1 ) ;

      vNumHiers = HierarchyCount( vDim ) ;

      vHierNum = 0 ;

      WHILE( vHierNum < vNumHiers ) ;

        vHierNum = vHierNum + 1 ;

        vHier = HierarchyN( vDim, vHierNum ) ;

        IF( HierarchyElementExists( vDim , vHIer, vElem ) = 1 ) ;

          HierarchyElementDelete( vDim , vHier , vElem ) ;

        ENDIF ;

       END ;

    ENDIF ;

     

    And at some point HierarchyIndex could be of use.

     

    Another example is that we have a set of standard subsets that we want to create for each hierarchy. If we want to add another subset to this standard we need a way to loop around all dimensions and all hierarchies within those dimensions. At present we create subsets for base level elements, consolidations, full hierarchy and a default subset being the top level element. In }ElementAttributes cubes, we also want to be able to create a default view for each hierarchy, which again requires the ability to loop for each hierarchy in a dimension.

     

    So I think that these functions do have a context in a TI process.

     

    The new Named Hierarchies do add a lot of functionality to TM1, and they are definitely a good thing. However, like anything new it is difficult to think of every possible ramification in advance.

     

    Regards

     

    Paul Simon

     

     

     






  • 11.  RE: Multiple hierarchy and leaves hierarchy combination

    Posted Mon October 12, 2020 11:07 AM
    Hi Paul,

    So what you are really trying to tell me here is that it has always been 'hard' to iterate anything in TM1 but typically you had a way around it, often by iterating elements of some control dimension. With the introduction of hierarchies, due to its internal implementation, we've not only neglected to give you a way to iterate the hierarchies of a dimension that way but made it harder to simply iterate the dimensions at the same time? Correct?

    So I get your ask, to offset that 'gap' you'd like to have those functions to be available in TIs as well, albeit their implementation would have a different 'context'! I.e. HierarchyCount wouldn't return the number of the dimension's hierarchies currently in the context of the rule execution, but rather return the number of hierarchies that that particular dimension actually has. To keep things similar the Hierarchy function would only return a hierarchy if the specified dimension had only one (as opposed to having only one of them in the context, a much more common case). HierarchyN and HierarchyIndex would return the Nth hierarchy of the specified dimension and that Nth position of the hierarchy respectively.

    And don't get me wrong, not disagreeing at all! But presuming my assumption here is right, wouldn't you agree we should have similar functions for Dimensions, Cubes and other object types as well???

    As for "Hallbjorn", he's the guy that started this thread ;-)

    As for versioning... don't get me started. Only thing I'll say is that Planning Analytics is the 'offering', which comes in various shapes, an offering which is powered by TM1 (see: https://www.ibm.com/nl-en/products/planning-analytics). TM1 (server) is a component in various of these 'packages' but I'll be the first to admit that Planning Analytics Local smells like the 'powered by TM1' only portion (albeit that all traditional clients did get rebranded). And since not every version needs to pick up a new version of TM1 while at the same time they decided to introduce a new digit in the version number, starting with 2.0.9.0, there is no fixed mapping between the version either indeed. 

    Cheers,

    ------------------------------
    Hubert Heijkers
    ------------------------------



  • 12.  RE: Multiple hierarchy and leaves hierarchy combination

    Posted Mon October 12, 2020 04:50 PM

    Hi Hubert

     

    Having to use a WHILE loop to iterate is not too bad. A FOR loop might be nicer or even a FOR EACH, but it can be done with a WHILE. I spend half my time these days programming in JavaScript and sometimes too many options is not a good thing. I accept that TI is a scripting language not a full programming language so I lower my expectations. The only feature I really miss is the ability to call a function and get a return value. Other than that TI has the basics of any programming language sequence, selection and iteration.

     

    I don't think IBM have made it any harder to iterate over dimensions. I have been doing this for years to iterate over elements in Dimensions, and Dimension Subsets, components of consolidations, parents or elements, etc.

     

    However, at present iterating over the Hierarchies within a Dimension is impossible in native TI. You have to build work around cubes and dimensions to do it.

     

    You can ignore the Hierarchy() function. As a bare minimum I would just need:

     

    HierarchyCount() to return the number of hierarchies in the dimension. The documentation would need to specify whether this included the Classic Default Hierarchy and / or the Leaf Hierarchy.

     

    HierarchyN() to return the Name of the Hierarchy of the given dimension at the given Nth position.

     

    Please have a word with someone at IBM about versioning. When we log a support call we actually have to wait for instructions on how to tell them what version we are on, and it generally involves delving into the file system and sending over a bunch of files. What is so difficult about having one number for TM1 and one for PAW/PAX/TM1 Web? IBM needs to learn to become a more customer centric company.

     

    Regards

     

    Paul Simon

     

     

     






  • 13.  RE: Multiple hierarchy and leaves hierarchy combination

    Posted Tue October 13, 2020 06:40 AM
    Hi Paul,

    If there is an obvious reason why this wouldn't work apologies.

    Each dimension already has an associated }Hierarchies_Dimname dimension, doesn't this already give you everything you need?

    HierarchyCount= DimSiz(}Hierarchies_Dimname) +1;  (I don't think it lists the all leaves hierarchy) and obviously you can step through the hierarchies once you have the count using Dimnm.

    It's not as explicit as the functions you've asked for but, in theory, this should work?

    ------------------------------
    Steven Rowe
    ------------------------------



  • 14.  RE: Multiple hierarchy and leaves hierarchy combination

    Posted Tue October 13, 2020 06:11 PM

    Hi Steve

     

    The problem is that IBM have said that all these extra curly bracket dimensions that they have put in for hierarchies, subsets, views, etc, might not all be staying, so I have deliberately not used any of them.

     

    Regards

     

    Paul Simon

     






  • 15.  RE: Multiple hierarchy and leaves hierarchy combination

    Posted Wed October 14, 2020 05:17 AM
    Hmm, OK that would explain your request for the functions.  How strong is the info on those dims being dropped?
    We use them in a lot of our code base,  I don't really understand why they would be dropped, the more of the guts of TM1 that are exposed to developers the better.
    There doesn't seem to be an value in dropping them, especially as they have been in the product for a number of years now.
    Would be good if someone from IBM can give some clarification.

    ------------------------------
    Steven Rowe
    ------------------------------



  • 16.  RE: Multiple hierarchy and leaves hierarchy combination

    Posted Wed October 14, 2020 07:56 AM
    Hi Steven,

    Albeit that message did not come from me I'll take you up on saying something about it. I have nothing against exposing the 'guts' of TM1 but IMHO it should be at least one level of abstraction between the actual 'implementation' and the interaction/consumption of such [meta-]data. I.o.w. I'd prefer to expose an object model which you could script against and which you could use to enumerate any and all objects as opposed to you going off and interpreting, in this case, elements of dimensions, as the definition of such objects. 

    With that said, knowing what we are looking to do in the next couple of major versions of TM1, I wouldn't be too surprised if changes (and a major cleanup) would occur in the true system 'tables' area, but I'll promise that we'll offset that with new capability to allow you to continue to do what you need to do and potentially make things easier.
    But who knows, like with many bad design decisions from the past, we might have to live with them indefinitely (like the name of the elements in these hierarchies dimensions which, instead of just the hierarchy new use the name of the internal dimension representing the hierarchy<sigh>) ;-)

    ------------------------------
    Hubert Heijkers
    ------------------------------



  • 17.  RE: Multiple hierarchy and leaves hierarchy combination

    Posted Wed October 14, 2020 12:32 PM
    Hi Hubert,
    Thanks for the update, hope all is well with you in these exciting times?

    As far as the object model goes, I think I can understand that from your side of the fence that having the available hierarchies stored in a dimension as members is little ugly.  Totally agree with the object model being one step away from the 'guts', that way you can change the 'guts' without breaking everyones references to the object model.

    From my side of the fence the dimension holding the list of hierarchies gives us that abstraction and crucially since it is a TM1 object we already have a complete set of functions and so forth that allows us to interact with it in a well understood way.

    I'm guessing / hoping the changes to the way the object model is presented to developers is quite far down the road but a break in the backwards compatibility of the object model could be a significant barrier to upgrading for many sites.
    Cheers,

    ------------------------------
    Steven Rowe
    ------------------------------