Planning Analytics

 View Only
Expand all | Collapse all

PAW Git repository integration Issues

  • 1.  PAW Git repository integration Issues

    Posted Mon October 09, 2023 10:54 AM

    Dear Planning Analytics Community,

    I am using PAW Version 2.0.86 and tried to use the newly introduced feature to integrate Git for Planning Analytics database objects.
    During that process I have noticed a couple things that I'd like to address and get your opinion on:

    - tm1project.json
    This file is used to decide which Objects should be included and which ignored when committing/pushing to Git. In the new Git feature, where you can select what you want to commit over the GUI, many Control Cubes aren't included and there is no way to include them other than manually editing the tm1project.json (by TM1 API) right before committing the changes. This is hacky and surely not designed to work this way. Does anyone know the background why not all objects are considered in the GUI and if they plan to change this in the future?

    - Deletion/Removal of obsolete objects
    When pulling from a Branch to update my current model, objects that do not exist in the repository anymore are not being deleted in the model. Only changes and newly added objects are reflected in the model, which means that I still need to manually write down all the objects that I want to delete, then remove them manually. Any ideas how to accommodate this issue or if this will be improved in the future?

    - Author
    This might be an Azure DevOps-specific issue, but since I log in with an "admin" account to PAW and make my commits from that account, in DevOps the Author is simply displayed as "Admin" and doesn't provide the required information of who actually made the changes. Even though I provide an Azure E-Mail Address when prompted, but it's being ignored. Does anyone have any experience on this? I have not been able to find any info online.

    Generally, I am missing a lot of documentation/information on how the Git integration process is supposed to work... With lots of trial and error I have now been able to make some sense of it, but it still feels very shaky.

    Looking forward to hearing some inputs from the community and also those that have been using the Rest API and if you've noticed any differences.

    Thank you !



    ------------------------------
    ------------------------------
    Vedrana Tomic
    Business Partner
    PM Factory Consulting
    Vienna
    ------------------------------
    ------------------------------


  • 2.  RE: PAW Git repository integration Issues

    IBM Champion
    Posted Tue October 10, 2023 01:29 AM
    Edited by George Tonkin Tue October 10, 2023 01:37 AM

    Hi Vedrana,

    Not sure I can offer too much at this stage but you may want to try see if there is anything additional in Bob's video and this follow-up

    Yuri also posted some of his experiences and challenges here.

    @Svetlana Pestsova may be able to give some feedback on what is planned for future releases.



    ------------------------------
    George Tonkin
    Business Partner
    MCI Consultants
    Johannesburg
    ------------------------------



  • 3.  RE: PAW Git repository integration Issues

    IBM Champion
    Posted Tue October 10, 2023 02:12 AM

    Hi Vedrana,

     

    You're right, currently GIT integration via PAW GUI doesn't include the control objects migration. This is planned for future release afaik. There are certainly a lot of improvements and features that are yet be incorporated for this functionality and I would be hoping to see them being added in the upcoming releases but currently as it stands, it does the basic migration of the projects pretty seamlessly and for advanced options such as cube data, attributes and its values migration, you would rather want to consider sticking to the manual approach. And I guess the same also holds true to how the objects are updated frm GIT in Tm1 database. It appears to only append the objects in tm1 and not rewrite it.

     






  • 4.  RE: PAW Git repository integration Issues

    Posted Tue October 10, 2023 03:55 AM

    Hi Amin,

    Thank you for your insights.

    When you say "basic migration" what exactly do you mean by that? As in how is IBM defining which objects they include and which they don't.

    While I can understand that committing cube DATA is not exactly the point of a Git repository, I fail to understand why they haven't included ElementAttribute Structures (which define the attributes my dimension has, not the attribute values) or for example Cell Security Rules, which I both consider a crucial part of PA models.

    As for the "appending vs rewriting" logic, yes, that would also be my experience.

    Thank you!



    ------------------------------
    ------------------------------
    Vedrana Tomic
    Business Partner
    PM Factory Consulting
    Vienna
    ------------------------------
    ------------------------------



  • 5.  RE: PAW Git repository integration Issues

    IBM Champion
    Posted Tue October 10, 2023 05:35 AM

    By that I meant the skeleton objects like cubes and dims (excl. most control objects and websheets) including TI processes and chores.

     






  • 6.  RE: PAW Git repository integration Issues

    Posted Tue October 10, 2023 03:50 AM

    Dear George,

    Thank you for the links.

    Indeed, I have already combed through these posts and other resources provided by Bob and Yuri (which were very helpful in figuring out what's going on compared to the IBM documentation - big Thank You!) and my questions are the result of this research.

    I'm looking forward to some more opinions or maybe some other reports on the experience of working with the GUI Git feature.

    Thanks again!



    ------------------------------
    ------------------------------
    Vedrana Tomic
    Business Partner
    PM Factory Consulting
    Vienna
    ------------------------------
    ------------------------------



  • 7.  RE: PAW Git repository integration Issues

    Posted Tue October 10, 2023 09:08 AM

    Tm1py implements all the available features of the git integration, if you want to go beyond what paw offers right now. 



    ------------------------------
    Ryan Clapp
    ------------------------------



  • 8.  RE: PAW Git repository integration Issues

    Posted Tue October 10, 2023 09:15 AM

    Hi Ryan,

    Thanks for your reply.

    I have played with the idea of introducing TM1py, however, our customers are not tech-savvy enough to work with Python. I can expect them to use a GUI but I couldn't expect them to use TM1py.

    Additionally, I have no guarantee that TM1py is supported by IBM or working correctly (no accountability), which makes it almost impossible to introduce for customers.

    Thanks for your input anyway.

    BR; 



    ------------------------------
    ------------------------------
    Vedrana Tomic
    Business Partner
    PM Factory Consulting
    Vienna
    ------------------------------
    ------------------------------



  • 9.  RE: PAW Git repository integration Issues

    IBM Champion
    Posted Tue October 10, 2023 04:36 PM
    Edited by Marius Wirtz Tue October 10, 2023 04:36 PM
    Hi Vedrana,
     
    If your users are not tech-savvy developers, they might be more comfortable with a tool like Pulse for TM1.
     
    I'd also like to provide some additional context regarding TM1py accountability and correct workings.
     
    Regarding TM1py, while it's true that it's not officially backed by IBM, it's actively maintained and supported by a robust user and developer community. Contributors come from some of the world's largest TM1 customers, and both developers like myself and organizations like IBM and Cubewise actively contribute to its development. To assess its reliability and support, I encourage you to explore the user base, contributors, and response times to raised issues on GitHub.
     
    For those seeking additional support beyond community contributions, Cubewise offers TM1py Support, a commercial support service, which can provide the compliance and assistance your customers might be looking for. You can find more information about TM1py Support by Cubewise here: https://code.cubewise.com/tm1py-support-by-cubewise
    If you would like to know more about TM1py and its applications, I invite you to participate in the 24-hour TM1 conference next week: https://horizon.cubewise.com/series/Horizon-Agenda/schedule
    To focus the agenda on TM1py, please select "Free Online Pass" under Access and "TM1py" under Product.
     
    I hope this provides more insight into TM1py and its place in the TM1 ecosystem. If you have any more questions or concerns, please feel free to reach out.
     
    Marius Wirtz



    ------------------------------
    Marius Wirtz
    ------------------------------



  • 10.  RE: PAW Git repository integration Issues

    IBM Champion
    Posted Tue October 10, 2023 10:02 AM

    Hi Vedrana,

    Our initial review of the Git integration left us feeling underwhelmed, there were many missing object types. 

    I didn't notice the tm1project.json file, I might review this again to see if some of the missing control objects can be added in.

    This was our assessment;

    Git Hub Integration with IBM Planning Analytics
    Cubes    
      Cube Security No }CubeSecurity
      Data No  
      Dimension List Yes  
      Rules Yes  
      Views Yes  
    Dimensions    
      Attributes No }ElementAttributes
      Dimension Security No }DimensionSecurity
      Element Security No }ElementSecurity
      Elements Yes  
      Hierarchies Yes  
      Subsets Yes  
    Processes    
      Code Yes  
      Process Security No }ProcessSecurity
    Chores    
      Chore Security No }ChoreSecurity
      Processes Yes  
      Time Yes  
         
    Control Objects    
      Only } items that can be included is non-system items.    
      No }ElementAttribute No  
      No }ElementSecurity No  
      No }cubeSecurity No  
         
      Only    
      }Drill processes Yes  
      user defined } dims/cubes Yes  

    I also hoped to add additional files, like csv or txt files, readme files, etc.

    We've started looking at the REST API to perform some of the replication type functions.

    Stuart King gives a good demo here; 

    Lets hope we see some updates to the Integration soon.

    Thanks



    ------------------------------
    Karl Blackler
    ------------------------------



  • 11.  RE: PAW Git repository integration Issues

    Posted Wed October 11, 2023 08:25 PM

    FYI...



    ------------------------------
    Stuart King
    IBM Planning Analytics Offering Manager
    ------------------------------



  • 12.  RE: PAW Git repository integration Issues

    Posted Wed October 11, 2023 09:04 PM
    Thank you for sharing this again. I love the pre-push / post-pull features, most of the time we have to migrate dimensions and cubes (usually admin cubes where we store certain business rules) and I find these features very useful 







  • 13.  RE: PAW Git repository integration Issues

    Posted Thu October 12, 2023 08:42 AM
    Edited by Hubert Heijkers Thu October 12, 2023 08:49 AM

    Great to see Planning Analytics starting to take full advantage of the GIT integration functionality provided by TM1 (a.k.a. PAE), love it!

    Just wanted to add thought that, whilst people seem to be looking for capabilities to simply 'move objects across', the GIT integration functionality was designed for scenarios where you'd not be storing the materialized database but rather the source of such database.

    Whilst a replication scenario can be described as source, as Stuart shows in the above demo, there are many more scenarios that that could written up that source from other places, as you normally would already, and code this up in the source to allow for automation of deployment/standing up such database from scratch.

    Another part of that is the notion of deployment environments (deployments) which allows developers to individual specify things like the security used in dev vs. test and production , which data is get loaded in each of these environments or even what objects to be deployed or not for example. Trust that PA, now that files are tackled, will add support for different deployments as well at some point in the future Stuart? 

    PS Noticed a comment above wrt not removing objects. That was a deliberate choose when we designed and implemented the GIT integration some 5-6 years ago to allow for objects to exist, like the Bedrock library, without having to be part of the source itself. Given that mature database typically don't change as much one could add a post-pull task to clear any evidence of past versions to coupe with situations where things should be removed.



    ------------------------------
    Hubert Heijkers
    STSM, Program Director TM1 Functional Database Technology and OData Evangelist
    ------------------------------



  • 14.  RE: PAW Git repository integration Issues

    Posted Thu October 12, 2023 10:11 AM

    Hi Hubert,

    Is there any plan to include the PAW Content (Books, Dashboards etc) in GIT deployment packages (at least Book Specifications)?

    Thanks



    ------------------------------
    Ardian Alikaj
    ------------------------------



  • 15.  RE: PAW Git repository integration Issues

    Posted Thu October 12, 2023 10:43 AM

    Hi Ardian,

    Not sure what you mean by a 'GIT deployment packages' but I if you are asking if there are plans to include PAW contents in source for a TM1 database I'd expect the answer would be no, that seems unlikely, but more a question for Stuart.

    Are you looking for a GIT based 'solution' for a complete PA[W] deployment??? Which presumably could contain multiple TM1 databases, and containing contents that might be sourced from those TM1 databases as well as external sourced?



    ------------------------------
    Hubert Heijkers
    STSM, Program Director TM1 Functional Database Technology and OData Evangelist
    ------------------------------



  • 16.  RE: PAW Git repository integration Issues

    Posted Thu October 12, 2023 11:17 AM

    Hi Hubert,

    Today, in order to migrate PAW content, we have to use the Lifecycle Manager from PAW Administration. That process creates a compressed file with all selected objects, and then we import that file in the destination. My question was, since the GIT-IntegrationMigration for TM1 backend objects (including Websheets) is introduced in PAW, is there any plan to integrate this "lifecycle management" capability with GIT

    Thank you



    ------------------------------
    Ardian Alikaj
    ------------------------------



  • 17.  RE: PAW Git repository integration Issues

    Posted Thu October 12, 2023 01:40 PM

    No plan to integrate LCM from Workspace Admin with the GIT feature in the Workspace Modeling Workbench.  We plan to keep these as distinct features as they are commonly leveraged by different user roles.

    That being said, we are thinking longer term about version control for Workspace content.



    ------------------------------
    Stuart King
    IBM Planning Analytics Offering Manager
    ------------------------------



  • 18.  RE: PAW Git repository integration Issues

    Posted Thu October 12, 2023 10:00 AM

    The above video covers how to move element attributes and attribute values with pre-push and post-pull tasks in the upcoming PAW 2.0.91 release.  Note that our user interface filters out most control cubes and dimensions in the model.  This is intentional.  If you examine an }ElementAttributes dimension as a dimension you will notice that it's member types are numeric and string.  However, we know that there are really three attribute types (numeric, string, and alias).  It doesn't make sense to push the }ElementAttributes dimensions as dimensions, since this would cause alias attribute types to be string attribute types in the model you pull the dimension into.  

    That being said, we do realize that it's possible to have rules on }ElementAttributes and other control cubes (rules on security cubes are not uncommon).  We are thinking carefully about how the Workspace modeling user interface we will support rules from the control cubes in the future.   

    Hopefully this provides some insight into the current state and roadmap for the GIT feature in Workspace.

    As Hubert mentioned earlier, the GIT capability set in TM1 can do much more than what is exposed in Workspace.  We are working towards eventually consuming all of the capabilities provided by TM1. 



    ------------------------------
    Stuart King
    IBM Planning Analytics Offering Manager
    ------------------------------



  • 19.  RE: PAW Git repository integration Issues

    Posted Fri October 13, 2023 11:10 AM

    Stuart,

    In your very good video you mention sharing the GIT TI processes.  Where might we find these?  Very powerful feature for v12!

    Dean Watts



    ------------------------------
    Dean Watts
    ------------------------------



  • 20.  RE: PAW Git repository integration Issues

    Posted Tue January 02, 2024 02:28 PM

    Hi,

    When you want to include a Cube Rule in the GIT Package, the only option I see is to Select the Cube itself, which will all all the Cube Dimensions, Cube Views and Cube Rules related to selected cube. When I go to Target TM1 Server and Pull the objects from GIT Repository, it will pull the Dimensions also. But I don't want to update the dimensions in TEST environment from Dimensions in DEV environment.

    Is there a way to Include only the RUX files into the GIT Branch?

    Thank you



    ------------------------------
    Ardian Alikaj
    ------------------------------



  • 21.  RE: PAW Git repository integration Issues

    Posted Wed January 03, 2024 06:05 PM

    You can config the settings by unchecking "Select dependencies" and checking "Exclude cube views", then select the cube and push it.



    ------------------------------
    Larry Ding
    ------------------------------



  • 22.  RE: PAW Git repository integration Issues

    IBM Champion
    Posted Wed October 11, 2023 08:17 PM

    Hi @Vedrana Tomic, could you please provide more details about your requirements? Are you looking to use GIT for version control or automate deployments between servers? What objects/data would you like to include?



    ------------------------------
    Vlad Didenko
    Founder at Succeedium
    TeamOne Google Sheets add-on for IBM Planning Analytics / TM1
    https://succeedium.com/teamone/
    Succeedium Planning Analytics Cloud Extension
    https://succeedium.com/space/
    ------------------------------