Planning Analytics

Planning Analytics

Get AI-infused integrated business planning

 View Only
  • 1.  TM1 Git - ignore .github directory

    Posted 3 days ago

    Has anyone found a meaningful way to ignore the .github directory when managing database assets with git?

    The TM1Project file only enables you to ignore files within the TM1 Source directory directly.  It seems the only other file that can automatically be updated is the README.md file as noted below.



    ------------------------------
    Roman Harasymiak
    ------------------------------


  • 2.  RE: TM1 Git - ignore .github directory

    Posted 2 days ago

    Hi Roman,

    Can you elaborate a bit more on your ask? The ignore, or not ignore, section in the project file indeed only applies to objects in your project/model.

    As it stands only the README.md file, if it exists, is retained, otherwise only the model content is pushed on any subsequent push. We are in the process of adding two additional capabilities, one of which would allow you to selectively push only changes made to selective objects and, secondly, allow for additional files to be included in the GIT repository used for a TM1 model/project. The latter would allow you to retain such .github directory, which I for one needed to retain my GitHub action definitions for the repository, but also other files like data pipelining definitions, editor specific setting (.vscode directory for example) etc. etc. as well.

    I.o.w., reading your request above I wondered; how did you get a .github directory and where would that be (as it stands TM1's GIT integration would ignore it) and, presuming you deliberately added it to the repository to begin with, why would you want to ignore it (as it stands any push from TM1 would remove it anyway - which is what we'll prevent from happening in the future).



    ------------------------------
    Hubert Heijkers
    STSM, Program Director TM1 Functional Database Technology and OData Evangelist
    ------------------------------



  • 3.  RE: TM1 Git - ignore .github directory

    Posted 2 days ago

    Hi Hubert,

    We are manually adding/re-adding the .github directory post each push to maintain a CODEOWNERS file, so that we can have an approval mechanism for each PR to deploy thru our pipeline from DEV-->QA-->STAGE-->PROD.

    I think the second option you mention would work for us, is there an ETA on availability on v11?  Like you mentioned that directory get's removed each time we do a push to the repository, and we need to manually perform another commit to add it back as our repository rules requires a review from CODEOWNERS prior to merging.

    We're trying to think of alternatives to manually repush a global copy via our deployment scripts, but something built in would be preferred.



    ------------------------------
    Roman Harasymiak
    ------------------------------



  • 4.  RE: TM1 Git - ignore .github directory

    Posted 2 days ago

    Hi Roman,

    I've been doing the same in my (demo) environments, resetting the last commit, undoing the delete of the '.github' folder and pushing the commit once again.

    There is currently no ETA for this, but, with more customers starting to looking at a 'database as code' approach and automating the deployment of such databases applying CI/CD principles, it is on the shortlist.

    Cheers,



    ------------------------------
    Hubert Heijkers
    STSM, Program Director TM1 Functional Database Technology and OData Evangelist
    ------------------------------



  • 5.  RE: TM1 Git - ignore .github directory

    Posted 2 days ago

    Hello, I'm also currently working on implementing Git as a VCS for PA.

    I'm not sure I understand the ask but Github does allow you to ignore specific files in your repository. You'll have to create a ".gitignore" file at the root of the repo. In my case, working on macos, my repo was filled with the os system files (.DS_Store).

    We do however intentionally keep the ".github" folder as it contains the script that syncs the repo with the Dev server (through "Gitpull") after each commit.

    It's good news that IBM is still working on further improving PA/Git integration, I can't stand shutting down UAT, drag & dropping files from Dev and restarting the service like cave people...



    ------------------------------
    Paul C
    ------------------------------



  • 6.  RE: TM1 Git - ignore .github directory

    Posted 2 days ago

    Hi Paul,

    GIT integration, currently limited to TM1 (read: there is no GIT integration for PA assets), is vital for anybody that wants to use GIT for change tracking (embracing a 'TM1 database as code' mentality) and even more so for anybody that wants to automate deploying changes to TM1 models using CI/CD type methods. Sounds like you are there already.

    Both Roman and myself have ran into the situation where we'd like additional files in our repos (the repos that contain the 'code' for the TM1 database like yours), only to find that if you use GitPush from TM1 to create/update a branch would remove all those files in the commit it creates. That's what the work I'm referring to above is going to prevent and essentially going to leave those files alone. Your '.gitignore' file is yet another example of a file which, if you'd GitPush from TM1 after making some changes, would end up being removed as well (which neither of us want to happen going forward).

    Just out of curiosity, are you making all your changes directly to the source and only pulling (GitPull) into TM1? 



    ------------------------------
    Hubert Heijkers
    STSM, Program Director TM1 Functional Database Technology and OData Evangelist
    ------------------------------



  • 7.  RE: TM1 Git - ignore .github directory

    Posted 2 days ago
    Edited by Paul C 2 days ago

    Ah ok I get it,

    In your current system you develop directly on the Dev TM1 instance and Push/Pull to UAT right? It is different from what we're planning to implement indeed (not sure we're doing the right thing though).

    In our system, we plan on developing the model directly in VScode and pushing the changes directly in Dev for unit testing, so basically the "Dev Repo" is always synced with the Dev environment. For UAT/Prod deliveries, we'll use another branch or repo that will contain only the changes validated by the business team.

    The reason we're planning on developing the model directly on VScode is because we struggle with concurrent developements on the same objects. The dev team is a bit large, the changes are many and it's impossible to keep track on who is modifying what. A VCS system would allow to detect conflicts and merging parallel dev on the same object.

    There's still a lot of work and testing to be done before implementing that solution, we'll have to add some pre-commit scripts to check the file validity and develop a language git extension (found one on the marketplace but it's a bit old and it only applies to the server files like .pro and .rux, not the representations created by "TM1git" such as .ti or .rules)

    I did notice some strange behavior though in my tests. If I "illegally" manually modify an object in Dev directly from PAW ti editor, my next gitpull doesn't detect the difference with the git repo and that modification stays.



    ------------------------------
    Paul C
    ------------------------------



  • 8.  RE: TM1 Git - ignore .github directory

    Posted yesterday

    Love it!

    Yes, in my ideal picture, no different then how one develops software essentially (are anything-as-code for that matter I'd imagine), you'd use a protected branch from which you deploy to (at least) production and you'd use feature/work branches to get particular new things developed, reviewed, presumably tested and approved before those get merged to that protected branch after which a final CI/CD pipeline would apply changes into your production environment.

    Note that this is completely different then what your Planning Analytics friends apparently imagined when they designed their UI for GIT integration which I'd argue is more like movement of objects using GIT as storage (not exactly what we, the TM1 team, had in mind, nor what one would expect for typical CI/CD pipelines). This applies a pattern where you'd end up using orphaned branches in one repo (which is perfectly fine for this purpose), but you'd never merge those branches in some primary branch as you'd loose all other objects in the model that wouldn't be part of the particular 'package' you created on such branch.

    But I see you are all aligned with the envisioned pattern already so that's great!

    Making all model changes directly in VSCode (my preferred tool as well) is perfectly valid as, as you've found out already, using one TM1 instance to make changes in requires knowing what all developers are doing and, until we add the support for selective push, doesn't allow you to push the work of one separate from others, and that is even presuming that different developers aren't making changes to the same objects, which they often end up doing (adding elements to the same dimension, adding/updating rules of the same cube, fixing/extending/amending code in the same TI process, etc. etc.). For that I've seen customers stand up separate TM1 instances per developer, all sharing the same code repository, but taking advantage of the deployments that can be specified in your project whereby they only stand up a minimal/light weight version of the main model for each. Especially now with TM1 v12, where standing up a new database is only one REST call away, its become super easy to do this as well as doing the same for testing purposed, for each and every branch one has, to do some automated testing.

    Finally, you are right that we don't overwrite objects, even if you changed them, if no changes were made to them in the source. However, there is a 'Force' parameter in GitPull (I know the documentation is very minimalistic on this topic) that should allow you to do what you want to do. Also you can use the UseForce() function in your preconditions for a pull to check the value that was specified for that Force parameter (default is false).



    ------------------------------
    Hubert Heijkers
    STSM, Program Director TM1 Functional Database Technology and OData Evangelist
    ------------------------------