Planning Analytics

 View Only
  • 1.  Planning Analytics Engine - Drill Through to AsciiOutput File

    Posted Thu October 19, 2023 06:13 AM

    Over in the blogs section I have been slowly adding new content around the Planning Analytics Engine, in one of those posts @George Tonkin raised a good point that we could potentially Drill Through to AsciiOutput files as part of the Debugging process.

    As many do, I tend to run a series of AsciiOutput statements into my TI Processes to test all manner of different conditions/ values/ logic etc..

    In the Planning Analytics Engine, we no longer have access to the underlying folder structure and as such any debugging will default to the file manager. We can add folders at will to the file manager but fundamentally we need to download the debug file to check it's contents.

    We could debug into a 'debug cube' but the point of AsciiOutput is it is quick and easy and flexible.

    George mentioned that we could use 'ReturnCSVTableHandle' which in turn relates to 'ExecuteRelationalDrillThrough' which can operate on an ODBC data source or a csv table

    The theory is that we could add an API call to the end of a development process to automatically pull up the details of an AsciiOutput file generated as part of the debugging process instead of continually accessing the file manager and downloading the files



    ------------------------------
    Edward Stuart
    ------------------------------


  • 2.  RE: Planning Analytics Engine - Drill Through to AsciiOutput File

    Posted Thu October 19, 2023 10:48 AM

    Success of sorts, jumped on a call with @George Tonkin and the drill through process was simple enough and worked as expected, I've not used "ReturnCSVTableHandle" before so it was a new one for me:

    "To support drillthrough to CSV, the TI function ReturnCSVTableHandle returns the handle of the CSV datasource. The REST API reads the datasource and converts the data to drillthrough rows."

    Drill.To.Debug.File Process:

    #Section Prolog

    DatasourceASCIIDelimiter = ',';

    DataSourceType = 'CHARACTERDELIMITED';

    DatasourceNameForServer = 'GetCubeDetails.txt';

    #Section Epilog
    ReturnCSVTableHandle;
    and a DrillThrough Rule referencing the above process
    [] = S: 'Drill.To.Debug.File';
    The resultant Drill Through returns:
    Drill to Debug File Example
    "The naming scheme of the CSV columns follows the Excel scheme: A, B, ... Z, AA, AB, and so on."
    This drill through to CSV/ text file works well for static files or for listing all files in a /Logs folder for example but the hope is to run a request to IBM Planning Analytics to immediately run this drill through to file to review an AsciiOutput.
    We tested running an ExecuteProcessWithReturn to the Drill Through process which returned successfully but did not open the grid window.
    We tested running ExecuteRelationalDrillThrough to the Drill Through process which returned successfully but did not open the grid window
    Subsequently I have tried ExecuteRelationalDrillThroughWithRowset and again, was successful but did not open the grid window
    For the time being I will make a ProcessLog cube with StatusCode and Body for HttpRequests to get this feedback and will continue to investigate the API calls to see if we can trigger the grid view which could give immediate feedback if a call has not been successful


    ------------------------------
    Edward Stuart
    ------------------------------



  • 3.  RE: Planning Analytics Engine - Drill Through to AsciiOutput File

    Posted Fri October 20, 2023 09:23 AM

    I tried using ReturnCSVTableHandle in a drill process, to allow end users to open an Audit file that gets created when they run an Action button that uploads data into a cube. The Audit file contains the lines that were skip and an error description so they can fix the data and load the file again.

    When I use ReturnCSVTableHandle, to return the Audit File through a Drill Process, it throws an error as below:

    Drill error

    Drill Process returned an error. The element or other information might be empty.
    And the file remains locked, so it seems like TM1 accesses the file but it fails to display it in the grid
    As an alternative, I have created a cube, with a few dimensions (Client, LOB, Business Function, Line Item, Measures), and the rejected rows are stored in the cube. Then in the Drill Process, we create a view with Data related to that Specific User/Business Function, and use RETURNVIEWHANDLE to return the View. This work fine, but even though we secure the LOB dimension, this is not enough. Our security model is more complicated, so users should not be able to see the data other users are loading. 
    We don's have a Security Group for each user, so we cant apply the security in the Client dimensions (which is a copy of }Clients dim)


    ------------------------------
    Ardian Alikaj
    ------------------------------



  • 4.  RE: Planning Analytics Engine - Drill Through to AsciiOutput File

    Posted Fri October 20, 2023 09:31 AM

    Hi Ardian,

    The drill process should be a fairly simple one, or is on my side:

    Prolog:

    DataSourceType = 'CHARACTERDELIMITED';
    DataSourceNameForServer=pFilename;

    Epliog;

    ReturnCSVTableHandle;

    pFilename in my case happens to be an element from a dimension containing TM1ProcessError logs.
    Performing the drill then passes the element to the parameter and that is used as the data source.
    I did remove the path statement I join as a prefix to pFilename to keep the example simple.
    I also removed a lookup to a cell where the user can specify the DatasourceASCIIDelimiter, again to keep the example simple.

    HTH



    ------------------------------
    George Tonkin
    Business Partner
    MCI Consultants
    Johannesburg
    ------------------------------



  • 5.  RE: Planning Analytics Engine - Drill Through to AsciiOutput File

    Posted Fri October 20, 2023 10:03 AM
      |   view attached

    Hi George,

    I just tried to create a very simple Drill Process as per your case and I still get the same error.

    As soon as I click on the Drill --> Open CSV File (as per screenshots attached) I get the error and the CSV File remains locked

    Ardian



    ------------------------------
    Ardian Alikaj
    ------------------------------



  • 6.  RE: Planning Analytics Engine - Drill Through to AsciiOutput File

    Posted Fri October 20, 2023 10:08 AM

    Have you tried this with an existing file that is created via a different process to avoid the file not being found because it is still a .csv$

    I also see that your path has forward slashes - are you on Linux? Have you tried backslashes?



    ------------------------------
    George Tonkin
    Business Partner
    MCI Consultants
    Johannesburg
    ------------------------------



  • 7.  RE: Planning Analytics Engine - Drill Through to AsciiOutput File

    Posted Fri October 20, 2023 11:22 AM

    Hi @Edward Stuart and @Ardian Alikaj

    Here are my 2 cents on how to enable users to see custom logs with the security model in mind, as Ardian mentioned: ...users should not be able to see the data other users are loading.

    Option 1: You can add the }Clients dimension to your custom logs cube and implement a cell cube security rule using the TM1USER function, something like this:

    []=S: IF( !}Clients @= TM1USER(), CONTINUE, 'NONE' );

    Option 2: If you are trying to read the system log files, indeed they are often locked by the running TM1 server. Your data upload process can write logs to a separate file for each user. Then you just pick that log file in your drill-through process based on the current user.

    Option 3: If you are considering a paid solution to enable non-admin users to see their own Audit logs (even Transaction and Message logs), it is implemented in SPACE: https://succeedium.com/space/features.html#audit-logs

    Actually, SPACE provides way more value than that. You can store your logs in an external database and search records without overloading your IBM PA\TM1 server. Why? Try searching for anything that happened a few weeks ago in your production environment (Transaction logs) and check the server CPU. Now imagine you have active users working with IBM PA data at the same time.



    ------------------------------
    Vlad Didenko
    Founder at Succeedium
    TeamOne Google Sheets add-on for IBM Planning Analytics / TM1
    https://succeedium.com/teamone/
    Succeedium Planning Analytics Cloud Extension
    https://succeedium.com/space/
    ------------------------------