Planning Analytics with Watson

 View Only
Expand all | Collapse all

REST API Data Load into IBM PA

  • 1.  REST API Data Load into IBM PA

    Posted Tue January 17, 2023 07:58 AM

    Our customer is evaluating API approach for loading and extracting data from IBM PA cloud. Currently we are loading data through csv files transferred to IBM FTP location. The idea is to  switch to REST API based integration.

    We are trying to understand;

    1. Is it feasible to to use REST api post method to update our sales cubes on a daily basis. Currently 3-4 csv files uploaded every day with avg size 1-2 mb , 15-20k rows maxing 300k rows (once in a while). How would be the performance with REST Api?

    2. Is "Cognos command center" the best choice for accessing API basd sources and orchestrating with other TI processes? Can we deploy Cognos command center to the same IBM PA cloud (without additional server) and access external REST Json sources and post it to IBM PA?

    Thanks in advance, would be great to hear your experience with REST based data integration on cloud.

    BR,
    James

    ------------------------------
    James Lavierre
    ------------------------------

    #PlanningAnalyticswithWatson


  • 2.  RE: REST API Data Load into IBM PA

    IBM Champion
    Posted Tue January 17, 2023 08:18 AM
    Edited by System Fri January 20, 2023 04:10 PM
    HI James,

    REST API will handle what you are looking to process. Have you had a look at TM1Py?
    This should do most of the heavy lifting for you and leaves you with the orchestration.

    In terms of orchestration, it depends. If you are on cloud, you already have CCC and could use it to perform various activities/actions.
    There are other options like Apache Airflow with significant options.

    I am sure others will be able to add further insights...

    ------------------------------
    George Tonkin
    Business Partner
    MCI Consultants
    Johannesburg
    ------------------------------

    Original Message


  • 3.  RE: REST API Data Load into IBM PA

    Posted Tue January 17, 2023 10:11 AM
    Hi George,

    Thanks for your answer. Have you experienced yourself with similar amount of daily data update using REST POST commands? Some says that putting all the data tuples into post body isn't feasible for that type of data. Is tm1PY better faster than REST API post action for cell update?

    BR,

    ------------------------------
    James Lavierre
    ------------------------------

    Original Message


  • 4.  RE: REST API Data Load into IBM PA

    IBM Champion
    Posted Tue January 17, 2023 10:17 AM

    Hi James,

    I do not have adequate use cases to give a definitive answer as my testing is largely in development environments.

    Ultimately whether TM1Py, Postman or some other language, you are always using a REST API post action.

    I am sure Marius Wirtz will add some commentary to this discussion as he has real-world experience and will likely be able to give you far more clarity than I can. We will both likely learn something!



    ------------------------------
    George Tonkin
    Business Partner
    MCI Consultants
    Johannesburg
    ------------------------------

    Original Message


  • 5.  RE: REST API Data Load into IBM PA

    IBM Champion
    Posted Tue January 17, 2023 10:57 AM
    TM1py has an optimized write mode that is faster than the current API functions. I'm using it the in the sample below.
    You can read more about the different write functions in TM1py here: https://cubewise-code.github.io/tm1py-tales/2021/write-with-tm1py.html

    Given that your dataset doesn't exceed 300k records, I wouldn't be concerned about performance or feasibility.

    Marius

    ------------------------------
    Marius Wirtz
    ------------------------------

    Original Message


  • 6.  RE: REST API Data Load into IBM PA