Platform

Platform

A place for Apptio product users to learn, connect, share and grow together.

 View Only
  • 1.  Processing large datasets

    Posted 05/24/18 04:47 PM

    Used the hosted datalink to get azure billing data. Totally easy to set up, love it. It downloaded 121 MB a couple of hours ago. The table showed up within about 20-30 minutes, but with nothing in it. I assume it's still working on the data. The calc queue has a complete record from about the time the download finished. Will it insert another when the table is accessible?

     

    What kind of performance can I expect when uploading large quantities of data? I'm really just curious so I don't keep checking every few minutes.

     

    Oh. What if I check the table out before there's data in it?





    #Platform


  • 2.  Re: Processing large datasets

    Posted 05/25/18 03:56 AM

    Hi, we have several large datasets and schedule the uploads for 8pm so that the calculations run overnight. some of our clac times have been 6-8 hours. It might be worth looking at the training for project optimisation as there are a lot of factors that influence calculation time. contact the trainers like @Debbie Hagen to get more help on which training would be best.


    #Platform


  • 3.  Re: Processing large datasets

    Posted 05/25/18 10:59 AM

    Thanks, Gary. I can see we're going to have to do the same thing. There's still no data there and the calc queue is empty. I submitted a support ticket this morning since it's been over 18 hours.

    Having said all of that, I'm definitely looking into the optimization. I saw the class you're referencing and want to take it because we could stand to streamline our calculations with what we already have.

    Happy Friday!


    #Platform


  • 4.  Re: Processing large datasets

    Posted 05/25/18 03:55 PM

    Update - the table landed last night, it took about 6 hours to process in. The reason I didn't see it this morning is because the data is for April and my project was set to March.

    IN MY DEFENSE.... I am an idiot. Thank you.


    #Platform


  • 5.  Re: Processing large datasets

    Posted 05/29/18 03:07 AM

     We've all been there ... In R11 I also loaded into the wrong environment and wondered where all my data had gone...


    #Platform


  • 6.  Re: Processing large datasets

    Posted 10/07/18 10:42 AM

    Sometimes it's all about Allocation Strategy used which impacts the overall performance of the system and calculation time.. 


    #Platform