IBM Apptio

 View Only

 Multiple Datalink check-ins timing

Jump to  Best Answer
  • CostingStandard(CT-Foundation)
  • Datalink
Apptio Community Member's profile image
Apptio Community Member posted 05/02/22 10:12 AM
I'm hoping someone has a clever answer to this situation ... 

We have our default Datalink (Classic) agent which handles all of our web and cloud based connections as well as an on-premise agent for internally-produced files.  Most of the data is refreshed Sunday nights but since there are two agents, they check in consecutively. I would like to reduce the number of check-ins so that we don't have a long queue of data updates waiting for the previous load to finish calculating, when it will just have turn around and do calculations again - but I need to keep the process automated. Any suggestions?
#Datalink
#CostingStandard(CT-Foundation)
Rene Norskov's profile image
Rene Norskov  Best Answer
In a similar situation I have pushed files to folders on Datadrop for the first group and then extended the second group to pick up files in addition to running "its own" connectors.  Datadrop then functions as a temporary holding area and ensures there is only one check-in / one calculation.
#CostingStandard(CT-Foundation)
Jenny Franklin's profile image
Jenny Franklin

Excellent question, @Robert Krauss!  I'm curious as well.  I haven't dabbled too much with the new Datalink, but wondering if there's a way to do that there.  Just adding to your question. ;)  Hopefully someone can hop on an enlighten us! 😊​


#CostingStandard(CT-Foundation)
Apptio Community Member's profile image
Apptio Community Member
Hi Robert, do you have these connectors running in groups or are they running individually? I have 11 connectors that run in a group and I believe they all check in at once.
#CostingStandard(CT-Foundation)
Apptio Community Member's profile image
Apptio Community Member
Hi @Jessica Matos - they are in connectors but since some are under the default agent and some are under the on-premise agent, the best I can get to is two connector groups.​
#CostingStandard(CT-Foundation)
Apptio Community Member's profile image
Apptio Community Member
Hi Robert,
not certain if this will work but have you tried enabling check in on just the last connector that fires? I can't be certain if that checks in just the respective table or the entire dev space that gets created by the DL jobs.
Worth a try?
Regards Mark
#CostingStandard(CT-Foundation)
Apptio Community Member's profile image
Apptio Community Member
@Jenny Franklin - in regards to the new Datalink, I can confirm that enabling check in for a connector only works on the table that is the target.

Example scenario:
Connector 1 -> Table A (check in disabled)
Connector 2 -> Table B (check in enabled)

Only Table B gets checked in, regardless of the ordering of the connector jobs. It's a tad annoying as we have multiple connectors running daily so a multitude of check ins occur overnight. They are all "daisy-chained" (great functionality), so one job completes before the next commences. Thankfully our stage calc times are under 3 hours so I typically find when I start the day the last build is pretty much winding up.​

Not sure on groups here - I was not able to use this functionality due to limitations with other aspects of the new Datalink and workarounds I had to come up with to manage these.
#CostingStandard(CT-Foundation)
James Funk's profile image
James Funk
You could just disable Staging on Friday evening before you log off, then turn it back on on Monday morning.

Admittedly, this "solution" has some drawbacks (it's not automated since you have to manually turn it off and on again, it means that Staging would need to start calculating when you turn it back on on Monday morning rather than on Sunday so that the data is ready to go on Monday morning, etc.), but if your goal is simply to reduce the number of Staging calcs that your environment has to perform, then I think this would help.
#CostingStandard(CT-Foundation)
Apptio Community Member's profile image
Apptio Community Member
@Rene Norskov - that's an interesting idea. I'll have to play around with the concept but it sounds like what I want to do.​
#CostingStandard(CT-Foundation)