We all might have differing opinions about running to the store to pick up items. But one thing is for sure -- when we have a busy day, we would rather get everything in one trip vs. having to go home and then turn around and go back for more items and “burn more daylight”.
The same can be said when automating data collection.
IBM Rapid Network Automation can be used in a number of different ways to automate your busy day, including automating tasks such as taking structured data files (I was working with Ericsson MME XML files) and turning them into JSON objects, before ingesting them into IBM’s SevOne NPM performance monitoring solution.
While you can get this job done in a workflow, there are definitely some best practices to improving the efficiency and performance of a workflow, especially when API endpoint calls are involved.
One of the ways you can improve your workflow’s performance (think of our shopping example) is to get more work done by sending more data to the API endpoint with a single request (go to the store once) vs. calling the API more frequently to send smaller chunks of data (go to the store multiple times).
This can often be accomplished by restructuring your workflow logic to bundle up the data into a single payload when possible (objects loaded into an array) and then sending the data on to the API endpoint once at the end of your workflow.
Consider this workflow: There is a list of “Items_I_need”. For each item in my list, if I build the payload containing just one of the items and then “go_to_the_store”, I will end up having to go through the same process many times, thus hitting my “go_to_the_store” api endpoint a lot.
Now, if we restructure things a bit so that we first build our shopping list in its entirety, then we only need to make one “go_to_the_store” API endpoint call, thus being much more efficient.
As a real-world example reveals, when I ingest my XML data into SevOne one object type at-a-time using a “For Each” automation action block loop, it takes 78 API endpoint requests. If, instead of ingesting the data one object type at a time, I bundle up my payload to include all data for all object types, I can get it done in one API endpoint request (one trip to the store instead of multiple trips).
The proof is in the pudding: You can see in the output below that while you can get the job done by taking 78 trips, there is network and processing latency introduced when you do it on a per object type basis (26.6 seconds) vs. when you get it done in one trip with a single API request (16.1 seconds). You may say, “So you saved 10 seconds -- big deal...” When collecting data, there are typically many devices to collect from, so with data volume, seconds matter.
So, next time you are constructing your automation workflows, consider how you organize things, and look for ways to make fewer trips to the store!