“Are my runbooks improving?” or “What feedback did I get for the runbooks?“ could be question you are asking yourself from time to time. After all, the quality of the runbooks impacts your operational efficiency. In this blog entry, we’ll show you how to leverage the Runbook Automation API within Cloud Event Management, to get that information in a quick and nice way.

The Goal

In our little scenario we will be looking at:

• The ratings of runbooks over time
• The ratings of runbooks across versions
• The amount of comments we get

The API

After we established our goals, we can look at the API. By reading the documentation we can easily see where we need to look:

• GET /runbookinstances will let us retrieve all the data
• The following fields from the output are interesting: comment, rating, _runbookId, _runbookVersion
• We want our call to include the filter options type=manual and status=completed to only get records which are actually finished and may have a rating/comment from an actual user.

For this example, we will be looking at the data from this year. This article was written on 2019-07-09, so we will specify the timeframe 2019-01-01 to now. We can use the parameters from and to to achieve that. By taking all of this into account, our final call looks like this:

curl -X GET "https://rba.mybluemix.net/api/v1/rba/runbookinstances?from=2019-01-01&status=complete&type=manual&fields=_runbookId%2C_runbookVersion%2Crating%2Ccomment" -H "accept: application/json" -H "Authorization: Basic XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX=="

Here is an example of how the output may look like:

[ { "_runbookId": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa", "_runbookVersion": 1, "rating": 3 }, { "_runbookId": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa", "_runbookVersion": 1, "rating": 2, "comment": "Step 2 didn't work" }, { "_runbookId": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa", "_runbookVersion": 1, "rating": 2, "comment": "Hard to follow" }, { "_runbookId": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa", "_runbookVersion": 2, "rating": 4 }, { "_runbookId": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa", "_runbookVersion": 2, "rating": 5, "comment": "Previous problem has been solved" }, { "_runbookId": "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb", "_runbookVersion": 1, "rating": 3, "comment": "Ok" }, { "_runbookId": "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb", "_runbookVersion": 1, "rating": 4 }, { "_runbookId": "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb", "_runbookVersion": 1, "rating": 2 }, { "_runbookId": "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb", "_runbookVersion": 1, "rating": 4 }, { "_runbookId": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa", "_runbookVersion": 0, "rating": 4 }, { "_runbookId": "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb", "_runbookVersion": 0, "rating": 4 }, { "_runbookId": "ccccccc-cccc-cccc-cccc-cccccccccccc", "_runbookVersion": 0, "rating": 4 } ]

Post-Processing the Data

This data is nice and all, but we want our result to be easy to read, so we do some fancy JavaScript work. After all, the API already gave us a good result, with only the fields we care about. For this example we set our own rules. Yours may vary, but some of those might be something you want to do as well:

• Ignore all entries from drafts
• Group entries by runbook and then by version
• Count the executions
• Calculate the average rating
• Display all the comments

After the processing our data is nice and tidy. It looks this way:

{
"aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa": {
"comments": [
"Step 2 didn't work",
"Hard to follow",
"Previous problem has been solved"
],
"versions": {
"1": {
"executions": 3,
"ratings": [
3,
2,
2
],
"ratingaverage": "2.3"
},
"2": {
"executions": 2,
"ratings": [
4,
5
],
"ratingaverage": "4.5"
}
}
},
"bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb": {
"comments": [
"Ok"
],
"versions": {
"1": {
"executions": 4,
"ratings": [
3,
4,
2,
4
],
"ratingaverage": "3.3"
}
}
}
}

The Post-Processing Script

If you are interested in how we did it, here is the node.js JavaScript code for you. It’s not that difficult, if you already understand JavaScript:

"use strict";

const fs = require("fs");
// and we assume the data is stored in a file named data.json
let data = JSON.parse(fs.readFileSync("./data.json", "utf8"));
//console.log(data);
let result = {};

// Iterate through all data
for(let record of data) {
//console.log(`> ${JSON.stringify(record)}`);
if(record._runbookVersion == 0) {
//console.log("Version = 0, skipping");
// Skip all draft executions, identified by version number 0
continue;
}
// Initialize the data structure by runbookId, if it’s not there, yet
result[record._runbookId] = result[record._runbookId] || { "comments" : [], "versions" : {} };

// Store the comment, if it is available
if(record.comment) {
result[record._runbookId].comments.push(record.comment);
}

// Store rating and perform the necessary calculations for number of executions and average rating
result[record._runbookId].versions[record._runbookVersion] = result[record._runbookId].versions[record._runbookVersion] || { "executions" : 0, "ratingsum" : 0, "ratings" : [] };
result[record._runbookId].versions[record._runbookVersion].executions += 1;
result[record._runbookId].versions[record._runbookVersion].ratingsum += record.rating;
result[record._runbookId].versions[record._runbookVersion].ratingaverage = Number.parseFloat(result[record._runbookId].versions[record._runbookVersion].ratingsum / result[record._runbookId].versions[record._runbookVersion].executions).toPrecision(2);
result[record._runbookId].versions[record._runbookVersion].ratings.push(record.rating);
}

console.log(JSON.stringify(result, function(key, value) { if(key == "ratingsum") { return undefined; } return value; }, 4));