I have had a couple of projects recently that involved building a custom identity adapter for IBM Security Identity Manager utilizing REST APIs of the target system. Custom Identity Adapters utilize IBM Security Directory Integrator (SDI) as a platform. SDI’s HTTP Connector in Call/Reply mode provides excellent platform to fetch or send data and to provide advanced error handling if REST service allows it. Establishing connection and checking APIs was a relatively easy task. The first challenge was to decide either to use common Java frameworks or SDI’s own JavaScript engine to process JSON responses.

SDI provides the means to process JSON objects, although it is not as straight forward and obvious process as using Java frameworks. The problem with Java frameworks is that the correct framework needs to be selected and then needs to be maintained separately. The maintenance is usually the biggest challenge. One can expect regular updates from the reliable Java framework but updating it on a regular basis demands quite high maturity level from the maintaining organisation.

One of problems with SDI is that scripting is not well documented. One can find bits and pieces here and there, but rarely the complete picture on where to start and how to get to the working prototype. Hence this blog post was born. It will only process the scripting part of the integration to the REST service. If you are interested in HTTP Client Connector usage and setup, please refer to the one of the useful links at the end of the article. I will publish the post in two parts, one describing processing of JSON objects and the other one dealing with building JSON objects. So, let us get started.

Processing simple JSON objects

Java Script Object Notation (JSON) object provides a text-based format to transfer object data between interfaces. Let us pretend we have a simple JSON object describing colours of various fruits:

var json = '{"apple":"green", "peach":"yellow", "strawberry":"red"}';

This is basically the response string, which might be fetched by HTTP Connector. There are two ways to change this string to an object in SDI. First one is to define it as hierarchical entry:

jEntry = work.fromJSON(json);

or as a JavaScript object:

jObj = fromJson(json);

Please not the letter cases in fromJSON() and fromJson() methods. Both ways have their advantages and drawbacks, but the main difference is how attributes are retrieved from the object. Let’s print both objects using task.logmsg() method and see how they look in the log:

task.logmsg("simple TDI entry : " + jEntry);

INFO  - simple TDI entry : {
	"apple": "green",
	"peach": "yellow",
	"strawberry": "red"
}

And the other one:

task.logmsg("simple JSON object :" + jObj);

INFO  - simple JSON object :[object Object]

We can see that JS objects cannot be printed as is and additional steps are required to retrieve object properties. Here is simple code snippet to access all attributes in JS object created from JSON string:

for (prop in jObj){
	task.logmsg("Printing pair " + prop + " : " + jObj[prop]);
}

And the result will look like this:

INFO  - Printing pair apple : green
INFO  - Printing pair peach : yellow
INFO  - Printing pair strawberry : red

In this code there is one way to access value of the attribute within the object if attribute name is known. There is also the other one which is quite similar. Let us look at them together:

task.logmsg("simple JSON object apple:" + jObj.apple);
task.logmsg("simple JSON object strawberry:" + jObj["strawberry"]);

and the result of printing above will look like this:

INFO  - simple JSON object apple:green
INFO  - simple JSON object strawberry:red

So, if the attribute names for the JSON are known, JS object provides an easy way to access values.

Accessing values within hierarchical entries is easy too.  You still will need the name of the attribute to access its value directly, but there is convenient method available to make them known, if printing the whole object is not feasible for some reason.

task.logmsg("attribute  names : " + jEntry.getAttributeNames());
task.logmsg("get attribute value by name : " + jEntry.getAttribute("apple")[0]);

The result looks like this:

INFO  - attribute  names : apple,peach,strawberry
INFO  - get attribute value by name : green

Using those two methods elements of JSON object can be accessed in loop. Bear in mind that getAttributeNames() method returns an array of strings and getAttribute() method will return an Attribute object. Attribute object can be treated as an array of objects. That is why we need [0] to access the first value. Look in SDI Documentation to explore other available methods of com.ibm.di.entry.Entry class (link at the end of the article). Value of the entry attribute can be also accessed directly as jEntry.apple.

Processing complex JSON objects

Processing more complex JSON objects requires more processing, but underlying logic stays the same. Let us complicate an example we have used before. New string will look like this:

{
	"fruits": [{
		"apple": [{
			"shape": "round",
			"colors": {
				"1": "green",
				"2": "yellow",
				"3": "red"
			},
			"size": "medium"
		}],
		"peach": [{
			"shape": "round",
			"colors": {
				"1": "yellow",
				"2": "red"
			},
			"size": "medium"
		}],
		"strawberry": [{
			"shape": "conic",
			"colors": {
				"1": "red"
			},
			"size": "small"
		}]
	}]
}

Processing this JSON object requires understanding its structure. Changing JSON String into SDI objects is done the same way:

jObj = fromJson(json); // Creates JS object
jEntry = work.fromJSON(json); // Creates SDI Hierarchical Entry

Let us play with JS object first. Just printing JS object

task.logmsg("entry as js object : " + jObj);

will give the following result:

INFO  - entry as js object : [object Object]

First element of the JSON was fruits and it is an array of different fruit objects. This element can be accessed as jObj.fruits, but it still will not be printing anything meaningful. After fruit element there will be an array of different fruits. First element can be accessed like this jObj.fruits[0]. This element resolves to apple object. As an example, apple object is presented as an additional array, containing only one element. So this code jObj.fruits[0].apple[0] will access the actual apple object. To retrieve shape feature of the apple object the following can be used:

task.logmsg("entry as js object, printing value : " + jObj.fruits[0].apple[0].shape);

which will print the following:

INFO  - entry as js object, printing value : round.

Furthermore, colours attribute of the JSON string is a multivalued attribute, which can be accessed directly by specifying a number of the colour:

task.logmsg("entry as js object, printing value : " + jObj.fruits[0].apple[0].colors["1"]);

INFO  - entry as js object, printing value : green.

To access other values or objects one can just change numbers in the expression above. Accessing fruit parameters one by one is usually done in a loop, the same way as for the simple JSON. This time though it will be a nested loop. If the structure of the JSON is known and unchanged, then some elements/objects can be accessed directly. For example, in this case the fruits object can be accessed directly, since we know that there is only one fruits object in the JSON. Usually it is not the case. So, to process each level of the JSON the following loop can be used:

for (prop in jObj){
	// processing code
{

To access the next level of the JSON the code similar to the one below can be used:

for (prop in jObj){
	// processing code
	// Initialize subobject for the next loop
	subObj = jObj[prop];
	for (el in subObj){
	// processing code
}
{

Using loops to access each individual value is a goto solution dealing with complex JS objects. Not all levels need to be processed, for instance one may skip processing jObj level and process jObj.fruits[0] object instead. This will save one iteration. The resulting code with all iterations will look somehow like this:

for (prop in jObj){
	task.logmsg("Starting to print values of the " + prop + " object");
	// Initialize subobject for the next loop
	subobj = jObj[prop];
	for (fruit in subobj){
		// Nothing to print here (accessing [0] element)
		// Initialize subobject for the next level
		frobj = subobj[fruit];
		for (nprop in frobj){
			task.logmsg("Printing features of a " + nprop + " fruit");
			// Initialize subobject for the next level
			num = frobj[nprop];
			for (int in num){
				// Nothing to print here, accessing fruit array
				// initializing subobject
				feature = num[int]
				for (ft in feature){
					// Printing single value attributes as is
					if (ft != "colors"){
						task.logmsg(nprop + " fruit's " + ft + " is " + feature[ft]);
					}
					// processing multi-value attributes
					else {
						// printing colors
						// initializing subobject
						colours = feature[ft];
						task.logmsg(nprop + " fruit's " + ft + " :");
						for (col in colours){
							task.logmsg("   " + colours[col]);
						}
					}		
				}
			}
		}
	}
}

Executing this code in SDI will produce the following result:

INFO  - Starting to print values of the fruits object
INFO  - Printing features of apple fruit
INFO  - apple fruit's shape is round
INFO  - apple fruit's size is medium
INFO  - apple fruit's colors :
INFO  -    green
INFO  -    yellow
INFO  -    red
INFO  - Printing features of peach fruit
INFO  - peach fruit's shape is round
INFO  - peach fruit's size is medium
INFO  - peach fruit's colors :
INFO  -    yellow
INFO  -    red
INFO  - Printing features of strawberry fruit
INFO  - strawberry fruit's shape is conic
INFO  - strawberry fruit's size is small
INFO  - strawberry fruit's colors :
INFO  -    red

Working with hierarchical Entry is a bit easier in my view. First of all, one can print it right away in the SDI logs like this:

task.logmsg("Printing JSON object as hierarchical entry : " + jEntry);

The output will look like this:

INFO  - Printing JSON object as hierarchical entry : {
	"fruits": {
		"fruits": {
			"apple": {
				"apple": {
					"shape": "round",
					"colors": {
						"1": "green",
						"2": "yellow",
						"3": "red"
					},
					"size": "medium"
				}
			},
……………

There are multiple ways to access values for hierarchical entry and organize the logic to print them out. Lets look at the output of the getAttributeNames() method. It will print a String array:

task.logmsg("Printing attribute names : " + jEntry.getAttributeNames());

INFO  - Printing attribute names : fruits.fruits.strawberry.strawberry.shape,fruits.fruits.peach.peach.shape,fruits.fruits.peach.peach.colors.2,fruits.fruits.apple.apple.colors.3,fruits.fruits.apple.apple.colors.2,fruits.fruits.peach.peach.colors.1,fruits.fruits.strawberry.strawberry.colors.1,fruits.fruits.apple.apple.colors.1,fruits.fruits.strawberry.strawberry.size,fruits.fruits.apple.apple.size,fruits.fruits.peach.peach.size,fruits.fruits.apple.apple.shape

Values of the entry can be directly accessed using members of this array. There are multiple other ways to access the elements of the entry. For instance getElementsByTagName() method (returning NodeList) can extract the whole fruit object like this:

task.logmsg("Testing the getElementsByTagName method : " + jEntry.getElementsByTagName("apple")[0]);

INFO  - Testing the getElementsByTagName method : "apple": {
	"apple": {
		"shape": "round",
		"colors": {
			"1": "green",
			"2": "yellow",
			"3": "red"
		},
		"size": "medium"
	}
}

There are other useful methods available for processing Entries. It is up to developer which road to choose. The code below will do the trick, but it is not easiest nor smartest way to do things.

 // Starting to pring values of JSON array
task.logmsg("Printing values of the fruit array")
// getting the attributes name and sorting the array so it's elements have an order
var names = jEntry.getAttributeNames().sort();
// initializing string attribute to process current fruit
var curFruit = "";
// initializing counter for printing header for the fruit being processed
var fCount = 0;
// iterating through the attributeNames array
for (value in names){
	// getting the fuit name and its feature as array of strings
	valArray = value.split(".");
	// if it is a new fruit, set values for variables
	if(curFruit != valArray[3]){
		curFruit = valArray[3];
		fCount = 0;
	}
	// Print a header for a new fruit
	if(fCount == 0){
		task.logmsg("Printing values for " + curFruit);
		fCount++;
	}
	// Printing actual values. 
	// Printing single value features
	if(valArray[4] != "colors"){
		task.logmsg("  " + curFruit + " fruit's " + valArray[4] + " is " + jEntry.getAttribute(value));
	}
	// Printing multivalue features
	else{
		task.logmsg("    " + curFruit + " fruit's color may be " + jEntry.getAttribute(value));
	}
		
}

The result of executing this script will look like this:

INFO  - Printing values of the fruit array
07:53:04,327 INFO  - Printing values for apple
07:53:04,328 INFO  -     apple fruit's color may be green
07:53:04,329 INFO  -     apple fruit's color may be yellow
07:53:04,330 INFO  -     apple fruit's color may be red
07:53:04,330 INFO  -   apple fruit's shape is round
07:53:04,331 INFO  -   apple fruit's size is medium
07:53:04,332 INFO  - Printing values for peach
07:53:04,332 INFO  -     peach fruit's color may be yellow
07:53:04,333 INFO  -     peach fruit's color may be red
07:53:04,334 INFO  -   peach fruit's shape is round
07:53:04,334 INFO  -   peach fruit's size is medium
07:53:04,334 INFO  - Printing values for strawberry
07:53:04,335 INFO  -     strawberry fruit's color may be red
07:53:04,335 INFO  -   strawberry fruit's shape is conic
07:53:04,336 INFO  -   strawberry fruit's size is small

These examples do not show the whole range of possibilities of SDI but provide a working example. Other approaches are possible and, in some cases, will offer a better solution.

Useful Links:

SDI JSON and XML processing guide:

http://www.tdiingoutloud.com/2016/11/json-and-xml-tutorial-part-4-json.html

Entry method reference:

https://www.stephen-swann.co.uk/javadoc/tdi7.0/com/ibm/di/entry/Entry.html

TDI hierarchical entry documentation:

https://www.ibm.com/support/knowledgecenter/SSCQGF_7.1.1/com.ibm.IBMDI.doc_7.1.1/tdihierarchicalentryobjects.htm

Part 2:
Building JSON objects with SDI