Graphical data mapping for REST APIs with JSON schema

With the availability of IBM Integration Bus v10 fixpack 4 you are now able to fully implement REST API operations with simple Graphical data mapping for REST APIs with JSON schema data model provided directly from the REST APIs Open API initiative Swagger 2.0 document. Additionally you can implement that use JSON data defined in JSON schema conforming to the draft 4 specification.

IBM Integration Bus v10 fixpack 4 also now provides the ability to create a REST API from scratch by using the IBM Integration Toolkit.

Note this post covers providing a REST API, I covered the facility for consuming REST APIs with in the post Graphically issuing REST requests in IBM Integration Bus.

Lets take a look at these facilities to create a simplistic “echo” or “hello world” REST API entirely graphically. Firstly we create the new REST API selecting the option to define the resources and operations ourself.

Figure 1. Create a new REST API and define it yourself

This opens the REST API editor, and we can proceed to define a Resource “Echo” with a “GET” operation, which will take a single “InputText” query parameter and return a “Message” JSON object, which we’ll define in the Model definition having a “Text” and “Time” string children. Refer to task topic “Defining resources, models, and operations in a REST API” for details on this.

Figure 2. Simple Echo REST API Resources and Model

Next we can move into implementing the “GetEcho” operation, which starts with clicking the create subflow action button to create and open the operation subflow. We will then drag Mapping node from the Transformation folder of the subflow editor palette and drop it into the subflow. Simply connect this Mapping node to the input and output of the subflow and double click it to open the New Message Map wizard.

The Integration Toolkit recognises that the map is being created for a subflow that is implementing the REST API “GetEcho” operation and offers a new option to create a message map with input and output for the REST API operation. By simply clicking the “Finish” button the message map is created for us and automatically populated with the data we defined for this operation.

Figure 4. Automatically populated Message Map

The map input provides us with access to the REST API operation input data in the “LocalEnvironment”, including any path and/or query parameters. For operations that support a request body, this would be set as the input body data. In this case the Get operation has no request body, so the map input body is created as unused “BLOB” type. A “Task” transform is provided to simply locate the REST API input data. On the output side of the map the operation response data, “Message” JSON object, is provided for you to map to.

To complete the implementation, we can simply graphically create transforms to populate the output response JSON object.

The “Text” output data is set from the input query parameter “InputText”. This can be performed by drag drop connection, or by right clicking the “Text” target and selecting the “Quick link from input” menu action, and entering “InputText” into the pop-up search bar. This will place a “Move” transform, which can then be changed, by clicking the transform type picker icon “v”, and selecting a “fn:concat” to prefix the input text with ‘Echo: ‘. To set “Time” right click it and select the “Add Assign” option, then click the transform type picker “v” adjacent to the “Assign” and select the “fn:currentDateTime()” function.

That’s it, our Echo REST API is completed and ready to deploy and test.

Note if you’ve not already done so you’ll likely need to do the set-up for permitting web browsers to access the REST API in the Integration server by using Cross-Origin Resource Sharing

Once deployed the Echo REST API application “GetEcho” operation can be invoked via:
http://localhost:7800/Simpleechorestapi/v1/Echo?InputText=My Message
and will respond with the defined JSON “Message” object:
{"Text":"Echo: My Message","Time":"2016-03-15T19:14:25Z"}

I’ve provided this Simple Echo REST API implementation using Graphical Data Mapping in the project interchange available (attached below)

Building on from simplistic example we can quickly see these capabilities will enable the rapid implementation of REST APIs.

Lets take a use case were we have data in comma separated value files that we wish to expose through REST APIs. We’ll exploit the ability to perform Graphical data mapping with a Global cache to enable the REST API to serve the data efficiently.

Again we start by creating a new REST API and using the REST API editor to this time define the two levels of Resources for the create operation passing a “FileLocation” JSON object Model and then from this resource Get, Patch and Delete operations on the “Row” JSON object Model.

The “postCvsFile” operation will be responsible for reading the REST API input request body “FileLocation” JSON object data and using this to read the CVS file data and load it into the Global Cache. We can chose to implement this Graphically by using Mapping nodes around a “File Read” node:

Figure 7. Post CSV File Operation Subflow

The first Mapping node “SetFileLocationFromRest” is created using the new option to create a message map with input and output for the REST API operation and the transformation is to simply use the input request body “FileLocation” JSON object data to set the local environment overrides for the File Read node. The created map can be customized to include the local environment tree on the output, as by default it is only added to input to allow the transforms to read the REST API input data.

The File read node is configured to read the file and parse it using a Data Format Description Language (DFDL) model for Comma Separated Values. Hence the input to the “LoadToCache” Mapping node is this DFDL CSV model and it’s output will be the REST API response, which in this operation is simply the HTTP status code 201. So in this case the Map is created as normal selecting the DFDL data model for input and the “BLOB” message to reflect the empty output body.

Figure 8. New Message Map for Loading the Cache

The created map will need to be customized to include the local environment tree so that it can access the REST API input parameters and set the HTTP response status code. The mapping uses a Cache Put transform inside a For each to add the rows from the CSV file to the Global Cache.

Note if the REST API operation did have a response body, we could again create the map for the operation, so that the output would be defined from the REST API swagger definition, and then switch the input to a required intermediary format as detailed here.

Error handing is provided in the “postCvsFile” operation subflow by wiring Mapping nodes to the failure terminals to simply report file or cache error status along with a 404 HTTP status code.

The implementation of the Get, Delete and Patch flows each follow a similar pattern with their subflow simply requiring a single Mapping node that has a map created using the new option to create a message map with input and output for the REST API operation and the transformation logic centred around the Cache Get, Cache Put, in update mode, and Cache Remove transforms respectfully.

For example the “getRow” operation subflow and map is shown in the following figure:

Note that in the Cache Return the existence of a “value” from the Cache Get is tested to determine if the response HTTP status should be set to 200, and the JSON object “Row” populated, or a HTTP status of 404 returned. The optional Cache Failure transform is also included in each of these maps to “catch” any failure in the cache operation, such as attempting to patch on a non-existent resource name or Row key value, and return the 404 HTTP status responses.

In the CSV data model I defined the first two columns as “minOccurs=1” to provide the “required” “Key” and “Value” for the Row model object in the REST API. A third column in the CSV file provides the “OptionalValue” in the Row model, which is not required. Since the Graphical Data Map Cache transforms can only get and put a single simple type value to the Global Cache, the Maps use the XPath functions “concat” and “tokenise” within “If” “Else” branches to join and split out the “Row” “Value” and “OptionalValue” from the single value in stored to the global cache.

I’ve provided the full implementation of this REST API Graphically Serving Cached CSV Data in a project interchange (attached below), so that you can explore it in more detail and deploy and test it using for example the swagger.io editor. Note prior to running this you need to have enabled the Global Cache in your integration node, for example by setting the “cachePolicy” parameter to “default”.

When exposing such an API you would typically look to exploit API Management to administer security policies, scale and monitor it’s resources.

IBM Integration bus introduced support for graphical data mapping of JSON data in v10. At that point the definituon of the JSON objects and arrays could be defined by either using the “Add User Defined function” or by providing a “equivalent XSD model“.

From fixpack 4, in addition to having the model for graphical data maps be defined by the JSON schema in the REST API swagger documents as discussed above, you can now create message maps using JSON schema types to define the map input or output. These JSON data types can be provided by loading “.json” files in a shared library container and creating you message map in the same container.

The JSON schema files must be self contained, that is not have external “$Ref” and there are some additional requirements noted here.

In this post I have reviewed the new capabilities for using Graphical data mapping for REST APIs with JSON schema in IBM Integration Bus fixpack 4. These facilities provide the ability to rapidly develop REST APIs graphically as well as implement transformations based on JSON schema models.