Explore the new features in IBM Integration Bus version 10.0.0.8

Late last year, IBM announced the arrival of a new Bluemix experimental service called Hybrid Connect, for helping register and track the usage of several on-premise IBM products such as IBM Integration Bus, IBM MQ, WebSphere Application Server, WebSphere Liberty, Operational Decision Manager and Business Process Manager. The service lets you:

• Register on-premise IBM software products with a Bluemix service in the IBM cloud.
• Gather product information and usage data from enabled and configured on-premise software, such as IBM Integration Bus.
• View information about all enabled and configured products through a single centralized dashboard in Bluemix.
• Gain a greater insight into the workload of the enabled and configured products, by accessing the runtime usage data.

The latest refresh of this Bluemix service is called IBM Cloud Product Insights. Product Insights provides improved features:

• Product grouping views There is now a customizable way to group products in the dashboard so that IT Administrators can organize product instances in a more logical way. IIB lets you tag integration server instances, so at a glance you can easily separate different environments for example such as Production, QA, Integration Test, and Development. This facility also includes aggregating runtime usage data.
• Advisor recommendations Product Insights now recommends other Bluemix services that may be of interest
• Revocable API keys and new geos The Product Insights service is now available in other Bluemix regions beyond North America and revocable API keys make it easier to authenticate with the service
• Getting Started The Getting Started experience for product registration has become even easier so you can see meaningful data from your product instances within a few minutes.

IBM Integration Bus provides the mqsichangebluemixreporting command to allow administrators to quickly register integration servers with the service. In line with the Product Insights service moving its status from an Experimental Bluemix service to a fully supported offering, with IIBv10.0.0.8 we have chosen to change some of the command flags, so even if you’ve already played around with the Hybrid Connect service using the IIBv10.0.0.7 release then you may wish to check out our documentation in the IIB Knowledge Center.

IIB now provides three product usage metrics which are displayed graphically in the Product Insights UI:

• Number of active CPU processors which can be used by the Integration Server/li>
• CPU time (ms) which shows the CPU usage of the Integration Server
• Resident Set Size which shows the amount of memory used by the Integration Server

The service also still captures basic information about the registered IIB Integration Servers:

• IBM Integration Bus product ID, defined in the IBM License Metric Tool (ILMT)
• IBM Integration Bus installation root directory
• Host name
• Host operating system name
• Host operating system version
• IBM Integration Bus integration node name
• IBM Integration Bus integration server name
• IBM Integration Bus product version
• All APARs that have been applied
• The URL of the IBM Integration Bus web user interface for this integration server.

For more details check out the IIB Knowledge Center.

Callable message flows were introduced into IBM Integration Bus about a year ago as part of IIBv10.0.0.4. This capability provides IIB users with a Hybrid Cloud solution which can be used to easily burst workload between IIB installed in an on-premise data center and the IBM managed IIB on Cloud service. You can split flows between different applications in an integration server or between different integration servers, which can be on different integration nodes. A calling flow processes information in one location, then calls another flow in a different location. Servers can be hosted on separate physical machines and the IIB switch component used to connect them. This original feature used a synchronous link between the invoking flow and the flow being called. IIBv10.0.0.8 expands the callable flow capability to provide an asynchronous linkage. The new CallableFlowAsyncInvoke node is paired with a new CallableFlowAsyncResponse node by assigning the same uniqueId property to each node. The invoke node issues a call to a separate callable message flow which contains a CallableInput node and a CallableReply node. When the callable message flow completes its processing, control is passed back to the CallablFlowAsyncResponse node. If you wish, you can also choose to share data between the calling flow and the response flow by storing and retrieving data in the UserContext folder in the Environment.

For more details check out the IIB Knowledge Center.

In recent fix packs IIB has focussed on extending our capabilities for interacting via REST and JSON. You can use our Graphical Data Mapping editor to create and transform JSON messages. You can define the data model from a JSON schema or from a REST API swagger document, which can be in JSON or YAML format. IIBv10.0.0.8 adds support for three further JSON constructs: allOf, anyOf and oneOf. These combination keywords facilitate the reuse of defined structures in your JSON schema.

Combinations using allOf:
The allOf keyword lets you extend a structure with additional fields. For example, consider extending the definition of a person object to be an employee who is categorised as either an architect, a developer, architect or a manager:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "definitions": {
    "person": {
      "type": "object",
      "properties": {
        "first_name": {"type":"string"},
        "last_name": {"type":"string"},
        "age": {"type":"number"}
      },
      "required": ["first_name","last_name","age"]
    },
    "type": "object",
    "properties": {    
    "employee": {
      "allOf": [
        {"$ref": "#/definitions/person"},
        {"properties":
          {"type": {"enum": ["architect","developer","manager"]}},
          "required": ["type"]
        }
      ]
    }
  }
}

JSON schema containing allOf like the one above are no longer rejected by the IIB Toolkit, and are now available for mapping as shown in the screenshots below:

Combinations using oneOf:
The oneOf keyword lets you specify a structure which is valid if it matches exactly one of the defined options. For example, consider the example JSON schema definition below which has an array of vehicles, each of which is either a car or a bike:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "definitions": {
    "car": {
      "type": "object",
      "properties": {
        "transmissionType": {"type":"string"},
        "engineSizeCC": {"type":"string"}
      },
      "required": ["transmissionType", "engineSizeCC"]
    },
    "bike": {
        "type": "object",
        "properties": {
          "style": {"type":"string"},
          "numberOfGears": {"type":"number"}
        },
        "required": ["style", "numberOfGears"]
    },
    "vehicle": {
        "oneOf": [
                  { "$ref": "#/definitions/car" },
                  { "$ref": "#/definitions/bike" }        
                ]
    },
    "vehicleList": {
    	"type": "array",
    	"items": {"$ref": "#/definitions/vehicle"}
    }    
  }
}

JSON schema containing oneOf like the example above are no longer rejected by the IIB Toolkit, and are now available for mapping as shown in the screenshot below:

Combinations using anyOf:
The anyOf keyword is similar to oneOf, but specifies that one or more of the contained structures must validate against the instance value (rather than exactly one). For example, consider the very similar example JSON schema definition below where we have exchanged oneOf for anyOf in the previous example. This is also now acceptable to the IIBv10.0.0.8 graphical mapper.

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "definitions": {
    "car": {
      "type": "object",
      "properties": {
        "transmissionType": {"type":"string"},
        "engineSizeCC": {"type":"string"}
      },
      "required": ["transmissionType", "engineSizeCC"]
    },
    "bike": {
        "type": "object",
        "properties": {
          "style": {"type":"string"},
          "numberOfGears": {"type":"number"}
        },
        "required": ["style", "numberOfGears"]
    },
    "vehicle": {
        "oneOf": [
                  { "$ref": "#/definitions/car" },
                  { "$ref": "#/definitions/bike" }        
                ]
    },
    "vehicleList": {
    	"type": "array",
    	"items": {"$ref": "#/definitions/vehicle"}
    }    
  }
}

For more details on this topic, check out the IIB Knowledge Center.

IIBv10.0.0.6 introduced asynchronous REST nodes to the product which can be used in a message flow to issue a REST request to an external REST API, and return control to the flow without waiting for a response. The response is received by a RESTAsyncResponse node, which can be in a separate message flow but must be in the same integration server. The nodes are used as a pair, and responses are correlated with the original requests using a unique identifier, which is specified by a developer as a property of both nodes.

IIBv10.0.0.8 extends the capability of this area of the product. You can now store user context data which you have available in the message flow prior to the RESTAsyncRequest node so that it can be used when the reply message is returned to the RESTAsyncResponse message. You can set a value for the variable OutputLocalEnvironment.Destination.REST.Request.UserContext, or create and populate variables below this part of the LocalEnvironment hierarchy.

For example, you could set values using ESQL expressions like these:

DECLARE Ref REFERENCE TO OutputLocalEnvironment;
CREATE LASTCHILD OF OutputLocalEnvironment AS Ref 
  TYPE Name NAME 'Destination';
SET Ref.REST.Request.UserContext = 'Important info';
SET Ref.REST.Request.UserContext.Name.FirstName = 'Fred';
SET Ref.REST.Request.UserContext.Name.Surname = 'Bloggs';		

For example, you could set values in a JavaCompute node like this:

MbMessage LocalEnvTree = outAssembly.getLocalEnvironment();
int type = MbElement.TYPE_NAME_VALUE;
MbElement LocalEnv = LocalEnvTree.getRootElement();
MbElement Dest =
  LocalEnv.createElementAsFirstChild(type, "Destination", null);
MbElement REST = 
  Dest.createElementAsFirstChild(type, "REST", null);
MbElement Req = 
  REST.createElementAsFirstChild(type, "Request", null);
MbElement UserContext = 
  Req.createElementAsFirstChild(type, "UserContext", "Important info");
UserContext.createElementAsFirstChild(type, "FirstName", "Fred");
UserContext.createElementAsFirstChild(type, "Surname", "Bloggs");

You can use a graphical Mapping node to set a value, by mapping to the mixed child element underneath OutputLocalEnvironment.Destination.REST.Request.UserContext:

Alternatively you can create sub-elements using the mapper’s user defined elements or cast options:

To access the user context data after a RESTAsyncResponse node, read the value or children of the LocalEnvironment.REST.Response.UserContext environment variable.

For more details check out the IIB Knowledge Center.

The KafkaConsumer node and KafkaProducer node have been enhanced to include support for message keys. You can define a string value as a key which is to be associated with a message when it is published. If a message is published to a topic with multiple partitions, this key value can be used to select the partition on which the message is stored. All messages published using the same key value are sent to the same partition. If no key is provided, messages are distributed across the topic partitions by Kafka. The key value can be defined in the LocalEnvironment structure which is passed in to a KafkaProducer node. For example, you can do this using Compute node ESQL like this:

SET OutputLocalEnvironment.Destination.Kafka.Output.key = 'keyval1';

After an instance of a KafkaProducer node, the key which was used is placed alongside the other Kafka values in the WrittenDestination area of the LocalEnvironment at LocalEnvironment.WrittenDestination.Kafka.key.

The KafkaConsumer node places the key that was associated with the received message (if there was one) in the LocalEnvironment too. This field exists only if a key was associated with the message when it was published. The received key value is received as a string by the KafkaConsumer node and is placed in the position LocalEnvironment.Kafka.Input.key.

For more details check out the IIB Knowledge Center.

Since our last fix pack, we have added a further ten built-in tutorials which you can access directly from your IIB Toolkit. The hyperlinks below provide a summary for each tutorial. For the actual tutorials, open an IIB Toolkit and go to Help – Tutorials Gallery.

• Producing and Consuming Kafka messages
• Call a REST API using the RESTRequest node
• Call a REST API using the RESTAsyncRequest node
• Using Bluemix Product Insights to view IIB Registration and Usage
• Using Bluemix Kibana Dashboards to view IIB Logs
• Aggregation nodes using MQ nodes with back-end services
• Aggregation nodes using HTTPAsyncRequest nodes with back-end services
• Aggregation nodes using SOAPAsyncRequest nodes with back-end services
• Aggregation nodes using RESTAsyncRequest nodes with back-end services
• Aggregation nodes using CallableFlowAsyncInvoke nodes with back-end services

This fix pack also includes upgrades to a couple of the internal IIB architectural components – The embedded Node.js implementation is now upgraded to version 4.7.2.1 and the MQ File Transfer Edition libraries which were previously at MQ FTE version 7.0.4.4 (+ APAR IT07171) have been upgraded to MQ v9.0.0.0 (+ APAR IT19645).