Integration Bus (IIB) isn’t just the same old WebSphere Message Broker dressed up with a new name. It has lots of new features like .NET integration mobile patterns and business rules to help ensure enterprise application integration success. Andrew Humphreys Senior Product Manager for IBM Integration Bus recently spoke with the Global WebSphere Community to detail some of the new features in IIB v9.0.0.2 and the upgrade path from WebSphere Message Broker v6.1-8. Humphreys also provided 3 tips to help optimize IIB performance as well as how IIB fits in with mobile and IoT.

GWC: In a recent webcast on the GWC about IBM Integration Bus almost half of the attendees said they were running v7. What features and functionality are they missing out on in v9.0.0.2?

Humphreys: Customers upgrading from v7 will get access to a huge number of new features and enhancements. IBM Integration Bus v9 is a significant evolution of the WebSphere Message Broker technology base with innovation to incorporate an increasing number of integration use cases such as BPM business rules Microsoft .NET and workload management. IBM has published a comparison of the features available in the different versions of IBM Integration Bus.

Particular features I would highlight include:

• .NET Integration – IBM Integration Bus now embeds a Common Language Runtime to invoke any .NET logic as part of an integration. It also includes full support for the Visual Studio development environment including the integrated debugger and code templates. IBM Integration Bus includes a comprehensive set of patterns and samples that demonstrate bi-directional connectivity with both Microsoft Dynamics CRM and Microsoft Message Queuing (MSMQ). This allows you to take advantage of existing Microsoft skills and connect simply to Microsoft applications.
• Policy-Based Workload Management – Many IT systems struggle to cope with peaks in demand and IBM Integration Bus now includes features to control the rate at which these systems are loaded in order to improve overall system reliability. You can form sophisticated workload management algorithms which can be easily configured on individual integration flows. They can also be defined as workload management policies using a powerful new editor in the IBM Integration Bus web UI. This latter approach to configuration lets you associate thresholds with groups of integration flows enabling specific classes of service (such as batch or online work) to be enforced across a range of scenarios. IBM Integration Bus can also detect and manage unresponsive integrations. You can define a threshold in the policy at which an integration is considered unresponsive and then define an action when that threshold is crossed such as restarting the integration server.
• Global Cache – The new global cache capability uses WebSphere eXtreme Scale technology to provide an in-memory data cache that can be shared across execution groups and brokers. A longstanding requirement has been a mechanism for sharing data between different processes.
• Record and Replay – The edit record and replay capabilities in IBM Integration Bus provide new and powerful browser-based tooling which allows suitably authorized users and administrators to view edit record and replay data flowing through the broker from all sources including messaging web services and Enterprise Information Systems.
• Mobile Patterns – IBM Integration Bus now provides built-in patterns to mobile-enable any enterprise service in as few as 2 clicks enabling you to build robust solutions with integrated caching and security in less than 10 minutes.
• Business Rules – Business rules empower decision makers to make dynamic changes to IT system operations without requiring costly development efforts. IBM Integration Bus let you deploy and invoke such rules as part of integration logic enabling rules to affect integrations in a robust high-performance manner. A Decision Service connector implements this feature translating elements from the input data into parameters for the business rule. The rule is then invoked using the embedded rule engine and the output is captured for downstream processing. This feature uses the separately licensed IBM Operational Decision Manager (ODM) product and uses the ODM Decision Center for full business rule management scenarios such as rule governance.
• Message Modelling and Mapping – IBM Integration Bus uses standard non-proprietary file formats to describe solution elements wherever possible in order to minimize rework when transferring between integration products. Data Format Description Language (DFDL) is an industry-standard way to model non-XML data that is used across many IBM and non-IBM solutions. The DFDL web community on Github hosts an increasing number of DFDL models for industry standard message formats including HL7 ISO8582 and TLOG. IBM Integration Bus has strong support for the DFDL standard including user-defined variables packed numeric fields delimited binary data and field lengths given by regular expressions. IIB v9 provides better performance when parsing and serializing DFDL messages faster deployment times and improvements in the DFDL Editor that making it easier to develop and work with DFDL models. The IBM Graphical Data Mapper is an easy-to-use yet sophisticated editor to visually map and transform source to target data. A rich feature set and simple design make IBM Graphical Data Mapper a good default transformation choice and its implementation inside IBM Integration Bus includes full stored procedure support to complement existing database selects inserts updates and deletes. It also includes further migration of maps from the WebSphere Message Broker V7 mapping tool.

GWC: What is the upgrade path from v7 to v9?

Humphreys: You can migrate from WebSphere Message Broker v6.1 7 or 8 directly to IBM Integration Bus v9.0.0.2. You can migrate v7 products at Fix Pack 5 (v7.0.0.5) or later but you must first ensure your WebSphere MQ installation is at a supported version (v7.1 7.5 or 8) by upgrading if necessary.

You can migrate an exisiting Broker in-place by simply running the mqsimigratecomponents command to migrate the broker. Migrating development resources is equally simple. When you start IBM Integration Toolkit v9 for the first time you are prompted to enter a workspace location. Enter the directory that contains the WebSphere Message Broker Toolkit workspace that you want to migrate and click OK. The workspace and resources are automatically migrated and are now available in IBM Integration Toolkit v9.

GWC: What features separate IIB Advanced from the standard IIB offering?

Humphreys: Standard mode is intended to be used to configure a small environment with low capacity requirements. It is aimed at clients with a broad set of connectivity requirements who do not need to scale to large numbers of flows or run high volumes of traffic - for example a single line of business or small business with few applications. In Standard mode the broker operates with all functional features enabled but you are limited in the number of resources that you can create and maintain in that you are limited to creating one execution group.

Advanced is the fully functional and scalable edition of IBM Integration Bus. It is aimed at Enterprise clients seeking an offering which provides high performance connectivity to the broadest range of data and applications those looking to manage large volumes of business critical traffic clients implementing a “Shared Service” infrastructure connecting multiple LOBs and applications with proper segregation and operational and clients seeking the most flexible capabilities in managing their data including the optimum solution in managing scheduled and unscheduled outages. In Advanced mode IBM Integration Bus operates with all features enabled and no operational limits on the creation of execution groups or on the number of flows that are deployed to an individual execution group are enforced.  Typical uses are for separation of individual department’s data and flows through utilisation of Multiple Execution Groups which maintains integrity of sensitive data and ensures minimum downtime across all users (planned or unplanned).

Note that the operational limitation is not related to total processing capability in terms of processor power. In single core deployments Standard and Advanced have access to the same CPU cores and memory so exhibit the same performance profiles. The limitation is related to the total number of in-flight messages ready for work that can be active. As Advanced can spawn multiple Execution Groups the total worker threads (message flows) = 256*(number of Execution Groups) assuming a theoretical limit of 256 threads per operating system process. In contrast Standard can spawn just 1 Execution Group meaning the total worker threads (message flows) = 256. Once 256 threads (message flows) are in-flight IIB Standard will have to wait for a thread to be released (complete its processing) before the next piece of work can be dispatched.

GWC: What are your top three tips to optimize IIB performance?

Humphreys: Optimizing your broker’s performance can lead to real savings in lower hardware requirements maximised throughput and improved response time. IBM publishes performance reports and tuning recommendations for IBM Integration Bus deployed in a variety of different topologies.

IBM also publishes a variety of performance planning guides and hints and tips on how to optimize performance. You can see their performance section in our Integration Community.

For my three tips I would say first understanding what IBM Integration Bus is and - perhaps more importantly - what it is not will help significantly in ensuring that you build an efficient application. It is a sophisticated product and there are often several ways to do things. Understanding how the broker works allows you to write more efficient message flows and debug problems more quickly.

Second message flow design is critical to achieve high performance and scalability.  It is better to design your message flows to be as efficient as possible from day 1 as there is only so much tuning at a hardware level that can be achieved. The most common performance issues are resolved by better message flow design.  To design effective message flows it is important to understand where the main performance costs are within a message flow. There are several areas that incur a processing cost: parsing and serialization of messages as they enter and exit the flow business logic such as transformations executed during message processing transformations tree navigation and tree copying and interactions with external resources such as databases and queue managers.

Finally it is important to look at all aspects of IBM Integration Bus implementation. These include the ways in which the external applications will interact the messages that will be processed the configuration of the broker and the broker queue manager and any other queue managers which form part of the configuration the transports over which messages are carried and the high-level characteristics of the hardware and software which you choose to run the message flows on. These all play a part in overall performance and as such they should all be considered from day 1. For example it helps in planning if you are able to understand whether processing is likely to be CPU or I/O bound. This is particularly important when you need to achieve high volumes of processing and so achieve higher CPU utilisations. You do not want to be I/O bound and limited to 10% CPU utilisation for example.

GWC: What is the advantage of hosting IIB in a cloud environment? Is configuration similar to on premises?

Humphreys: Cloud creates new opportunities for businesses and enables execution with a speed ease and cost that has not been possible before.  Cloud allows companies to move more quickly bringing new products services and capabilities to market in less time than ever. It changes the economics of IT moving from capex to opex reducing risk of new technology and increasing the ability to test new capabilities and technologies out.

As cloud adoption grows businesses are increasingly facing hybrid integration challenges needing to integrate a mix of on premise applications and cloud applications.  As enterprises adopt cloud technologies integration becomes critical if they are to realise the full potential for their business. IaaS PaaS and SaaS all have higher value if they are integrated with systems of record and with each other.  This drives a need for an integration solution which enables integrations between endpoints both from within the enterprise to the cloud and from the cloud reaching into the enterprise.

As a result enterprise IT organizations are looking to extend their existing on premise integration solutions to run in and connect to the cloud. They want to be able to reach up and include all of the organization’s cloud-based environments integrating them as part of the enterprise with appropriate governance. To them the cloud represents another deployment option for what has traditionally been an on premise solution.

IBM Integration Bus it is fully supported under the existing licence T&Cs to run in SoftLayer Microsoft Azure or Amazon EC2 public clouds using a bring your own software licence (BYOSL) model  which basically means if you have IIB entitlement you can run IIB on those clouds. We also have an article on the Integration Community describing how to set up IIB on SoftLayer.

GWC: What is the key to effectively integrating mobile apps with existing enterprise applications?

Humphreys: The explosion of mobile has created a huge business opportunity but also created new challenges for integration. The accommodation of more mobile devices has substantially increased the amount of work and cost required for for integration.

There are multiple challenges with effectively integrating mobile apps and existing enterprise applications. Enterprise applications and mobile apps are fundamentally different. The value of enterprise applications is to provide the source of truth and as a result the pace of change is ideally low with an application lifecycle that is permanent and foundational. In contrast the value of mobile apps is in dialog and discovery. The pace of change is ideally high with a transient and fluid lifecycle.

The challenge of integrating the two is that the pace of mobile development is often hampered by the complexity of integration with backend systems and the time taken to do so. Mobile developers can often end up spending too much time creating an integrated experience with backend systems.

IBM Integration Bus provides patterns that reduce the time and skills needed to integrate mobile apps. The patterns enables the mobile developer do all the hard work of connecting to backend systems in less than a minute generating Web services that expose the desired operations. It even generates a ready-to-run sample mobile application that makes testing easy. 

A second challenge is how to deliver end-to-end performance required for mobile users. IBM Integration Bus v9 provides built-in caching which can dramatically cut response times and load on backend systems. Its new workload management features also provide the ability to control mobile traffic entering enterprise and load balance with backend systems helping to deliver an more robust and responsive mobile experience.

GWC: How do you see the Internet of Things (IoT) affecting enterprise application integration?

Humphreys: Similarly to the mobile challenges discussed above the Internet of Things presents new challenges for integration. Gartner have predicted that over two-thirds of all new integration flows will extend outside the enterprise firewall to integrate with IoT devices and data.

The challenge for businesses will not just be to connect it will be how to use these new sources of data to generate new opportunities and gain new insights into their business. The goal is not to just collect all this new data; it is to analyse and act upon it. Enterprise application integration will need to grow to encompass integrating enterprise applications and IoT devices as well as to perform analytics on in flight data as it passes through the integration platform enabling businesses to anticipate the changes needed to proactively optimize the business.

IBM Integration Bus v9 includes many features to support IoT integration. There is now a fully supported MQTT connector available on our Github site that can be used to connect natively with MQTT applications.  We provide patterns that enable out-of-the-box integration with IBM MessageSight which is an appliance-based messaging server that is designed to handle large numbers of connected clients and devices and process high volumes of messages. We provide an embedded rules engine allowing business decisions to be made on data flowing through the bus and we also provide a similar feature on our Github site that embeds an R analytics engine allowing data flowing through the bus to be scored against a predictive model.

GWC: What sessions and activities will we see around IIB at InterConnect in February?

Humphreys: We have a very full agenda for IIB at InterConnect in our Enterprise and Cloud Integration track. This track will focus on the challenge that the technology that businesses rely on is diverse from enterprise systems and packaged applications to cloud-based services and ensuring that this technology is properly integrated is critical.
We will have sessions within this track to hear best practices on integration from successful clients and for the latest technology updates in this important area.




Andrew Humphreys is the Product Manager for IBM Integration Bus Product Management Team (formally known as WebSphere Message Broker). He is responsible for ensuring the success of the product by defining and executing the vision strategy and product roadmap to maximize the product's position the in the marketplace. Andrew has worked for IBM for 18 years and prior to joining the Product Management team he held a variety of roles in GBS Software Services Product Strategy and Technical Sales which has helped him build a combination of broad architectural skills deep specialist knowledge in the areas of messaging SOA integration and BPM and extensive customer-facing experience which allows him to engage effectively at all levels within IBM and customer organizations.