App Connect

App Connect

Join this online user group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only

Explore the new features in App Connect Enterprise version 11.0.0.3 

Thu July 09, 2020 09:18 AM

We’re thrilled to announce the availability of IBM App Connect Enterprise v11.0.0.3. This is the third fix pack released for App Connect Enterprise software. We provide regular fix packs, approximately once per quarter, a cadence which we intend to continue through 2019. Fix packs provide both regular maintenance for the product, and also new functional content. This blog post summarizes all the latest and greatest capabilities:

  • Multi-instance High Availability for integration nodes
  • Support for Global message flow coordination (XA 2 Phase Commit) now available across ODBC, JDBC, JMS and CICS
  • User-defined Message Flow Nodes written in Java and C
  • Dynamic monitoring and statistics via commands mqsichangeflowstats, mqsireportflowstats, mqsichangeflowmonitoring and mqsireportflowmonitoring
  • Enhancements to make Policy information accessible from Java outside a JavaCompute node
  • ACEv11.0.0.3 provides updated language translation

Expand the sections below to find out more!

Multi-instance High Availability for integration nodes

Although many users are adopting App Connect Enterprise for use in container based architectures which take a very different approach to the architectural challenge of offering constant availability for integration capabilities, it is also the case that a section of ACE users require the product to be arranged in a highly available configuration. To achieve this, it is possible to set up your integration nodes to work with an external high availability manager such as a Red Hat cluster, Veritas Cluster or a Windows Cluster for example. In addition to these existing options, fix pack 3 of App Connect Enterprise v11 re-introduces the out-of-the-box concept of multi-instance high availability. With this mode of operation, there is no need for an external HA co-ordinator. You can now configure multi-instance integration nodes as an MQ Service dependency such that when the queue manager becomes unavailable the integration node stops, and when the queue manager starts the integration node is also started.

To support multi instance style High Availability you require one of:

  • Shared file system on network storage such as NAS
  • Cluster file system such as IBM GPFS
  • SAN (dedicated network providing access to consolidated block level data storage)

These restrictions stem from 3 fundamental requirements (due to the way multi-instance works):

  • Data write integrity
  • Guaranteed exclusive access files
  • Release of locks on failure

IBM has made a testing statement for IBM MQ multi-instance queue manager file systems, which also applies for App Connect Enterprise. A common mistake when configuring and using MQ multi-instance queue managers is to assume that the NFS server and NFS client are at version 4, which is the required version, but in reality at least one of these components is using version 3. NFS V3 is NOT supported with MQ multi-instance queue managers. This technote will help you to verify whether an NFS server and client are at version 4. With the multi-instance technology, a file server is configured to host both the shared work path directory for the multi-instance integration node and the shared directories for the multi-instance queue manager. The file server is shared between two computers, each of which has a licensed copy of the IBM App Connect Enterprise and MQ products installed.

To create a multi-instance node which runs as an MQ service, you can use the -d option on the mqsicreatebroker command:

  mqsicreatebroker MyBroker -q MyQMGR -d defined

When you have configured an integration node to start and stop as an MQ service:

  • The integration node starts and stops automatically when its associated queue manager starts and stops. For a multi-instance integration node, this action can occur during failover of the active queue manager.
  • A multi-instance integration node cannot be started in standby mode when its MQ service is defined as active.
  • You can stop the integration node manually by using the mqsistop command, but the integration node does not restart until the queue manager is stopped and started again.

For full steps for the configuration of the multi-instance technology, you can follow the Knowledge Center pages here.

Support for Global message flow coordination (XA 2 Phase Commit) now available across ODBC, JDBC, JMS and CICS

App Connect Enterprise can be configured to make a message flow transaction “globally coordinated” so that external resource managers are controlled using a two-phase commit approach using the XA protocol standard. This means that the same transaction can be used to coordinate updates to more than one external resource, such as IBM MQ, CICS, Databases (when communication is via ODBC or JDBC) and JMS providers. The default behaviour of an ACE integration node is to manage message flow transactions using local transactions (sometimes referred to as a one-phase commit approach). This approach will be sufficient for the majority of transactions, but if your business requires assured data integrity and consistency (eg for financial transactions or for audit compliance) then you can use a local MQ queue manager to globally manage the ACE transaction. When deploying a message flow which you wish to globally coordinate, in your BAR file select the message flow and ensure that the Coordinated Transaction property is selected.


To globally coordinate transactions, there must be a queue manager specified on the integration node, and that queue manager performs the transaction manager role. The queue manager that is specified on the integration node is used as a globally coordinated resource. Other queue managers cannot participate in the message flow transaction. Further detailed instructions for how to configure ACE’s relationship with various resource managers are available at the links below:

User-defined Message Flow Nodes written in Java and C

Despite ACE having in excess of 100 message flow nodes available in the flow palette out of the box, there may be occasions where you would like to author your own message flow node, either as an Input node to mark the beginning of a message flow, or in a mid-flow position as a message processing or output node. Over the years, there have also been several community contributions for user-defined nodes for previous versions of the IBM Integration Bus software such as for example the Log4j node available as SupportPac IAM3 and the JobExecution node available as SupportPac IA9Z. From ACEv11.0.0.3, you can author User-defined nodes or deploy message flows which contain User-defined nodes which were authored and built using earlier versions of IBM Integration Bus. Conceptually, there are two required tasks when creating a User-defined node:

  • Authoring the Eclipse plugin which represents the node in the ACE Toolkit message flow palette
  • Authoring the runtime implementation of the node which is provided to the ACE runtime

To help you author the Eclipse plugin which represents your User-defined node in the ACE Toolkit, the Toolkit itself provides some simple wizards which take configuration settings such as the name of the node you wish to use, the picture icons to represent it in a message flow, the input and output terminals you would like, and the basic properties of the node. The screen shots below show some of this configuration:

Once you are happy with the Toolkit representation of your User-defined node, you can package it into a plugin (Follow the menu option File > Export > IBM Integration > User-defined node plug-in) and then copy the generated jar file into the dropins folder of the tools directory in your ACE installation. When you restart your Toolkit, you should find a new drawer (your node category) in the flow palette, and you should be able to drag-and-drop and then wire up your User-defined node in a message flow just like a standard message flow node:

When developing the runtime implementation code for a user-defined node in Java, you should use the MbNodeInterface interface, implement a constructor and methods for evaluate and getNodeName. The screen shot below shows a trivial example to assist with the structure of your code. Full instructions can be found using the links at the base of this section.

For detailed instructions, you can refer to the following pages in the App Connect Enterprise Knowledge Center:

Dynamic monitoring and statistics via commands mqsichangeflowstats, mqsireportflowstats, mqsichangeflowmonitoring and mqsireportflowmonitoring

ACEv11 lets you configure the collection of accounting and statistics data by modifying the settings in the configuration file for your integration node or server (node.conf.yaml or server.conf.yaml). The extract below shows the relevant section of the configuration file:

Statistics:
  # Application message flows will by default inherit Snapshot and Archive values set here
  Snapshot:
    #publicationOn: 'inactive'   # choose 1 of : active|inactive, default inactive
                                 # Also set Events.OperationalEvents.MQ|MQTT for outputFormat json,xml to be published to MQ/MQTT
    #accountingOrigin: 'none'    # choose 1 of : none|basic
    #nodeDataLevel: 'none'       # choose 1 of : none|basic|advanced
    #outputFormat: 'usertrace'   # comma separated list of : csv,bluemix,json,xml,usertrace
    #threadDataLevel: 'none'     # choose 1 of : none|basic
  Archive:
    #archivalOn: 'inactive'      # choose 1 of : active|inactive, default inactive
                                 # Also set Events.OperationalEvents.MQ|MQTT for outputFormat xml to be published to MQ/MQTT
    #accountingOrigin: 'none'    # choose 1 of : none|basic
    #majorInterval: 60           # Sets the interval in minutes at which archive statistics are published
    #nodeDataLevel: 'none'       # choose 1 of : none|basic|advanced
    #outputFormat: 'usertrace'   # comma separated list of : csv,xml,usertrace
    #threadDataLevel: 'none'     # choose 1 of : none|basic

If you activate the publication of statistics events in the .conf.yaml file, then all deployed message flows will have the publication of statistics events activated when the integration server is restarted. With ACEv11.0.0.3, the mqsichangeflowstats and mqsireportflowstats commands are provided to enable you to dynamically switch on and report statistics settings without a restart of the server being required. For example, consider a stand-alone integration server. The following command sequence shows snapshot statistics being switched on:

C:\Program Files\IBM\ACE\11.0.0.3>mqsireportflowstats --admin-host localhost --admin-port 7600 --snapshot
BIP15075I: Snapshot: Integration Server 'MyServer' is currently 'inactive'. All settings: 'inherit'.
BIP8071I: Successful command completion.

C:\Program Files\IBM\ACE\11.0.0.3>mqsichangeflowstats --admin-host localhost --admin-port 7600 --snapshot --control active
BIP8071I: Successful command completion.

C:\Program Files\IBM\ACE\11.0.0.3>mqsireportflowstats --admin-host localhost --admin-port 7600 --snapshot
BIP15075I: Snapshot: Integration Server 'MyServer' is currently 'active'. All settings: 'inherit'.
BIP8071I: Successful command completion.

Note that if the server is subsequently restarted, the statistics settings will be returned to their initial state dictated by the server.conf.yaml file. In a future ACE fix pack we intend to further enhance the ACE administrative API to provide PATCH capabilities which will be persisted to disk and remain in force when a server is restarted.

You can also check on the current flow statistics settings by issuing a query using the ACE administrative REST API. Following the example above, if you were to issue a GET request using curl, or an equivalent REST client, then the settings will be returned as shown by the JSON response below:

  curl -X GET http://localhost:7600/apiv2/statistics/snapshot
  {
    "hasChildren":false,
    "name":"snapshot",
    "type":"snapshot",
    "uri":"/apiv2/statistics/snapshot",
    "properties":{
      "accountingOrigin":"inherit",
      "name":"Snapshot",
      "nodeDataLevel":"inherit",
      "outputFormat":"inherit",
      "publicationOn":"inherit",
      "threadDataLevel":"inherit",
      "type":"Snapshot"
    },
    "descriptiveProperties":{},
    "active":{
      "publicationOn":"active"
    },
    "actions":{
      "unavailable":{
        "start-collection":"/apiv2/statistics/snapshot/start-collection"
      },
      "available":{
        "stop-collection":"/apiv2/statistics/snapshot/stop-collection"
      }
    },
    "children":{},
    "links":[]
  }

This fix pack also introduces the mqsichangeflowmonitoring and mqsireportflowmonitoring commands which can be used (in a similar way to the examples shown for flow statistics) to dynamically activate monitoring for the message flows deployed to a running integration server.

Enhancements to make Policy information accessible from Java outside a JavaCompute node

App Connect Enterprise v11 provides developers with the ability to create Policies inside Policy Projects using the ACE Toolkit. Policies are used to control connection properties and operational properties which are required by the ACE runtime. A policy can be used by an administrator to override or abstract some specific property values. For example sensitive data which might differ between runtime environments such as Dev / Test / Production. Policies can be deployed to the ACE runtime alongside message flows as part of a Broker Archive (BAR) file. In addition to the built-in message flow nodes use of policies, a user can also write code as part of their flow to access policy information to drive their behaviour. In particular, policies of type “user defined”, which contain properties and values of the user’s choice, might be useful to be accessed in this way. The equivalent capability in IBM Integration Bus was offered by user-defined configurable services. The code snippet example below demonstrates how to access properties and values from a policy using Java:

  MbPolicy myPol = MbPolicy.getPolicy("UserDefined","{MyUserDefinedPolicyProject}:MyUserDefinedPolicy");
  for (String propertyName : myPol.getProperties())
    userpolicy.createElementAsLastChild( MbElement.TYPE_NAME_VALUE, 
                                         propertyName,
                                         myPol.getPropertyValueAsString(propertyName) );

The MbPolicy class and the methods shown are included as part of jplugin2.jar which is available to both the ACE Toolkit and runtime. From ACEv11.0.0.3, these methods can be accessed by the ACE runtime outside of a JavaCompute node, so if you have ESQL logic which invokes a Java method for example, then from this fix pack you can still take advantage of code such as the above example.

ACEv11.0.0.3 provides updated language translation

App Connect Enterprise v11 is now available with up to date translated message catalogues for the following languages:

German, Spanish, French, Italian, Japanese, Korean, Polish, Portuguese Brazilian, Russian, Turkish, Simplified Chinese, and Traditional Chinese

From ACEv11.0.0.3, the ACE Toolkit, the ACE Web UI and messages which are issued in response to commands in a command console are available with translated text. When you launch the IBM App Connect Toolkit for the first time (or if you return to the welcome screen from the menu Help > Welcome) with fix pack 3 you will now find a link you can follow for help installing a language pack:


#AppConnectEnterprise(ACE)
#ACEV11
#FixpackV11.0.0.2
#FixpackV11.0.0.3

Statistics
0 Favorited
34 Views
0 Files
0 Shares
0 Downloads
Global message icon