App Connect

OpenTelemetry is an incubating Cloud Native Computing Foundation project which provides an open source framework for tracking and correlating data as it passes between disparate applications. Designed to complement the existing App Connect Enterprise monitoring features, this new feature provides a light-weight method of instrumentation which enables an ACE integration server to export span data to OpenTelemetry collectors. OpenTelemetry collectors can be thought of as vendor-agnostic proxies that receive data before sending it on to one or more observability back-ends. This new feature makes it easy to use tools such as IBM Instana or Jaeger to help trace data as it flows across multiple protocols and applications. Support is provided for the Linux x86-64 platform and the feature works when ACE is installed on-premises using integration node owned servers or when ACE is installed into a container environment using independent integration servers.  OpenTelemetry is enabled at the integration server level, and spans are created for all message flow nodes that support OpenTelemetry:

• MQInput, MQOutput, MQReply, MQGet, MQPublication
• HTTPInput, HTTPReply, HTTPRequest, HTTPAsyncRequest, HTTPAsyncResponse
• RESTRequest, RESTAsyncRequest, RESTAsyncResponse
• SOAPInput, SOAPReply, SOAPRequest, SOAPAsyncRequest, SOAPAsyncResponse
• CallableInput, CallableReply, CallableFlowInvoke, CallableFlowAsyncInvoke, CallableFlowAsyncResponse

To switch on OpenTelemetry, before starting your integration server uncomment the relevant properties in the template server.conf.yaml file which you will find in your integration server's working directory as shown below:

To better understand the kind of observability which is possible through this new feature, let's consider a situation that involves a set of ACE message flows which are available through the Toolkit Tutorials Gallery in an example which shows how to use MQ and HTTP to link message flows in a coordinated request-reply scenario:

App Connect Enterprise provides the FTEInput and FTEOutput message flow nodes which provide an easy to use interface to the IBM MQ Managed File Transfer capabilities. IBM MQ Managed File Transfer transfers files between agent processes by dividing each file into one or more messages and transmitting the messages through your IBM MQ network. Previously this feature has required ACE to be co-located with an MQ Queue Manager. From ACEv12.0.7.0, when you use an independent integration server (either in a development environment or within a container) in combination with a remote default queue manager, this remote default queue manager can also be used for the use case of sending or receiving files by using IBM MQ Managed File Transfer.  This use case is offered alongside the other features of remote default queue managers such as using the Event Driven Architecture message flow nodes which have been available since ACEv11.0.0.7

To use this new feature, ensure that your integration server specifies an MQEndpoint Policy to define a remoteDefaultQueueManager in its server.conf.yaml file:

Update the server.conf.yaml file to specify the location of the IBM MQ File Transfer Agent configuration files and the IBM MQ File Transfer Agent name, which must match the name in the agent.properties configuration file. If you specify a coordination queue manager in the server.conf.yaml file, it must also match the name in the coordination.properties configuration file. An example server.conf.yaml configuration is as follows:

The property agentConfigurationDirectory, specifies the directory on the file system where the integration server initially locates the properties files that are required by the IBM MQ File Transfer Agent that runs as part of the IBM App Connect Enterprise integration server. When the integration server is first started, these properties files are copied into the working directory of the integration server (into the config/FTE/default subdirectory structure). The copying process happens only the first time that the integration server is started, so if you make configuration mistakes you must correct the files in the config/FTE/default subdirectory structure of the server's working directory.

The IBM MQ File Transfer Agent configuration involves other properties files (agent.properties, coordination.properties and command.properties) and the queue manager also requires the scripted creation of some queues. We won't go into the details here, but you can read all about it with full instructions in our documentation. 

With your server configured, you can send files using simple message flows. Here's an example using our set up as described above, utilizing an independent integration server on Windows and a remote queue manager on Linux:

The message flow shown is triggered by an HTTP input message, and uses a while loop in the Compute node to propagate multiple lines into an example output file.

The message flow in our example constructs an HTTPReply message which includes information from the LocalEnvironment tree which summarises the file transfer:

App Connect Enterprise (and IBM Integration Bus before it) helps you split processing between different integration servers. These servers can be deployed in your own data center, in containers, or in the public cloud. Given this, message flows need to communicate securely and to help with this they use an ACE component which we refer to as a Switch server. The Switch server is a special kind of integration server that routes data. The connectivity agents contain the certificates that your flows require to communicate securely with the Switch server:

Switch servers have been a capability of the product for many years now and you can read more about them here.  Over time, we have expanded the initial usecases for the Switch to go beyond connection from the public cloud, to also encompass callable flows, remote debugging and remote administration:

Due to this expansion of use cases, the configuration and terminology of this area of the product has become more complex, so from ACEv12.0.7.0 we have made some small changes which aim to simplify this area of configuration moving forward. The json configuration files which provide the different agent types have now been amalgamated together for greater simplicity. In future you will simply have Switch Servers and Switch Clients. The existing configuration files remain supported and will continue to work for back compatability purposes so existing users need not be concerned.

ACEv12.0.7.0 introduces a new property on the HTTPReply node to specify the compression of data which is sent in the HTTP response.  You can uniquely select this setting on a message flow node by message flow node basis.


If you prefer to achieve the same outcome but through product configuration without changing your message flow's properties then you can do that too, by enabling the compression of response data for all HTTPReply nodes being used within any given integration server.  A default HTTPReplyWithCompression policy is provided as part of the product installation, which you can use to set up compression properties for all HTTPReply nodes on an integration server. To enable the default properties in the policy, uncomment the HTTPReplyWithCompression property in the server.conf.yaml configuration file, as shown in the following example:

For more information, check out our documentation here.

Ease of use when viewing Message Formats in the Flow Exerciser

The Flow Exerciser has been enhanced to make it easier to read the data being displayed. New radio buttons have been added to allow a choice between Raw text and Formatted text. Some simple JSON and XML examples are shown below:

Deploying referenced shared libraries

When using the Toolkit to deploy an application to an integration server, if the application has a dependency on a shared library, from ACEv12.0.7.0 you can choose whether to include the shared library in the BAR file that is generated when the application is deployed. If you select the option to include the shared library, any nested shared libraries are also included in the BAR file. This option is available only if the shared library exists in the Toolkit workspace.  This new capability has been introduced as a development feature to help simplify and speed up development. It's acknowledged that in some very occasional circumstances this behaviour may be undesirable for a small group of Toolkit users, so this behaviour has been specifically implemented within the Toolkit and doesn't apply to other situations such as using the mqsideploy command. A confirmation message will also be shown as an added protection: