Explore the new features in App Connect Enterprise version 11.0.0.9

When ACEv11.0.0.8 was released, provided a new capability to carry out local execution of IBM Operational Decision Manager business rules, using an embedded rules engine inside the integration server process.

You can execute rulesets which have parameters of XML type using the ODMRules node:

You can execute rulesets which have Java parameters or XML parameters using a JavaCompute node:

This feature was initially defined as Technology Preview Code and at the time we blogged about our desire to lift these restrictions on its use in a forthcoming fix pack. ACEv11.0.0.9 now declares the feature ready for production usage.

IBM ODM business rules are organized into rulesets. Fix pack 9 introduces a new syntax variant for how the ruleset path can be used to refer to a version of a rule. The ruleset path incorporates the name of both the RuleApp and RuleSet. Both these sections can include a version. From ACEv11.0.0.9 you can also use the string “Latest” rather a defined version number. When first referenced, the “Latest” version of a given RuleSet or RuleApp is loaded and cached in memory, at the ACE Application level. This means that all ODM Rules node instances in all flows within any given application will evaluate the same result no matter when they are first referenced. Currently, the only mechanism you have to load updates and re-evaluate what the “Latest” version means, is to stop and start the application. Although enhancements can never be guaranteed, in the future we expect to expand this functionality to allow for automatic polling of rule updates.

Another enhancement in this area which is provided with fix pack 9 is the ability to avoid reliance on an ODM server and instead use your ODMServer policy to reference a ruleset archive on the local file system of the integration server. To achieve this edit the resServerURL property of the ODMServer policy:

• For example on a UNIX operating system, file:/Users/admin/ACE/rulesets/ruleApp_Sale_1.0.jar
• For example on a Windows operating system, file:///C:\Users\Administrator\Downloads\ruleApp_Sale_1.0.jar

The ODM capability also inserts entries into the Activity Log in the browser based ACE Web UI, and now provides Resource Statistics indicating the number of successful and failed rule executions:

From ACEv11.0.0.9 we are now offering a display of message flow activity logs in the Web User Interface. Activity Logs help you to understand what your message flows are doing by providing a high-level overview of how they interact with external resources. Normally, this doesn’t require detailed product knowledge, and aims to help enhance your understanding of the overall system. Activity Log messages are concise and avoid technical complexity, although more information is provided in the message detail. Because the log entries are short, uncomplicated, and focused on single activities, they can be quickly scanned and understood. Activity Logs have been available for writing to a circular file system throughout the ACEv11 release using an ActivityLog policy to define the configuraiton details. This policy approach also provides filtering capabilities, based on tags. Because Activity Logs are enabled by default, they can be useful for troubleshooting. For example, if the Activity Log for a message flow shows a prolonged period of inactivity, review the Activity Logs for associated resource types. You might discover that a problem with an external resource is the reason why messages are no longer being processed by your message flow. All message flows emit activity log entries. This capability is always enabled by default because it has an extremely low performance overhead, however if you wish you can disable it using the integration server’s YAML configuration.

In the web UI for any flow, you can now see a new tab named “Activity log”, as shown in the picture below:

On the activity log tab, we don’t immediately fetch the activity log. You’ll see a message stating Fetching the activity log causes the message flow to momentarily stop processing messages. Click the button and you should see table entries:

On the right side, just above the table, there is a display showing the date and time when the data was last updated. Clicking the Refresh button will fetch the updated activity log entries. Depending on how long it has been since you last refreshed and how active your flow has been, you may see some of the same data entries when you refresh or an entirely separate set of entries. This is because the activity log is a circular log of 1000 entries by default.

Starting from ACEv11.0.0.9, we are now providing a new dedicated component for helping you test your deployed REST APIs. This Web User Interface capability was originally developed for our sister product, IBM API Connect and it is now used in various parts of IBM’s integration portfolio, including App Connect Enterprise. Eagle-eyed ACE users may also recognise this REST API explorer as the same one we were already using to let you explore ACE’s own administration REST API, which you can find on the URL route of /apidocs from a started integration server or integration node.

In the example further below, we are looking at a stand-alone integration server to which we have deployed a REST API named CustomerDatabaseV1. This REST API is available from the ACE product tutorial called Using a REST API to manage a set of records. You can navigate to this tutorial from inside the ACE Toolkit using the menu Help > Tutorials Gallery. The tutorial has instructions and links for importing and deploying the example REST API:

The picture below shows the REST API has been deployed to an integration server, and we have clicked on one of the available operations, and also switched to the Try it tab.

From here, you can invoke the REST API and view the response:

If you are using an integration server with default settings, when you attempt to invoke your REST API you will receive an error. This will most likely be due to a need to enable your integration server to permit cross-origin requests from a web browser by enabling Cross-Origin Resource Sharing (CORS). There are quite a few control options you can select, depending on how open you would like your CORS settings to be. To help you get up and running the example below shows one very open set of configuration options in the integration server’s server.conf.yaml file. Remember you need to restart your integration server after you have editted and saved the configuration file:

Fix pack 9 of App Connect Enterprise provides a richer set of administration REST API options and commands for reporting and changing trace settings for both stand-alone integration servers and node-owned integration servers. Users may have noticed in previous ACE fix packs the mqsireporttrace command reported BIP8385E: This command is not currently available in IBM App Connect Enterprise. but now with this new capability enabled, the command provides a rich set of options. If you use a simple command variant with the name of your integration node you will be presented with a hierarchical layout showing all the currently active settings:

C:\Program Files\IBM\ACE\11.0.0.9>mqsireporttrace ACEv11009Node
ACEv11009Node
  trace
    agentTrace
      serviceTrace
        level='none'
        traceSizeKb=1048576
        mode='fast'
  servers
    MyServer
      trace
        userTrace
          level='none'
          traceSizeKb=1048576
          mode='safe'
        serviceTrace
          level='none'
          traceSizeKb=1048576
          mode='safe'
          stopOn=''
        traceNodes
          enabled='on'

This fix pack also now provides some new options on the mqsichangetrace command including a new --non-persist option. This reflects the fact that the administrative REST API now has a set of PATCH methods available too. New methods have been provided for both standalone integration servers and node-owned integration servers. Full documentation of these possibilities can be viewed in the admin REST API documentation which is served directly from a running server using the /apidocs route, but a summary of some of the paths which are now available are listed here for convenience:

• GET /apiv2/trace/agent
• POST /apiv2/trace/agent/service-trace
• GET /apiv2/trace/agent/service-trace
• PATCH /apiv2/trace/agent/service-trace
• GET /apiv2/servers/{server}/trace
• POST /apiv2/servers/{server}/trace/service-trace
• GET /apiv2/servers/{server}/trace/service-trace
• PATCH /apiv2/servers/{server}/trace/service-trace
• POST /apiv2/servers/{server}/trace/user-trace
• GET /apiv2/servers/{server}/trace/user-trace
• PATCH /apiv2/servers/{server}/trace/user-trace
• POST /apiv2/servers/{server}/trace/trace-nodes
• GET /apiv2/servers/{server}/trace/trace-nodes
• PATCH /apiv2/servers/{server}/trace/trace-nodes

From fix pack 9, you can now secure your integrations by implementing basic authentication on deployed message flows, so that incoming requests must supply a username and password, which are then authenticated against credentials that are stored locally in App Connect Enterprise (as opposed to utilising a third party system such as LDAP for example).

We will consider a very simple example flow here, which does nothing more than echo an HTTP message back to the requesting client:

The HTTP Input node’s properties have a Security tab which includes the allocation of a security profile:

The referenced security profile policy is used to specify that the flow would like to take advantage of local enforcement:

Define the credentials which you would like to have the flow authenticate locally, using either the mqsisetdbparms command or the mqsicredentials command:

• mqsisetdbparms -w workDir -n local::LocalCredentialAlias –u user1 –p password123
• mqsicredentials --create --work-dir workDir --credential-type local --credential-name LocalCredentialAlias --username user1 --password password123

Having defined these settings and deployed the artifacts, you should find that when you invoke the simple flow, for example in the picture below we use Curl for this, the identity is authenticated. To demonstrate the example here, the password has first been incorrectly set to the value password before the second attempt uses the correct value of password123

For more details, please check out the Knowledge Center page on Authenticating incoming requests by using credentials stored in the vault

Fix pack 9 introduces the ability to dynamically load a new server security certificate. The server certificate can be applied without the need to restart the integration server. When receiving secured inbound connections over HTTPS, integration servers use one of their components called the HTTPS Connector resource manager. You can query and update the HTTPS Connector resource manager by using the App Connect Enterprise administration REST API. You do not have to restart the integration server for such updates to take effect. There are a number of situations that might require updates to the properties of the HTTPS Connector resource manager in an integration server. You can dynamically update the properties in question by using the PATCH verb in the administration REST API. Updates that are made by using the PATCH verb are persisted in the overrides sub-directory of the integration server working directory meaning that, if the integration server is restarted, the updates are not lost. Several HTTPS Connector resource manager properties can be updated using these new REST API methods:

• KeyAlias
• KeystoreFile
• KeystorePassword
• KeystoreType
• TruststoreFile
• TruststorePassword
• TruststoreType

Being able to dynamically update these properties, and the KeystoreFile property in particular, is useful in the situation where you are using certificate authorities that refresh their certificates on a frequent basis but you do not want to restart the integration server whenever the refresh occurs. Updates take effect on the first inbound connection following the update; preexisting connections will retain the original configuration. You do not have to restart the integration server for the changes to take effect. For example, to update the KeyAlias, KeystoreFile, KeystorePassword, and KeystoreType properties for the HTTPS Connector resource manager for an integration node owned integration server, use the following curl command with the PATCH verb:

curl -X PATCH -H "Content-Type: application/json" http://hostname:port/apiv2/servers/{server}/resource-managers/https-connector -d "{\"properties\": {\"KeyAlias\":\"server1cert\",\"KeystoreFile\": \"C:\\temp\\myserver_keystore.p12\",\"KeystorePassword\": \"server::keystorePass\",\"KeystoreType\": \"p12\"}}"

If the command is successful, an HTTP status code 204 is returned and you should see the updated values in the overrides sub-directory of the integration server working directory.

You can also reuse the same keystore file but with a different value for the server security certificate to refresh the server certificate to be used by the integration server without having to restart the integration server. If you are in this situation where the keystore file name has not changed, but its contents have, then instead of updating the server certificate by using the PATCH verb (as discussed above) in the REST API, you must use the POST method to load a new server certificate that you have defined in a certificate file with the same name as one that you have previously used. For example, for a node-owned integration server you could use the Curl command below:

curl -X POST http://hostname:port/apiv2/servers/{server}/resource-managers/https-connector/refresh-tls-config

We have been enhancing App Connect Enterprise with new capabilities for interacting with Kafka across several fix pack releases. For example you can:

• Use the KafkaRead node to read an individual message published on a Kafka topic, by specifying the offset position of the message in the topic.
• Attach a Kafka policy to one or more Kafka nodes in a message flow to control, at run time, the properties used to connect to the Kafka cluster.
• Set and retrieve Kafka custom header properties including custom header properties to access an Apache Avro schema from a schema registry
• Configure security credentials for connecting to Kafka

In this most recent fix pack ACEv11.0.0.9 we have added support for using the SASL/SCRAM authentication mechanism for securing connections between Kafka message flow nodes and a Kafka cluster. Salted Challenge Response Authentication Mechanism (SCRAM), also known as SASL/SCRAM, is an SASL mechanism that performs password-based authentication between the client and server, and resolves some of the security concerns that are associated with SASL_PLAIN authentication. To authenticate a connection between Kafka nodes and a Kafka cluster that uses SCRAM for authentication, you must configure the nodes to use a Kafka policy that is configured with the connection details.

To take advantage of this new feature, when you create a Kafka policy in a policy project:

• Set the SASL config property to org.apache.kafka.common.security.scram.ScramLoginModule required;
• Set the Security mechanism property to either SCRAM-SHA-256 or SCRAM-SHA-512, as required by the Kafka cluster.

Also, remember to use either the mqsisetdbparms command or the mqsicredentials command to associate a username and password with a security identity.

• mqsisetdbparms -w workDir -n kafka::myKafkaSecId -u myUsername -p myPassword
• mqsicredentials --create --work-dir workDir --credential-type kafka --credential-name myKafkaSecId --username myUsername --password myPassword

For more details, please check out the Knowledge Center page on Authenticating connections to a Kafka cluster by using SASL/SCRAM