This release provides a new Business Transaction Monitoring capability which helps ACE users monitor the lifecycle of a message payload through a business transaction, which can include travelling through multiple message flows. IBM Integration Bus 10 provided a similar Business Transaction Monitoring feature, but this was not carried forward into App Connect Enterprise until now… when it returns with a new-look user-interface and the improved ability to monitor flows deployed across multiple integration servers.
We consider a Business Transaction to mean a set of events or actions that form a self-contained business use case; such as booking an airline ticket or the order, dispatch and delivery of a parcel. A Business Transaction can incorporate several different technical transactions or units of work, and can include data being exchanged across multiple protocols between message flows.
You can monitor multiple instances of a business transaction which may all be running concurrently. To extend our example, this could include monitoring when multiple customers are booking airline tickets, or ordering parcels all at the same time.
Business Transaction Monitoring is based upon the ability of ACE to publish monitoring messages over MQ, and the feature also uses a relational database to store the progress of the transactions. To use Business Transaction Monitoring, you must create a Business Transaction Definition, which specifies which monitoring events signify the start, progress and end of the business transaction. The Business Transaction Definition itself is saved into the integration server’s server.conf.yaml file. It refers to a Business Transaction Policy which stores the details of the event types and which flows they will originate from.
The ACE web user interface helps you configure and monitor Business Transactions. There are two tabs: Configure and Monitor.
- Use the Configure tab to create or modify a business transaction definition, and to define the business events that participate in the business transaction.
- Use the Monitor tab to view instances of the business transaction, and the status of those instances and associated events.
For detailed information about setting up Business Transaction Monitoring, refer to the details in the IBM documentation here.
Once configured, and with events flowing you can view the current status of instances of a Business Transaction, with the web user interface’s donut charts which look like this:
User-defined nodes allow users to extend the function of IBM App Connect Enterprise. User-defined nodes can be written in Java, C or they can be constructed using a subflow. If using a subflow, until ACE 12.0.2.0, there were restrictions placed on this kind of user-defined node as a result of it needing to be deployed as a compiled artifact (with the .cmf extension) in a BAR file. This mod release lifts these restrictions, simplifying the authoring and packaging process for a subflow user-defined node, and also providing the following benefits. You can now:
- Deploy a subflow user-defined node without the need to compile and inline resources to produce cmf in a BAR file.
- Use other resources, such as ESQL, maps, subflows, and message models within the function of a subflow user-defined node.
- Use subflow user-defined nodes within subflows so that you can easily take advantage of the subflow user-defined node feature and other features such as REST APIs and Integration Services.
- Use the WebUI to show the true structure of the subflow used within the subflow user-defined node.
- Use the visual flow debugger and step into the subflows.
- See the subflow displayed in the palette in the Message Flow editor.
This new feature for subflow user-defined nodes has been created with two distinct roles in mind:
- Subflow user-defined node authors who want to develop, test, and package a subflow user-defined node
- Subflow user-defined node consumers who want to use a subflow user-defined node as part of their message flow development
Focussing on the first of these roles, you would begin by creating your subflow implementation inside a static library. A right-click action on the subflow provides a subflow user-defined nodes menu which helps you add the subflow user-defined node to the flow palette.The dialog can be used to provide a name for the node, a tooltip and bespoke icons:
You will notice that this step causes some new files to be added to the static library project as shown below:
The subflow user-defined node is now available ready to be inserted into a message flow
Further options are available by right-clicking from the static library project level.
You can set the palette category, or as shown in the dialog below, package the subflow user-defined node ready for distribution to another user.
App Connect Enterprise helps you to push the definition of deployed REST APIs into IBM API Connect, which can then be used to manage and publish your APIs. API Connect has a vast array of features for promoting and monitoring the usage of REST APIs, including securing them by authenticating access requests from users and applications, and monitoring and fine-grained control over access and workload management, such as the ability to control the number of requests per second for a client application.
This established feature, which provides two options - the mqsipushapis command, and an action in the administrative web UI - has been further expanded in App Connect Enterprise 12.0.2.0 to also support pushing to API Connect 10. The screenshot below shows the new menu option:
For further information check out this topic in the documentation.
You can deploy TCPIP Client and Server policies to an integration server to control the connection details used by one or more TCPIP message flow nodes at run time, when sending or receiving data over a connection to or from an external application across a raw TCPIP socket.
Until this release of App Connect Enterprise, it has not been possible to redeploy any policy project containing a TCPIP Client policy or TCPIP Server policy which has previously been deployed to the same integration server with different properties. If you attempted to do this on ACE 12.0.1.0 you would have received error BIP9328E. These two policy types were previously classified as non-dynamic policies. Non-dynamic policy types lack operational flexibility as if you wish to change their properties, it is necessary to delete all deployed resources from an integration server before deploying the updated policy and any other artifacts which were previously deployed to the server which you wish to preserve.
With ACE 12.0.2.0 you can now redeploy TCPIPClient and TCPIP Server policies. Regular observers will be aware that it has been our long held strategy to make all policy types dynamic and redeployable within App Connect Enterprise. We are making good progress with this aim – 27 policy types are now dynamic, with 4 more non-dynamic policy types remaining which we hope to address in future releases.
Dynamic policy types (27): ActivityLog, Aggregation, CDServer, CICSConnection, Collector, CORBA, EmailServer, FTPServer, IMS, JDEdwards, Kafka, MonitoringProfile, MQTTPublish, MQTTSubscribe, ODMServer, PeopleSoft Connection, Resequence, SAPConnection, SecurityProfiles, SiebelConnection, SMTP, TCPIPClient, TCPIPServer, Timer, UserDefined, WorkloadManagerment, WXSServer
Currently remaining non-dynamic policy types (4): DotNet Application Domain, Java ClassLoader, JDBCProviders, JMSProviders. HTTPConnector and HTTPSConnector policies are now deprecated (these configuration settings were moved to the server.conf.yaml file).
ACE 12.0.2.0 includes an enhancement to report the time at which an application was deployed and the name of the BAR file which was used for its deployment. This information is made available through a command console session using the mqsilist command, or by examining the artifact in the development Toolkit or administrative Web UI as shown in the screen shots below.
Please note that this new deployment information will only be shown for freshly deployed applications after 12.0.2.0 has been installed and that the information will not be shown for any existing resources which were deployed at 12.0.1.0 or migrated applications created using the mqsiextractcomponents command.
This mod release continues our new ibmint command strategy for restructuring the product’s commands. This new set of commands helps you to automate administrative actions using a simple verb-first approach and provides greater consistency with common parameters between different commands. ACE 12.0.2.0 provides the following enhancements in this area:
The ibmint package command is provided with three new options:
- --keywords-file supplies a path to a file that contains keywords (such as version, authoring or custom information) which can be applied to deployable resources.
- --compile-maps-and-schemas enables the compilation of maps and schema into the BAR.
- --project supplies the name of the project to package (which can be provided multiple times for multiple projects).
There is a new ibmint delete node command:
- Use the ibmint delete node command to delete a stopped integration node.
- The following example deletes MyNode, and all the files in its work directory whilst sending verbose internal trace to the specified file:
- ibmint delete node MyNode --delete-all-files --trace C:\files\tracefile.txt
There is a new ibmint generate msgindex command:
- Use the ibmint generate msgindex command to generate an index of recorded messages.
There is also a handy cheat-sheet for all the new simplified ibmint commands.
From 12.0.2.0, IBM App Connect Enterprise expands the types of data which can be sent to an Elasticsearch, Logstash and Kibana (ELK) stack to include BIP message logs, message flow monitoring events, and activity log events. The logstash input protocol which is used for sending the data can be beats, beatsTls, http or https.
You can enable message flow monitoring events to be sent to the configured ELK Connection by setting properties in the Events.BusinessEvents.ELK section of the server.conf.yaml file. Remember you must also ensure that monitoring has been enabled in the Monitoring.MessageFlow section too:
The name of the ELK connection, must be defined elsewhere in the server.conf.yaml file (it is not shown in the snippet above). The settings which make up this connection are the same as those used in previous releases of the product when sending data to ELK. Shown below is the template for this section of the yaml file:
From 12.0.2.0, in addition to the ELK stack mentioned in the above section, IBM App Connect Enterprise also enables the writing of monitoring events to a file (in addition to the MQ and MQTT protocols which were previously supported in earlier software releases).
- The events are written to rotatable log files which you will find located in the MQSI_WORKPATH/common/monitoringEvents directory.
- For example, on Windows platforms by default this will be at the location C:\ProgramData\IBM\MQSI\common\monitoringEvents
- The files written to this directory are named MonitoringEvents_YourIntegrationServerName.json
In order to turn on the publication of monitoring events to file, you must edit the following stanza in the node.conf.yaml file:
Activity logs contain information about message flows and how they interact with external resources. You can use an Activity Log policy to control the data that is written to the activity logs, and to control how the logs are written to a file, published to Logstash in an ELK stack, or both. The screenshot below shows the most recent changes to the Activity Log policy to accommodate writing to ELK. The property called Name of the ELKConnection for logging to ELK refers to the same ELK connection information as we have dicussed in the other ELK sections of this blog post.
In previous versions of App Connect Enterprise, it has been necessary to restart the integration node in order for authorization changes to be applied. For example adding new permissions for a particular integration server required the whole integration node to be restarted in order to pick up changes, which had a much wider operational impact than necessary. To rectify this situation, with ACE 12.0.2.0 a new property can be used in the node.conf.yaml file to drive an integration node's behaviour to be dynamic in this regard. Consider the following steps which will demonstrate the new configuration property:
- Launch a command console and create a new integration node which has 3 associated servers. These commands will also create a webuser named ben, with the role iibadmin, which will have view access to both server1 and server2:
ibmint create node MyNode
ibmint start node MyNode
ibmint create server server1 --integration-node MyNode
ibmint create server server2 --integration-node MyNode
ibmint create server server3 --integration-node MyNode
mqsiwebuseradmin MyNode --create --user ben --password passw0rd --role iibadmin
mqsichangefileauth MyNode -r iibadmin -p read+,write+,execute+
mqsichangefileauth MyNode -e server1 -r iibadmin -p read+,write-,execute-
mqsichangefileauth MyNode -e server2 -r iibadmin -p read+,write-,execute-
- Edit the node.conf.yaml configuration file for MyNode (on Windows for example this would be at C:\ProgramData\IBM\MQSI\components\MyNode\node.conf.yaml) by adding the following settings for RestAdminListener.basicAuth and RestAdminListener.authorizationEnabled which turn on authorization for the Web UI:
basicAuth: true # Clients web user name and password will be authenticated when set true
authorizationEnabled: true # Clients web user role will be authorized when set true
- Restart the integration node for all the above settings to take effect:
ibmint stop node MyNode
ibmint start node MyNode
- Open a web user interface, and log in with the user ben. The log in will be successful, and you will be able to see the properties of MyNode, and the fact that it owns server1 and server2 ... but other actions such as deploying to the servers will not be available at this stage
- Now, to demonstrate behaviour prior to ACE 12.0.2.0, execute the following command whose purpose is to give users with the role iibadmin (such as the ben webuser) further execute privileges when interacting with server1:
mqsichangefileauth MyNode -e server1 -r iibadmin -p read+,write+,execute+
The above command will require MyNode to be restarted. Prove this by logging in as ben to the integration node's web ui again, and note that this user still does not have the right to deploy to server1. Now restart the integration node and try again, and you will find that the mqsichangefileauth command will have now become effective and ben will be able to deploy to server1.
- Having demonstrated the old behaiour, now stop the node and apply the following new dynamicUpdate setting, which will cause further mqsichangefilauth changes to become effective without requiring a restart of the node:
ibmint stop node MyNode
Edit the node.conf.yaml file for MyNode by adding the property Security.dynamicUpdate and setting its value to true:
Security:
dynamicUpdate: true
ibmint start node MyNode
- The dynamicUpdate setting which has now been applied, means that further changes made via the mqsichangefileauth command will be picked up without requiring the node to be restarted. Make the same change to server2 in order to demonstrate this:
mqsichangefileauth MyNode -e server2 -r iibadmin -p read+,write+,execute+
- After a few seconds, log back in to the web UI and you will find you can now deploy to server2.
In summary then, the new node.conf.yaml property described above causes the integration node to check the last modified time of the YAML files and update its internal state when the file changes. This update is delayed for five seconds after the last change in order to avoid picking up partial sets of changes halfway through a set of scripted changes; instead the updates will be picked up after the files have stabilised.
Users of the product on Windows platforms may notice a small change we have made to the product's installer. When you download the product, you should extract the downloaded .zip file for ACE 12.0.2.0 in order to unpack the installation files into a local directory: For example, if you extracted the file ACESetup12.0.2.0.zip into C:\temp, the following extracted files will then appear in C:\temp\ACESetup12.0.2.0
- ACESetup12.0.2.0.exe. This file is the installation executable file.
- ACEtk1.cab. This file contains the product files for the IBM App Connect Enterprise Toolkit component.
- ACEae1.cab. This file contains the product files for the OpenAPI editor component.
- ACEws1.cab. This file contains the product files for the WebSphere Service Registry and Repository nodes component.
- readmes. This folder contains the readme files for all of the supported national languages.
When you run the installation executable file ACESetup12.0.2.0.exe, the separate cab files are used for the installation of different components of the product. Taking this architectural approach will help us with future strategies to make certain parts of the product installation optional. In the current release we are confident that these changes will also result in a slightly reduced time taken for the installation to complete.
The ACE 12.0.2.0 release also introduces support for POWER 10 hardware processor for both AIX 7.2 and RHEL 8.4. Full details can be found on the formal system requirements pages (start here)