The Designer authoring tool has been enhanced in ACE 13.0.6.0 to provide for the import of Open API documents which describe third party REST API interfaces that you would like to invoke from a Designer flow. From the Connect section of the main menu (on the left side of the browser window), navigate to the Applications and APIs section, and a list of apps will be displayed (sorted into two groups depending on whether you have provided connection info yet). To the right of the search box, click the plus icon, as shown below:
In the resulting dialog you can browse your local filesystem for a suitable Open API definition:
Alongside the new Designer feature discussed in the previous section regarding the import of Open API documents that can be used when invoking an external REST API, ACE 13.0.6.0 also supports using the OAuth 2.0 authorization framework when using outbound REST requests to communicate with external REST APIs. OAuth 2.0 is an authorization framework that allows third-party applications to access protected resources on a user's behalf without needing to directly share the user's credentials. In the context of REST APIs, OAuth 2.0 enables secure access to resources by delegating authorization to an authorization server and then using access tokens granted by the authorization server to check what an application is allowed to do, instead of relying on passwords.
The OAuth settings are exposed to a Designer flow developer after the Open API document which represents the external REST API has been imported. From within a message flow, the flow developer can scroll through the list of applications and select the API. In our previous example we called this MyApi and you can see in the screenshot below, that a developer then has an enumerated drop-down showing the various available Authorization types:
The Authorization type which you select from the enumerated drop-down will control the rest of the data fields that are presented to the flow developer:
With ACE 13.0.6.0 we have introduced a similar capability when making connections to third party applications from Designer message flows. This includes support at authoring time, and then also at runtime. To use an HTTP Proxy connection from Designer, users should begin by navigating to theConnect section of the main menu (on the left side of the browser window) and select the HTTP proxies option.
This will display the HTTP proxy configuration page, where you can see a table view of previously created proxies, or create a new one by clicking the button to Create HTTP proxy. The resulting Define proxy dialog will prompt you for a Proxy name (to identify the proxy), the Proxy URL, Username and Password. Having created a proxy configuration, it can then be referenced when connecting to several applications. The currently supported set of application connections which can exploit an HTTP proxy connection (both in Toolkit and in Designer) is as follows: Amazon DynamoDB, Amazon EC2, Amazon EventBridge, Amazon Kinesis, Amazon RDS, Amazon S3, Amazon SNS, Amazon SQS, Astra DB, AWS Lambda, Confluence, Databricks, GitHub, Google Chat, Google Drive, Google Gemini, Google Sheets, IBM DB2, Microsoft Azure Blob Storage, Microsoft Azure Event Hubs, Microsoft Azure Service Bus, Microsoft Dynamics 365 for Finance and Operations, Microsoft Dynamics 365 for Sales, Microsoft Entra ID, Microsoft Exchange, Microsoft SharePoint, Microsoft Teams, SAP Ariba, ServiceNow, Shopify, Slack, Snowflake, Splunk, Workday
The Define proxy dialog is shown below:
Once you have one or more proxies defined, when connecting an application, you can select your desired proxy from the enumerated drop-down list as shown in the example below with the Salesforce connector:
Continuing our mission to expand the available Toolkit message flow nodes for easy connection to third party applications, this quarter ACE 13.0.6.0 has added three new Discovery Request Message Flow nodes:
- Apache Pulsar Request node: Apache Pulsar is a cloud-native messaging and event streaming platform that supports high-throughput, low-latency, and real-time data processing. Use the Apache Pulsar Request node to connect to Apache Pulsar and issue requests to perform actions on objects such as publishing a message to a topic and retrieving messages from a subscription.
- Astra DB Request node: Astra DB is a serverless, cloud-native database from DataStax (now an IBM company after it was acquired earlier this calendar year). It provides scalable storage, search, and vector capabilities to help you build AI and data-driven applications. Use the Astra DB Request node to connect to Astra DB Request and issue requests to perform actions on objects such as Retrieve all collections and Create, Retrieve, Update or Delete documents. The Astra DB Request node operations are synchronous and non-transactional, which means that, if a message flow fails and rolls back after the Astra DB Request node, the operation on the data source will still complete.
- Databricks Request node: Databricks is a cloud-based platform for big data and AI. It combines Apache Spark with Azure services to simplify data engineering, analytics, and machine learning. It combines elements of data warehouses and data lakes to enable management and analysis of both structured and unstructured data. Use the Databricks Request node to connect to Databricks and issue requests to create, update, delete, or retrieve records in a table. The Databricks Request node operations are synchronous and non-transactional, which means that, if a message flow fails and rolls back after the Databricks Request node, the operation on the data source will still complete.
Each new type of connector also has a corresponding new policy type, which helps Toolkit users define configuration properties for easy connection to the applications. These policies also link to credential information that can be encrypted and stored in an ACE vault, enabling the ACE runtime to safely and securely connect to your applications. For example, here's a picture of the new Apache Pulsar policy type:
In ACE 13.0.6.0 we have added four new Discovery input nodes - one for each of the new applications mentioned in the previous section, and also a new SAP SuccessFactors Input node to go alongside the SAP SuccessFactors Request node which has been provided in prior ACE versions:
Earlier this year in ACE 13.0.3.0, we added capability to the Toolkit HTTPRequest node to provide for automatic retries in runtime situations where the invocation of an external service results in particular HTTP status codes or other socket errors being returned. A very similar capability has just been added to the REST Request node in this month's new ACE 13.0.6.0 mod release. The REST Request node in ACE 13.0.6.0 includes the following properties on a new Retry properties tab:
- Retry Mechanism: The new Retry mechanism property defaults to
No retry which will maintain pre-existing behaviour of the HTTP Request node for all existing message flows that were created in earlier ACE releases. Alternatively if flow developers select Short retry then the following additional properties become editable.
- Retry threshold: This defines the number of times that retries will be attempted, with each try separated by the short retry interval. Should all the retry attempts be exhausted without success, then the failure will be treated using the node's normal behaviour with the Failure terminal.
- Short retry interval: This specifies the time interval in seconds between each retry.
- Retry condition: This property stores a list of error conditions which cause retry to be attempted. The available error codes can be HTTP status code, literal POSIX error codes on Unix or Windows Sockets error code literals (eg
ECONNREFUSED). A selection dialog is provided via the Edit Condition button to help you make your desired selections.
ACE 13.0.6.0 includes some usability enhancements for the Toolkit which make it easier to work with policies. You may notice an improved look-and-feel whereby some policies now group together similar properties into twistie sections. For example, the MQEndpoint policy shown below has expandable twistie sections for Description, SSL and Reconnect properties:
Another example of the improved layout can be seen below with the REST Request policy type which has a table structure for the definition of security schemes:
Now that it is possible to add Version, Short Description and Long Description properties to a policy, you can include keywords within the description fields. These keywords can then be viewed in the Toolkit Properties panel, when a deployed policy is selected in the Integration Explorer view:
The same properties can also be displayed when using the mqsilist command to report on the deployed artifacts within an integration-node-owned integration server:
The ACE Toolkit's Scheduler node is used to trigger a message flow to run at specified intervals of time, or at specified times on specified days. In ACE 13.0.6.0 the node's Advanced properties tab has been expanded to incorporate a new setting named Missed Event Mode. When you configure a state persistence policy for the Scheduler node, the Missed Event Mode property becomes available. You can use this property to specify how the Scheduler node handles events that were missed while the integration server was stopped or unavailable. In general, message flow input nodes use the State Persistence policy to save their state, such as the last poll time, stream position, or last event time. In this way, a State Persistence policy helps a message flow to be able to resume from the same point after a restart. State Persistence can be useful when the integration server stops unexpectedly or if it is restarted during processing.
The options provided by the Missed Event Mode enumerated dropdown and their meanings are:
- None: Do not issue any missed events. The Scheduler node continues with the original schedule.
- Reset: Issue one missed event immediately and start a new schedule from the current time.
- Last: Issue one missed event and then resume the original schedule.
- All: Issue all missed events and then resume the original schedule.
A good way to describe these settings is to consider an example. Imagine the Scheduler node has been configured to emit events (i.e. trigger the message flow) every two minutes. Imagine the flow starts at 10:20. Everything is working perfectly and as you might expect, the flow runs at 10:22, 10:24, 10:26, 10:28 and 10:30. At this stage the integration server is stopped and it remains stopped until it is restarted at 10:41 (so in total it has been stopped for 11 minutes, during which time
For example, the Scheduler node is configured to emit an event every 2 minutes. The last event was emitted at 10:30. The integration server goes down and restarts at 10:41.
- None: No missed events are emitted. The next event occurs at 10:42.
- Reset: One event is emitted immediately at 10:40, and the next event occurs at 10:43 (based on the new schedule).
- Last: One event is emitted immediately at 10:40, and the next event occurs at 10:42 (following the original schedule).
- All: Missed events for 10:32, 10:34, 10:36, 10:38 and 10:40 are emitted immediately, then the next event occurs at 10:42 (following the original schedule).
On Windows platforms, the graphical installer for ACE 13.0.6.0 has an improved layout showing the optional (non-default) selection of components that can be made by users who do not want to install the full product:
OpenTelemetry is a Cloud Native Computing Foundation (CNCF) project that the CNCF accepted in May 2019 and then moved to the Incubating maturity level in August 2021. The adoption and popularity of the OpenTelemetry approach to observability has been gaining traction ever since. OpenTelemetry provides a vendor and tool agnostic framework for improving the observability of applications. Within the OpenTelemetry standards, ACE has a long track record of promoting the OpenTelemetry approach to tracing, which helps users to diagnose the elapsed time spent by a piece of data within different applications. ACE offers configuration for emitting trace events when interacting with the MQ nodes, HTTP nodes, REST nodes, SOAP nodes, Callable Flow nodes and Kafka nodes. A trace typically consists of many Spans, where a span typically represents the actions that occurred within a particular application and has information to make querying and debugging easier such as a name, a status, a span kind, and time-related data.
With this feature, when OpenTelemetry tracing is enabled for an ACE integration server, span-related details that are generated during message flow execution, are applied to Activity Log entries in the Tags section. The available OpenTelemetry tags include:
- TRACE_ID
- SPAN_ID
- TRACE_STATE
- TRACE_PARENT
These details are visible across all output modes of the activity log, including IBM App Connect Enterprise WebUI, and ibmjson format in console logs, and activity log file logging. This enhancement simplifies correlating events between OpenTelemetry-based observability platforms and the IBM App Connect Enterprise logging system, therefore improving diagnostics and troubleshooting.
Consider the example of a simple message flow HTTP Input > Compute > HTTP Reply When this flow is invoked, Activity Log entries relating to the message flow are tagged (in the column of the table named Tags) as shown in the screenshot below:
This feature was actually first shipped in the ACE 13.0.5.1 maintenance release, and is the subject of a separate dedicated blog here.
IBM App Connect Enterprise provides various logs to assist with problem determination and troubleshooting that each produce slightly different output file formats. From ACE 13.0.6.0 a new Log Analyser tool is provided within the product installation which can perform post-processing on log files to . The tool supports the processing of the following types of logs:
- Service trace or User Trace files
- Activity Log files in csv format
- Message Flow Accounting and Statistics files in csv format
- Parser Manager Logs
You can launch the Log Analyser tool from an ACE Command Console using this command:
java -Xmx2000m -jar ./server/tools/aceloganalyser.jar
Having selected a Tool Functionality, Input Files and Output Directory, when you click the Process button, the Status window will be updated as shown below:
If you then click the hyperlink to Open Output File a static HTML report will be opened in your browser:
The table below documents the varous Tool Functionality options which are currently available: