App Connect

This latest release of ACE 13.0.3.0 introduces some updates which have been made to our data caching capabilities:

• The following new types of caching are introduced:
  • Embedded Global Cache
  • External Redis Global Cache
• The following types of caching are now deprecated and will be removed in a future release:
  • Embedded WebSphere® eXtreme Scale (WXS) grid
  • External WebSphere eXtreme Scale (WXS) grid

Previous versions of App Connect Enterprise have relied upon the WebSphere eXtreme Scale (WXS) technology in order to underpin our integration server's built-in caching capabilities. Unfortunately, the WXS technology is no longer receiving future functional enhancements, is not supported to run with Java 17 nor is it supported for use in containers. Given this, IBM App Connect Enterprise 13.0.3.0 introduces new capabilities for in-memory caching use-cases.

Users who do not want to involve a 3rd party product and who wish to cache data between separate ACE integration servers, or replace an existing embedded WXS grid, should check out the new Embedded Global Cache feature.  The new Embedded Global Cache feature aims to provide very simple configuration. To minimize disruption for users of prior versions of ACE, the existing programming interfaces are maintained for interacting with the global cache from a message flow's JavaCompute node and Graphical Mapping node, however the new implementation does require users to explicitly nominate which other servers should be involved when replicating both reads and writes to the cache. This configuration is made through the integration server's server.conf.yaml file, in the section shown below:

A detailed blog dedicated to describing the new Embedded Global Cache in ACE 13.0.3.0 has been published by Aaron Gashi here.

The Embedded Global Cache referred to above is hosted entirely within ACE integration servers. This cache is designed and optimized for use within (and between) integration servers. This Embedded Global Cache is supplied as part of IBM App Connect Enterprise and no further installation is needed to use the cache. By contrast, an External Redis Global Cache is not provided with IBM App Connect Enterprise; it must be obtained, installed, configured, and managed separately. With an external Redis Global Cache, you must create, administer, and manage the Redis components outside of IBM App Connect Enterprise. Therefore, you can separate the availability and management of your cache from IBM App Connect Enterprise. Considerations such as replication, clustering, and use of Redis API-compatible alternatives are up to you.  For an external Redis global cache, App Connect Enterprise provides limited support restricted to ensuring App Connect Enterprise's correct usage of the Redis API.

So, for users who want to share their cache with other products in addition to ACE, or for those users who want to replace an existing external WXS grid solution, a better solution would be to cache their data externally in an External Redis API compatible cache.

If you wish to use an External Redis Global Cache, you must first create a Redis Connection policy using the Toolkit Policy editor:

The Security identity property references a Redis credential:

Integration servers which have the required credential and policy defined can be used to interact with the external Redis cache by using a Mapping node or a JavaCompute node.

ACE 13.0.3.0 introduces 4 new Discovery Request message flow nodes:

• Azure Cosmos DB Request node: Azure Cosmos DB is a globally distributed, multi-model NoSQL database service from Microsoft, which is designed to provide high availability, scalability, and low-latency access to data for modern applications. It can handle unstructured, semi-structured, structured, and vector data types. Use the Azure Cosmos DB Request node to issue requests to create, retrieve, update, or delete items.
• Milvus Request node:  Milvus is an open-source, high-performance vector database designed to efficiently store and search large volumes of unstructured data. The Milvus Request node issues requests to retrieve collections, retrieve all databases, insert vector, delete vector, retrieve vectors and search vector and hybrid search vector.
• Pinecone Vector Database Request: Pinecone Vector Database is a purpose-built platform for indexing and searching dense vectors. It supports use cases like semantic search, recommendation systems, and AI-powered data analysis. Use the Pinecone Vector Database Request node to issue requests on indexes or vectors. For indexes, the node supports operations such as create index, delete index, retrieve all indexes and retrieve index details. For vectors, the node supports operations such as delete vectors, query vectors, retrieve vector IDs, retrieve vectors, update vector and update or create vector.
• Workday Request:Workday delivers cloud-based solutions that incorporate human capital management, financial management, financial performance management, analytics, and other services.

Those who are very familiar with App Connect, will know that the Workday Request node has been available for use in Designer flows for quite a long time, and despite many other discovery connectors being introduced to the Toolkit, release of ACE 13.0.3.0 will be the first time we have taken this step for the Workday Request node. One of the reasons for this delay has been related to our concerns over the large volume and size of the schema files which are required by the various supported versions of Workday. Curious readers or users of ACE, may notice some small differences with installation files compared to other discovery connector nodes. This is because the Toolkit discovery implementation and the integration server runtime implementations have been coded to not require the full set of metadata for all operations to be present on every installation of the product, and to exploit local caching mechanisms to improve performance.

Depending on your company's security posture, discovery connectors deployed to some on-premises systems might sometimes be required to go through an HTTP proxy when connecting to public endpoints. We have gradually been expanding the set of discovery connectors that can operate in this fashion and with ACE 13.0.3.0 we now have 24 discovery connector message flow nodes that can behave in this way. Full documentation is provided here.  The Salesforce Input and Salesforce Request nodes have now had this capability added. When going through connector discovery, the connection properties now have an additional Proxy name property, which is optional, as shown below:

The specified property is used to reference the name of the Policy Project and Policy which will describe the connection information for the HTTP Proxy:

The credential name defined in the policy is used when authenticating with the HTTP Proxy.

ACE 13.0.3.0 further extends the capabilities of message flows created in our Designer authoring tool to also incorporate actions involving Kafka, the real time event streaming platform that enables you to publish/subscribe, store, and process events. Kafka nodes are provided which trigger the flow (for example when new messages are published to a topic) and which involve end or mid-flow actions such as sending a message to a topic.

ACE 13.0.3.0 provides a new Container Explorer view which may be helpful for users of ACE in containers. This view lets you connect to your ACE Certified Container dashboard on the Red Hat OpenShift platform (or alternate Kubernetes), which in order to use this feature must be running the IBM App Connect Operator at version 12.10.0 or higher. In the Red Hat OpenShift plaform you can consult the list of Installed Operators:

In the ACE Toolkit, from ACE 13.0.3.0 you will find the Container Explorer on the next tab along from the Integration Explorer in the bottom left corner:

The Add connection menu option causes a connection dialog to be launched:

Fill out the Dashboard API Base URL, and the rather more obvious Username and Password. Clicking Finish will cause the Container Explorer view to be populated:

The right-click menus from the Container Explorer provide some basic administrative functions, which may be expanded over time to also encompass deeper tasks involving configuration files:

• Container Dashboards: Add connection
• Specific Dashboard: Remove Connection
• Specific Dashboard: Disconnect
• Specific Dashboard: Open Web User Interface
• BAR Files: Upload BAR
• Specific BAR File: Update BAR
• Specific BAR File: Delete BAR
• Specific Integration Runtimes: Add BAR
• Specific Integration Runtimes: Remove BAR

For more information check out the ACE documentation here.

In November 2024, IBM announced our future intention to end marketing and discontinue access of IBM App Connect Professional on Cloud. It is our plan to discontinue access to the cloud service at the end of March 2026. Users of the service are recommended to:

• Investigate IBM App Connect Enterprise as a Service. For most users of IBM App Connect Professional on Cloud, this will be the most appropriate replacement service to move to.
• Review your existing IBM App Connect Professional on Cloud deployments.
• Convert your IBM App Connect Professional on Cloud orchestrations. The IBM App Connect Professional on Cloud Studio can be used to export your orchestrations as a project archive file (.par). This file can be used as the source for the new conversion tool provided in the App Connect Enterprise Toolkit. Most users of the Studio will be best served by moving to Toolkit message flows using this conversion tool. However, if you prefer you could also create new flows from scratch using the Designer.

The App Connect Professional Design Studio defines a set of Activities.  Activities are subcategorised into a Transform folder, a Logic folder and then many additional folders, one for each 3rd party application with which you may wish to integrate.  The App Connect Enterprise Toolkit defines a palette of Message Flow Nodes.  Message Flow Nodes are grouped into two sections of the palette – the Toolbox (which provides functions that are App Connect centric) and Connectors (which provide functions for connecting to 3rd party applications).  When expanded, each drawer in the Toolkit Message Flow Palette contains multiple individual Message Flow nodes.

When you run the Toolkit conversion tool (from the menu File > Import), an application project will be created which contains one message flow for each converted orchestration. 

In most cases, the message flow will initially show errors and some additional manual configuration will be required. 

The application project will also contain an HTML file in a folder called documents.  If you select specific nodes in the message flow, each node's Description properties will include some further brief instructions. A new Open Link button on this panel can be clicked which will take you to the relevant section of the generated HTML file in the documents folder.

The generated HTML file will have instructions to help with further configuration:

The Task view is also populated with TODO items to help with further configuration:

In the last ACE mod release (13.0.2.0) the ibmint deploy command was extended with some new properties to make it easier when communicating with a remote secured integration server over SSL. With ACE 13.0.3.0, a similar set of new parameters have also been applied to a wide selection of other ibmint commands:

• ibmint create server command
• ibmint delete server command
• ibmint start server command
• ibmint stop server command
• ibmint display credentials command
• ibmint set credential command
• ibmint unset credential command

In general, it is our strategic direction to use the ibmint style of command for all new commands, but given the widespread popularity of some mqsi* style commands and the fact that the server's admin interface in v13 is now secured via HTTPS by default, on this occasion we have also chosen to apply the same security settings to a core set of mqsi* style commands as well.

• mqsicredentials command
• mqsideploy command
• mqsistart command
• mqsistop command
• mqsistartmsgflow command
• mqsistopmsgflow command
• mqsireloadsecurity command
• mqsipushapis command
• mqsichangeflowuserexits command
• mqsireportflowuserexits command

Full details of the new security parameters are available in the ACE documentation, but for quick convenience here's a summary of the new options:

--output-uri URI
URI for a remote integration server in the form tcp://[user[:password]@]host:port or in the form ssl://[user[:password]@]host:port.

--https

Specifies that HTTPS will be used for the connection to the integration node or server. If neither --https nor --no-https is specified, the connection is tried first with HTTPS and then without using HTTPS if the first attempt fails. The --https parameter is valid only if the --output-host parameter is specified, or if the --output-uri parameter is specified with a URI that starts with ssl://.

--no-https

Specifies that HTTPS will not be used for the connection to the integration node or server. If neither --https nor --no-https is specified, the connection is tried first with HTTPS and then without using HTTPS if the first attempt fails. The --no-https parameter is valid only if the --output-host parameter is specified, or if the --output-uri parameter is specified with a URI that starts with ssl://.

--cacert cacertFile

Specifies the path to the certificate file (in either PEM, P12, or JKS format) to be used to verify the integration node or server. If no cacert file is specified and default admin-ssl is enabled, the cacert file defaults to the default pem file for admin-ssl.  The --cacert parameter is valid only if HTTPS is used for the connection, so it cannot be set together with the --no-https parameter. You can set --cacert when the --https parameter has been set or when neither the --https nor --no-https parameter has been set (in which case SSL is used by default).  The --cacert parameter can be set only if the --output-host parameter is specified, or if the --output-uri parameter is specified with a URI that starts with ssl://.

--cacert-password cacertPassword

The password for password-protected cacert files. The --cacert-password parameter is valid only if HTTPS is used for the connection and if the --cacert parameter has been set. You cannot set it together with the --no-https parameter.  The --cacert-password parameter can be set only if the --output-host parameter is specified, or if the --output-uri parameter is specified with a URI that starts with ssl://.

--insecure

Specifies that the certificate that is returned by the integration node or server will not be verified.  The --insecure parameter is valid only if HTTPS is used for the connection, so it cannot be set together with the --no-https parameter. You can set --insecure when the --https parameter has been set or when neither the --https nor --no-https parameter has been set (in which case SSL is used by default).  The --insecure parameter can be set only if the --output-host parameter is specified, or if the --output-uri parameter is specified with a URI that starts with ssl://.

The HTTPRequest node in ACE 13.0.3.0 introduces a new set of Retry properties:

• Retry Mechanism: The new Retry mechanism property defaults to No retry which will maintain pre-existing behaviour of the HTTP Request node for all existing message flows that were created in earlier ACE releases. Alternatively if flow developers select Short retry then the following additional properties become editable.
• Retry threshold: This defines the number of times that retries will be attempted, with each try separated by the short retry interval. Should all the retry attempts be exhausted without success, then the failure will be treated using the node's normal behaviour with the Failure terminal.
• Short retry interval: This specifies the time interval in seconds between each retry.
• Retry condition: This property stores a list of error conditions which cause retry to be attempted. The available error codes can be HTTP status code, literal POSIX error codes on Unix or Windows Sockets error code literals (eg ECONNREFUSED). A selection dialog is provided via the Edit Condition button to help you make your desired selections.

App Connect Enterprise provides callable flow nodes which enable one message flow to easily call another message flow directly. Callable flow nodes can be considered a convenient way to split flow processing between different integration servers which can be hosted in ACE installations in entirely separate locations, perhaps split between a traditional on-premises installation and others in containers or deployed to IBM's managed ACE service on the AWS public cloud.

Callable flows are very versatile as hey work with logical message trees of every type - when callable flow nodes communicate, they take care of serializing and deserializing the logical tree at each end of the connection. This flexibility can also sometimes make callable flows harder to understand for new developers or for those who are familiar with strongly typed declared interfaces (such as Open APIs or XML schema). With this in mind, ACE 13.0.3.0 introduces new properties on the CallableInput and CallableReply nodes. The Supported Domains table on each node shows the message domains that are supported, together with information about the supported message models, message types, and physical message formats. The nodes now also contain an option to check messages against the supported domain information.

If the Check messages against supported domains option is selected on the CallableInput node, any incoming messages that are received by the node are checked against the details in the Supported domains table to see if they match. If they do not match, the message is rejected. If this option is not selected, incoming messages are processed without checking against the entries in the table, and the information in the Supported domains table is used only in the registration of the end point to provide extra details of the types of messages that can be sent into the endpoint.

For both the CallableInput and CallableReply nodes, if the Check messages against supported domains option is selected, the last folder in the message tree is accessed and is assumed to be the message body. If the body folder is owned by a parser, the external name of the parser is used to check against the domains in the supported domains table. When a match is made to a row in the supported domains table, the other columns are checked. A check is made only if the fields contain a valid character value. If a column does not contain a value, no checks are made for the column. Messages are not validated against schemas as part of the message checking. If you require schema validation, you can configure it as part of the message flow logic.

More detailed information is available in the ACE documentation