IBM webMethods Hybrid Integration

IBM webMethods Hybrid Integration

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.



#Automation

 View Only

Embedded global cache and external Redis cache in IBM webMethods Hybrid Integration

By Srikant V S posted 18 days ago

  

Introduction

A global cache is a repository for data that you want to reuse. The cache facilitates the sharing of data within an App Connect runtime and between App Connect runtimes, and eliminates the need for an alternative solution, such as a database. You can use one message flow node to store data in the global cache, then a second node (in the same message flow or a separate flow), can retrieve that data from the global cache. For a deeper understanding of caching concepts and architecture, see the data caching overview in the product documentation

The embedded global cache is hosted in App Connect runtimes. The embedded global cache is supplied as part of IBM App Connect Enterprise and no further installation is needed to use the cache. This embedded cache is a new feature in App Connect Enterprise that replaces the previous implementation based on IBM WebSphere eXtreme Scale. By contrast, an external Redis global cache is not provided with App Connect Enterprise; it must be obtained, installed, configured, and managed separately. Both options are available for users of App Connect in IBM webMethods Hybrid Integration, and we will explore both in this blog.

Embedded global cache in webMethods Hybrid Integration

The embedded global cache in App Connect Enterprise 13.0.3.0 is enabled by default and consumes virtually no CPU or memory until data is stored. It can operate without replication, similar to the local cache, making it lightweight and efficient. Unless your App Connect runtime is explicitly configured to use the local cache by default, or you’re using a Redis Connection policy to connect to an external Redis instance, any global maps accessed will automatically use the embedded global cache. In this article, we’ll take a hands-on journey into the power of embedded global cache in App Connect by building message flows that demonstrate how to store, retrieve, and manage data in-memory—unlocking faster, smarter integrations without relying on external databases.

Prerequisites

I have included all artifacts required to set up the scenario which will be described in detail here.
To follow along and setup a similar system, you will need:

  • An instance of webMethods Hybrid Integration, with the following capability enabled:
    • App Connect
  • The following App Connect Enterprise artifacts:

Building the message Flows

To explore the capabilities of the embedded global cache in App Connect, we’ll build three message flows using the App Connect Enterprise Toolkit. Each flow will expose an HTTP endpoint and perform a specific cache operation: write to cache, read from cache, and clear the cache. These flows use an HTTPInput node to receive requests, a JavaCompute node to interact with the cache programmatically, and an HTTPReply node to send responses back to the client. If you’re new to creating message flows, refer to create and manage message flows in the product documentation for detailed steps.

1. Import the Project Interchange File

  • Download the attached PI file.
  • In the Toolkit, go to File → Import → Project Interchange.
  • Select the PI file and import it.
    That’s it! Your application and flow are ready to use.

The following examples show the evaluate() methods for each JavaCompute node, which showcase how to interact with the cache programmatically. Later, we’ll package these flows into a BAR file and test them using cURL commands.

evaluate method snippet to write to embedded global cache

MbGlobalMap globalMap = MbGlobalMap.getGlobalMap("abbreviation_map");
String key = "ACE";
if (globalMap.get(key) != null) {
	msg = "key already exists";
} else {
	globalMap.put(key, "App Connect Enterprise");
	msg = "added to cache";
}

MbElement outRoot = newMessage.getRootElement();
MbElement jsonData = outRoot.createElementAsLastChild(MbJSON.PARSER_NAME)
		.createElementAsLastChild(MbElement.TYPE_NAME, "Data", null);
jsonData.createElementAsLastChild(MbElement.TYPE_NAME_VALUE, "result", msg);

evaluate method snippet to read from embedded global cache

MbGlobalMap globalMap = MbGlobalMap.getGlobalMap("abbreviation_map");
String key = "ACE";
String value = (String) globalMap.get(key);

if (value != null) {
	value = "Found in cache : ".concat(value);
} else {
	value = "Cache Empty, Try adding something..:/";
}
MbElement outRoot = newMessage.getRootElement();
MbElement jsonData = outRoot.createElementAsLastChild(MbJSON.PARSER_NAME)
		.createElementAsLastChild(MbElement.TYPE_NAME, "Data", null);
jsonData.createElementAsLastChild(MbElement.TYPE_NAME_VALUE, "result", value);

evaluate method snippet to clear embedded global cache

MbGlobalMap globalMap = MbGlobalMap.getGlobalMap("abbreviation_map");
String key = "ACE";
if (globalMap.get(key) != null) {
	msg = "embedded cache map cleared.";
	globalMap.remove(key);
} else {
	msg = "nothing to clear";
}
MbElement outRoot = newMessage.getRootElement();
MbElement jsonData = outRoot.createElementAsLastChild(MbJSON.PARSER_NAME)
		.createElementAsLastChild(MbElement.TYPE_NAME, "Data", null);
jsonData.createElementAsLastChild(MbElement.TYPE_NAME_VALUE, "result", msg);

The following example shows what your flows look like.

IBM App Connect Enterprise Toolkit showing a message flow named globalcacheflow with three HTTP input nodes connected to Java compute nodes for writing, reading, and clearing global cache, each followed by HTTP reply nodes.

2. Exporting the application as broker archive (BAR) file

To deploy your integration flows to App Connect on webMethods Hybrid Integration, start by bundling your message flows into a BAR file. To generate a BAR file for your message flows, follow the steps in Creating a BAR file in the product documentation. Once the BAR file is created, it can be deployed to App Connect on webMethods Hybrid Integration.

3. Deploying the BAR file in App Connect in webMethods Hybrid Integration

To deploy our integration on App Connect, we’ll begin by creating an integration runtime, which acts as the execution environment for our flows. Next, we’ll upload the packaged BAR file in the runtime.

Create runtime page in webMethods Hybrid Integration showing fields for runtime name, integrations, configurations, version selection, and replica options, with runtime named test-embedded-cache and version 1.3.0.3 selected.

Let's explore the integration which we have deployed.The image shows the deployed integration named gcdemo1 also we have three message flows. Each flow starts with an HTTP Input node on the left, connects to a central node that performs a cache operation, and ends with an HTTP Reply node on the right. The first flow writes to the global cache, the second reads from it, and the third clears it.

Layout and properties tab in webMethods Hybrid Integration showing an active flow named gcdemo1 with three HTTP input nodes connected to operations WriteToGlobalCache, ReadFromGlobalCache, and ClearGlobalCache, each followed by HTTP reply nodes.

You can invoke these message flows using any REST client or simply via cURL from the command line. Each flow is exposed as an HTTP endpoint, allowing easy interaction with the embedded global cache.

➜  ~ curl https://test-embedded-cache-https-acdev3802719.a-vir-d1.appconnect.ipaas.ibmappdomain.cloud/clear
{"result":"Cache Empty, nothing to clear"}


➜  ~ curl https://test-embedded-cache-https-acdev3802719.a-vir-d1.appconnect.ipaas.ibmappdomain.cloud/write
{"result":"Key : ACE added to cache"}


➜  ~ curl https://test-embedded-cache-https-acdev3802719.a-vir-d1.appconnect.ipaas.ibmappdomain.cloud/read
{"result":"Found Value : App Connect Enterprise, for Key : ACE"}

This application demonstrates how simple it is to writeread, and clear from the embedded global cache—making it easy to test and integrate cache operations into your flows. A key advantage of this caching mechanism is that data stored in the embedded global cache is shared across different message flows, enabling efficient reuse of common information. This shared memory model helps optimise performance, especially in scenarios where multiple flows need access to the same data.

Within the integration runtime, developers can monitor flow execution and performance using the Activity Log under the Resource Manager tab. The image shows the Activity Log for a resource named global-cache, listing details like timestamp, message, thread ID, node, and resource manager. Each entry captures operations such as reading or writing data to cache maps (e.g., abbreviation_map), along with success messages and tags for quick filtering. This detailed view helps troubleshoot issues, analyze flow interactions, and fine-tune performance across your integration landscape.

Activity log for global-cache in webMethods Hybrid Integration showing multiple entries with timestamps, BIP codes, messages, message details, thread IDs, nodes, resource manager, and tags for operations like getting and putting data in abbreviation_map.

The embedded global cache in App Connect is effective for sharing data across flows within a single integration runtime, however, as the cache is scoped to the runtime replica, for multiple runtimes or runtimes with multiple replicas it may not be appropriate for your use-case. In contrast the external Redis cache offers a robust and reliable solution across all deployment scenarios, including single replica, multiple replica, and across multiple integration runtime setups.

Redis provides a centralized and scalable caching solution that allows different runtimes to access and update shared data reliably. This setup is especially beneficial for scenarios involving cross-runtime coordination, caching frequently accessed reference data. By decoupling the cache from the runtime, you also gain flexibility in managing cache lifecycle, persistence, and performance tuning.

External Redis cache support in webMethods Hybrid Integration

To test cross-runtime caching using Redis, we’ll demonstrate how data written from one integration runtime (redis-ir) can be read by another (redis-ir2)—showcasing Redis as a centralized, shared cache across runtimes. We'll connect the App Connect runtime to an external Redis instance and create a map called abbreviation_map, where key-value pairs e.g ACEaaSAppConnect Enterprise as a Service will be stored. You can use any hosted Redis service but make sure to configure Transport Layer Security (TLS) for secure communication. This requires the Redis server certificate in .pem format (e.g., servercerti.pem). With TLS in place, your integration flows can securely read and write to Redis, enabling scalable and performant data sharing across runtimes.

Prerequisites

I have included all artifacts required to set up the scenario which will be described in detail here.
To follow along and setup a similar system, you will need:

  • An instance of webMethods Hybrid Integration, with the following capability enabled:
    • App Connect
  • The following App Connect Enterprise artifacts:
    • The Redis_Cache.bar file contains an integration flow which is triggered by the cURL command.
    • Project interchange RedisCacheDemo.zip containing the source for all integrations used.

To connect to external Redis, we have to create a vault and store Redis credentials in it.

Create an external directory vault

mqsivault --create --ext-vault-dir C:/test3 --ext-vault-key changeit --vault-options no-export

Set the Redis credentials

ibmint set credential --external-directory-vault C:/test3 --external-directory-vault-key changeit --credential-type redis --credential-name redis --username our_username --password our_password --server-certificate @/servercerti.pem

Export the vault

mqsivault --ext-vault-dir C:/test3 --ext-vault-key password --export --archive-location C:/vault.zip --archive-key changeit

Building the message flows

To explore the capabilities of an external Redis cache in App Connect, we’ll build three message flows using the App Connect Enterprise Toolkit. Each flow will expose an HTTP endpoint and perform a specific cache operation write to Redis, read from Redis, and clear entries from Redis. These flows use an HTTPInput node to receive requests, a JavaCompute node to interact with Redis programmatically via the configured Redis connection policy, and an HTTPReply node to send responses back to the client. If you’re new to creating message flows, refer to create and manage message flows in the product documentation for detailed steps.

1. Create a Redis Connection policy

  • Create a policy of type Redis connection in the App Connect Enterprise Toolkit, this will help us in connecting to the external Redis server.
  • Please note the security identity should be the same as credential name e.g. redis in this case, for more information on connection properties please refer to Redis connection policy properties in the product documentation.

The configuration details for the Redis connection named myredis are shown here. This setup uses the Redis Connection template and includes key properties such as the host name, port number, and database number. Security settings indicate that SSL is enabled (true) with the TLSv1.3 protocol, These parameters ensure secure communication between the integration runtime and the Redis instance, providing encrypted data exchange and reliable connectivity.

2. Import the Project Interchange File

  • Download the attached PI file.
  • In the Toolkit, go to File → Import → Project Interchange.
  • Select the PI file and import it.
    That’s it! Your application and flow are ready to use.

The following examples show the evaluate() methods for each JavaCompute node, which showcase how to interact with the Redis cache programmatically. Later, we’ll package these flows into a BAR file and test them using cURL commands.

evaluate method snippet to write to Redis cache

MbGlobalMap globalMap = MbGlobalMap.getGlobalMap("abbreviation_map","{redis}:myredis");
String key = "ACEaaS";
if (globalMap.get(key) != null) {
	msg = "key already exists";
} else {
	globalMap.put(key, "App Connect Enterprise as  a Service");
	msg = "added to cache";
}

MbElement outRoot = newMessage.getRootElement();
MbElement jsonData = outRoot.createElementAsLastChild(MbJSON.PARSER_NAME)
		.createElementAsLastChild(MbElement.TYPE_NAME, "Data", null);
jsonData.createElementAsLastChild(MbElement.TYPE_NAME_VALUE, "result", msg);

evaluate method snippet to read from Redis cache

MbGlobalMap globalMap = MbGlobalMap.getGlobalMap("abbreviation_map","{redis}:myredis");
String key = "ACEaaS";
String value = (String) globalMap.get(key);

if (value != null) {
	value = "Found in cache : ".concat(value);
} else {
	value = "Cache Empty, Try adding something..:/";
}
MbElement outRoot = newMessage.getRootElement();
MbElement jsonData = outRoot.createElementAsLastChild(MbJSON.PARSER_NAME)
		.createElementAsLastChild(MbElement.TYPE_NAME, "Data", null);
jsonData.createElementAsLastChild(MbElement.TYPE_NAME_VALUE, "result", value);

evaluate method snippet to clear Redis cache

MbGlobalMap globalMap = MbGlobalMap.getGlobalMap("abbreviation_map","{redis}:myredis");
String key = "ACEaaS";
if (globalMap.get(key) != null) {
	msg = "redis cache map cleared.";
	globalMap.remove(key);
} else {
	msg = "nothing to clear";
}
MbElement outRoot = newMessage.getRootElement();
MbElement jsonData = outRoot.createElementAsLastChild(MbJSON.PARSER_NAME)
		.createElementAsLastChild(MbElement.TYPE_NAME, "Data", null);
jsonData.createElementAsLastChild(MbElement.TYPE_NAME_VALUE, "result", msg);

3. Exporting the application as BAR file

To deploy your integration flows to App Connect on webMethods Hybrid Integration, start by bundling your message flows into a broker archive (BAR) file. To generate a BAR file for your message flows, follow the steps in Creating a BAR file in the product documentation. Once the BAR file is created, it can be deployed to App Connect on webMethods Hybrid Integration.

4. Deploying the BAR file in App Connect in webMethods Hybrid Integration

To securely connect App Connect runtimes with Redis, we’ll begin by accessing App Connect and creating two configurations: one of type vault archive and another of type vault archive key. These configurations are essential for managing sensitive credentials and certificates used in Redis TLS connections. Using the previously exported vault archive, we’ll create the Vault Archive configuration. Then, we’ll generate a vault archive key using the same password that was used during the vault export process. Once these configurations are in place, we’ll import the BAR file (containing our integration logic) and create an Integration runtime. During integration runtime setup, we’ll attach both the vault archive and vault archive key configurations to ensure secure access to Redis. To test cross-runtime caching, we’ll repeat this process for a second integration runtime—allowing both runtimes to securely interact with the same Redis instance and share cached data seamlessly.

The Configurations tab in the Manage section of webMethods Hybrid Integration provides a centralized interface for creating and managing configurations for deployed integrations. In this example, a search for “vault” returns two entries: vault (Vault archive) and vaultkey (Vault archive key).

Upload the BAR file using Import BAR file button in Manage section, The BAR files tab in the Manage section of webMethods Hybrid Integration provides a centralized view for importing and managing BAR files used to deploy integrations.

BAR files tab in webMethods Hybrid Integration listing redis_cache_demo BAR file, not deployed, associated with default-deployment-location, with last modified timestamp.

The Create runtime page in webMethods Hybrid Integration allows users to define a custom runtime environment for their integrations. In this example, the runtime is named redis-ir, with the integration redis_cache_demo selected for deployment. Under Configurations, two entries—vault and vaultpass—are added to provide secure credentials and related information.

Create runtime page in webMethods Hybrid Integration showing runtime named redis-ir, integration redis_cache_demo, configurations vault and vaultpass.

We will wait for integration runtimes (redis-ir) and(redis-ir2) to be ready and access the deployed integration, In this example, the flow named redisflow is active and consists of three key operations: Write_To_Redis, Read_From_Redis, and Clear_Redis_Cache. Each operation is connected to HTTP Input and HTTP Reply nodes, enabling seamless interaction through REST endpoints.

Layout and properties tab in webMethods Hybrid Integration showing active flow redisflow with three HTTP input nodes connected to Write_To_Redis, Read_From_Redis, and Clear_Redis_Cache operations, each followed by HTTP reply nodes.

Once the integration runtimes are ready, you can interact with the deployed integrations through their respective HTTP endpoints, each mapped to a specific cache operation. These endpoints allow external systems or test clients to trigger flows for writing to Redis, reading from it, or clearing cached data—making it easy to validate cross-runtime caching behavior. Lets' write to redis using /write endpoint of redis-ir and read using /read endpoint from redis-ir2.

➜  ~ curl https://redis-ir-https-acdev3802719.a-vir-d1.appconnect.ipaas.ibmappdomain.cloud/write
{"result":"Key : ACEaaS added to cache"}


➜  ~ curl https://redis-ir2-https-acdev3802719.a-vir-d1.appconnect.ipaas.ibmappdomain.cloud/read
{"result":"Found Value : App Connect Enterprise as  a Service, for Key : ACEaaS"}

We successfully validated cross-runtime caching by writing data to the Redis database from one integration runtime (redis-ir) and reading it from another (redis-ir2). This confirms that Redis can reliably serve as a shared caching layer across multiple runtimes hosted on webMethods Hybrid Integration. This confirms that Redis can effectively serve as a centralized caching layer even within a single integration runtime, enabling modular flow design and efficient reuse of shared data.

The Resource statistics tab in webMethods Hybrid Integration provides real-time performance metrics for resources used in your integration flows. In this example, the statistics for GlobalCache are displayed, showing the Total Map Actions over the last 5 minutes.

Resource statistics tab showing GlobalCache total map actions graph over last 5 minutes.

The Statistics tab for the global-cache resource in webMethods Hybrid Integration provides detailed operational metrics for cache performance. The summary section shows key properties such as ConnectionFailures (0), Connects (4), FailedActions (0), MapReads (4), and MapRemoves (1), indicating stable connectivity and successful cache operations. Below, the Redis-specific metrics for myredis mirror these values, confirming consistent behavior across the cache instance. This view helps developers monitor cache health and validate integration flows using Redis.

Statistics tab for global-cache showing summary metrics: 4 connects, 4 map reads, 1 map remove, and no failures, with similar values for Redis connection myredis.

Conclusion

The embedded global cache in App Connect Enterprise is an excellent choice for lightweight, in-memory caching within a single integration runtime or a small-scale setup. It’s enabled by default, requires no extra installation, and delivers fast performance for local caching scenarios without external dependencies.

For deployments that need cache sharing across multiple replicas or runtimes, you can extend this capability by backing the cache with an external Redis cache. Redis provides enterprise-grade features like clustering, replication, and high availability, ensuring consistency and scalability across distributed environments.

Until next time, happy integrating!

#AppConnect  #Applicationintegration

0 comments
123 views

Permalink