App Connect

Automation of business transaction reconciliation using App Connect and IBM MQ across application

By OMPRAKASH UPADHYAY posted Mon June 14, 2021 06:57 AM

  

Co-Author - Abhinav Priyadarshi

Reducing time for disaster recovery is crucial for organizations needing to achieve operational resiliency. This article helps CIOs, CTOs and IT development teams understand the complexity of maintaining stateful transaction processing with participating applications, auto reconciliation and visualization and provides information for organizations setting up high recovery mechanisms for transaction reconciliation and settlement in case of failure, as well as transaction disruption events during network failure.

Problem statement

Many organizations spend huge amounts of time manually settling business transactions when there is complete fail-over of an application or site zone failure to another application site zone. The fail-over condition arises due to uncontrolled network failure, OS failure, hardware failure or DR drill. The availability and processing of time bound and business critical transactions across application site zones are needed by many organizations, especially financial institutions. The automation of this requirement needs technology support and solution implementation.

This article covers an approach to achieve message availability, processing, and settlement of unprocessed messages across application sites during application site zone fail-over. Here the application site zones are Mumbai, London, and New York.

Steps Required

The following are the steps necessary to achieve message availability, processing, and settlement of unprocessed messages:

  1. Perform gateway message availability by replicating the messages across all application zones.
  2. Once the gateway message is successfully consumed by the receiving application at the primary application zone, generate the acknowledgement message for the successfully consumed gateway message by the receiving application at the primary application zone.
  3. Use the acknowledgement message to remove the successfully processed gateway messages from all other secondary application zones.
  4. If the primary application zone goes down or becomes unavailable, then process the replicated message at Step 1. from one of the secondary application zones.
  5. The receiving application consumes the replicated message and generates the acknowledgement message for the successful processing of the message.
  6. Reconcile the successfully acknowledged message from the other available application zone. The secondary zone which generates the acknowledgement message removes those successfully processed messages from all other available application zones.
  7. Once the primary application zone is up, the secondary zone reconciles the successfully processed and acknowledged message from the primary site and removes it from the primary application zone so that it is not reprocessed.



Architecture setup for AOS



alt text

alt text

Let’s understand the product placement in the diagram above with geolocation.

Position Product Mechanism Remarks

Block Row 1

IBM MQ Advance with Aspera fasp.io

Pub/Sub model for receiving raw data across Mumbai-London-New York

MQ Gateway Cluster

Block Row 2

IBM App Connect with IBM MQ Advance with Aspera fasp.io

Message flow replicating transaction across Mumbai-London-New York

IBM App Connect managing the stateful of transaction

Block Row 3

Participating application

Settlement system

AC1

IBM App Connect in Mumbai

Work in Progress Queue (WIP) and Clash Queue (CQ)

AC2

IBM App Connect in Mumbai with High Availability

Work in Progress Queue (WIP) and Clash Queue (CQ)

AC3

IBM App Connect in London

Work in Progress Queue (WIP) and Clash Queue (CQ)

AC4

IBM App Connect in London with High Availability

Work in Progress Queue (WIP) and Clash Queue (CQ)

AC5

IBM App Connect in New York

Work in Progress Queue (WIP) and Clash Queue (CQ)

AC6

IBM App Connect in New York with High Availability

Work in Progress Queue (WIP) and Clash Queue (CQ)



























Scenario

Mumbai receives the transaction at its gateway and the message is replicated across the data centre using IBM App Connect replicating the message across London and New York with IBM Message Queue.

Steps to implement the above scenario

Step 1. Copy the gateway message across the geo location circled in the diagram below of Block 2. Here AC1, AC3 and AC5 are present in three different geo locations. AC1 and AC2 are active–passive servers within the same geo location. QM1, QM3, QM5 are the destination queue managers for the receiving application.

alt text


Step 2. Message copy circled below in the diagram of Block 3 for the application consumption. The receiving application consumes the message and acknowledges the successful consumption.

alt text


























Step 3. The message is removed from all geo-locations after application consumption and successful acknowledgement.

alt text



















Failure scenario for Mumbai (site down) and London (site up)


alt text



Step 1. The work in progress Intermittent Transaction starts flowing from London.
The receiving application starts processing the incoming request from the London site and acknowledges transaction completion


alt text



























Step 2. WIP London controls and starts the settlement for successful completion with the application.
After successful completion of the transaction processed by the receiving application, IBM App Connect starts the settlement.

alt text

























Step 3: Clash Queue for London control to retry in case of an unsuccessful transaction with the application. Any unsuccessful transaction will be retried from the Clash queue.

alt text



Handling successful processing at Primary site.


alt text



Handling primary site failure:


alt text



Smart Benefit

✓ The Reconciliation of the System of records is automatic hence reducing entire RPO & RTO for the business with respect to Business Transaction


✓ Settlement of the system is automatic

Conclusion

This article explains a high-level overview of the reconciliation of transaction across sites during primary zone failure. Here this scenario could be implemented using IBM MQ and App Connect products features. These products provide features to replicate messages to many destinations, apply aggregation for the fan-in and fan-out messages. More details about the implementation will be published in the next version of this article.

References:

Introduction to IBM MQ

IBM App Connect Enterprise introduction

Introduction to IBM WebSphere MQ publish/subscribe messaging

GroupScatter node


#AppConnect
#MQ
#Businesstransaction
#messageprocessing

Permalink

Comments

Wed June 16, 2021 08:20 AM

This article should help many banking industry and other industries who are using IBM MQ and App Connect for transaction processing.