Acknowledgement
This article was written in conjunction with IBM ANZ Professional Services Safer Payments expert, Roger Robins.
Introduction
This article discusses how IBM App Connect Enterprise (ACE) can be used as part of a Safer Payments implementation.
IBM Safer Payments MCI interface only supports a subset of the XML standard. It requires the first element be an IRIS tag; does not support single quotes; does not support spaces between elements; does not support CDATA sections.
Additionally, the core banking system, or message provider often won’t be able to provide the message in the expected XML format. The message needs to be transformed into a format Safer Payments understands.
What is IBM App Connect Enterprise (ACE)?
ACE (formerly IBM Integration Bus (IIB)) is a heterogenous integration engine designed to facilitate any to any protocols connection and mediate data of any format XML, JSON, fixed format, tagged or delimited. Unlike most other integration vendor offerings, the IBM ACE engine is C-based and optimized for which ever operating system or hardware it is deployed into and hence highly performant. These performance characteristics are a match for the IBM Safer Payments engine. It is lightweight with no pre-requisite database or Java platform required. ACE satisfy the broadest set of integration patterns from traditional Enterprise Application Integration (EAI) and Enterprise Service Bus (ESB) constructs to modern agile integration architectures. It is these qualities that make it the perfect host for Safer Payments connectors.
For example; a message may be read in as a record from a file on the file system or a message over MQ with comma delimited values. ACE could then translate the message into an XML message; and send it out over TCP/IP sockets to a remote systems such as Safer Payments and then manage the response back to the calling original calling system.
Figure 1. Calling SP from ACE
Under the covers the high speed ACE runtime is delivering the protocol connection on input and output plus parsers capable of dealing with any data format that normalize that data into a logic structure that can be operated upon easily to perform tasks like transformation and unique id matching for correlation.
Figure 2. Ace Runtime
ACE implementation at an Australian Bank
The organization providing the messages was able to provide us with an XML representation of an ISO 8583 card message which had the necessary IRIS tags. However; the reply message they expected had to be prefixed with the length of the message as a binary number; and certain values sent in the request message had to appear in the response. Values such as CardAcceptorTerminalIdentification had leading and trailing spaces which were removed by Safer Payments processing.
We used the ACE toolkit to build and deploy a message flow that stored the values for the reply message and calculate and prepend the length value.
In this example message flow for an Australian Bank, we are taking card messages from the message provider in on TCPIP Server Input. (The Trace nodes are only capturing message logging to troubleshooting purposes) The first node that does any work is StrLenHeader (a compute node); which looks for in the stream to identify end of message. It then parses the message as XML and stores away four values that need to be added to the reply message. TCPIP Client Output sends the message to Safer Payments.
TCPIP Client Receive receives the response message from Safer Payments and the compute node adds the stored values into the response message and prepends the length value. TCPIP Server Output sends the Safer Payments response back to the message provider. The introduction of ACE solved the problem of message formatting and the ease with which it was deployed and set up helped bring the project in on time and under budget. It's overall impact on the end to end solution was negligible in terms of introduced latency. Response times through Safer Payments with ACE still approximately 20ms.
Figure 3. ACE Message Flow
Architectural Deployment as an adapter for Safer Payments
Architecturally it was decided to install ACE on each Safer Payments instance and use the load balancer to redirect traffic if either ACE or the MCI interface was not available. Each ACE instance only forwarded messages to it’s local Safer Payment instance.
While it was possible to have just one ACE instance per data center; that would have introduced a single point of failure within the data center.
Figure 4. ACE Deployed as Safer Payments Adapter
What can ACE do for Safer Payments Clients?
ACE comes with a toolkit that can be used to build and deploy message flows.
Creating a message flow is as easy as creating an ACE application in toolkit and creating a msgflow under that application. Once you have a msgflow; you can drag and drop an input and an output adapter from the left menu bar and connect them up, with or without a transformation node.
Adapters
As you can see; there are a lot of adapters to choose from. Important adapters to Safer Payments are File; TCPIP; HTTP and IBM MQ. Which one you use depends on your business needs. If you are sending messages to MCI interface; then use TCPIP (as in the screenshot below). If using MQ; then use the MQ adapters. If ACE is being used as a middle man between Safer Payments and an external case system; then HTTP and Web Service adapters will be important to you.
Figure 5. ACE Message flow
Transforming Data
Data transformation can be achieved in a variety of ways:
- Graphical Mappings in a message flow
- Extended SQL in a compute node
- Java (including JAXB) in a message flow
- XSL in a message flow
- PHP
Mappings.
If you have meta-data that describes you data formats in XSDs, DTD, COBOL copybooks, C headers etc. these constructs are used by ACE to generate logical models the input and output models can then be used in the graphical mapper to translate data or add additional data to enrich the outgoing message without coding. The sophisticated DFDL parser and tooling extends this meta-data capabilities to encompass tagged and delimited data and complex industry formats such as ISO8583 for example.
Figure 6. Ace Graphical Mapping
Extended Structured Query Language.
ESQL uses a Compute node to execute ESQL language to copy and transform data from the input message to an output message.
Figure 7. Ace Extended Structured Query Language
Java.
Java uses a Java Compute node to execute java code to transform the input message.
Figure 8. ACE Java
XSLT.
XSL uses an XSL stylesheet to transform values
Figure 9. ACE XSLT
ACE for Safer Payments Summary
Deployed a lightweight, easy to configure adapter, ACE can take any data feed over any protocol and provide request/reply adaption to Safer Payments. It will transform the data to the Safer Payments XML data model and take care of the context specifics of matching response messages to their original requests. Where necessary it will also manage the semantics of adapting between asynchronous source systems to the synchronous Safer Payments interface and vice versa. ACE can be configured to connect to Safer Payments via its HTTP, TCPIP sockets or MQ interfaces.
Figure 10. ACE Universal Adapter
Exploring ISO 8583 payment format
Most Card messages are in an ISO8583 format or an AS2805 format which is the Australian version of an ISO8583. There are primarily three flavors of ISO8583:
ISO8583:1987; ISO8583:1993 and ISO8583:2003
ISO8583 is a binary file containing two bitmap fields. The bitmap fields determine which of the 128 fields (for 1987) or 193 fields (for 1993 & 2003) are present in the message. To parse a message this complex is difficult to say the least.
Fortunately; ACE comes with a DFDL parser that allows you to create an XSD to parse ISO8583 messages and derive an XML message.
Example transforming ISO8583 with choice of bitmap to XML equivalent.
Figure 11. ISO8583 to XML
CREATE COMPUTE MODULE ISO8583_TCPIP_to_WMQ_Transform_ISO8583_to_XML
CREATE FUNCTION Main() RETURNS BOOLEAN
BEGIN
SET OutputRoot.Properties = InputRoot.Properties;
-- Create the XMLNSC output domain
CREATE LASTCHILD OF OutputRoot DOMAIN('XMLNSC');
-- Copy the parsed fields from the input DFDL domain
-- into the output XMLNSC domain
SET OutputRoot.XMLNSC = InputRoot.DFDL;
-- The PrimaryBitmap and SecondaryBitmap are not needed
-- in the XML output format, so delete these complex elements
-- from the output tree
SET OutputRoot.XMLNSC.ISO8583_1987.PrimaryBitmap = NULL;
SET OutputRoot.XMLNSC.ISO8583_1987.SecondaryBitmap = NULL;
RETURN TRUE;
END;
CREATE PROCEDURE CopyMessageHeaders() BEGIN
DECLARE I INTEGER 1;
DECLARE J INTEGER;
SET J = CARDINALITY(InputRoot.*[]);
WHILE I < J DO SET OutputRoot.*[I] = InputRoot.*[I]; SET I = I + 1; END WHILE; END; CREATE PROCEDURE CopyEntireMessage() BEGIN SET
OutputRoot = InputRoot; END; END MODULE;
Example transforming XML to ISO8583 with choice of bitmap
Figure 12. XML to ISO8583
CREATE COMPUTE MODULE ISO8583_WMQ_to_TCPIP_Transform_XML_to_ISO8583
CREATE FUNCTION Main() RETURNS BOOLEAN
BEGIN
SET OutputRoot.Properties = InputRoot.Properties;
SET OutputRoot.Properties.MessageType = '{}:ISO8583_1987';
CREATE LASTCHILD OF OutputRoot DOMAIN('DFDL');
DECLARE InRef REFERENCE TO InputRoot;
CALL RemoveUnrequiredXMLAttributes(InRef);
SET OutputRoot.DFDL = InRef.XMLNSC;
CREATE NEXTSIBLING OF OutputRoot.DFDL.ISO8583_1987.MTI_MessageOrigin NAME 'PrimaryBitmap'; DECLARE DFDLPointer REFERENCE TO OutputRoot.DFDL.ISO8583_1987.PrimaryBitmap;
CALL PopulatePrimaryBitmap(DFDLPointer);
DECLARE SecondaryBitmap BOOLEAN FALSE;
IF CAST(SUBSTRING(FIELDNAME(OutputRoot.DFDL.ISO8583_1987.*[<]) AFTER '_') AS INT) > 64 THEN
SET SecondaryBitmap = TRUE;
END IF;
IF SecondaryBitmap THEN
-- This means a SecondaryBitmap will be required
CREATE NEXTSIBLING OF OutputRoot.DFDL.ISO8583_1987.PrimaryBitmap NAME 'SecondaryBitmap'; MOVE DFDLPointer TO OutputRoot.DFDL.ISO8583_1987.SecondaryBitmap;
CALL PopulateSecondaryBitmap(DFDLPointer);
-- SecondaryBitmap is present, so set Bit001 to 1
SET OutputRoot.DFDL.ISO8583_1987.PrimaryBitmap.Bit001 = 1;
ELSE
MOVE DFDLPointer TO OutputRoot.DFDL.ISO8583_1987.PrimaryBitmap;
-- SecondaryBitmap is not present. Bit001 will already be initialised to 0.
END IF;
-- Having created the Bitmap structures with all bits set to 0, now set the Bit values ...
CALL UpdateBitmapsBasedOnAvailableFields(DFDLPointer);
RETURN TRUE; END;
-- GenerateThreeDigitIndex
CREATE PROCEDURE GenerateThreeDigitIndex(IN IndexInt INTEGER, OUT IndexChar CHAR)
BEGIN
SET IndexChar =
CASE
WHEN IndexInt < 10 THEN ('00' || CAST(IndexInt AS CHAR))
WHEN IndexInt < 100 THEN ('0' || CAST(IndexInt AS CHAR))
ELSE CAST(IndexInt AS CHAR)
END;
END;
--RemoveUnrequiredXMLAttributes
CREATE PROCEDURE RemoveUnrequiredXMLAttributes(INOUT InRef REFERENCE)
BEGIN SET InRef.XMLNSC.*:XmlDeclaration = NULL;
DECLARE NumberRootAttributes INTEGER CARDINALITY(InRef.XMLNSC.ISO8583_1987.(XMLNSC.Attribute)*[]);
DECLARE I1 INTEGER 1;
WHILE I1 <= NumberRootAttributes DO SET InRef.XMLNSC.ISO8583_1987.(XMLNSC.Attribute)*[1] = NULL; SET I1 = I1 + 1; END WHILE;
END;
--PopulatePrimaryBitmap
CREATE PROCEDURE PopulatePrimaryBitmap(INOUT DFDLPointer REFERENCE)
BEGIN
DECLARE I2 INTEGER 1;
DECLARE IndexChar CHAR;
-- Create PrimaryBitmap structure, with all bits set to 0
WHILE I2 < 65 DO CALL GenerateThreeDigitIndex(I2,IndexChar); CREATE LASTCHILD OF DFDLPointer NAME ('Bit' || IndexChar) VALUE 0;
SET I2 = I2 + 1; END WHILE; END;
--PopulateSecondaryBitmap
CREATE PROCEDURE PopulateSecondaryBitmap(INOUT DFDLPointer REFERENCE)
BEGIN
DECLARE J INTEGER 65;
DECLARE IndexChar CHAR; WHILE J < 129 DO
-- Create SecondaryBitmap structure, with all bits set to 0 CALL GenerateThreeDigitIndex(J,IndexChar); CREATE LASTCHILD OF DFDLPointer NAME ('Bit' || IndexChar) VALUE 0; SET J = J + 1; END WHILE; END;
--UpdateBitmapsBasedOnAvailableFields
CREATE PROCEDURE UpdateBitmapsBasedOnAvailableFields(INOUT DFDLPointer REFERENCE)
BEGIN
-- Loop over the fields which are present to update the Bitmaps
MOVE DFDLPointer NEXTSIBLING;
WHILE LASTMOVE(DFDLPointer) DO
DECLARE CurrentFieldName CHAR FIELDNAME(DFDLPointer);
DECLARE BitName CHAR ('Bit' || SUBSTRING(CurrentFieldName AFTER '_'));
IF CAST(SUBSTRING(CurrentFieldName AFTER '_') AS INT) < 65 THEN -- Current field relates to PrimaryBitmap SET
OutputRoot.DFDL.ISO8583_1987.PrimaryBitmap.{BitName} = 1; ELSE -- Current field relates to SecondaryBitmap SET
OutputRoot.DFDL.ISO8583_1987.SecondaryBitmap.{BitName} = 1; END IF; MOVE DFDLPointer NEXTSIBLING; END WHILE;
END; END MODULE;
DFDL Schemas can be found on Github at:
https://github.com/DFDLSchemas/ISO8583