Introduction
As IBM Expert Labs we often get asked to review IBM App Connect Enterprise (ACE) flows for reliability, ease of consumption and most importantly for performance. In this article, we describe the most common mistakes we tend to see and how you can avoid them.
1. Message Tree Copying
The message tree is a structured representation of a message as it passes through the flow. In ESQL and Java nodes, we can manipulate the message tree, but often they are copied without updates.
Recommendation: Avoid message tree copying if no updates to that message tree as message tree copies are expensive.
Action: For an ESQL compute node use the ‘compute mode’ property to control which message trees (components) are used in the output, ‘Message’ (meaning InputRoot) is the default:
Example: If you only update the LocalEnvironment in a compute node, set the ‘compute mode’ to ‘LocalEnvironment to avoid copying the unchanged Input message tree to the output:
2. Default behaviours
ACE sets default values in order to help customers use the system quickly and efficiently. However, these are not always the most performant, secure or error handling as individual integration requirements need to be met. Not all scenarios can be templated.
Recommendation: Review all default values and code to ensure that the message flow is performing well and as expected.
Example: When an ESQL node is added to a message flow, the commented out lines for – CALL CopyEntireMessage() will be used, but this copies the whole message tree.
Example Action: Avoid using the statement: SET OutputRoot = InputRoot; and then not updating any fields in the message tree via ESQL, for example:
CREATE COMPUTE MODULE TEST_MF_Compute
CREATE FUNCTION Main() RETURNS BOOLEAN
BEGIN
-- CALL CopyMessageHeaders();
CALL CopyEntireMessage();
RETURN TRUE;
END;
.
.
END MODULE;
3. Adjacent Compute Nodes
It can be tempting to modularise a compute or Java compute node and split functions and processes into their own nodes. However, this can cause multiple message tree copying events which as we have discussed is expensive!
Recommendation: Avoid adjacent compute nodes all performing message tree copying. If needed, ensure there is justification and note the potential performance impact for future maintainers of the code.
For example, if both the ESQL compute nodes ‘Check Input’ & ‘Format Output’ update the message tree then two message tree copies occur:
4. Long tree paths
When ACE processes a message tree path e.g., InputRoot.XMLNSC.Message… it has to navigate through each step until it gets to the target path. This causes elongated processing time, especially with greater flow utilisation.
Recommendation: Avoid repeated long tree paths in ESQL statements, use and declare a reference instead.
Example:
A long tree path:
SET OutputRoot.XMLNSC.Message.CustomerDetail.Address.HouseNumber = InputRoot.XMLNSC.UpdateMessage.HomeAddress.HouseNameNumber;
SET OutputRoot.XMLNSC.Message.CustomerDetail.Address.Street = InputRoot.XMLNSC.UpdateMessage.HomeAddress.Street;
SET OutputRoot.XMLNSC.Message.CustomerDetail.Address.Locality = InputRoot.XMLNSC.UpdateMessage.HomeAddress.Locality;
SET OutputRoot.XMLNSC.Message.CustomerDetail.Address.Town = InputRoot.XMLNSC.UpdateMessage.HomeAddress.Town;
SET OutputRoot.XMLNSC.Message.CustomerDetail.Address.Postcode = InputRoot.XMLNSC.UpdateMessage.HomeAddress.Postcode;
References used instead:
Declare inAddressRef REFERENCE TO InputRoot.XMLNSC.UpdateMessage.HomeAddress;
Declare outAddressRef REFERENCE TO
OutputRoot.XMLNSC.Message.CustomerDetail.Address;
SET outAddressRef.HouseNumber = inAddressRef.HouseNameNumber;
SET outAddressRef.Street = inAddressRef.Street;
SET outAddressRef.Locality = inAddressRef.Locality;
SET outAddressRef.Town = inAddressRef.Town;
SET outAddressRef = inAddressRef.Postcode;
5. Message Flow Loops
A message flow with nodes that cause a flow loop can have severe impacts on the Integration Server, causing it to crash and run out of memory. This is because the cyclic processing can loop with no exit strategy.
Recommendation: Do not create a loop by connecting the output terminal of a message flow node to the input terminal of previous node. If you need to execute a succession of message flow nodes repeatedly, use the ESQL propagate statement.
Example:
The following wired loop shows the out1 terminal of the compute node ‘Process additional data’ being wired to the input terminal of the preceding ‘Fetch Additional Data’ HTTP Request node.
6. Message Parsing
Parsing a message can be expensive, especially for large message trees. This is particularly true for large payloads, complex structures and certain parser types because parsing the whole message can take time and resources, especially if you don’t need the whole tree.
Recommendation: Use the default ‘On demand’ parsing setting in Input nodes wherever possible – this parses the message up to the point of the last reference required reducing the amount of parsing ACE needs to do.
7. Static Message Tree Updates
In some cases, the message tree updates applied in a message flow node may not be carried forward. This means that messages are either incorrect when passed to the flow consumer or sometimes developers will rewrite the message tree logic in the next node meaning more processing.
Recommendation: Review the message tree updates to ensure they are being carried forward through the flow.
Example:
Incorrect setting of Compute node ‘mode’. The default setting is ‘message’ (InputRoot) meaning that changes made to OutputLocalEnvironment won’t be carried forward to the following nodes in the message unless the compute mode selected includes: ‘LocalEnvironment’.
8. Unconnected output terminals
Whilst this might seem obvious, this does not cause an error in the development environment. As such, some message flows may have unconnected output terminals. If this is used to catch edge cases, sometimes testing won’t catch that this terminal is throwing messages away, meaning a potential loss of data.
Recommendation: Review output terminals as part of the standard flow review process, and/or include output terminal checks in static code analysis performed as part of routine testing
Example:
In this flow, the ESQL compute node ‘Handle Error” output terminal is not connected, especially not to any output node. If an exception is passed to ‘Handle Error’ then the exception message will be discarded after ‘Handle Error’ has been executed. This maybe what is desired if – for example – ‘Handle Error’ contains a call to some logging code.
9. Multiple Output terminal connections are not guaranteed to execute in any order
When configuring an output terminal for a node, it is possible to generate multiple output connections. This could mean processing is not in order, impacting processing of the message flow.
Recommendation: When multiple connections are required, use the ‘FlowOrder’ node to set the correct processing path for the flow.
Example:
The order of execution for ‘Do this first’ and ‘Do this second’ ESQL is not guaranteed in the first flow.
The second example shows how we can ensure the functions are run in the correct order.
10. Cardinality in Loops
Cardinality refers to the number of items (instances of a data element) that a particular node or mapping can handle during execution. If Cardinality is configured inside a loop, this can be a performance problem with large arrays where the cost of evaluating CARDINALITY is expensive. If the array is large there are more iterations of the loop, hence more resources used.
Recommendation: Avoid using CARDINALITY inside a loop, plaxe the statement before the loop.
Example:
CARDINALITY inside the WHILE causes large performance impact:
WHILE (I < CARDINALITY (InputRoot.XMLNSC.TestData.GroupItem.Array[]))
CARDINALITY statement before the loop:
DECLARE currentItem, totalItems INTEGER;
SET currentItem = 1;
SET totalItems = CARDINALITY(OutputRoot.XMLNSC.MyMessage.Array[]);
WHILE currentItem <= totalItems DO
SET ....
SET currentItem = currentItem + 1;
END WHILE;