“Flow Hopping” is the practice where you create a solution in IIB and you break it up into smaller message flows with some communication like MQ or SOAP calls as the connecting mechanism between the smaller message flows.
In this blog I’m going to illustrate the performance penalty, if any, you pay if you do this. Please make sure to read the results as well, because maybe the answer is not 100% intuitive.
1.1 Hardware and Software versions
My trusty laptop, ThinkPad W540 with 16Gb of memory, running IBM Integration Bus V10.0.0.4 and IBM MQ V8.
1.2 Example
In this example, I’m going to use a very simple setup. Stringing together 4 message flows with MQ queues in between to create a message flow hopping example, see Figure 1. I’m starting the sequence by putting a dummy message into the first queue. Every one of the last three message flows will use a mapping node to transform the input data.
Figure 1- Flow hopping
As a counter I’ll have one consolidated message flow that is consuming an MQ message, it will then transform the data with the help of the same three mapping nodes used in Figure 1 and the result will be written back into an MQ queue, see Figure 2.
Figure 2 - Consolidated Flow
In both scenarios above the first flow consumes the dummy message out of the queue and then loops 25 000 times to populate the input queue for the real performance test. The message flow used for this looks like Figure 3.
Figure 3 - Loop x times
The ESQL code in the “LoopX” node is shown in Figure 4. In this example, it will loop 25000 times and create the same message as shown in Figure 5 in the downstream queue “Hop1”. I’ve spaced the data in order to make it more readable for this write-up.
Figure 4 - LoopX code
Figure 5 - Input data
The mapping nodes in all the flows use two XSD files, generated in the Toolkit from sample XML, see Figure 6 and Figure 7.
Figure 6 - HOP1 XSD
Figure 7 - HOP2 XSD
The setup of the message flows 2 and 3 in Figure 1 look as follows, see Figure 8 and message flow 4 looks as in Figure 9. The reason the fourth message flow has a node in to remove the MQ header is because I’m using the put timestamp in the message to calculate the overall time taken to execute, and by removing the MQ header I’m forcing a new timestamp into the final message.
Figure 9 - Hopping flow
Figure 9 - Last hopping flow
Each of the smaller hopping message flows will consume a message from a queue, map the input and write the resultant to an output queue. The mapping in this message flow looks like Figure 10.
Figure 10 - Simple mapping message flow 2
For completeness, the mapping in the third message flow looks like Figure 11 and the mapping for message flow four looks exactly the same as in Figure 10.
Figure 11 - Simple mapping message flow 3
The consolidated flow from Figure 2 has the message flow layout as in Figure 12. Please note as before I’m using the removal of the MQ header to force a new timestamp in the MQ message going to the queue “Hop4”.
Figure 12 - Consolidate flow message flow
In Figure 12 the mappings inside “Mapping” and “Mapping2” looks the same as in Figure 10 and the mapping in “Mapping1” looks the same as in Figure 11, in fact, in the application they are 100% the same mappings shared in a library.
DOWNLOAD THE TEST FILE HERE READ PART 2