The recently released Business Transaction Monitoring (BTM) allows a business user to track the life cycle of a business transaction that has been processed by multiple message flows. Events are captured, correlated and a summary, the business transaction state, is calculated. Previous articles have described how you get started with BTM and how customize it further.
IBM Integration Bus advanced users might recognise some of these concepts in the existing Record and Replay (RR) feature that have been available in the product since version 8.0.
This article compares and contrasts the two sets of capabilities with a view to guiding you when you might want to use one versus the other and explains a few considerations to take into account if you use them side by side.
Features
Here are the two features at a glance:
Let’s explore the common area and the differentiators for each of these technologies.
BTM and RR common points
An event is a processing step that occurs during an IIB integration. It contains information about the source of the event such as which message flow and node and which action was performed e.g. transaction started/ended or a node terminal fired. The event may include the message payload and exception list. When a message flow is processed, the events configured on its nodes are emitted to the MQ topic.
To add events to monitoring flows you will need to use either the toolkit or the mqsichageflowmonitoring command.
Both BTM and RR are consumers of these monitoring events. Other consumers might also be configured. Both BTM and RR capture events. They set up subscriptions to the corresponding topics for message flows like
$SYS/Broker/integrationNodeName/Monitoring/integrationServerName/flow_name
that receive the events and put it to a destination queue. Messages are read from this queue and recorded into the database.
As this capturing mechanism is shared by both BTM and RR, they both prereq MQ and can only be configured on integration nodes that have a queue manager specified.
Both BTM and RR use tables of the same structure to record events. However, as explained later, it is better to use different schemas when using both BTM and RR and not to have them write to the same table as it will make it more difficult for correlation capabilities in BTM.
In both cases, you need to configure the database for use by IIB. This involves running the provided scripts to create the WMB_MSGS table for recording monitoring events and the WMB_BINARY_DATA table for storing payload data and exception information. Additionally, you must configure the environment to allow IIB to make ODBC connections to the database and use mqsisetdbparms to set a user identifier and password for that connection.
BTM only capabilities
Below are some characteristics that are BTM specific and cannot be done by RR.
Correlation and Global ID
BTM correlates events in order to report the state of a business transaction in a way RR cannot. BTM correlation works by analysing a received event to see if it is mentioned in any business transaction definition. If that is case, the global ID is used to determine the particular business transaction instance this event belongs to and its effect on the business transaction state.
As described, the global ID is essential for BTM. The monitoring events need to have a global ID specified with an Xpath as opposed to selecting the ‘automatic’ option when specifying the event. RR does not depend on a global ID being specified.
So both the correlation and the requirement to have a global ID specified are extras of BTM over RR.
BTM correlation does not come for free and has a cost. You can choose what which monitoring events to include in the business transaction and depending on your requirements, so you might want to flag only those events that you deem critical for monitoring your business transaction. As the correlation is a BTM only, the extra correlation costs are incurred only when using BTM.
Summarises events
BTM summarises correlated events in the business transaction states. You can see at a glance the state of the business transaction. It is easier to understand what happened to a given request by seeing the events that make a business transaction.
These are the main use cases for BTM, so clearly you cannot achieve this with RR and these are some keys differentiators.
Configuration
Once the monitoring events have been defined with the globalIDs set up, the required part of the configuration is database related. The database needs to be configured as described above in the common parts. In addition, to the common database configuration described above a new table for business transactions, WMB_BUSTRANS, also needs to be created by running an sql script server\ddl\db2\BusinessCaptureSchema.sql
BTM uses a default data capture policy for its configuration. A default policy is available out of the box and only needs a database name set to be operational. Even taking into account the extra table that needs to be created, configuring BTM is easier and relies on a set of defaults and runtime decisions on servers to use for recording and viewing.
Grouping and filtering of results
The results of business transactions are grouped by the type of BTD and you cannot mix and match, say PurchaseOrder business transactions with StockAlert business transactions. You can filter by the status of a business transaction, failed, in progress, which is a new BTM specific capability.
Record and Replay only
Here are the Record and Replay characteristics that cannot be done by BTM.
Event filtering
With RR, it is possible to filter on any event fields, including time based queries based on timestamps, and to combine any number of conditions. With BTM, you see the events for a particular business transaction instance, but there is no further event filtering. The only filtering is at the business transactions level when you can choose a specific transaction ID and a transaction state.
Configuration
There is more flexibility in distributing the recording and viewing work on the servers but at the cost of more complicated configuration via Configurable Services, Data Capture Store and data capture source and Data Destination.
Event View per store
You can see all the events captured to a store, regardless of their source or whether they are related or not. This can be an advantage or not.
Replaying events
Another difference has a clue in the name: with RR you can select events to replay. Not only does BTM not support replay, but replaying certain messages can interfere with the summarising capabilities and reporting of the state of the business transaction. Say a start message of a completed business transaction is replayed. This would cause the business transaction to be reported as inconsistent.
Finer grained access rights
RR can be configured for a datacapture type of user. Combining these with broker access right, it is possible to configure set up a user that can only do RR operations. This is not supported for BTM.
To summarise:
Using both BTM and RR
With these characteristics, you can hopefully choose better what technology is right for you. What if you have deployed IIB applications with different monitoring requirements and need to run both BTM and RR?
Then you need to take these points into account.
Separation at data level
It is recommended that you do not share the event tables that RR and BTM use even if they have the same structure. The reason for this is that if you have flows monitored by both RR and BTM you might have the same event recorded twice which might give you unexpected results for your RR queries.
One way to achieve the data separation is to have different schemas, so you would have two sets of tables, say BTM.WMB_MSGS, BTM.WMB_BINARY_DATA and RR.WMB_MSGS and RR.WMB_BINARY_DATA
Alternatively, you can use different databases, one for BTM, the other for RR.
Separation at server level
It is recommended that you distribute your BTM recorder and your RR recorder to different integration servers. The main reason is performance.
The simplest way to achieve this is to have different integration nodes dedicated to the different types of monitoring. If you need to mix and match the technologies on the same node, you will need to specify different servers. For RR in the dataCaptureStore configurable service you will need to specify an egForRecord and egForView. This is described here https://www-01.ibm.com/support/knowledgecenter/SSMKHH_10.0.0/com.ibm.etools.mft.doc/bn28727_.htm
BTM has a default data capture policy that means the runtime decides which server to use for recording and viewing. To overwrite these, you will need to modify the default data capture policy to assign the recording and viewing to different servers than those used by RR. How to do this is describe here
Separation at queue level
Again for performance reasons, you might want to configure BTM and RR to use different queues for recording and backout. When a monitoring event is emitted it is published to a MQ Topic on the associated Queue Manager. The relevant subscription picks up the event and puts it to a destination queue. For BTM, the destination queue is defined in the default Data Capture policy to be SYSTEM.BROKER.DC.RECORD. A further queue is also needed to handle events that cannot be at process time be written to the database, and thus are backed out onto this queue. The default is SYSTEM.BROKER.DC.BACKOUT. The same concepts exist for RR. They are specified in the queueName and backoutQueue properties of the Data Capture Store Configurable Service. For improved performance it is recommended that you overwrite one of these defaults, say in the Data Capture Store Configurable Service to have BTM and RR use different recording and backout queues.
Note that other configurations are also possible.
The article has described the similarities and differences between BTM and RR, the means to choose or to combine them.