CAM

ITCAM for Transactions Tracking: Tips, Best Practices, and Pitfalls Part 1

By IMWUC Community Team posted Tue October 15, 2019 02:32 AM

  

by Eric Montagnon 

Here are some tips concerning ITCAM for Transactions Tracking with Tx collector (TU) / Tx reporter (TO) agents used :

 

A)  Are you planning to use ITCAM For Transactions Tracking to monitor your portal application? Yes? Here is some tips for you.

A1) Components to install:
Generally, you install a Data collector plug-in associated to your domain on the Application Server system; for example if you want to monitor your Portal  served by WebSphere App Server, you install the Data collector plugin on WAS app server machine.

 If you have a HTTP server in front of WAS app server, you install also a Data collector plugin on it.


Like, If you have the following architecture in use :
IHS on server1 <=> WAS on server2
Then generally you need to install :
for IHS on server 1 : WRT 7.4 agent (with 'Transaction Tracking' option enabled) see (1)
for WAS on server 2 : WAS Data Collector from ITCAM for Application Diagnostics product (or WASTT) see (2)

On server3 which is generally the ITM infrastructure server, install :
- ITM 6.3 TEMS and TEPS
- ITCAM4tx Tx collector agent v7.4 and its application support
- ITCAM4tx Tx reporter agent v7.4 and its application support
- ITCAM4Tx AMC agent v.74 and its application support
- WRT v7.4 application support (no need for WRT agent on it)

Useful link : Installing and configuring Transaction Tracking Data Collector plug-ins
https://www.ibm.com/support/knowledgecenter/en/SS5MD2_7.4.0.1/com.ibm.itcamt.doc/tt/dita/concept/kto_dom_in_oview.html

(1) WRT transaction tracking enabled:
https://www.ibm.com/support/knowledgecenter/en/SS5MD2_7.4.0.1/com.ibm.itcamt.doc/rt/Install_Guide/wrt_config_win.html

(2) WAS Data Collector
https://www.ibm.com/support/knowledgecenter/en/SS5MD2_7.4.0.1/com.ibm.itcamt.doc/tt/dita/concept/kto_dom_was.html
https://www.ibm.com/support/knowledgecenter/en/SS5MD2_7.4.0.1/com.ibm.itcamt.doc/tt/dita/cam_ad/installingthedatacollector.html
https://www.ibm.com/support/knowledgecenter/en/SS5MD2_7.4.0.1/com.ibm.itcamt.doc/tt/dita/cam_ad/configure_dc_to_monitor_app_server_unix.html

or WASTT:
https://www.ibm.com/support/knowledgecenter/en/SS5MD2_7.4.0.1/com.ibm.itcamt.doc/tt/dita/task/kto_dom_wastt_inst.html

You can also refer to some articles published on IBM developerworks regarding ITCAM for Tx Best Practices :

https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli%20Composite%20Application%20Manager/page/Best%20Practices%20for%20ITCAM%20for%20Transactions

like :
Troubleshooting and Tuning Best Practices for Web Server Monitoring in Appliance Mode
https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli%20Composite%20Application%20Manager/page/Troubleshooting%20and%20Tuning%20Best%20Practices%20for%20Web%20Server%20Monitoring%20in%20Appliance%20Mode

or
Tivoli Composite Application Manager for Transactions Best Practices for Web Server Monitoring
https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli%20Composite%20Application%20Manager/page/Tivoli%20Composite%20Application%20Manager%20for%20Transactions%20Best%20Practices%20for%20Web%20Server%20Monitoring
or

Web Response Time - End 2 End Best Practices
https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli%20Composite%20Application%20Manager/page/Web%20Response%20Time%20-%20End%202%20End%20Best%20Practices

A2) ports used 
- The various ITCAM4Tx agents (tx collector (TU) , tx reporter (TO), WRT agent (T5), AMC agent (T3) ) are using default ITM TEMA port to communicate with the TEMS.
By default, this port is 1918 for IP.PIPE or 3660 for IP.SPIPE. see : 

http://www.ibm.com/support/knowledgecenter/SSTFXA_6.3.0/com.ibm.itm.doc_6.3/install/unix_configms.htm

- The various Data collector plug-ins communicate with the Tx Collector aggregation agent using default port 5455. see :
https://www.ibm.com/support/knowledgecenter/en/SS5MD2_7.4.0/com.ibm.itcamt.doc_7.3/tt/dita/reference/kto_tools_datacoll_coll.html
 

 

 

B) How to evaluate if filtering settings review in AMCe is needed for Tx Reporter?

One way to check Tx Reporter data is to use AggregationAnalyzer.jar tool also known as "Transaction reporter configuration checker"

see link :

https://developer.ibm.com/apm/resources/ibm-tivoli-composite-application-manager-transaction-reporter-configuration-checker/

latest version I've got of this tool (provided as is, no official support) :

readme:

https://ibm.box.com/s/hj8m6onfhpjoc3pqedtu7pmispb10jwz

tool:

https://ibm.box.com/s/6aagas3y2iwi84udi71kvp2dsvdta2eb

 

How to use this tool?

1) copy AggregationAnalyzer.jar into a java bin directory of a local machine where java is installed

e.g. I copied AggregationAnalyzer-6Mar2014.jar into Windows system C:\IBM\ITM\java\java70_x64\jre\bin

2) copy the ./todata directory content into local machine where AggregationAnalyzer.jar was installed

3) run Tx reporter configuration checker

java -jar AggregationAnalyzer.jar

 

Example :


C:\IBM\ITM\java\java70_x64\jre\bin>java -jar AggregationAnalyzer-6Mar2014.jar 93478.842.758.todata

 

Reading 93478.842.758.todata\topology\MetaData.xml
Processing 1386 records
0%       25%       50%       75%     100%
|---------+---------+---------+---------|
|.......................................|
Found 1386 records in 1.535s. There were 0 records with missing RecordIdentity.xml
Writing HTML report to C:\IBM\ITM\java\java70_x64\jre\bin\todata_summary.html
Writing CSV report to C:\IBM\ITM\java\java70_x64\jre\bin\todata_raw.csv


4) review the report produced by this tool
open C:\IBM\ITM\java\java70_x64\jre\bin\todata_summary.html file content using a browser

and review in particular results seen as 'Critical', like for example:

TxReporter-Report-example.png

 

5) Filtering guidelines can be found here:

Aggregation Transaction Tracking nodes by using reporting and filtering:

https://www.ibm.com/support/knowledgecenter/en/SS5MD2_7.4.0/com.ibm.itcamt.doc_7.4.0.0/rt/AMC_Editor/kto_amc_tt_aggreg_nodes.html

 

Best Practices when using "Agentless Tracking" (a.k.a. TCP Monitoring):
https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli+Composite+Application+Manager/page/Best+Practices+for+Agentless+Tracking

AMC Editor filtering of TCP traffic via Components:
https://www.ibm.com/support/knowledgecenter/en/SS5MD2_7.4.0.1/com.ibm.itcamt.doc/rt/AMC_Editor/AMCE_components.html

AMC Editor filtering of Tx :
https://www.ibm.com/support/knowledgecenter/en/SS5MD2_7.4.0.1/com.ibm.itcamt.doc/rt/AMC_Editor/AMCE_filters.html

Filtering and Reporting Best Practices guide  :
https://www.ibm.com/developerworks/community/wikis/form/anonymous/api/wiki/f3edd794-17c1-4ea0-8ea8-674752b3ece2/page/ee837f29-fa16-48e9-99f2-474e829ce831/attachment/6e0f2ad8-8b28-4ff9-94f0-d8eb590a5780/media/filtering%20and%20reporting%20best%20practices%20v1.0a.pdf

 

If you see in to_summary.html report the Total Aggregates with a high value, like > 5K, or Critical red status, then filtering must be applied. Check under each category in the Results of such report which data collector type is the most contributor to such big Total Aggregates value. Generally, it's often WRT (t5) agents that have not enough filtering in place, and they flow too much data into Tx Reporter agent.

You can also search in the TO ras1 agent log file, and search for "topologydeterminationcollection"  line entries like :

topologydeterminationcollection.cpp,417,"logTotalStats") records total 5855. edges total 11537. verticals total 0, interactions total 3.

a "records total" of around 5K is the usual limit

a sum of edges total, verticals total, and interactions total of about 25K is the usual limit

 

6) Gradual approach:

One recommendation is to check in TEP Tx Reporter  "Reporter status" workspace; does it show many (WRT/T5s?)
aggregation agents connected to it ?
example : Aggregation agent contacts available = 22

Then do NOT connect ALL the T5s to the Tx Collector agent; to go and connect them gradually, use the "Enable Transaction Tracking Integration" and untick/disable this option available in WRT agent configuration settings. Then review todata_summary.html report, apply extra filtering if needed, then connect more aggregation agents (WRT, Data collectors), and continue to enable filters to keep todata files and topologies readable in TEP and under control.

 

C) " SqlException: The row length of the table exceeded a limit of "4005" bytes " message is seen in TEP "Errors" view of Summarization and Pruning Agent workspace

If you are using ITCAM for Tx Tracking V7.4.0.1 with Tx Reporter agent and historical data being warehoused in TDW via S&P and Warehouse ITM agents, and you are seeing the above message reported in TEP, then check below ..

example of message seen in TEP
 54010 | com.ibm.db2.jcc.am.SqlException: The row length of the table exceeded a limit of "4005" bytes. (Table space "USERSPACE1".).
SQLCODE=-670

Check your Summarization and Pruning (SY) agent log,

== 103187 t=Thread-5 Summarization and pruning agent started
== 103188 t=Thread-5 TAKE0120 schedule :

..
..

== 103498 t=work5 SQL IN ERROR :
  statement name     : Transaction_Instances_H.Alt1
  SQL                : ALTER TABLE "XS2INST1"."Transaction_Instances_H" ADD "AVG_Initial_Time_Deviation" DECIMAL (31, 2)
  template SQL       :
== 103499 t=work5 SQLException occured. SQL State = 54010 , SQL Error Code = -670
== 103500 t=work5 com.ibm.db2.jcc.am.SqlException: The row length of the table exceeded a limit of "4005" bytes. (Table space "USERSPACE1".). SQLCODE=-670, SQLSTATE=54010, DRIVER=3.57.82


This problem is matching this APAR # IV86223 :
Summarization and Pruning agent reports "Size Row Exceeded" whenwarehousing Transaction Reporter tables

This APAR is addressed with test fix : 7.4.0.1-TIV-CAMTT-TF0030. Contact IBM support if you need to obtain such test fix.

As stated in its 7.4.0.1-TIV-CAMTT-TF0030's readme :
II. PREQUISITES AND CO-REQUISITES
Transaction Reporter version 07.40.01.00

III. CONTENT DESCRIPTION
The MDV contains a set of SQL files to be executed by Tivoli DataWarehouse (TDW) administrator.  The scripts will migrate the Transaction Reporter tables
that exceeds 4K in table width.

Fixes included in 7.4.0.1-TIV-CAMTT-TF0030 includes the following APARS:
IV80137 Summarization and Pruning agent reports "Size Row Exceeded" when warehousing Transaction Reporter tables.

IV. INSTALLATION INSTRUCTIONS
This patch is to overcome an edge case where the TDW was created pre ITM 6.2.0 and the table spaces were 4K (ITMREG4K) wide instead of 8K (ITMREG8K) wide.
It should not be applied on any TDW where the warehousing tables are already at 8K.
 

D) TEP GUI "Agent Management Services" (AMS) workspace shows alert : 'Agent overutilizing CPU'. Why?

if ktucma process (Transaction Collector agent) or ktocma process (Transaction Reporter agent) is stopped and restarted after some time of run, unexpectedly, it's very likely that the AMS has been activated for this process. Check

See for reference : http://www-01.ibm.com/support/docview.wss?uid=swg21514839

example of symptom as seen in TEP :

 

AMS2.png

0 comments
2 views

Permalink