AIOps

AIOps

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only

System P Agents 

Fri March 13, 2020 09:35 AM

Table of Contents

  1. Required AIX Levels (perfagent.tools fileset)
  2. General Considerations
  3. Latest Interim Fixes
  4. Best Practices
  5. Common Tracing
  6. Common Problems
  7. Common Questions
  8. How HMC Agents Collects Data
  9. User's Guides
  10. Reports
  11. Blog Posts

 

Required AIX Levels (perfagent.tools fileset)

  1. The UNIX OS agent now integrates the AIX Premium agent function, so only the UNIX OS agent is needed (see General Considerations below).  However, both agents require certain levels of the AIX perfagent.tools fileset in order to collect accurately the performance data.  See Technote #1447016 for more details.
  2. The HMC Base agent v6.2.2.3, which replaces the CEC Base agent, does not require the AIX perfagent.tools fileset because it gets the data directly from the HMC.
  3. The CEC Base agent should no longer be used, but if it is, all the AIX LPARs on the server must be at the AIX perfagent.tools fileset level in the technote above.

General Considerations

  1. The UNIX OS agent replaces the AIX Premium agent.
    1. UNIX OS agent v6.3 Fixpack 2 (available September 13, 2013) integrates all the AIX Premium attributes except for the Active Memory Expansion and Workload Manager attribute groups.  See Appendix B in the IBM Tivoli Monitoring UNIX OS Agent Reference Version 6.3 Fix Pack 2.pdf.
    2. Previous versions:
  2. System P Agents v6.2.2 Interim Feature 3 is now available (08-Mar-2013) and includes HMC agent v6.2.2.3 and AIX, CEC and VIOS agents v6.2.2.2.  The package descriptions and part numbers on Passport Advantage are:

     

    IBM Tivoli Monitoring for System p V6.2.2 Interim Feature 3, English (CIH93EN) (Both agent and support files are included in this one package)

    IBM Tivoli Monitoring for System p V6.2.2 Interim Feature 3, Language Support, Multiplatform, Multilingual (CIH94ML)

     

     

    1. The HMC Base agent 6.2.2.3 replaces of the CEC Base agent (see this blog post) and does not require the AIX perfagent.tools fileset.
    2. New in the HMC Base agent 6.2.2.3:
      • The HMC Base agent was updated to retrieve CPU usage sampling events using the lslparutil HMC command. The HMC must be configured to collect these samples by using the chlparutil HMC command. See the HMC documentation for more information about the lslparutil and chlparutil commands.
      • In V6.2.2.3, the HMC Base agent represents each server as an IBM® Tivoli® Monitoring subnode. With these subnodes, each Server is represented by a separate node in the Tivoli Enterprise Portal navigation tree under the HMC Base agent node. When a situation affects an individual server or resources for the server, the affected server is immediately identifiable. Representing each server as a subnode also provides for the creation and association of server-specific situation threshold values
      • The HMC Base agent includes Self Describing Agent support when the agent is installed in the same CANDLEHOME as an IBM Tivoli Monitoring V6.2.3 Fix Pack 1 Tivoli Enterprise Monitoring Agent.
      • New attribute groups and attributes were added to monitor CPU utilization for Servers, CPU_Pools, and LPARs.
      • New workspaces that can be used to drill down from Servers to Pools to LPARs through IBM Tivoli Monitoring dynamic workspace links were added to the agent.
      • For more, including Reports changes, see the HMC Base Agent 6.2.2.3 Users Guide - New in this release.
    3. The VIOS Premium agent is already installed on the VIOS.  See the Users Guide for configuration steps.  Use the download packages above to install the application support files on TEMS and TEPS.
    4. The AIX Premium agent is replaced by the UNIX OS agent, as described above.
    5. The CEC Base agent is replaced by the HMC Base agent.
    6. For APARs included in this release, see Technote: #1512471

For information on previous releases of the System p agents, go here: Old Release Information for System p Agents

 

Latest Interim Fixes

 

 
Agent Interim Fix Link
AIX Premium 6.2.2.2-TIV-ITM_AIX_PREM-IF0005 (released 12/06/12) http://www.ibm.com/support/docview.wss?uid=swg24033859
CEC Base 6.2.2.2-TIV-ITM_CEC BASE-IF0005 (released 08/25/14) http://www.ibm.com/support/docview.wss?uid=swg24038249
HMC Base 6.2.2.3-TIV-ITM_HMC_BASE-IF0004 (released 06/14/16) http://www.ibm.com/support/docview.wss?uid=swg24042283
VIOS Premium 6.2.2.2-TIV-ITM_VIOS_PREM-IF0006 (released 10/24/14) http://www.ibm.com/support/docview.wss?uid=swg24038578
Reports 6.2.2.2-TIV-ITM_SYSP_RPT-IF0003 (released 08/20/12) (should upgrade to release 6.2.2.3) http://www.ibm.com/support/docview.wss?uid=swg24033306
Language Pack 6.2.2.2-TIV-ITM_SystemP_LP-IF0002 (released 01/21/13) http://www.ibm.com/support/docview.wss?uid=swg24034229 

 

 

Best Practices

  1. Install and run the following agents to monitor your System p environment (see General Considerations above for details):
    • HMC Base agent v6.2.2.3 on any AIX LPAR to monitoring the HMC and the Servers, including CPU metrics on the servers and LPARs.
    • UNIX OS agent v6.3 FP2 or above on each AIX LPAR to monitor all LPAR metrics, including specific AIX metrics that have been ported from the AIX Premium agent.
    • VIOS Premium agent v6.2.2.2 on every VIOS (pre-installed), and install Interim Fix 6.2.2.2-TIV-ITM_VIOS_PREM-IF0005.
    • Remove the CEC Base agent if running the HMC Base agent v6.2.2.3.
    • Remove the AIX Premium agent if running the UNIX OS agent on each AIX LPAR.
  2. Read the System p Virtualization Best Practices (a little dated but still useful): System p Virtualization Best Practices Link
  3. Read Historical Collection Best Practices: Historical Collection Best Practices Link

 

Common Tracing

Common Tracing
  1. Each System P agent is comprised of two processes, the factory agent and the data provider. For all System p agents, do the next step to setup tracing for both processes, but there will be an additional step for the HMC Base agent v6.2.2.3.
  2. For release 6.2.2.2 (6.2.2 Interim Feature 2 or 06.22.02.00), KBB_RAS1 tracing should be set to "ALL" (For example, KBB_RAS1=ALL) in $CANDLEHOME/config/xx.ini file where xx is:
    • px - AIX Premium agent
    • pk - CEC Base agent
    • va - VIOS Premium agent.
    • ph - HMC Base agent (for 6.2.2.3, the file is $CANDLEHOME/config/ph_<instance>.config)
  3. For the AIX, CEC and VIOS agents, the data provider tracing relies only on KBB_RAS1=ALL in the files listed above.
    1. The data provider messages will go in the $CANDLEHOME/logs/<hostname>_xx_*DataProvider*.log. There will be no separate .trc file as there was in earlier releases.  Usually, the data provider logs are of most interest.
    2. The factory agent messages will go in the $CANDLEHOME/logs/<hostname>_xx_kxxagent*.log.  The logs will show the communication flows between agent and TEMS and between agent and data provider.
    3. Most problems are in the data provider, but if the problem is in the agent process, you might need to increase the number of log files, since the files will wrap when full.  In the ini or config file, modify this line to increase the COUNT and MAXFILES: KBB_RAS1_LOG='%(CTIRA_LOG_PATH)/<hostname>_ph_<instance>_%(systask)_%(sysutcstart)-.log INVENTORY=%(CTIRA_LOG_PATH)/<hostname>_ph_<instance>_%(systask).inv COUNT=15 3 LIMIT=5 PRESERVE=1 MAXFILES=15'
  4. For the HMC agent v6.2.2.3, reconfigure the agent to set the log level to FINEST to log more detailed messages to the $CANDLEHOME/logs/kph_data_provider_hmc2_x.log:
    1. Use the itmcmd command to reconfigure the agent and answer "7" for Finest when prompted with "Level of Detail in Data Provider Log [ 1=Off, 2=Severe, 3=Warning, 4=Info, 5=Fine, 6=Finer, 7=Finest, 8=All ] (default is: 4):"
    2. Here is an example from the lab:

      # ./itmcmd config -o hmc2  -A ph   (Note: hmc2 is the instance name)
      Agent configuration started...
      Edit "Monitoring Agent for HMC Base" settings? [ 1=Yes, 2=No ] (default is: 1): 1
      Edit 'HMC Information' settings? [ 1=Yes, 2=No ] (default is: 1): 1
      HMC Hostname (default is: itmhmc1):
      HMC Username (default is: hscroot):
      Edit 'Data Provider' settings? [ 1=Yes, 2=No ] (default is: 1): 1
      Maximum Number of Data Provider Log Files (default is: 10):
      Maximum Size in KB of Each Data Provider Log (default is: 9999):
      Level of Detail in Data Provider Log [ 1=Off, 2=Severe, 3=Warning, 4=Info, 5=Fine, 6=Finer, 7=Finest, 8=All ] (default is: 4): 7

      Will this agent connect to a TEMS? [1=YES, 2=NO] (Default is: 1):
      TEMS Host Name (Default is: itmaix71n):

      Network Protocol [ip, sna, ip.pipe or ip.spipe] (Default is: ip.pipe):

           Now choose the next protocol from one of these:
           - ip
           - sna
           - ip.spipe
           - 0 for none
      Network Protocol 2 (Default is: 0):
      IP.PIPE Port Number (Default is: 1918):
      Enter name of KDC_PARTITION (Default is: null):

      Configure connection for a secondary TEMS? [1=YES, 2=NO] (Default is: 2):
      Enter Optional Primary Network Name or 0 for "none" (Default is: 0):
      Agent configuration completed...
      As a reminder, you should restart appropriate instance(s) for new configuration settings to take effect.

      #

       

  5. For tracing of previous releases of the System p agents, go here: Old Release Information for System p Agents
  6. If an agent or data provider process core dumps, Tivoli Support will ask for snapcore.

Common Problems

  1. HMC agent: TEP shows "Not Collected" for CPU attributes and "No Samples" in Sample Timestamp.
    • The HMC administrator must use the chlparutil command to define the sample interval for each server.
    • For example: chlparutil -r config -m <managed system> -s 300
    • To check the current configuration, run: lslparutil -r config -m <managed system> -F sample_rate
    • For chlparutil syntax, go here: chlparutil syntax and for lslparutil, go here: lslparutil syntax
  2. HMC agent: The chlparutil was run, but TEP still shows "Not Collected" for CPU attributes, and there are over 100 LPARs per server.
  3. VIOS agent: The TEP shows no data and errpt shows aixDataProvider process core dumped.

 

Common Questions

  1. HMC Agent: How do you configure the ssh connection to the HMC?
    • From the LPAR where the HMC Base agent is installed:
    • cd <install dir>/aix526/ph/bin
    • ./setup_hmc_key.pl and answer the prompts

How HMC Agent Collects Data

  1. HMC Commands by Attribute Group (high-level)
  2. HMC Commands per Attribute (detailed)

User's Guides

Reports

  • See Chapter 8 of the HMC Base Agent v6.2.2.3 User's Guide for information about System p reports.
  • The reports are available in the ITM_AGT_SYS_P_6.2.2_IF3_AIX_EN.tar file from the Passport Advantage package "IBM Tivoli Monitoring for System p V6.2.2 Interim Feature 3, English (CIH93EN)".  Untar the file and use the \REPORTS directory.

Blog Posts

 

 

Statistics
0 Favorited
17 Views
0 Files
0 Shares
0 Downloads
Global message icon