Engineering

 View Only

ELM - Simple Application Supervision

By Jérôme Desquilbet posted Tue September 01, 2020 09:08 AM

  

Note :

  • screenshots from RTC 6.0.6.1 iFix11 (installed with Liberty and Derby)
  • replace https://my.jazz:9443 by your server subdomain.

Introduction

ELM proposes a JMX MBeans mechanism thats offers application supervision points to activate and read from.

For each enabled observation point, CLM itself is responsible for refreshing the information at regular intervals, the frequency depending on the information collected.

References (on Jazz.net)

How To

“Repodebug”

See Introduction to using the repodebug service for debugging server issues.

From Server Administration > Advanced Properties, Enable repodebug service:

Enable repodebug service



The activation of collection tasks will now be effective.

A measure exploration can be done from the JMX management beans service page.

Note: do the same for CCM.

Collection Tasks

Visit the Server Administration > Serviceability Properties page.

For example:

Enable License Consumption Metrics Mbean


Enable Server Metrics MBean


Note: the frequency (Delay between invocations) is configurable but it is advised not to change the default values (in seconds).

Test

Observation can now be done from the outside, either using a JMX connector (like Splunk’s one); either with a simple HTTP GET, for example calling https://my.jazz:9443/jts/repodebug/mxBeans/com.ibm.team.foundation.server:name=jts,type=serverMetrics/attributes/freeMemory to obtain the available memory:

Free Memory



or, a shell command line:

curl -v -k -b $COOKIES 'https://my.jazz:9443/jts/repodebug/mxBeans/com.ibm.team.foundation.server:name=jts,type=serverMetrics/attributes/freeMemory?Accept=text/json'


would return:

3412420736

The measure point is part of the com.ibm.team.repository.service.internal.serviceability.MetricsCollectorTask > Enable Server Metrics MBean componenet, as seen above, and its recommended refresh frequency is 900 seconds.

A simple implementation can therefore be to write a script for each measure to be done: log in to ELM, then read and write to a CSV file with a timestamp. The file (freeMemory.tsv in the example below) can the retrieve at any moment as a spreadsheet.

Simple freeMemory.sh script (here in /opt/Jazz/scripts/monitoring/):

#!/bin/sh
cd /opt/Jazz/scripts/monitoring
COOKIES=./cookies.txt
rm -f $COOKIES
USER=the_user # replace with the JTS (technical) user
PASSWORD=the_passwd # replace with their password
HOST="https://my.jazz:9443/jts"
FILE=freeMemory
# Login
/usr/bin/curl -k -c ${COOKIES} ${HOST}/authenticated/identity
/usr/bin/curl -k -L -b ${COOKIES} -c ${COOKIES} \
  -d j_username=${USER} -d j_password=${PASSWORD} \
  "${HOST}/authenticated/j_security_check"
# Date with seconds precision
Date=`date --iso-8601=seconds`
# https://my.jazz:9443/jts/repodebug/mxBeans
# Domain: com.ibm.team.foundation.server
FreeMemory=`/usr/bin/curl -k -b ${COOKIES} \
  ${HOST}'/repodebug/mxBeans/com.ibm.team.foundation.server:name=jts,type=serverMetrics/attributes/freeMemory?Accept=text/json'`
echo -e "${Date}\t${FreeMemory}" >> ${FILE}.tsv #


A better version will avoid to log in each time, share the session cookie, and trap the errors.

This script is to be run every 900 seconds (15 minutes) by the cron service (see man crontab and man cron) of a (small) server observing the ELM server. Note: to not send an email each time, modify the deamon configuration using sudo vi /etc/sysconfig/crond to set CRONDARGS=-s -m off; then restart the daemon using sudo systemctl restart crond.service

Edit the /etc/crontab configuration:

chmod u+x freeMemory.sh crontab -e


and add the following line (see man 5 crontab) :

0,15,30,45 * * * * /opt/Jazz/scripts/monitoring/freeMemory.sh

Conclusion

As it very difficult to anticipate the sizing of the ELM servers, especially when the number of users continuously grows, it is necessary to monitor what is happening. ELM can easily be monitored, it is “built-in”.


#Engineering
#Sustainability

Permalink

Comments

Fri September 25, 2020 02:13 AM

Many thanks to Jérôte, helpful information 👍

Sun September 06, 2020 09:49 AM

Thanks Jérôme for the nice overview and the related links collection.