Author: @Sorna Sarathi N
Co-authored by: @GIREESH PUNATHIL
Introduction
In part 1 of this Java application monitoring series, Java Monitoring 101: Basic Concepts, we introduced essential ingredients in JVM monitoring, metrics and their constituent events, and mechanisms to capture those events. In part 2 of this series, Java Monitoring 101: Tools, we introduced common monitoring tools that are used in the Java ecosystem, their advantages and challenges and how they can be used effectively to monitor Java applications. Then, in part 3 of this series, Java Monitoring 101: Debugging, we explored how we can utilize these monitoring tools and methodologies to debug common problems and issues our cloud-native Java applications may face.
Finally, in this last article of this Java monitoring series, we’ll take a look at some monitoring best practices to make our monitoring more efficient and useful.
Monitoring Best Practices:
Let’s have a look at some best practices you should keep a note of when monitoring a Java application:
Monitoring selection and configuration
- There are a range of monitoring tools that are available and each tool may provide different insights, so it is advisable to use a more than one tool to monitor to obtain a holistic view of the application. However, once you have become familiar with your application’s runtime behaviour and efficiency, you may pick up one tool that is proved to best represent the measurement of the application’s characteristics.
- Before starting to monitor based on the project’s goals and requirements, set a baseline for most of the profiles by collecting metrics from the application in its healthy state. This helps to detect anomalies easily and weed out false positives.
- Re-configure the monitoring environment based on the changing application requirements, as using stale / static configuration might produce unexpected results. For this, whenever there is a new deployment, review the application’s code and configuration changes and adjust the configuration of the monitoring accordingly.
Production
- Reduce amount of monitoring in production and use it only when needed. monitoring is not free, and it comes with a performance penalty by itself. Several monitoring tools provide options to attach to a running JVM and collect data on the fly, so leverage that facility whenever applicable.
- Use circular buffers for recording monitoring data to decongest the disc. Understand the tradeoff between collecting data for a longer duration (which gives more information while bloating up the disc) and recycling data buffers at a predefined capacity (which gives efficient disc usage while runs the risk of missing out vital monitor data) and strike a right balance for your application’s characteristics.
- Always upgrade your application and Java versions to stay latest and greatest, ensuring you can use modern monitoring tools and reducing the risk of errors/bugs that have been fixed by fixpacks in later versions.
Visualisation and Interpretation
- Know when to monitor and when not to. We have seen unwanted and micro-level monitoring produced outlier results which later proved to be harmless. As described in our earlier blog, the key is to arrive at baseline data for each metric and then compare the corresponding values at production to see the degree of deviation before jumping to conclusions.
- Define reasonable frequencies for monitoring certain metrics. For example, collecting GC logs every second on an application that runs GC every 20 minutes is overkill. Understanding the change patterns and life cycles of each metric helps to determine what to measure and when, in an efficient manner, making the overall monitoring as light weight as possible.
- Over a period of time, use the learnings from monitoring of your application and build a document that captures the relationship between workload patterns and visual patterns. This will help in identifying anomalies rapidly.
Summary and next steps:
Through this article, you should now be aware of best practices you can try when monitoring your Java applications to ensure your monitoring is as efficient and useful as possible. If you’ve been following along the whole series, you should also now be familiar with basic monitoring concepts, tools you can make use of for effective Java monitoring and problems we can debug using monitoring tools and data.
If you’re interested in continuing your learning of monitoring, check out this free, in-depth learning course on edx: [Monitoring and Observability for Application Developers](https://www.edx.org/learn/devops/ibm-monitoring-and-observability-for-application-developers). This course provides a comprehensive overview of monitoring and observability, and teaches you the hands-on skills to employ monitoring, observability, and logging for your application.
Alternatively, take a look at the interactive guides available on the Open Liberty website that take you through how to use further monitoring/observability tools for your Java applications at the application level: https://openliberty.io/guides/#observability.
