AIOps: Monitoring and Observability - Group home

IBM Z OMEGAMON for JVM Automated Archive and Extract for Analytic Insights


The IBM Z OMEGAMON family of products expands its capabilities with continuous integration/continuous delivery (CI/CD) PTFs.  In March 2022, IBM Z OMEGAMON for JVM PTF UJ08099 delivered a new feature improving the breadth of its z/OS Connect EE API monitoring. The Archive, Extract, Transform (AET) feature allows customers to automatically archive z/OS API monitoring detail records from z/OS System Logger log streams to long-term archives on  generation data group (GDG) sequential files. These archives may be stored on low-cost DASD devices or mass media cartridge tape images. The archive data sets are stored in a compact, binary format.

Figure 1. OMEGAMON for JVM automated archiving process for z/OS Connect EE API data

A utility program is provided that is used to extract records from these archives and transform the data into formats that can be easily ingested by analytic insight (AI) tools such as Elastic Search with Kibana. The formats that are provided in the initial release are CSV, JSON and NDJSON. The extract function of the utility allows filters to be applied on just about any attribute of the data. Select all APIs executed during a specific time range from an archive for example. Or extract records for specific APIs matching a regular expression. This capability allows a reduction in size of the interim datasets required to load into the AI tools.


OMEGAMON for JVM provides an excellent platform for real-time monitoring of z/OS Connect EE API response times and errors. ITM situations provide instant awareness of problem conditions. A wide range of dashboards and workspaces in the enhanced 3270UI, the Tivoli Enterprise Portal and the IBM Z Service Management Explorer are perfect for daily operations and short-term history analysis of z/OS Connect EE APIs. For customers with very large volumes of API activity it may not be practical to maintain API data in log streams for periods longer than 24 hours. Without an archive process in place, the data in log streams is automatically deleted once the log stream data sets exceed the logger policy retention period parameter.


The AET archive process uses ITM automation to submit batch jobs at daily intervals (usually soon after midnight) which copy the previous day’s records to a GDG version data set. Customers can specify the type of storage medium they wish to store the archive data and the maximum age of the archives before they are automatically deleted. This is done by tailoring a JCL template file used to generate the batch jobs. Customers must also define the names of the log streams they wish to archive.  Configuration of the feature is simple, straightforward and is enabled with support in PARMGEN and IBM Z Monitoring Configuration Manager (IZMCM). You will need PARMGEN PTF  UJ08103for this support.


For a full description of this feature, including installation and configuration refer to IBM Documentation at