AIOps: Monitoring and Observability - Group home

Unlocking OMEGAMON Data for Analytics


Is your IT environment becoming more complex? Are there not enough skilled subject matter experts to rapidly find performance issues? Are there too many applications that require monitoring? If you’ve experienced any of these concerns, then the new IBM® Z OMEGAMON® Data Provider (ODP) feature may be what you are looking for to help address those problems.
There is a wealth of subsystem data available within the IBM OMEGAMON family. Businesses are looking to leverage analytics, artificial intelligence, and machine learning to automate processes and act based on myriad operational data available within their enterprises. The goal of ODP is to “unlock” that data and make it available in industry standard, self-describing formats that can be quickly consumed by a wide variety of analytics applications.
ODP is a simple, but very efficient service. It takes an input, transforms it to a requested output format and makes it accessible for analytic processing. It can run within z/OS as a zIIP enabled Java application or on a supported non-z system. When first delivered, ODP will take data that is prepared for Near Term History collection by the IBM Z OMEGAMON Monitor for z/OS agent and then transform it to be sent via JSON to Elastic, Splunk or Kafka  servers. No end user programming required to exploit those analytics servers.   Some programming is required to pull the data or access it via applications using the Prometheus  Metrics API so the data can be loaded into a Grafana  dashboard.
Figure 1: Architecture view of OMEGAMON Data Provider 

Configuration Files use YAML

The input from OMEGAMON is described in a simple YAML file. The target platforms are also described via YAML enabling the business to customize the specific data that will be sent to each analytics platform to reduce the volume of data and focus on rapid response within the targeted analytics platform.

Figure 2: ODP Data Broker KAYOPEN.YAML

Figure 3: ODP Data connect server connect.yaml

Analytics Dashboards

With general availability of ODP, only the z/OS agent is supported via open dashboards out of the gate. Our process is to review the data tables and key attributes for each OMEGAMON agent and create sample dashboards that will independently be made available. The goal is for Db2® and CICS® to be done next by the end of this year. Stay tuned for additional info for each new collection of published dashboards. 
We’ve already seen the power of these new dashboard models. Recently, a customer asked for an enhancement to the existing user interfaces within OMEGAMON. This would normally take a few months to schedule, develop, test, and deliver those enhancements. Using an Elastic dashboard, however, it took less than an hour to provide the requested view of data.  

Figure 4: Elastic Dashboard sample
Using the open analytics dashboards enables a business to create application specific dashboards with z/OS and non-z components. Better for the business to monitor end-to-end application performance to rapidly identify where issues may be occurring. Executive level dashboards can be built for systems, subsystems and application topologies using speedometer, bar, line graph and other models that wouldn’t be possible in traditional 3270 interfaces.

Figure 5: Grafana Dashboard sample

A History of Sample Dashboards

And this is not the first IBM monitoring offering that works with analytics dashboards. Similar technology has been deployed with IBM Z CICS Performance Analyzer, IBM Transaction Analysis Workbench and IBM IMS® Connect Extensions.

Figure 6: IMS Connect Extensions Splunk sample dashboard

Where to find OMEGAMON Data Provider

OMEGAMON Data Provider is only available in the IBM Z Monitoring Suite 1.2.1 or IBM Z Service Management Suite 2.1.1 and above monitoring suites. If you are at a different level, you may be able to trade up, via a credit for current purchases, as well as rapidly deploy using the new IBM Z Monitoring Configuration Manager. The ODP feature can be installed upon that in less than two hours. Another blog provides highlights of the installation experience. Installation Tips blog  
It’s a new day in performance monitoring. Analytics and ultimately, machine learning, are the order of the day. OMEGAMON Data Provider is an excellent way to unlock your data and feed the analytics offerings of your choice.