AIOps on IBM Z - Group home

Analytics dreams really do come true with the new IBM Z OMEGAMON Data Provider

  
Analytics dreams really do come true with the new IBM Z OMEGAMON Data Provider

Analytics professional with feet up his desk, with bubble above his head, showing his dreams of analytics graphs


With the holidays approaching, maybe its true, that dreams can come true!  As a mainframe developer, application administrator, and data scientist, my dream has been for easy access to mainframe performance metrics for advanced use cases such as special monitoring based on machine learning.  With the availability of the IBM Z OMEGAMON Data Provider (IZODP), my dream is coming true!

I’m also a believer in the rule of three, and in this case, the dream has three parts:  built-in, modern mainframe metric streaming, easy set up and management of a getting-started data warehouse, and built-in instrumentation for the streamer itself, for easy management of performance, capacity, and uptime.

When I came to the IBM Z brand a few years back, OMEGAMON was my go-to portal for learning the mainframe, and managing my mainframe workloads.  It gave me great eyes on the internals.  I quickly learned of the OMEGAMON SOAP API, and then the REST API, for gathering real-time metrics.  I built a simple, bash script data streamer, that served me well over the years, for ad-tech side projects, some of which can be seen on my drdavew00 YouTube channel, such as streaming metrics into the Elastic Stack, and trying the Elastic Stack X-Pack Machine Learning capabilities:  https://www.youtube.com/watch?v=aGn8ruVxq3E.

Dream come true part 1:  Simple, built-in streaming:

Now, with the IZODP, the first part of my dream come true is a reality! OMEGAMON metric streaming is built into the IBM Z Systems Management Suite 2.1.1 and IBM Z Monitoring Suite 1.2.1 as part of IBM Z OMEGAMON Integration Monitor 5.6.  A modern broker and connector has been implemented to do real-time streaming of all the same OMEGAMON metrics that have served the IBM Z monitoring and diagnostic community for years.  The broker is based on Zowe the leading edge approach to z/OS applications, specifically leveraging a plug-in to the Zowe Cross Memory Server.  A full Zowe install isn’t required.  The connector is a Spring Boot Java application.  Simple components for making OMEGAMON metrics available to the outside world, easily integrated into z/OS and USS.  All your existing agents can easily stream their data, or you can selectively enable agents and metrics you desire.  The connector can stream in TCP mode for Logstash, and can also stream to Prometheus and Apache Kafka endpoints.  In short, self-defining, well curated, data science-ready metrics, easily streamed in modern JSON format, is a dream come true!

Overview of IZODA broker and streamer:

Block diagram showing the IZODP broker feeding data to three types of output in the connector: TCP for Logstash, Kafka, and Prometheus.

Dream come true part 2:  Easy data warehousing

The second part of my dream come true is a quick start approach to a data warehouse.  The world’s simplest Logstash configuration file is all that’s necessary to connect IZODP streams to any output that Logstash supports, such as Splunk, Elasticstack, InfluxDb, etc.  A docker repo is available to easily stand up an Elastic Stack data warehouse, and import index templates and  sample Kibana dashboards.  Follow the README, and you too can be up and running quickly with a data warehouse, and set of OMEGAMON z/OS metric sample dashboards.  Here’s the github repo:  https://github.com/z-open-data/z-omegamon-analytics-elastic-docker

Address Space CPU Utilization sample dashboard:

Time series line graph of high CPU users, histograms of top users

 

Dream come true part 3:  Turn-key monitoring of the streamer

The third part of my dream come true is eyes on the streamer itself.  As a Spring Boot application, the IZODP connector is easily monitored in great detail with Prometheus.  Here too, there’s a getting started package available that quickly stands up a Prometheus instance, and provides Grafana dashboards:  https://github.com/daveyc/prometheus 


Prometheus monitoring of IZODP:

Time series line graphs, horizontal bar graph gauge, showing stats over time, cumulative data transfer

 

Looking ahead

I’ll say more about aspects of this dream come true!   Out of the box, 10 IBM Z OMEGAMON Integration Monitor 5.6 (KM5) workspaces or tables are enabled, with default collection intervals, that OMEGAMON customers are likely already greatly familiar with.  There are sample Kibana dashboards provided for these 10 workspaces or tables.  The remaining 90 IBM Z OMEGAMON Integration Monitor 5.6 workspaces or tables can be enabled, by quickly adding them to the simple list specified in the RKANPARU(KAYOPEN) dataset member, and the collection interval can be changed in this dataset member also.  Sample dashboards are not provided for these 11 other tables. 

RKANPARU(KAYOPEN)  table selection dataset member:

3270 view of KAYOPEN configuration data set, with table names and attributes like collection interval



With Prometheus monitoring of the connector’s heap, throughput, etc., users might want to tune their collection intervals to get the best balance of resource footprint and time resolution.  Initial look at performance show that IZODP is quite capable of streaming metrics for dozens, and dozens of LPARs, but that’s just for the initial support of z/OS metrics – stay tuned!!

While I like IBM Z OMEGAMON Integration Monitor 5.6 some of my favorite OMEGAMON workspaces are for Db2 and CICS®.  I look forward to IBM adding many of the other OMEGAMON workspaces, along with sample dashboards, in the near future!

›  I hope the 2021 holiday season is a time for all our dreams to come true.  Let’s look forward to the new year, with a great new way to collect, analyze, and share mainframe performance metrics, where we can let our imaginations drive new techniques to find new insights in our IBM Z operating environment.
› I have three resolutions for 2022 to integrate IZODP with IBM Z Anomaly Analytics, IBM Watson AIOps, and a leading third party analytics platform like Databricks.

 

Learn more, get involved: 


⇒ Mark your calendars, for a great webinar from IIBM and Rocket Software, coming on Dec 2.  Do you want to know how you can use your existing CICS data tools with modern, open dashboards for new visualizations? See IZODP in action!!   Join us on Dec 2 http://ibm.biz/CICS-1202

⇒  What to compare notes, hints, hit me up on social media:  drdavew00!

⇒ Try the Elastic Stack container for an easy Proof of Concept after installing IZODP  https://github.com/z-open-data/z-omegamon-analytics-elastic-docker

⇒ Try the Prometheus Github to monitor your IZODP connector data flow https://github.com/z-open-data/odp-prometheus-grafana

⇒ Learn more and do some planning for install:  https://community.ibm.com/community/user/ibmz-and-linuxone/blogs/james-porell1/2021/11/07/installation-considerations-for-omegamon-data-prov