Introduction

In this article we provide an overview of how z/OS Connect EE audit data can be viewed in a Splunk dashboard.

Common Data Provider for z Systems

IBM Common Data Provider for z Systems provides the infrastructure for accessing IT operational data from z/OS systems and streaming it, in a consumable format, to an analytics platform such as Elasticsearch, Apache Hadoop or Splunk. It monitors z/OS log data and SMF data and forwards it to the configured destination. Over 140 different data sources are supported including all standard SMF records, Syslog and application logs from CICS, IMS, Netview and other subsystems. A web-based configuration tool is used to specify what data you want to collect from your z/OS system, where you want the data to be sent, and what form you want the data to arrive in at its destination. This configuration information is contained in a policy.

In March 2019, Common Data Provider for z Systems added the ability to consume SMF data (record type 123) written by the z/OS Connect EE audit interceptor. Auditing provides traceability and accountability allowing you to capture and record which APIs have been called by which users. The example below shows an extract of the output from the audit interceptor.

See here for a definition of the format of z/OS Connect EE audit records.

Figure 1 shows a Common Data Provider for z Systems policy for streaming z/OS Connect EE audit records (SMF 123 records) to Splunk.

Figure 1 Streaming z/OS Connect EE audit records to Splunk

The Common Data Provider for z Systems Splunk application (https://splunkbase.splunk.com/app/3822/) demonstrates how to use mainframe operational data that has been streamed to a Splunk environment. It provides Splunk dashboards that show near real-time data from SMF 30 and Syslog data. IBM Z Operations Analytics (https://splunkbase.splunk.com/app/4498/) includes a broader set of IBM Z operational dashboards and insights across a variety of subsystems.

Sample scenario

In this sample scenario, we have created a Splunk dashboard (Figure 2) that provides a view of a z/OS Connect EE API workload.

Figure 2 Sample Splunk dashboard

Each of the charts in the dashboard are associated with search strings defined in the Splunk user interface.

• The API Requests chart shows the number of requests per minute for the different APIs of the workload (catalog, customer and phonebook APIs). The search string for this chart is shown below:

• * sourcetype="zos-smf_123*" | timechart count(SM123SSI) by API_SERVICE_NAME​

  The sourcetype defines the type of data being searched, timechart defines the type of chart, SM123SSI represents the type of subsystem (z/OS Connect EE in this case) and records are counted by the API-SERVICE-NAME which represents the name of the API.

• The customer API by HTTP method chart shows the different HTTP methods used to invoke the customer API. The search string for this chart is shown below:

• The catalog API by version chart shows a breakdown of requests for the different versions of the catalog API. The search string for this chart is shown below:

• The catalog API by channel chart shows a breakdown of requests for the catalog API across different channels. The search string for this chart is shown below:

Summary

This article shows a simple example of how z/OS Connect EE SMF data can be viewed in a Splunk dashboard. Splunk is capable of ingesting, correlating, and visualizing data from across your entire enterprise so the range of dashboards that you can create is almost limitless.

More information

See the links below for more information on using the Common Data Provider for z Systems with Splunk:

The IBM Common Data Provider for z Systems Dashboards on Splunk
IBM Common Data Provider for z Systems

This article was written by Systems Lab Services Montpellier. Thanks to John Strymecki of the IBM Z Advanced Technology Group for the creation of the Splunk dashboard.