IBM Z and LinuxONE - IBM Z

IBM Z

The enterprise platform for mission-critical applications brings next-level data privacy, security, and resiliency to your hybrid multicloud.

 View Only

IBM Z Common Data Provider 2Q2021 PTF Supports batch load of SYSLOG data

By Zhen Yi Jia posted Wed May 19, 2021 03:42 AM

  

In the IBM Z Common Data Provider 2.1.0 2Q2021 continuous delivery PTF, the Log Forwarder can collect batch data of zOS-SYSLOG in data sets.

With this function, IBM Z Common Data Provider can send historical log data to zAware through Kafka for initial training.

Key changes for this function:

  • Add z/OS SYSLOG from ARCHIVE data stream under the Batch Data Group in the Configuration Tool.
  • Add a batch mode startup parameter -b to the Log Forwarder.
  • Add an HBOIN DD in the Log Forwarder job to specify data sets to collect

Configure a batch data stream

The z/OS SYSLOG from ARCHIVE is a brand-new data stream with a different color from the z/OS SYSLOG data stream. Therefore, the first step is to add the batch data stream under the Batch Data Group.

Figure 1 shows a batch data stream on Configuration Tool

Shows a batch data stream on Configuration Tool

In the configuration window, there are some fields that need input.

  • Name: The name that uniquely identifies the data stream to the Configuration Tool. If you want to add more data streams of the same type, you must first rename the last stream that you added.
  • Data Source Name: The name that uniquely identifies the data source to subscribers.
  • File path: A unique identifier that represents the data origin. For a batch data stream, the File Path is used to bind datasets in the job of Log Forwarder instead of a real data set name.

Figure 2 shows the configuration window for z/OS SYSLOG from ARCHIVE

Shows the configuration window for z/OS SYSLOG from ARCHIVE

Configure the Log Forwarder

You can create a new job file for batch mode, so that it will not affect your current Log Forwarder job. These two jobscan run at the same time.
In the Job file, there is an HBOIN DD that specifies the data sets to collect. The data set must be specified with the file path and data set name format. In each line, you can specify one data set only. In this way, the data set will be streamed to a specific data stream mapped by the given file path. If there are multiple lines in the HBOIN, the Log Forwarder will collect them one by one to ensure the data are streamed in order.

 

Figure 3 shows the HBOIN DD in the job file
Shows the HBOIN DD in the job file

 

To run the Log Forwarder in batch mode, you must add a "-b" parameter to the start command. In batch mode, the Log Forwarder will only collect batch data streams and stop after sending all data sets.

 

IBM Z Common Data Provider provides detailed step-by-step guidance on how to use the batch load of SYSLOG data.

The detailed procedure on how to configure z/OS SYSLOG from ARCHIVE data stream can be found here:

https://www.ibm.com/docs/en/zcdp/2.1.0?topic=forwarder-zos-syslog-from-archive-data-stream

The detailed procedure on how to create the Log Forwarder batch job:

https://www.ibm.com/docs/en/zcdp/2.1.0?topic=clf-creating-log-forwarder-batch-job-sending-syslog-data-data-streamer

 

For additional information on what’s new in the IBM Z Common Data Provider 2Q2021 PTF see:

https://www.ibm.com/docs/en/zcdp/2.1.0?topic=whats-new-in-version-210-may-2021

 

 


 

0 comments
18 views

Permalink

https://community.ibm.com/community/user/blogs/zhen-yi-jia1/2021/05/19/ibm-z-common-data-provider-2q2021-ptf-batch-load