IBM Destination Z - Group home

Monitoring Your z/OS Network Traffic

By Destination Z posted Mon December 23, 2019 03:34 PM


Imagine being directed by your manager to identify the overall quality of the cryptographic protection for your z/OS network. What exposures do you have? Who is using unapproved protection protocols, or worse, not using any protection at all? Where would you start?

There are numerous methods on z/OS that can cryptographically protect TCP/IP traffic, such as system SSL, Java Secure Sockets Extension (JSSE), integrated IPSec, z/OS OpenSSH and Application Transparent Transport Layer Security (AT-TLS). The typical configuration of these methods allows for the specification of multiple permutations of acceptable security attributes.

But even if you acquired a deep understanding of each method’s configuration, you would only know which security attributes were possible in your network. You wouldn’t know which attributes had been negotiated by the security endpoints for any given connection. Existing SMF and trace records might give you some hints, but you would still have to work to turn those hints into a comprehensive understanding of the security of your network’s traffic patterns. In the end, you would still probably have some missing pieces.

Protection Monitoring

z/OS Communications Server V2.3 introduces a new feature to provide you with what you need to answer your manager’s request: z/OS Encryption Readiness Technology (zERT). zERT allows a z/OS network security administrator to determine which TCP and Enterprise Extender (EE) traffic patterns to and from their z/OS systems meet approved network encryption policies and which don’t. It does this by collecting and recording, in SMF record format, the cryptographic protection attributes for Transport Layer Security (TLS), SSL, SSH and IPSec security sessions that terminate on the local stack.

Zero or more security sessions can protect a given application connection at any given time. For example, a connection might be flowing in the clear (e.g., no security sessions), it might be protected by a TLS session only (e.g., one security session) or it might be protected by both TLS and IPSec at the same time (e.g., multiple security sessions). Additionally, the type and level of security coverage might change during the life of the connection. For instance, an FTP control connection can transmit data as clear text, change to use TLS security protocols and return to transmitting data as clear text.

Security sessions take into account how data is protected for a given traffic pattern. Multiple application connections using identical cryptographic protection and matching the same traffic pattern map to the same security session. A single application connection using multiple security protocols map to multiple security sessions. Figure 1 shows the relationship between application connections and security sessions.

Figure 1: Examples of Application Connections and Security Sessions

Monitoring Protection

zERT provides two functions at no charge to assist network administrators in determining how data in flight is being protected:

 1. zERT Discovery. This tracks and records significant events regarding the security sessions that are used to protect each individual application connection. zERT Discovery is available with z/OS V2.3, obtains security information in two ways:
  • Limited observation of an application connection's data stream. Stream observation obtains basic cryptographic information that’s exchanged during an initial TLS, SSL or SSH handshake, and only applies to TCP connections.
  • Direct notification from z/OS zERT-enabled cryptographic protocol providers. These are z/OS IPSec, System SSL and z/OS OpenSSH. Observation yields less comprehensive information than what’s provided by cryptographic protocol providers, but it does provide coverage for security sessions that use providers that aren’t zERT-enabled such as the JSSE or OpenSSL.
2. zERT Aggregation. This provides summary information for security sessions terminating on the local stack. zERT Aggregation uses the information accumulated by zERT Discovery as a basis for the summary records. Since multiple application connections can use the same security session over time, zERT Aggregation typically results in the creation of significantly fewer SMF records than zERT Discovery, while still providing significant cryptographic detail. zERT Aggregation is available on z/OS V2.3 with APAR PI83362.

Both zERT Discovery and zERT Aggregation record collected information as SMF type 119 records. zERT Discovery information is recorded as zERT connection detail (subtype 11) records. These records are written when application connections start, end or have a change in cryptographic coverage. A zERT connection detail record contains information about the zero or more security sessions currently protecting an application connection.

zERT Aggregation information is recorded as zERT summary (subtype 12) records. These records are written at user-defined intervals. Each zERT summary record represents a single security session, and includes both significant security characteristics and usage statistics for the security session.

Figure 2 shows the basic layout of the zERT subtype 11 and subtype 12 records.

Figure 2: SMF Type 119 Subtype 11 and 12 Record Formats

Each SMF record provides information about the connection endpoints as well as appropriate information about the local z/OS socket owner, such as user ID and job name. For each protocol included in the SMF record, a protocol specific section is included that contains important cryptographic attributes of the security session being recorded.

For instance, the protocol specific section can include values identifying the security session’s authentication method, the cryptographic algorithms used for encryption, message authentication, key exchange and the key lengths used for each algorithm (where appropriate). However, it’s important to understand that zERT doesn’t store or record the values of secret keys, initialization vectors or any other secret values that are negotiated or derived during cryptographic protocol handshakes. For complete details on the SMF 119 zERT connection detail record formats, see the zERT connection detail record page within the IBM Knowledge Center.

Enabling zERT

Enabling the zERT functions is simple. The zERT Discovery function is enabled by specifying the GLOBALCONFIG ZERT parameter in the TCP/IP profile data set, while the zERT Aggregation function is enabled by coding GLOBALCONFIG ZERT AGGREGATION. Independent SMFCONFIG and NETMONITOR controls in the TCP/IP profile data set are available to direct zERT to write the SMF records to the z/OS SMF, to a network management application by using the real-time Network Management Interface for zERT information, or both. Figure 3 summarizes the configuration tasks required to use zERT.


TCP/IP Profile statement


Enable zERT Discovery function


Enables zERT monitoring.

Enable zERT Aggregation function


To enable the aggregation function, the discovery function must also be enabled.

Enable recording of SMF 119 subtype 11 records (only if you want to record these records)


The discovery function must be enabled before SMF 119 subtype 11 records are written.

Enable recording of SMF 119 subtype 12 records (only if you want to record these records)


The aggregation function must be enabled before SMF 119 subtype 12 records are written.

Enable collection of SMF 119 subtype 11 records via SYSTCPER service (only if you have an application that uses this real-time NMI)


The discovery function must be enabled before SMF 119 subtype 11 records are generated.

Enable collection of SMF 119 subtype 12 records via SYSTCPES service (only if you have an application that uses this real-time NMI)


The aggregation function must be enabled before SMF 119 subtype 12 records are generated.

Figure 3: Summary of zERT-Related Configuration Tasks

Since zERT Aggregation generally produces far fewer SMF records than zERT Discovery, we suggest recording the zERT summary records (SMFCONFIG TYPE119 ZERTSUMMARY) on a regular basis. Recording zERT Detail records is only suggested when per-connection details are required to investigate a specific scenario.

Coming Soon

IBM zERT Network Analyzer, a new web-based interface that will be available in the future, will give z/OS network security administrators the ability to query and analyze the data recorded in SMF 119 subtype 12 “zERT summary” records. zERT Network Analyzer is a z/OS Management Facility (z/OSMF) plug-in that administrators can use to determine which z/OS TCP and Enterprise Extender traffic is or isn’t protected according to the specific query criteria. See the statement of general direction in the May 15, 2018 IBM z/OS V2R3 enhancements announcement letter for more information.

More Information

·       For more detailed information on configuring zERT, see z/OS Communications Server: IP Configuration Reference.

·       For more information on the real-time zERT NMI service, see Network Management Interfaces

As you plan to use the zERT functions, it’s important to realize that while zERT Discovery provides the most thorough level of cryptographic coverage information, the number of zERT connection detail records can be large. This depends on the number of connections that are supported by your z/OS system and the frequency with which those connections are created, deleted or changed from a cryptographic coverage perspective. Using the zERT Aggregation function typically reduces the number of SMF records generated, but provides less information at an individual application connection level. This combination of functions might be what you need to provide your manager with an in-depth analysis of your network cryptographic coverage.

Chris Meyer, CISSP, is a senior software engineer in Research Triangle Park, N.C. He is the z/OS Communications Server security architect and has been developing IBM operating systems, file systems and security-related software products for 35 years.

Dave Wierbowski is a senior software engineer in Endicott, N.Y. A member of the z/OS Communications Server development team, his area of expertise is IPSec and he currently serves as the z/OS Encryption Readiness Technology technical lead.

Michael Gierlach is an advisory software engineer in z/OS Communications Server in Research Triangle Park, N.C. Mike has worked on VTAM, TCP/IP and z/OS Communications Server for 25 years, most recently on z/OS Encryption Readiness Technology.