This is the first in a short series of blogs about MQ for z/OS and the types of data that can be collected using System Management Facilities (SMF). It will discuss why you might want to collect the data, how to enable data collection and which SMF destination you could choose.
Subsequent blogs will discuss:
- What data to collect, as well as the implications to cost and data volumes of collecting that data.
- How to use data that has been collected, for example tooling that is available.
- What’s new in V924 relating to SMF data collection.
These blogs will not discuss the interpretation of the data being collected by MQ, but there are a set of blogs that do walk through the process of understanding the data, and these can be found:
- Understanding the costs reported by MQ accounting
- Understanding the costs reported by MQ accounting – part 2
- Determining the costs of workloads that move messages
Why are we talking about MQ and SMF?
In an ideal configuration, your MQ resources will be able to handle the workload that is required of them. MQ provides job log messages to advise you when there is a problem, but of course there will be times that this does not give sufficient warning to prevent impact to the workload, such as when there are spikes in workloads or system outages.
As such, monitoring your MQ resources is advisable, even when MQ is behaving as required, as this allows you to build an understanding of good behaviour as well as notice when bad behaviour starts to occur.
The “Introduction to monitoring IBM MQ for z/OS” section provides an overview of the monitoring facilities available to MQ for z/OS.
This series of blogs is focused on the use of SMF with performance statistics and accounting trace, rather than on-line monitoring, event messages and getting snapshots of IBM MQ using MQSC DISPLAY commands.
What is SMF?
System Management Facilities (SMF) collects and records system and job-related information that your installation can use for many purposes, including reporting reliability, evaluating data set activity and analysing your configuration.
The following diagram illustrates the functional overview of SMF, with an explanation of each item in the table below.
|
#
|
Description of process
|
|
1
|
Routines (SMF, system, program-product including MQ, installation-written) collect and format data into records and pass the records to the SMF writer.
|
|
2
|
SMF routines copy records to SMF buffers and transfer records from the SMF buffers to either:
- SMF data sets
- SMF logstreams
|
|
3
|
If you write your SMF data to SMF data sets, the data sets are filled one at a time; while SMF writes records on one data set, other SMF data sets can be dumped or cleared.
If you write your SMF data to SMF logstreams, you can keep writing data to your logstream. The logstream will offload to DASD data sets when the logstream coupling facility structure (or local storage buffers for a DASD-only logstream) fills. You define the thresholds for how much data is held before offload in the logstream definition.
|
|
4
|
The SMF dump programs copy data from either SMF data sets or logstreams to tape or DASD for permanent storage. You can also use the SMF dump programs to dump to a temporary data set for immediate use.
|
|
5
|
Analysis and report routines, either user-written or those such as MP1B or Tivoli Decision Support for z/OS, process information records.
|
Note: The SMF documentation “SMF Introduction” contains the original diagram with a more detailed explanation of the SMF processes and with additional references.
Why use SMF?
As mentioned earlier, the data this blog pertains to is performance statistics and accounting data trace.
By default MQ for z/OS statistics and accounting traces use SMF as the destination, although this can be overridden – see the MQSC “START TRACE” command documentation for alternative destinations.
Which SMF destination-type should I use?
Both SMF data sets and SMF logstreams are appropriate for the collection of MQ performance statistics and accounting trace, and it may simply be a decision based on what is already in use at your installation.
During MQ performance testing, we found advantages to the use of logstreams and these are detailed in the performance report MP16 under section “IBM MQ and zEnterprise Data Compression (zEDC) with SMF”. Whilst measuring the performance benefits of zEDC with logstreams, we found that logstreams generally were able to collect data at a significantly higher rate than SMF data sets, meaning that in environments with high data collection rates, data was not being lost.
Additionally, separate logstreams can be configured for specific SMF types such that MQ SMF types 115 (statistics) and 116 (accounting) can be isolated from other SMF types, and indeed from each other if so desired.
Since z/OS v2r2, a third type of SMF destination has been made available – which allows real-time streaming using in-memory logstreams. These can be used in parallel with logstreams. In-memory logstreams will be discussed in detail in a later blog.
How do I enable MQ to write to SMF?
Statistics and accounting trace are two separate entities – and do not need to be collected together. Statistics trace relates to how the queue manager is working, for example buffer pool usage, whereas accounting relates more to the interactions between applications and the queue manager, for example whether an application has put or got a message.
Statistics trace data is written as SMF type 115 records.
Accounting trace data is written as SMF type 116 records.
Prior to MQ for z/OS V924, the frequency of collection of both statistics and accounting records is controlled by the CSQ6SYSP STATIME attribute. By default, this is set to 30 minutes but the supported range is 0 (uses the default SMF collection broadcast interval) to 1440 minutes. There are some nuances to this, particularly with both class 1 and 3 accounting data which perform writes at the end of task (disconnect) – but this will be discussed in more detail in the second blog in this series.
The STATIME attribute can also be set using the SET SYSTEM command, or the PCF SET SYSTEM command.
As statistics and accounting trace generate different types of SMF records, the traces need to be enabled separately.
Enabling at queue manager start:
To enable SMF data collection from the start of the queue manager, you can use the CSQ6SYSP SMFSTAT and SMFACCT attributes. To enable the appropriate data collection, the attribute needs to be changed from the default value of NO.
See https://www.ibm.com/docs/en/ibm-mq/9.2?topic=module-using-csq6sysp
For SMFSTAT, selecting a value of YES will enable accounting data to be collected for class 1 only. To request additional classes, a range of integers can be specified, e.g.
SMFSTAT=(1,3,4), X
Note: For statistics classes 2 and 3, class 1 must also be specified.
For SMFACCT, selecting a value of YES will enable accounting data to be collected for class 1 only. To request additional classes, a range of integers can be specified, e.g.
SMFACCT=(1,2,3), X
Enabling on a running queue manager:
To enable SMF data collection against a queue manager that is already running, it is necessary to start the appropriate trace option, for example:
START TRACE(STAT) CLASS(01) DEST(SMF)
START TRACE(ACCTG) CLASS(03) DEST(SMF)
The START TRACE command documentation shows the available options.
Note:
When using the START TRACE command for statistics and accounting traces, specifying CLASS(*) will only start traces for classes 1 to 3. Channel initiator statistics and channel accounting data are not started with CLASS(*) and must be started with CLASS(4).
In summary
Hopefully this blog has helped to understand why you might choose to enable statistics and accounting trace in your MQ queue manager and if so, the subsequent blog should help determine the impact of enabling trace options.
Enabling statistics and accounting trace does have a cost but statistics trace in particular has a small cost and the benefit is that you are able to identify when your systems are not running as well. Accounting trace classes 1 and 3 can be relatively expensive for certain workloads and this is discussed in more detail in the subsequent blogs in this series.