zPET - IBM Z and z/OS Platform Evaluation and Test

zPET - IBM Z and z/OS Platform Evaluation and Test

zPET - IBM Z and z/OS Platform Evaluation and Test

Experiences and tips from a team of system programmers and testers who run a Parallel Sysplex on which we perform the final verification of a z/OS release and System z hardware and System Storage before they become generally available to clients.

 View Only

Modernizing Z Workload Scheduler Automation with Python and Ansible

By Justin Largo posted 22 days ago

  

z/OS Platform Evaluation and Test (zPET) runs customer-like workloads in a Parallel Sysplex environment to perform the final verification of new IBM Z hardware and software.

Summary

This post explores how we transformed a carefully structured, manual Z Workload Scheduler (zWS) process into a streamlined, automated workflow using Python, Ansible, and IBM Z Open Automation Utilities (ZOAU). By integrating modern DevOps tools with mainframe operations, we significantly reduced manual intervention and improved test efficiency.

Audience

The expected audience of this blog post are z/OS system programmers, DevOps engineers working with mainframes, and Automation architects.

Helpful Terms & Definitions

  • WAPL = Workload Automation Programming Language. Used in Z Workload Scheduler to automate functions and access the Programming Interface (PIF).
  • Occurrence = An application that can be found in a current plan. Referred to as CPOC in zWS WAPL definitions.
  • Operation = A job that is found in an occurrence in a current plan. Can be identified by its job name or id in JES as well as an operation number. Referred to as CPOPCOM in zWS WAPL definitions.
  • SDSF = System Display and Search Facility. This is a helpful utility in z/OS for monitoring, controlling and viewing output of jobs in a system.
  • zWS = Z Workload Scheduler. Formerly known as Tivoli Workload Scheduler (TWS) or Operations Planning and Control (OPC).
  • ZOAU = IBM® Z Open Automation Utilities
  • PIF = Programming Interface. Used in zWS to make API requests to the product.

Introduction

As a new member of zPET, I’ve been assisting with enhancing our weekly automations. One pain point was the need to wait several hours on Tuesdays for a scheduled application— referred to as an occurrence in Z Workload Scheduler (zWS) terms. Then, once that occurrence and its operations have finally completed, we then must go through a multi-step evaluation process that includes carefully checking individual operations and resubmitting or completing them, running a CF structure REALLOCATE, and then submitting another job to complete the test scenario. Our testers understandably found this process quite time-consuming and undermines our goal of fully automated weekly tests, as someone would need to resume the test later in the day. This prompted us to explore zWS documentation in depth to identify better automation strategies and save our team hours of manual work each week.

Problem Statement

Given our Tuesday automation scenario above, we needed an easy way to find information in zWS that included the following:

  • List and filter application data from the database
  • List and filter occurrences from the current plan
  • List and filter operations inside an occurrence in the current plan
  • Resubmit operations
  • Mark operations with a new status

Although we have experience using REXX scripts to extract data from PIF, we didn't have a clearly defined path to give us a way to expand upon it for all the use cases we have planned for it.   Additionally, retrieving data programmatically from zWS is something our team has not fully explored.

Solution

After conducting dedicated research about all things zWS, I found that the path to success would come from accomplishing the following steps:

  1. Install EQQJOBs to access the required batch-job skeletons and WAPL samples for EQQYJPX to work properly.
    1. Ensure that the PTF for APAR PI79321 was applied so that we can use the new and improved procedures for invoking Workload Automation Programming Language (WAPL) through batch.
  2. Write the required JCL used to invoke EQQYXJPX with customized WAPL snippets to find and modify data in zWS.
  3. Write a series of Python methods that invoke ZOAU to submit the above JCL.
  4. Execute the automations via an Ansible playbook.
    1. While ZOAU provides the necessary automation capabilities on the mainframe, it doesn't handle remote execution. That’s where Ansible comes in—it allows us to remotely orchestrate and execute those ZOAU-based scripts within USS.

These steps will allow us to achieve our goals and help our teams become more productive. What follows are some simple examples of how we implemented these steps to show how easy it is to do. So, without further ado, let’s get into the details.

Step 1: Install EQQJOBS to Access Required Libraries

We installed the required batch job skeletons and WAPL samples with the EQQJOBS installation aid. Note: these instructions assume that a zWS Controller is configured, and the PTF for APAR PI79321 is applied.

After logging on to z/OS, navigate to ISPF and execute the command TSO EQQJOBS. This will bring up a panel that looks like the following:

From there, enter option 2 for generating batch job skeletons and option 4 for the WAPL samples. For reference in this tutorial, we can assume that the batch job skeletons were installed to LARGO.ZWS and the WAPL samples were installed to LARGO.ZWS.EQQJOBS. After this I did have to make two manual changes in order to have the batch execution of WAPL work.

  1. The jobs must run on the zWS Controller LPAR. This means that jobs or scripts need to target <LPAR NAME> and subsystem <zWS Controller Name> in the SUB parameter of EQQYXJPX. This can be done easily in the JCL header with SYSAFF=<LPAR NAME>.
  2. The generated EQQYXJPX procedure was modified to remove the STEPLIB DD statement in the JCL. We needed to do this because of some caveats:
    1. STEPLIBs are not needed for modules in the LNKLST since they are searched by default.
    2. It’s APF authorized if it’s used via LNKLST.
    3. If a library is APF authorized, it’s authorized by default as a LNKLST dataset, when LNKAUTH is specified.
    4. The library loses APF authorization if it has a STEPLIB.

Refer to https://www.ibm.com/docs/en/zos/3.1.0?topic=functions-apf-authorized-libraries for more information on APF-authorized libraries.

At this point, we can begin writing JCL and executing WAPL snippets!

Step 2: Automating JCL Execution with WAPL

Previously, for our team to retrieve information from zWS, we had a REXX script that essentially accessed the PIF directly as if it was an assembler program accessing each segment in memory based on an offset. Now with WAPL, we can access this information in about 2-3 lines of code inside a generic job. Each job requires the following information:

  1. The SYSAFF parameter must be specified in the JCL header. This will ensure we execute this job on the proper zWS controller image.
  2. Specify the JCLLIB that contains the WAPL snippets; as mentioned here for our examples, I installed it in LARGO.ZWS.EQQJOBS.
  3. The RUNWAPL EXEC, which is how we call the EQQYXJPX procedure, will in turn run the WAPL code. It can contain a few different parameters such as ARGS and SUBSYS which specify how the data is to be returned and the name of the zWS Controller Subsystem, respectively. More information about this can be found here.
  4. A LOADDEF statement to know what segment of data we would like returned and where to save that data. In the examples below, that can range from ADCOM (application database entries), CPOC (occurrences in current plan), and CPOPCOM (operations in current plan). Next, I specify MYDATA to be where I want my results to be returned. More information about the OUTPUT data can be found here.
  5. Lastly, SYSIN contains our WAPL code.

Below, you can see different ways to access and modify different data from zWS.

Step 2a: Finding applications in the zWS Database

Below, the WAPL snippet is going into the application database to find all valid applications that contain the name JUSTIN. More details can be found here.

//FINDAPP  JOB MSGLEVEL=(1,1),MSGCLASS=H,CLASS=A,

//             TIME=1440,REGION=0M,MEMLIMIT=3G,SYSAFF=<CONTROLLER LPAR NAME>

//**********************************************************************

// JCLLIB ORDER=(LARGO.ZWS.EQQJOBS)

//RUNWAPL  EXEC EQQYXJPX,

//         ARGS='FIELDSEP(,) SHOWDFLT(N) STRIP(Y)  SELECT(Y)',

//         SUBSYS=<ZWS CONTROLLER NAME>

//MYDATA   DD SYSOUT=*,LRECL=2048

//OUTBL    DD SYSOUT=*

//SYSIN    DD *

LOADDEF ADCOM DATA(MYDATA)

LIST ADCOM ADID(JUSTIN) VALID(=)

This provides us with the following CSV output given our arguments:
ADCOM,ADID=JUSTIN,ADSTAT=A,ADTYPE=A,ADFROM=250306,ADMONITOR=N,ADTO=111,ADDESC=Justin's Test App,ADGROUP=,ADOWNER=SETUP,ADODESC=,ADPRIOR=3,ADCAL=,ADLDATE=250429,ADLTIME=1552,ADLUSER=LARGO,…

Now anyone can access data in an easy known format, whether it’s data analysis with pandas or cherry-picking fields like ADID to feed into automation like our Tuesday scenario.

Step 2b: Finding occurrences in the current plan

Similarly, we can find occurrences (applications) in the current plan by only changing the data retrieved from the CPOC block. The application id referred to as ADID stays the same. Another interesting feature here is that we can use wildcard operations when searching for data in WAPL. We can also add filters based on the input arrival (IA) by placing that in the LIST command. More information and options can be found here.

//FINDCO  JOB MSGLEVEL=(1,1),MSGCLASS=H,CLASS=A,

//             TIME=1440,REGION=0M,MEMLIMIT=3G,SYSAFF=<LPAR NAME>

//**********************************************************************

// JCLLIB ORDER=(LARGO.ZWS.EQQJOBS)

//RUNWAPL  EXEC EQQYXJPX,

//         ARGS='FIELDSEP(,) SHOWDFLT(Y) STRIP(Y)  SELECT(Y)',

//         SUBSYS=<SUB NAME>

//MYDATA   DD SYSOUT=*,LRECL=2048

//OUTBL    DD SYSOUT=*

//SYSIN    DD *

LOADDEF CPOC DATA(MYDATA)

LIST CPOC ADID(RRS*)

Step 2c: Finding operations in zWS

Again, you’ll start to notice a pattern of how simple it is to find data. The snippet below finds operations inside an occurrence. Here you can see that I’m looking through all the operations in all RRS* occurrences that have a status of E, or error. This is where WAPL becomes pretty powerful so that we can find exactly the data we need for modification operations. More information can be found here about listing CPOPCOM.

//FINDCP  JOB MSGLEVEL=(1,1),MSGCLASS=H,CLASS=A,

//             TIME=1440,REGION=0M,MEMLIMIT=3G,SYSAFF=<LPAR NAME>

//**********************************************************************

// JCLLIB ORDER=(LARGO.ZWS.EQQJOBS)

//RUNWAPL  EXEC EQQYXJPX,

//         ARGS='FIELDSEP(,) SHOWDFLT(Y) STRIP(Y)  SELECT(Y)',

//         SUBSYS=<SYS NAME>

//MYDATA   DD SYSOUT=*,LRECL=2048

//OUTBL    DD SYSOUT=*

//SYSIN    DD *

LOADDEF CPOPCOM DATA(MYDATA)

LIST CPOPCOM ADID(RRS*) STATUS(E)

Step 2d: Making modifications to items in zWS

Now, you might be wondering that this is great that we can find all that information, but how do I actually automate actions? Well, in two easy lines we can do that. Rather than a LIST command, we use MODIFY. If you’re curious, WAPL provides many different options to operate on your zWS data which can be found here. So, the biggest takeaway with modifications on CPOC or CPOP, is that it requires more specifications. First, we need to find the correct app/occurrence which requires an application id known as ADID as well as an IA or input arrival to know which application to modify since there could be duplicates. Then, we specify the operation or CPOP, which just requires the OPNO and then how I would like to modify that operation which in this was marking it as complete.

//RERUN  JOB MSGLEVEL=(1,1),MSGCLASS=H,CLASS=A,

//             TIME=1440,REGION=0M,MEMLIMIT=3G,SYSAFF=<LPAR NAME>

//**********************************************************************

// JCLLIB ORDER=(LARGO.ZWS.EQQJOBS)

//RUNWAPL  EXEC EQQYXJPX,

//         ARGS='FIELDSEP(,) SHOWDFLT(N) STRIP(Y)  SELECT(Y)',

//         SUBSYS=<SYS NAME>

//MYDATA   DD SYSOUT=*,LRECL=1024

//OUTBL    DD SYSOUT=*

//SYSIN    DD *

MODIFY CPOC ADID(JUSTIN) IA(2504291317)

MODIFY CPOP OPNO(3) STATUS(C)

Step 3: Leveraging ZOAU to Execute JCL

Now, the above is great by itself! Being able to execute JCL to perform these automations can open the door to many different possibilities. To expand automation capabilities further, we started bringing Python and ZOAU into the picture. Python can be easily leveraged to create classes based on the data definitions in CPOC, CPOPCOM, etc. as well as easily create WAPL templates to allow for our automations to be extensible. The simplest way to get Python to automate WAPL would be to do the following:

  1. Create a function to generate WAPL snippets
  2. Create a function to write JCL to a file
  3. Create a function to submit the JCL with ZOAU

WAPL templates can be generated using template strings like with the below code snippet that finds an application given an app name. Note that this app name can also include wildcard operators, so no additional logic is needed for that.

def find_app_template(app_name: str) -> str:

    """

    Returns the wapl template to find an app for the given app_name.

    Parameters:

    app_name (str): The name of the application.

    Returns:

    str: The application template.

    """

    return f"""LOADDEF ADCOM DATA(MYDATA)

LIST ADCOM ADID({app_name}) VALID(=)"""

Now that we have our WAPL being programmatically generated, we need to create another function that creates the JCL. This can be pretty complex, so we can keep it simple and just template in the WAPL with a bunch of default JCL values.

def find_app(app_name:str):

    # Create JCL template

    wapl_template = find_app_template(app_name)

    jcl = create_wapl_jcl(

        wapl_template=wapl_template,

        jcl_lib_dsn="LARGO.ZWS.EQQJOBS",

        sysaff="MYLPAR",

        controller="MYCONTROLLER",

    )

    # Write jcl to file

    jcl_path = "wapl.jcl"

    with open(jcl_path, "w") as file:

        file.write(jcl)

    # Submit WAPL Job

    from zoautil_py import jobs

    jcl_file = "/path/to/wapl.jcl"

    job_id = jobs.submit(jcl_file, is_unix=True)

    # Retrieve output

    job_dds: list[dict] = jobs.list_dds(job_id=job_id)

    raw_adcom_data: list[str] = job_dds["MYDATA"].split("\n")

    return raw_adcom_data

Lastly, we can use ZOAU to submit the JCL with them Jobs Python API. Assuming we are writing our JCL file to USS, we can submit it with the following code snippet from ZOAU:

from zoautil_py import jobs

jcl_file="/path/to/wapl.jcl"

job_id = jobs.submit(jcl_file, is_unix=True)

Putting it all together, we have Python function that finds an app that looks like this:

def find_app(app_name:str):

    # Create JCL template

    wapl_template = find_app_template(app_name)

    jcl = create_wapl_jcl(

        wapl_template=wapl_template,

        jcl_lib_dsn="LARGO.ZWS.EQQJOBS",

        sysaff="MYLPAR",

        controller="MYCONTROLLER",

    )

    # Write jcl to file

    jcl_path = "wapl.jcl"

    with open(jcl_path, "w") as file:

        file.write(jcl)

    # Submit WAPL Job

    from zoautil_py import jobs

    jcl_file = "/path/to/wapl.jcl"

    job_id = jobs.submit(jcl_file, is_unix=True)

    # Retrieve output

    job_dds: list[dict] = jobs.list_dds(job_id=job_id)

    raw_adcom_data: list[str] = job_dds["MYDATA"].split("\n")

    return raw_adcom_data

Step 4: Wrapping it up with Ansible

With our modular Python function in place, we can now create an Ansible playbook to execute this code remotely so that we can find apps from whatever system we’re currently on rather than having to login into USS or a 3270 terminal. Assuming we have Ansible set up correctly, we can create and run the following playbook:

- name: Run the WAPL automation script

  hosts: mysystem.ibm.com

  environment: "{{ environment_vars }}"

  tasks:

    - name: Copy scripts

      ansible.builtin.copy:

        src: ../python_wapl_snippets/

        dest: /tmp/python_wapl_snippets

        mode: '0755'

    - name: Run script

      shell: "python3 /tmp/python_wapl_snippets/main.py"

      register: result

    - ansible.builtin.debug:

            msg: "{{ result.stdout }}"

   

We have successfully written WAPL, wrapped it in a user-friendly Python script, and automated it with Ansible!

Conclusion

While this post walked through a relatively simple example, the techniques and tools demonstrated—Python, Ansible, ZOAU, and WAPL—unlock the potential for far more sophisticated automation across your zWS environment. From dynamically retrieving application and operation data to programmatically resubmitting jobs and updating statuses, this approach lays the groundwork for scalable, maintainable mainframe automation.

By integrating this solution into our weekly automation pipeline, we’ve eliminated an average of 4 hours of manual effort per week—time previously spent checking statuses and resubmitting operations. That is 200 hours saved annually, freeing up our team to focus on higher-value tasks and accelerating our testing cycles.

What’s Next?

If you’re looking to modernize your z/OS operations, this is a great place to start. Try adapting the examples to your own environment, explore more WAPL capabilities, and consider how Python and Ansible can help you scale automation across your enterprise.

Ready to dive deeper?

  • Explore the IBM Z Open Automation Utilities documentation
  • Check out the zWS Knowledge Center
  • Or start building your own automation playbooks today!

References

0 comments
21 views

Permalink