On mainframes, application programmers and system programmers operate in different circles. They have really different jobs. However, they do collaborate with each other, especially when system programmers launch a new middleware release or when application programmers are deploying a major new application. Testing is a place where they often unite.
In this article, the everyday practices of software testing in enterprise systems are surveyed for both system and application software with additional focus on new and innovative tools and practices.
Role of System Programmers
In mainframe IT organization, the system programmer has a critical role. He or she installs, tailors and maintains z/OS and other OSs as required
. Notably, they also install or upgrade products that run on the system like CICS
. The work is varied. Sometimes it’s upgrading the latest version of the OS or it could be something simpler like upgrading a single product, such as Enterprise COBOL
The toughest job, requiring great care and attention to detail, is upgrading the OS as this has the potential to impact every batch job, online transaction and started task in one or more systems.
Test Activities Focus on Systems
As soon as you discuss testing, you are exposed to language used by system programmers like IPL
. System programmers will IPL a new release in a production environment after they have tested it with a simulated production workload in a test LPAR. One of the challenges for system programmers is that the workload needs to include all applications and software products to ensure that service-level agreements can be met. After initial testing, system programmers usually IPL in a shared OS environment after having installed any PTFs needed for coexistence
System programmers perform a significant level of function and stress testing because testing by IBM on z/OS and middleware doesn’t replace the need for testing in their own environment. System programmers often start with basic function testing, including:
• Initialize the system
• Initialize JES2 or JES3
• Log on to TSO/E
• Run the product installation verification procedures that are typically found in the program directory
• Submit a job
• Check the job's output
A typical next step is customization of z/OS, for example, to use automation software since most z/OS systems today have automated startup, shutdown and recovery support provides by special products like IBM Automation Control for z/OS
Next, if CICS or IMS is installed, system programmers initialize a region and sign on to a terminal. Then, they might start bringing ISV
products into the test environment. As the administration and testing continues, systems programmers often ask application programmers for help running critical production jobs and testing support for a representative interactive workload. A more complete set of tasks
performed by system programmers are documented in the z/OS Planning and Installation manual. One system programmer doesn’t do all these activities. And, there is a degree of product specialization where different system programmers support the OS versus the transaction processing systems and the database software.
Application Programmers Design, Build and Test
Application designers and application programmers’ design, build, test and deliver mainframe applications for the company's end users. Based on requirements gathered from business analysts, the application designer creates a design specification that the programmer uses to construct application programs. In addition to creating new application code, programmers are responsible for maintaining and enhancing the company's existing mainframe applications. Today, this is often the main job for many mainframe application programmers. The New York Times reported
that “More than 200 billion lines of COBOL code are now in use and an estimated 2 billion lines are added or changed each year, according to IBM Research.”
Somebody needs to support the existing programs. While mainframe installations still create new programs with COBOL or PL/I, languages such as Java have become popular for building new applications on the mainframe. There are various estimates about the number of Java programmers
in the world but the number is probably more that 9 million active programmers.
Test Activities on Applications
During the application development process, the designer and programmer typically interact with other IT people in the organization. For example, a programmer might work on a team with other programmers who are building interface modules for related application programs. When development of an interface module is completed, this module would be passed through a testing process including function, integration and systemwide tests. Following these tests, the application programs must be acceptance tested by the user community to determine whether the code actually satisfies the original user requirement.
The difference between systemwide tests and the testing done during the development phase is that programmers and users are now testing the application as a whole, as well as in conjunction with other applications. Being performed is:
• User testing the application for functionality and usability
• Integration testing so the new application is tested together with other applications to see if they interface as expected
• Performance or stress testing using real production data or real production data volume to see how well the application performs when there’s high demand
Issues coming out of these tests need to be addressed before going into production. Sometimes, fixing problems can take more resourcefulness than the initial effort of creating the necessary parts of the application.
What Recent Innovations?
In general, automation now plays an important role in testing. Its importance has been growing gradually over the years. System programmers often take testing activities and collect them into job streams or commands issued in REXX programs from TSO. This approach saves time, reduces human error and offers the opportunity to build a repository of tests that can be run over again as needed. Application programmers are now using tools like Rational Quality Manager
using automation to reduce labor-intensive activities to accelerate project schedules.
Many testing tools have been developed for both system and application developers.
An example for system programmers is IBM z Systems Development and Test Environment.
It’s a platform for mainframe application demonstration, development, testing and education. It supports IBM z/OS middleware and other z/OS software to run on Intel-compatible platforms without IBM Z mainframe hardware. It’s based on the IBM System z Personal Development Tool (zPDT), which provides an emulated IBM Z architecture with virtual I/O and devices.
An example for application developers is IBM Developer for z Systems Enterprise Edition
that is a comprehensive solution for cost-effectively creating and maintaining z/OS applications. A useful set of COBOL, PL/I, C/C++, High Level Assembler, JCL and Java development tools and optimized tooling for batch, CICS, IMS and Db2 runtimes provide z/OS application developers a modern development environment that enhances their productivity. This solution consists of two modular offerings: IBM Developer for z Systems and IBM Debug for z Systems.
A Discrete Profession
It’s no wonder that testing has developed into a discrete profession. This is reflected on the popular job-finding sites. Indeed.com recently listed 161 jobs for a mainframe tester. Glassdoor.com recently listed 74 mainframe jobs. Salaries range from $40 to $100K. Titles are diverse—Mainframe Tester, Manual Tester, Automation Tester, Big Data Tester and Business Analyst/Quality Assurance Tester are examples. The Career Path of A Software Tester
infographic is useful in understanding the potential steps in the profession of a tester.