Fans of IBM Z Hub - Group home

My tour of the IBM Poughkeepsie system Z test lab

By Enzo Damato posted Wed August 10, 2022 10:20 AM


Recently, I obtained a z114 mainframe, and over the last two months, I have been busy getting the machine set up and configured. So far, I have gotten my machine configured, and loaded with and IOCDS that describes my hardware configuration. I have also received installation media for z/VM from IBM, which I have gotten installed and set up. I have also received media for the installation of VSEn from 21st century software. Later this August, I am going to be presenting my experiences at SHARE. Subsequent to this, Shari Chiara from IBM reached out to me to offer me a tour of IBM’s Poughkeepsie system Z test lab. I happily accepted this invitation, and I traveled out to the lab on August 2nd.

When I first arrived on site, I met up with my tour guides for the day: PJ Catalano and Paul Novak. Paul Novak has been my point person at IBM for getting the software and services that I have needed to get my z114 up and running, and PJ is one of the heads of the test lab. We then went inside, and into the raised floor computer room. Unlike some other datacenters, the IBM test lab had been set up with a 36 inch raised floor. This allows power and data cabling for the systems to be run under the floor, along with air conditioning for all of the systems. As we went up onto the raised floor, and into the test center, the first thing that I saw was a disassembled z16 CPC drawer. Inside, I was shown the methods used to assemble a complete processor system. Perhaps the most noticeable difference was the presence of liquid cooling, and the apparent absence of RAM. Unlike other servers, the larger processors (around twice the size of an AMD thread ripper) used in the z16 require liquid cooling, and more efficient thermal conduction material to operate at their fullest efficiency. Unlike commodity computers which use thermal paste, the z16’s CPUs are set up with highly conductive metal pads. Also of note was the custom RAM modules in use. Since the z16 uses its own memory controller, it also needs its own custom memory modules, which gives a more dense capacity, and allows the CPU to arrange the memory in a redundant configuration to minimize failures. Also in this part of the lab, was the “Mini-z16” This system, which consisted of an I/O cage, CPC and a support element, is the smallest possible configuration of a z16, and is used whenever a system needs to be shipped around the country for testing at a remote facility. Specifically, this system was sent to a facility that simulates cosmic rays.

After this, we went on to the patch, fiber, and network lab. In this section of the lab, we got to see many cool technologies. First, we saw DWDM modules, which are used to cram multiple fiber optic signals into one cable by splitting and un-splitting the wavelengths. This tech was being used in conjunction with high bandwidth Fiber channel, ficon, and Ethernet switches. These units, used in conjunction with a long-distance fiber optic loop, are used to test and validate any combination or configuration of fiber networking that may be encountered in the field. This is a fascinating look at how large enterprises are able to network everything from storage to compute over nearly infinite distances. We then proceeded into the central patch room in the middle of the datacenter. Here, we saw the termination point for all of the fibre optics in the entire data center. Instead of connecting each device directly together, or directly to the switch, all devices are instead connected to huge banks of patch panels, so that whenever a configuration change needs to occur, or a new system is added to lab, the jumper connections in the patch room can be changed, without having to worry about pulling new cable through the entire lab. This is one idea that I will definitely be stealing for my setup.

Next, we went on to see the mainframe peripheral section of the lab. Here, there were cabinets upon cabinets of DASD and tape units. Here, there was everything from All-flash DASD to a 4-frame tape library. All of this storage is uplinked to the central patch room from before, and then shared out among all of the Z systems in the lab. This configuration takes full advantage of the switched features of FICON, so that all of the z systems in the lab can have full redundant access to all of the peripherals present. Although storage units are typically not much to look at, I did enjoy watching the robot in the tape library shoot back and forth loading, grabbing and releasing tapes. It is quite visually interesting. Also in this part of the lab, there is a throwback to the 80s. Stashed off to the side in a closet was an old ESCON analyzer, built off of an original IBM PC. Although this device is almost completely useless now, it was a fun Easter egg on the tour.

After the storage section, we headed on to the real meat and potatoes of the tour. In the compute section of the lab, there was every version and configuration of system z imaginable. From a 4-frame, maximally configured unit, to a base model set up as a coupling facility, this was the part of the lab where the real magic happened. Here, the Z test engineers created and ran every form of test possible on the system. From sysplex failover testing to IO card testing, the systems here are put through their paces to ensure that there will be no unexpected failures or weirdness by the time the devices reach the clients. This is where we also got to see the z16 fully assembled and kitted out with its display covers. I was quite impressed with the new and updated system design, as compared to the older z114 that I have at home. Specifically, I think that the change to a standard 19 inch rack, rear-facing-only IO, and side-rack PDUs instead of the older bulk power adapters are a major improvement. I was also quite impressed with the density of the PCIe IO cards in the updated cages.

We finally ended the tour at the lab’s operation center. Here, I got to talk with some of the developers and testers, who showed me the machine’s system consoles and interfaces. I also got an opportunity to talk with Eric Reid, the lab’s z/tpf expert. He gave me an amazing demo of TPF on one of the lab’s mainframes. He showed me how the machines were able to process a truly amazing number of transactions and IO operations, all without instabilities or crashes. He also gave me an overview of what would be involved with getting TPF installed on my machine. I am hoping that I can get some install media, so I can get it set up. All in all, I found the tour to be highly impressive, with the lab implementing the latest and greatest in the mainframe and open systems world. I was also inspired by many of the configuration methods and wiring methods used in the lab, which I think will make management of my own mainframes and servers easier. I also want to thank Shari Chiara, Pasquale A Catalano, Paul Novak, and Eric Reid for taking the time to show me the lab, and help demo some of the systems. I also wanted to thank Connor Krukosky (the first mainframe kid) and Ernest Horn for taking the time to talk to me about the System Z platform and advice for my machine. Having them all as resources and mentors has been invaluable in helping me get my mainframe up and running. I look forward to a continued relationship with them, and I am greatly appreciative of all that IBM has done for me.