Python - Group home

“I've been working in IT for about 20 years. I spent the first 10 years working on Unix systems, primarily with non-IBM databases. I came to the mainframe when I joined Credit Suisse, more than 10 years ago. Furthermore, I have had the opportunity to partner with IBM on different topics like Db2, z/VM or running Linux on Z. I'm heading the team called ‘Mainframe Db2 and Analytics’ here at Credit Suisse in Switzerland. We look after all aspects around Db2 on z/OS, we take care of installation, housekeeping, analytics, consulting, and database administration.”

How did Python help you overcome the migration challenges?

At first, we ran Jupyter notebooks locally on our workstations. Everybody was using their personal installation and credentials to connect to the mainframe. Soon, we wanted to share the reports and jointly develop the scripts. Therefore, we had to set up a shared Jupyter environment on a server.

Since we’re a bank, there were many aspects to be considered from the start: Security integration, firewalls, e-mail integration for automating reports and notifications, etc. These are just the very basics to get the reports running without considering more advanced topics like high availability or high-performance.

I thought that having Python available on z/OS would make some of these challenges just magically disappear. And they really did! By running Python on z/OS we can reuse and leverage much of the existing Mainframe infrastructure.

Could you elaborate a little more on the use-cases for Python on z/OS?

We started looking into Python from a system administration and automation perspective. And we basically used Python to compliment REXX. But we also tried to go two steps further.
We have started building a performance dashboard in Python. And we also plan to implement some self-services options, where we make REST endpoints available for different activities on the system.

We currently have half a dozen applications running in Python. Two of these have implementations of REST endpoints in them and they access IBM Db2 data. They process the data and provide a report back to the caller. In one case in the form of a JSON document, in another case in the form of a properly formatted excel spreadsheet.

The third application is a web portal, fully integrated with our infrastructure, but also with our application-level security solution. It enables users to customize and generate reports and to download them directly as an excel sheet. Furthermore, this application performs REST call outs from the mainframe to do additional authorization checks.

Why did you choose to implement these in Python as opposed to other solutions?

Python claims for itself that it enables you to achieve superior results in less time. Productivity is therefore one factor; architecture simplification is another one. Application solutions that used to span across different components, platforms, and programming languages, can now be implemented in one place. Developers appreciate that this environment provides very tight security and high availability out of the box, and that they can benefit from many of the available infrastructure building blocks.

Third, is the availability of frameworks enabling the reuse of software components, which is a key aspect when it comes to software engineering efficiency.

Another important point to all these applications is data proximity. They all need to process very large amounts of data in very little time to deliver results to business divisions. Overnight reports based on yesterday’s data are often no longer sufficient. Instead, business divisions demand reports to be based on current data, available immediately and in a user-friendly format.

Have you found ways to collaborate with those application developers off platform? Are you able to write applications off platform then for z/OS?

Python is used in Credit Suisse outside of the mainframe in the areas of analytics, AI, and machine learning. So, when we started these python projects, we wanted to try and reuse the tool chain that is already in place for application development on distributed systems. And this worked out very well.

Portability is another important aspect because it allows us to use all the tools that are available for Python development and testing. Modern tools like Visual Studio Code or PyCharm offer great features in these areas.

As for portability, all our Python applications are developed on Windows and that’s where initial testing is performed. We created a small Db2 package to encapsulate the connection to Db2. It relieves the developer from knowing all the details like IP addresses, ports, and configuration settings for all the Db2 environments.

As a next step, applications undergo automatic testing managed by Jenkins pipelines and executed in a Docker environment. Once all tests passed, applications get deployed into their actual runtime environment on z/OS - without any modifications to sources or configurations.

So, knowing you have a background in Db2 and analytics and assuming you want to use Python at crunching those numbers perhaps in the data science space. Give us an idea of maybe what you couldn't do before and what can you do now. 

I think the exploitation of Python on z/OS for all types of machine learning use cases is certainly something that is on top of my mind. I believe the opportunities in this space, on the infrastructure side but also on the application side, are plentiful. However, there is also a broader range of opportunities.

Let's assume you have data in Db2 which you need to enrich with data coming from a REST service. You need to combine, clean, and process this data and provide it as a nicely formatted excel spreadsheet. And you want to provide this as report as a web based self-service.

This is an example of a very common use case these days and used to be rather difficult to implement. Thanks to the infrastructure that we have on z/OS and the availability of Python, such requirements have become easy to solve.

Have you used ZOAU (Z Open Automation Utilities) to augment your Python experience?

We installed ZOAU right after we installed Python. For system administration tasks I view it as an essential enhancement to Python. We use it in combination with Jinja, a powerful template engine to generate, prepare and execute batch jobs based on all kinds of input data, track their status and interface with z/OS data sets and functions.

The use of Python in this area is very attractive. One can exploit a state-of-the-art language with modern tools for development which is more efficient than using solely an ISPF editor, but it is also more fun, and it will typically lead to better code, thanks to all the built-in features and checks.

Python is generally described as a language that is easy to learn and easy to use. But Python is also a language that enables and supports concepts like modularity, code reuse, clean code, encapsulation, object-oriented concepts like inheritance and much more. “In short, Python allows you to write very elegant code that is of high quality, easy to understand and easy to maintain.” These are very important and compelling aspects of the Language which I feel are sometimes overlooked.

On top of all the technical benefits, how do you evaluate the possibility of bringing a whole new talent pool to IBM z/OS?

Skill shortages are often a hot topic in the Mainframe area. With Python, ranking among the most popular languages for several years now, the situation is quite the opposite. Python is being taught virtually everywhere these days, not only in computer science degrees but in many other fields as well. As a result, we have this large and growing talent pool of new joiners and career starters that bring Python skills with them, and they also know how to work with data. If we manage to provide these colleagues with a Python environment on z/OS, they will be ready to get work done on their first day and without ever touching a green screen or writing any type of JCL.

As a second effect, Python can also improve the collaboration between business divisions and IT. Thanks to the frameworks that come with Python, and the support for explorative programming, it can help to collaborate more closely, build MVPs faster and at earlier stages, possibly let business divisions work with some development artefacts and thus help build bridges between business and IT.

Overall, Python can help reduce the hurdles and the barriers that we have towards mainframe development. And I believe it also helps to make the IBM zSystems platform more attractive for development and to improve the overall perception of the platform.