IBM Z and LinuxONE IBM Z

IBM Z

The enterprise platform for mission-critical applications brings next-level data privacy, security, and resiliency to your hybrid multicloud.

 View Only
Expand all | Collapse all

Creating a new mainframe batch process - What toolset do I use? Traditional or Modern

  • 1.  Creating a new mainframe batch process - What toolset do I use? Traditional or Modern

    Posted 4 days ago
    Edited by Steve Spence 4 days ago

    Over the last couple of weeks l have been scratching my head over what to do in creating a new in house mainframe batch process.

    I could develop this process using the traditional software tools COBOL or PL/1 possibly REXX and JCL or do l go down the modern route using Python, z/OSMF and z/OSAU. I know the question might be it depends on what your process is doing but it's about using older mainframe tools over the more modern toolset.

    For me personally designing the process using the so called legacy toolset would be much more straightforward but the modern toolset ie: Python would probably involve less coding and a more robust process which would be easier for today's Mainframer's to implement and support.

    Any thoughts much appreciated!



    ------------------------------
    Steve Spence
    Expert Systems Engineer, Applications
    Ensono Ltd
    London
    ------------------------------



  • 2.  RE: Creating a new mainframe batch process - What toolset do I use? Traditional or Modern

    Posted yesterday

    Hello Steve, 

    I won't tell you what to do and, probably, my considerations have already passed through your head before writing here. In any case, it is a good opportunity to reflect myself. So, I apologize beforehand if they are not exactly what you expect. 

    First thing I thought was, cool, yes Python is a great tool and modernization is always a nice thing to do. I have worked myself with some customers to implement Python and substitute some old style tools, but not batch.

    But then, rethinking, I had some doubts. 

    1. Java is halfway, in performance terms, between Cobol and Python. It performs very, very well and you have a lot of information about how to implement Java as a batch player. Search for information coming from @David Follis in this IBM Community. This would give you part of both worlds: a good performance and a better ease of maintenance. A good option for regular datasources: Db2, IMS… and a good option if your batch has to interact with any other subsystem along the process. 
    2. If I had to choose between Cobol and PL/I I'd bet on Cobol just because a lot of compatibility initiatives are focused on Cobol more than PL/I. Also Cobol can interact easily with Java in the case you want to add some external pieces in the future. Batch with Cobol is a well known thing I won't discuss on this. 
    3. Python, I love Python. I started learning it long before it was used for data science in 2002. It is a powerful tool and you will find a lot of people to work on it. Python fitness depends, mainly on the funtions and data access you need to manage. If the data repository is Db2, good. If you have to access some files for the data you'll need ZOAU for that. ZOAU is like a shell on steroids to manage and work with z/OS datasets (and some more things). I would have to check about batch implementation and how to manage regular commits and checkpoints on the case that something fails, but this  might be my own ignorance. I see Python as a great tools to manage data pipelines more than a batch itself. Python + ZOAU is a good option to substitute REXX and achieve ease of maintenance, at least better than REXX. 
      Also, performance is the slower of them all. If your batch is extensive and you have execution time restrictions, I'd go for Java or Cobol.

    All of them can be triggered and managed using JCL, it can help to manage and organize the sequence of execution steps. I wouldn't drop JCL as a tool to execute any of the Java, Cobol or Python batch. It is an advantage of the platform that I would use to my benefit. 

    If you want to have a longer conversation, please, tell me. 

    Best regards, 



    ------------------------------
    Diego Cardalliaguet
    IBM Data&AI zStack Leader for EMEA
    IBM
    ------------------------------



  • 3.  RE: Creating a new mainframe batch process - What toolset do I use? Traditional or Modern

    Posted yesterday

    Thanks Diego for your thoughts.

    I think the dilemma I have is going to be around the supportability, and if I use say COBOL and JCL the question might come up as to why I didn't use Python because everybody uses it now it's available on the mainframe.

    Today most new mainframe developers don't necessarily learn COBOL or PL/1 like the old days, it tends to be the more established mainframer's who have many years of experience who would be happy to use traditional tools.

    The same could be said of using ISPF over z/OSMF or Zowe CLI, most comments I got back are ISPF uses the old green screen, this legacy, old stuff why would I want to use this, anyway this is a topic for another day. I am sure this has been discussed all the time.



    ------------------------------
    Steve Spence
    Expert Systems Engineer, Applications
    Ensono Ltd
    London
    ------------------------------



  • 4.  RE: Creating a new mainframe batch process - What toolset do I use? Traditional or Modern

    Posted 14 hours ago
    Edited by Diego Cardalliaguet 11 hours ago

    Use of Python would depend upon the job that batch would do and where. OK to work with Db2, difficult to use other datasources, you might be in the situation of having to access some of them using Db2 and DVM: VSAM for example. 

    You can access Db2 using JDBC and DDF using the JayDeBee package. If you use ibm_db you'll need to set up ODBC in the USS.

    If data is to be reworked using Pandas or any other packages, you'll have to download it to the USS and will need zFS sizes according to the size of datasets. Otherwise you'll have to use SQL as with Java. 

    To do a full analysis I would need to know more.

    CORRECTION: Python has Cobol interoperability on z/OS.

    Regards,  



    ------------------------------
    Diego Cardalliaguet
    IBM Data&AI zStack Leader for EMEA
    IBM
    ------------------------------



  • 5.  RE: Creating a new mainframe batch process - What toolset do I use? Traditional or Modern

    Posted yesterday

    Java: +1



    ------------------------------
    Sergio Samayoa
    ------------------------------



  • 6.  RE: Creating a new mainframe batch process - What toolset do I use? Traditional or Modern

    Posted 22 hours ago

    "Java is halfway, in performance terms, between Cobol and Python."

    I think you're underselling Java. Java performance is very good and I wouldn't be surprised if it outperforms COBOL for CPU intensive work (particularly on a zIIP). The one caveat is the overhead of JVM initialization. That's probably only a fraction of a second, but if it's something running thousands of times it will be significant compared to starting a regular program.

    The Java JZOS classes allow you to work with z/OS data and datasets. The JZOS batch launcher allows you to run java programs as regular z/OS batch jobs using DD statements from JCL etc. You can also run simple Java programs from a single source file without a separate compilation step i.e. as a script. They should run with the same performance as a compiled Java class or jar.



    ------------------------------
    Andrew Rowley
    ------------------------------



  • 7.  RE: Creating a new mainframe batch process - What toolset do I use? Traditional or Modern

    Posted 22 hours ago

    Hi Andrew,

    AFAIK there is nothing special instructions for JVM on zIIP processors just IBM charges less for workload on those processors.

    IMHO nothing beats the native code generated by COBOL, PL/I or C compilers.



    ------------------------------
    Sergio Samayoa
    ------------------------------



  • 8.  RE: Creating a new mainframe batch process - What toolset do I use? Traditional or Modern

    Posted 21 hours ago

    Hi Sergio,

    Many sites have CPs that are not running full speed so the zIIP has a built in advantage. zIIPs might also be less utilized so there is less wait for CPU.

    Java is also compiled to native code, and there are optimizations that the JIT compiler can apply at run time that cannot be done for pre-compiled programs. A few years ago I did a (very artificial) comparison between languages where Java was faster that C for that particular test.

    I use Java for SMF processing and it can easily process 1 GB/s. I/O becomes the bottleneck.

    If you have CPU intensive COBOL programs I would be interested in doing a comparison with Java to get some real numbers. It will depend on the program - but don't rule Java out.



    ------------------------------
    Andrew Rowley
    ------------------------------



  • 9.  RE: Creating a new mainframe batch process - What toolset do I use? Traditional or Modern

    Posted 20 hours ago

    Hi Andrew

    I'm not ruling our Java, actually I prefer Java for any new development even batch: Maintainability over speed.

    And now adays there is a lot of JSON and/or XML which, IMHO, is not fun in COBOL or PL/I.



    ------------------------------
    Sergio Samayoa
    ------------------------------



  • 10.  RE: Creating a new mainframe batch process - What toolset do I use? Traditional or Modern

    Posted 19 hours ago

    "Maintainability over speed"

    But Java gives you speed as well. Since you mentioned JSON, I did a test a while back where I converted a day's CICS SMF data into 26GB of JSON using Java. That took less than 5 minutes - which seems fast to me. (I'm not sure you want your CICS SMF data converted to JSON, but it's possible!)



    ------------------------------
    Andrew Rowley
    ------------------------------



  • 11.  RE: Creating a new mainframe batch process - What toolset do I use? Traditional or Modern

    Posted 14 hours ago

    Hi Andrew. 

    I have done Java on a mainframe since 2001. Starting with WebSphere v3.0.1. I've lived the transition from S/390 to Z, from less than 800MHz to 4GHz processors in z10. The introduction and disappearance of zAAP processors and the coexistence with zIIP that were intended just for Db2. The introduction of Java in CICS, the transition of tWAS to Liberty and the introduction of CICS Liberty… 

    There's no one as enthusiast of Java on IBM Z as I am. But I wouldn't state that it is faster than COBOL. You can make a Java transaction for less than 5MIPS that, in addition, runs on zIIP; but it won't outperform Cobol. 

    It is true that there are some instructions in zIIPs to accelerate Java and that zIIPs are always at full capacity. True that you have to "heat" the JVM because of the lazy compilation which, BTW, is there for historical performance reasons: when you started a WAS server, all requests to servant regions (remember them?) were retained in control regions until all classes where compiled and that triggered some timeouts for the first transactions coming in.

    JZOS is great to start and for small-medium tasks, but for full business class batch in Java you need Liberty with JSR-352 (you can see a big list of entries by David in this community).

    My recommendation to modernize batch is Java, more than Python. With Java you can achieve business class batch and will run better than in any other platform, there's no other place where Java runs better than in the IBM Z (as of today), but it is very difficult to beat Cobol, especially if you are in Cobol 6 and use the new compilers. 

    Python is great, as a substitute for REXX when you include ZOAU. It still has some limitations of access to some data repositories and a lack of interaction with other languages. Java can interact with Cobol (in 31 or 64 bits), other batch, CICS and IMS. Liberty gives you the WOLA interface for memory-fast transactional interactions… All that is yet to come with Python. 

    There are many nuances that @Steve Spence has not shared here. 

    Regards, 



    ------------------------------
    Diego Cardalliaguet
    IBM Data&AI zStack Leader for EMEA
    IBM
    ------------------------------



  • 12.  RE: Creating a new mainframe batch process - What toolset do I use? Traditional or Modern

    Posted 11 hours ago

    JSR-352 isn't really what I think of as batch. To me, batch on z/OS is generally a JOB running from JCL submitted automatically or manually, or perhaps a scheduled unix task. If it's running under Liberty I don't think of it as batch, in the same way as CICS isn't batch.

    "it is very difficult to beat Cobol"

    It's just speculation until someone measures it. It's a subject I find very interesting. I did some testing previously:

    https://www.blackhillsoftware.com/news/2021/08/10/java-vs-c-drag-racing-on-z-os/

    That test is very artificial and not really the sort of processing COBOL is intended for.

    The 1 billion row challenge might be more like a typical COBOL workload. Anyone care to knock up a COBOL solution to compare to Java?



    ------------------------------
    Andrew Rowley
    ------------------------------



  • 13.  RE: Creating a new mainframe batch process - What toolset do I use? Traditional or Modern

    Posted 7 hours ago

    It's really hard to beat Pl/1 from a true efficiency standpoint.  

    That is of course something folks seem to care less about these days, until of course you have to.  



    ------------------------------
    Kurt Acker
    zLOV Architect @ Velocity Software
    Sir Kurt, Angler of New Technology, Master of Modernization and IBM Champion
    ------------------------------



  • 14.  RE: Creating a new mainframe batch process - What toolset do I use? Traditional or Modern

    Posted 23 hours ago
    Edited by Mohamed Moussa 22 hours ago

     Strategic Overview: IBM Z Batch Toolset Comparison

    IBM offers both traditional and modern approaches to batch processing. Your choice depends on system architecture, team skills, and future goals.

    Traditional Toolset (COBOL, PL/I, JCL, REXX)

    • Best for: Extending legacy systems, maintaining existing batch jobs, and ensuring compatibility with long-standing infrastructure.

    • Strengths:

      • Mature, stable, and highly optimized for z/OS.

      • Deep integration with IBM schedulers and JES2/JES3.

      • Proven reliability for high-volume, mission-critical workloads.

    • Limitations:

      • Limited support for modern DevOps practices.

      • Scarcity of skilled COBOL/PL/I developers.

      • Less flexible for hybrid cloud or API-driven architectures.

    Modern Toolset (Java, Python, z/OSMF, z/OS Automation Utilities)

    • Best for: New development, hybrid cloud integration, and agile workflows.

    • Strengths:

      • Supports REST APIs, microservices, and cloud-native patterns.

      • Java workloads can be offloaded to zIIP processors for cost savings.

      • Easier onboarding for modern developers.

      • Compatible with DevOps pipelines and containerized environments.

    • Limitations:

      • May require additional configuration for performance tuning.

      • Integration with legacy systems may need wrappers or hybrid orchestration.



    ------------------------------
    Mohamed Moussa
    Strategic Operations & IT Manager | Digital Transformation Leader | Multilingual Consultant | Migrat
    NO Limits Co Ltd
    Istanbul
    ------------------------------



  • 15.  RE: Creating a new mainframe batch process - What toolset do I use? Traditional or Modern

    Posted 9 hours ago

    Very interesting comparison.

    • Scarcity of skilled COBOL/PL/I developers - does this really matter as there is tools which can interpret or convert the code to a modern language(especially around the world of AI)?



    ------------------------------
    Steve Spence
    Expert Systems Engineer, Applications
    Ensono Ltd
    London
    ------------------------------



  • 16.  RE: Creating a new mainframe batch process - What toolset do I use? Traditional or Modern

    Posted 15 hours ago

    Hi Steve,

    when modernizing the question about the programming language and tooling always comes up early. I personally think, this question is overrated. In IBM Z and z/OS, we're building applications and infrastructures that we expect to run and maintain for decades. In the same ways as COBOL, REXX, and alike are seen as legacy programming languages today, I'd expect the same for Python and Java in 10 years from now.

    A good programmer should be able to write code in any (most) programming language. Assuming that you don't have the development capacity to re-write your entire batch application, you'll find yourself in the situation with programs written in multiple different languages in some years. Finding people for each of those will be more difficult. Training them in multiple languages will take more time.

    What is more important for me is more about programming paradigms, e.g. functional programming like PL/I, COBOL vs. object oriented programming, for example in Java. That said, you can write "object oriented" also using functional programming languages, it's just not good supported by the language itself. Having good programming practices like clean code, and good documentation about your programs will improve maintainability by factors.

    That might not be the information you're looking for, but I think these consideration will help to decide.



    ------------------------------
    Michael Grötzner
    z/OS I/O Configuration Architect
    IBM Germany Research & Development GmbH
    ------------------------------



  • 17.  RE: Creating a new mainframe batch process - What toolset do I use? Traditional or Modern

    Posted 11 hours ago

    Hi Michael

    Thanks for your insight. The fact that Assembler, COBOL and PL/1 code is probably well imbedded in most processes throughout the world especially banking systems, there isn't a sudden need to have to convert it to a more modern language (Java or Python).

    At least from this discussion I can take all the feedback to help me decide. 



    ------------------------------
    Steve Spence
    Expert Systems Engineer, Applications
    Ensono Ltd
    London
    ------------------------------



  • 18.  RE: Creating a new mainframe batch process - What toolset do I use? Traditional or Modern

    Posted 13 hours ago

    Hi Steve,

    Thank you for raising this very relevant question – it's something we are hearing from almost every client today.

    I have been following the discussion closely and appreciate the valuable insights shared so far. I am particularly interested in better understanding the level of effort and complexity involved in converting batch workloads and underlying programs developed in COBOL, Assembler, REXX, JCL, etc.

    While many customers are eager to modernize, the key decision factors remain:
    • How do we accurately assess the business and technical impact in advance?
    • How do we quantify the expected performance improvements to justify the investment?

    I understand that success depends significantly on the skills and experience of the delivery teams – and many would advocate engaging IBM Consulting. However, I would like to know whether there are existing customer examples where such modernization has already been executed, along with indicative timelines and measured performance improvements.

    I am aware of IBM tools such as WCA4Z and ADDI that support discovery and analysis, but I would also like to confirm whether these tools can provide a comprehensive complexity evaluation and projected performance outcomes when transitioning to new programming languages or modernization frameworks.

    Looking forward to further insights from the group.

    Regards,

    Abhi



    ------------------------------
    Abhishek Anand
    ------------------------------