Master the Mainframe Global User Group

Expand all | Collapse all

Weekly t-shirt challenge: Repurposed Mini Test Jobs

  • 1.  Weekly t-shirt challenge: Repurposed Mini Test Jobs

    Posted Thu April 30, 2020 10:29 PM

    Repurposed Mini Test Jobs

    Thank you so much for all the great ideas for weekly challenges!  @Robert Prins wins his second t-shirt with his idea of sharing the ways you've "abused" your mini test jobs (or we can call it "code reuse").   But @Fábio Emilio Costa is on a streak winning for an 8th time with his idea to share your new working from home workspaces.  I'll have to start mailing him homemade cookies or something since he's getting bored of t-shirts!  

    For this week, let's go with Robert's idea.  Share your mini bits of code that do something fun or strange, let's see that code "abuse".


    The rules:
    1.  Every Thursday, I'll start a new discussion thread titled Weekly Challenge.

    2.  This week, reply to the thread with your repurposed little test jobs.  

    3.  Vote for your favorite entries by clicking the "Recommend" button next to your favorites.

    4.  The following Thursday, I'll announce the winner and start the next week's thread.

    5.  The person submitting the item with the most votes will receive one of these Legends of Z t-shirts (or equivalent)!

    ​​

    ------------------------------
    Misty Decker
    ------------------------------


  • 2.  RE: Weekly t-shirt challenge: Repurposed Mini Test Jobs

    Posted Fri May 01, 2020 04:31 PM
    Having proposed the challenge, let's give it a go...

    In 1978 I enrolled in the Technical University of Delft. As part of the curriculum we got 50 runs (of 3 seconds) on the IBM system, to write five (increasingly) more "complicated" programs, in Algol 60, and using, yes, punched cards! Once you had completed your five assignments, you could use the remaining runs to do whatever you wanted with them. I used them to write a few programs to calculate constants for Gaussian integration, and one to calculate the square root of 2 to, if I remember correctly, 50 digits. (And once a fellow student discovered we could run jobs in "CLASS=X", we had unlimited access, plus 10 seconds per JOB, but sadly soon after punched cards were phased out, and replaced by terminals, and you needed a valid ID to access the system)

    In 1985 (I never finished university, electronics in the end wasn't my cup of tea), I joined a Dutch insurance company as a trainee, and one of my more experienced colleagues gave me a bit of "Compile, Link, and Go" JCL, "so that you can try [work related] things". I mostly used it to do just that, but I did resurrect my 50-digit square roots program, and actually translated the version I had written for the TI-59 programmable calculator (which could calculate square roots to 800(!) digits) into PL/I.

    Then, in 1990, I moved to the UK (the result of a hitchhiking trip to Turkey, long story), joined an insurance broker, and yes, my try-out JCL came along. By that time my collection of "Extended Precision" programs on the TI-59 also contained two programs to calculate "pi" and "e", to respectively 1,297 and 1,300 digits, an amazing achievement, as that left only 160 bytes for the programs. The original "pi" program took, if I remember correctly, more than 25(!) days, and the "e" program took just over 24 hours, but in the end I translated another "e" program, one originally written for the HP-41C into PL/I. It used a much faster algorithm (returning, on the TI-59, 870 digits in "just" under 3 hours) that would work backwards, and extended the calculation to more digits when required. The program worked, but calculating "e" to 1,000 digits wasn't all that interesting, I think it took just a few minutes, and that was it.

    So I decided to become a little bit more ambitious... Well, actually a big bit more, like 1,000 times...

    Obviously, calculating one million digits of "e" takes time, unless you use something like Y-cruncher, when it takes seconds, so I changed the program to run for 10 (incredibly) CPU bound minutes at a time, after which it would write out the intermediate result out to a dataset. (For what it's worth Y-cruncer uses exactly the same formula for "e", the summation of inverted factorials)

    And then I started my, obviously copied to another bit of JCL, "test code". I don't remember how many hundreds (or even thousands!) of runs it took, during working days I submitted them probably at a rate of one or two per hour, but on weekends, when we had the test system completely at our disposal, I would just submit 50 or 60 jobs in one go and as they all had the same jobname, they would just run, and run, and run. During those weekends, I was single at the time and rented just a room in Ipswich, I also did more useful things, like writing many of the tools you can now find on the z/OS part of my website, Prino's z/OS Tools and utilities

    Eventually, sometime in 1992, the calculations finished, I had calculated 1,000,000 million digits of "e", probably some 18 months before the first public announcement of a similar achievement by Robert Nemiroff and Jerry Bonnell on The Number e to 1 Million Digits. My first million digits agree with theirs, they got the fame, but a year earlier I had already been given my own "15 minutes of fame" by being included into the UK edition of the Guinness Book of Records with my 24-hour hitchhike record.

    And yes, my current hitchhike statistics program also started as yet another copy of my little test program, and the current (35th) version still runs on z/OS, and what's more, this one single program has been responsible for several APARs applied to the various versions of Enterprise PL/I, the first back in 2006 or 2007, when Enterprise PL/I V3.5 choked on an "INIT (LIFT_LIST.WAIT ** 2)" in the allocation of a based structure!

    The current version uses features I suggested in IBM's RFE program, features I first implemented, in, yes, what else, my little test program. At the time (2007) my code required direct manipulation of the (undocumented) 16 byte header of PL/I "AREA" variables, but my RFE 31632, "Allow defined allocate in AREA" and a later follow-on, RFE 100073: Add FASTALLOC builtin for allocation in AREA properly added PL/I support and the potential to greatly speed up list and tree processing to PL/I, two features that have quite a bit of relevancy in this day and age, given that processing XML and JSON makes use of these type of structures! Using both RFE's and building linked lists and/or tree structures inside a PL/I "AREA" variable makes it possible to zap them with a single "AREA=EMPTY();" statement, which generates, in the Enterprise PL/I (V5.2.2) listing I've got here, just 5(!) instructions, and that's probably four more than would be required for an area that's declared STATIC or AUTOMATIC, my area variable is CONTROLLED.

    To stay closer to the little test program, during my year at KLM in the Netherlands I also used it to figure out how to save potentially significant amounts of CPU time in programs using "BASED REFER" (self-defining) structures, and my write-up was nearly literally copied by Peter Elderon, I think he's still IBM's lead maintainer of Enterprise PL/I, into the PL/I Programming Guide. Look at page 350 (396 in the PDF) of the "Enterprise PL/I for z/OS Programming Guide" (V5.2) and the second paragraph below is a near carbon copy of what I wrote to Peter in 1996:

    "When your code refers to a member of a BASED structure with REFER, the
    compiler often has to generate one or more calls to a library routine to map the
    structure at run time. These calls can be expensive, and so when the compiler
    makes these calls, it will issue a message so that you can locate these potential
    hot-spots in your code.

    If you do have code that uses BASED structures with REFER, which the compiler
    flags with this message, you might get better performance by passing the structure
    to a subroutine that declares a corresponding structure with * extents. This will
    cause the structure to be mapped once at the CALL statement, but there will no
    further remappings when it is accessed in the called subroutine."

    I used yet another version of the test program to optimize a KLM routine to calculate CRCs that were used to verify data uploaded to planes. My version cut the CPU time by, I know this is hard to believe, 99.7%! (Using unaligned bit(1) fields in OS PL/I V2.3.0 AD 1996 used to be bad, very bad, I'm not sure if this is still true in Enterprise PL/I AD 2020)

    Final words? Don't use your test programs to calculate "pi" or "e" on z/OS, my employer obviously found out, SMF is unforgiving, and wasn't too pleased about it, but at that moment my reputation was already so high that I got off with just a friendly(!) warning not to do something like that ever again, and about half-a-decade later I probably made up for it by writing the tools Willis used for its Y2K conversion. (And I still hitchhike)

    And as for the Compile, Link and Go JCL? It currently looks like: (JCL is somewhat mangled, thanks to the fact that this site needs me to run scripts from a zillion other sites, I use NoScript)

    //PRINOCOM JOB (PRINO),
    // 'RAHP COMP/LINK',
    // CLASS=A,
    // MSGCLASS=H,
    // MSGLEVEL=(2,0),
    // NOTIFY=&SYSUID
    //*********************************************************************
    //* PL/I COMPILE & GO
    //*********************************************************************
    //PROCS JCLLIB ORDER=(PRINO.RAHP.PROC)
    //*********************************************************************
    // INCLUDE MEMBER=IMOI MY LIBS
    //*
    // SET SRCE='TEST'
    //*
    // SET GO='16' JUST COMPILE
    // SET GO='00' COMPILE, LINK & RUN
    //*********************************************************************
    //IEFBR14 EXEC PGM=IEFBR14
    //SYSLIN DD DSN=&&OBJECT,
    // DISP=(,PASS),
    // UNIT=SYSDA,
    // SPACE=(CYL,(2,2,1)),
    // DCB=(RECFM=FB,LRECL=80,BLKSIZE=27920)
    //*
    //LIST DD DSN=&SYSUID..&SRCE..LIST,
    // DISP=(MOD,DELETE),
    // UNIT=SYSDA,
    // SPACE=(TRK,1)
    //*********************************************************************
    //IBMZPLI EXEC PGM=IBMZPLI,
    // REGION=0M,
    // PARM=('+DD:PLISTD +DD:PLIUSER')
    //*
    //* PL/I COMPILER STEPLIB
    //*
    // INCLUDE MEMBER=ISTEP@&PLIVER SET IN IMOI
    //*
    //SYSUEXIT DD DSN=NULLFILE,
    // DISP=SHR
    //*
    //SYSPRINT DD DSN=&SYSUID..&SRCE..LIST,
    // DISP=(,CATLG),
    // UNIT=SYSDA,
    // SPACE=(TRK,(60,90),RLSE),
    // DCB=(RECFM=VBA,LRECL=137,BLKSIZE=0)
    //*
    //SYSUT1 DD UNIT=SYSDA,
    // SPACE=(CYL,(5,5))
    //*
    //PLISTD DD DSN=&DATA(PXEP&PLIVER.Z), SET IN IMOI
    // DISP=SHR
    //*
    //PLIUSER DD *
    test(all,sym,separate,sepname) sysparm('TEST') opt(0)
    notest sysparm('') opt(3)
    //*
    //SYSIN DD DSN=&PLI(&SRCE),
    // DISP=SHR
    //*
    //SYSLIB DD DSN=&INC,
    // DISP=SHR
    // DD DSN=&PLI,
    // DISP=SHR
    //*
    //SYSLIN DD DSN=&&OBJECT(OBJ),
    // DISP=(OLD,PASS)
    //*
    //SYSPUNCH DD DUMMY
    //*********************************************************************
    //SORT EXEC PGM=SORT
    //*
    //SORTIN DD DSN=&SYSUID..&SRCE..LIST,
    // DISP=SHR
    //*
    //SORTOUT DD SYSOUT=*
    //*
    //SYSIN DD DSN=&DATA(SORTCOPY),
    // DISP=SHR
    //*
    //SYSOUT DD DUMMY
    //*********************************************************************
    //IFGO IF RC <= 4 & RC >= &GO THEN
    //*
    //IEWL EXEC PGM=IEWL,
    // PARM=('LET,LINECT=0,LIST,XREF',
    // 'AMODE=31,RMODE=ANY')
    //*
    //SYSLIN DD *
    SETCODE AC(0)
    INCLUDE OBJECT(OBJ)
    NAME TEST(R)
    //*
    //OBJECT DD DSN=&&OBJECT,
    // DISP=(OLD,PASS)
    //*
    //SYSLIB DD DSN=CEE.SCEELKED,
    // DISP=SHR
    // DD DSN=SYS1.CSSLIB,
    // DISP=SHR
    // DD DSN=&ISP.SISPLOAD,
    // DISP=SHR
    //*
    //SYSUT1 DD UNIT=SYSDA,
    // SPACE=(CYL,(5,1))
    //*
    //SYSLMOD DD DSN=&&LOAD,
    // DISP=(,PASS),
    // UNIT=SYSDA,
    // SPACE=(CYL,(2,2,1)),
    // DCB=(RECFM=U,LRECL=27988,BLKSIZE=0)
    //*
    //SYSPRINT DD SYSOUT=*,
    // DCB=(RECFM=FBA,LRECL=121,BLKSIZE=27951)
    //*********************************************************************
    //GO EXEC PGM=TEST,
    // REGION=0M
    //*
    //STEPLIB DD DSN=&&LOAD,
    // DISP=(OLD,PASS)
    //*
    //SYSPRINT DD SYSOUT=*
    //*
    //CEEDUMP DD SYSOUT=*
    //PLIDUMP DD SYSOUT=*
    //SYSUDUMP DD SYSOUT=*
    //*
    //IFGOE ENDIF


    with "IMOI" containing, for site-to-site portability:

    //*********************************************************************
    //* My standard JCL include member
    //*********************************************************************
    // SET LOAD='PRINO.RAHP.LOAD'
    // SET OBJ='PRINO.RAHP.OBJ'
    //*
    // SET REXX='PRINO.RAHP.EXEC'
    // SET DATA='PRINO.RAHP.DATA'
    // SET CNTL='PRINO.RAHP.CNTL'
    //*
    // SET PLI='PRINO.RAHP.PLI'
    // SET INC='PRINO.RAHP.INC'
    //*
    // SET TEXT='PRINO.RAHP.TEXT'
    // SET VEXT='PRINO.RAHP.VEXT'
    //*
    // SET DBRM='PRINO.RAHP.DBRM'
    // SET DEBUG='PRINO.RAHP.DEBUG'
    //*
    // SET DASD='SYSDA'
    //*
    // SET OUTC='*'
    // SET OUTD='*'
    //*
    // SET PLIVER='54' For options member

    ------------------------------
    Robert Prins
    Programmer/analyst-programmer
    Freelancer "between contracts"
    Vilnius
    +370 60549447
    ------------------------------