I am a Java programmer that has done some work on the mainframe and Unix end of WebSphere end-to-end, but now am working on a COBOL to Java conversion task, after not ever coding for z/OS directly, and IBMJZOS is still new to me.

I  am trying to see if I can improve read performance of fixed block datasets by not relying on IBMJZOS ZFile access that per code seen in this shop seems to read one record at a time.  Our code is built upon using the CobolDatatypeFactory object.

The file being read has over a million (1,001,042 to be exact) records, and has an LRECL of 120, and BLKSIZE of 27960.  Per JES output, there were  4298 block reads done on the file during a job run.

I've tried using ZFileRecordReader with slight success, but since its ZFile newReader method is deprecated, and the JavaDoc for it seems to point to using RecordReader, I went looking into FileFactory as an alternative.  The filename is being specified via //DD:

Using IBMJZOS 2.4.8 under JDK 8 64-bit on the USS system going to that fixed-block dataset, BufferedInputStream is not maintaining the 'fixed record' layout as data flows into the buffered stream, and is instead dropping trailing spaces.   Noted 0x15 (EBCDIC newline) delimiter characters separating the records, which was strange to me since the input file is fixed block.  I could not find a way to control this behavior, which may be fixable if there was a way to specify that the file is fixed-block.

Any suggestions on either improving read performance of fixed block records or figuring out why FileFactory BufferedInputStream  behaves this way would be appreciated.

Thanks.




------------------------------
Albert Stein
------------------------------

Albert,

Is there some reason you're opposed to RecordReader newReader (or newReaderForDD)? That's the recommended replacement. N.B. ZFileRecordReader is not the same thing as RecordReader.

As an aside, you may want to include the IBM Record Generator for Java in your toolbag if you have a COBOL copybook (or Assembler DSECT) for your data set layout.

https://developer.ibm.com/mainframe/products/record-generator-for-java/

------------------------------
Timothy Sipples
------------------------------

Original Message:
Sent: Mon March 09, 2020 03:47 PM
From: Albert Stein
Subject: IBMJZOS FileFactory BufferedInputStream drops trailing spaces from Fixed Block records

I am a Java programmer that has done some work on the mainframe and Unix end of WebSphere end-to-end, but now am working on a COBOL to Java conversion task, after not ever coding for z/OS directly, and IBMJZOS is still new to me.

I  am trying to see if I can improve read performance of fixed block datasets by not relying on IBMJZOS ZFile access that per code seen in this shop seems to read one record at a time.  Our code is built upon using the CobolDatatypeFactory object.

The file being read has over a million (1,001,042 to be exact) records, and has an LRECL of 120, and BLKSIZE of 27960.  Per JES output, there were  4298 block reads done on the file during a job run.

I've tried using ZFileRecordReader with slight success, but since its ZFile newReader method is deprecated, and the JavaDoc for it seems to point to using RecordReader, I went looking into FileFactory as an alternative.  The filename is being specified via //DD:

Using IBMJZOS 2.4.8 under JDK 8 64-bit on the USS system going to that fixed-block dataset, BufferedInputStream is not maintaining the 'fixed record' layout as data flows into the buffered stream, and is instead dropping trailing spaces.   Noted 0x15 (EBCDIC newline) delimiter characters separating the records, which was strange to me since the input file is fixed block.  I could not find a way to control this behavior, which may be fixable if there was a way to specify that the file is fixed-block.

Any suggestions on either improving read performance of fixed block records or figuring out why FileFactory BufferedInputStream  behaves this way would be appreciated.

Thanks.




------------------------------
Albert Stein
------------------------------

Thanks for the response Timothy. 

I did wind up coding around (via some deep logic) the newline characters while using the FileFactory's BufferedInputStream, and did not improve performance.  BufferedInputStream seemed the logical choice from my prior Java experience, but does not fit well reading BSAM in the current MVS storage world apparently, due to what I saw and reported in my initial query here.

I then went back to RecordReader (which I did not fully understand a few days ago, wish there were some working samples on the net) and did code a better buffering system than ZFile, and with some trial and error am currently at about an 8% improvement when using a BUFNO value that I took for a Java-side block size multiplier of 30 (vs values between 2 and 1024) that was originally set in the JCL used for the COBOL version prior to my Java re-code.

Do you or anyone else know of any other ways in improving read performance?   I'll be heading to working with RecordWriter for the 'write' side shortly.


------------------------------
Albert Stein
------------------------------

Original Message:
Sent: Thu March 12, 2020 02:41 AM
From: Timothy Sipples
Subject: IBMJZOS FileFactory BufferedInputStream drops trailing spaces from Fixed Block records

Albert,

Is there some reason you're opposed to RecordReader newReader (or newReaderForDD)? That's the recommended replacement. N.B. ZFileRecordReader is not the same thing as RecordReader.

As an aside, you may want to include the IBM Record Generator for Java in your toolbag if you have a COBOL copybook (or Assembler DSECT) for your data set layout.

https://developer.ibm.com/mainframe/products/record-generator-for-java/

------------------------------
Timothy Sipples

Original Message:
Sent: Mon March 09, 2020 03:47 PM
From: Albert Stein
Subject: IBMJZOS FileFactory BufferedInputStream drops trailing spaces from Fixed Block records

I am a Java programmer that has done some work on the mainframe and Unix end of WebSphere end-to-end, but now am working on a COBOL to Java conversion task, after not ever coding for z/OS directly, and IBMJZOS is still new to me.

I  am trying to see if I can improve read performance of fixed block datasets by not relying on IBMJZOS ZFile access that per code seen in this shop seems to read one record at a time.  Our code is built upon using the CobolDatatypeFactory object.

The file being read has over a million (1,001,042 to be exact) records, and has an LRECL of 120, and BLKSIZE of 27960.  Per JES output, there were  4298 block reads done on the file during a job run.

I've tried using ZFileRecordReader with slight success, but since its ZFile newReader method is deprecated, and the JavaDoc for it seems to point to using RecordReader, I went looking into FileFactory as an alternative.  The filename is being specified via //DD:

Using IBMJZOS 2.4.8 under JDK 8 64-bit on the USS system going to that fixed-block dataset, BufferedInputStream is not maintaining the 'fixed record' layout as data flows into the buffered stream, and is instead dropping trailing spaces.   Noted 0x15 (EBCDIC newline) delimiter characters separating the records, which was strange to me since the input file is fixed block.  I could not find a way to control this behavior, which may be fixable if there was a way to specify that the file is fixed-block.

Any suggestions on either improving read performance of fixed block records or figuring out why FileFactory BufferedInputStream  behaves this way would be appreciated.

Thanks.




------------------------------
Albert Stein
------------------------------

It's probably time to pull out a Java performance profiler to see what's going on and whether there's any opportunity for improvement. If you have OMEGAMON JVM then that should work nicely, or you could try this one that my colleague recommends:

https://www.ibm.com/support/knowledgecenter/SS3KLZ/com.ibm.java.diagnostics.healthcenter.doc/homepage/plugin-homepage-hc.html

Another possible option is a hybrid program, i.e. let COBOL handle the I/O and then layer Java atop/alongside if that's the best performance combination. If this program is running outside CICS Transaction Server then you can use Enterprise COBOL's INVOKE statement and other cross-language facilities. (Within CICS the INVOKE statement isn't available, but CICS facilities are.) Note that the latest Enterprise COBOL Version 6.3 compiler now offers the very first 64-bit support (AMODE 64). AMODE 64 COBOL comes with a few caveats at the moment, but if the current caveats are acceptable then you should be able to fashion a purely 64-bit dual language program.

------------------------------
Timothy Sipples
------------------------------

Original Message:
Sent: Fri March 13, 2020 07:29 AM
From: Albert Stein
Subject: IBMJZOS FileFactory BufferedInputStream drops trailing spaces from Fixed Block records

Thanks for the response Timothy. 

I did wind up coding around (via some deep logic) the newline characters while using the FileFactory's BufferedInputStream, and did not improve performance.  BufferedInputStream seemed the logical choice from my prior Java experience, but does not fit well reading BSAM in the current MVS storage world apparently, due to what I saw and reported in my initial query here.

I then went back to RecordReader (which I did not fully understand a few days ago, wish there were some working samples on the net) and did code a better buffering system than ZFile, and with some trial and error am currently at about an 8% improvement when using a BUFNO value that I took for a Java-side block size multiplier of 30 (vs values between 2 and 1024) that was originally set in the JCL used for the COBOL version prior to my Java re-code.

Do you or anyone else know of any other ways in improving read performance?   I'll be heading to working with RecordWriter for the 'write' side shortly.


------------------------------
Albert Stein

Original Message:
Sent: Thu March 12, 2020 02:41 AM
From: Timothy Sipples
Subject: IBMJZOS FileFactory BufferedInputStream drops trailing spaces from Fixed Block records

Albert,

Is there some reason you're opposed to RecordReader newReader (or newReaderForDD)? That's the recommended replacement. N.B. ZFileRecordReader is not the same thing as RecordReader.

As an aside, you may want to include the IBM Record Generator for Java in your toolbag if you have a COBOL copybook (or Assembler DSECT) for your data set layout.

https://developer.ibm.com/mainframe/products/record-generator-for-java/

------------------------------
Timothy Sipples

Original Message:
Sent: Mon March 09, 2020 03:47 PM
From: Albert Stein
Subject: IBMJZOS FileFactory BufferedInputStream drops trailing spaces from Fixed Block records

I am a Java programmer that has done some work on the mainframe and Unix end of WebSphere end-to-end, but now am working on a COBOL to Java conversion task, after not ever coding for z/OS directly, and IBMJZOS is still new to me.

I  am trying to see if I can improve read performance of fixed block datasets by not relying on IBMJZOS ZFile access that per code seen in this shop seems to read one record at a time.  Our code is built upon using the CobolDatatypeFactory object.

The file being read has over a million (1,001,042 to be exact) records, and has an LRECL of 120, and BLKSIZE of 27960.  Per JES output, there were  4298 block reads done on the file during a job run.

I've tried using ZFileRecordReader with slight success, but since its ZFile newReader method is deprecated, and the JavaDoc for it seems to point to using RecordReader, I went looking into FileFactory as an alternative.  The filename is being specified via //DD:

Using IBMJZOS 2.4.8 under JDK 8 64-bit on the USS system going to that fixed-block dataset, BufferedInputStream is not maintaining the 'fixed record' layout as data flows into the buffered stream, and is instead dropping trailing spaces.   Noted 0x15 (EBCDIC newline) delimiter characters separating the records, which was strange to me since the input file is fixed block.  I could not find a way to control this behavior, which may be fixable if there was a way to specify that the file is fixed-block.

Any suggestions on either improving read performance of fixed block records or figuring out why FileFactory BufferedInputStream  behaves this way would be appreciated.

Thanks.




------------------------------
Albert Stein
------------------------------