Our COBOL V4.2 compiler has NUMPROC(MIG) specified.  For V5.2 and now V6.2 (and 6.3) we have NUMPROC(NOPFD) and ZONEDATA(MIG).  Yesterday a program was implemented with a minor change, but caused major havoc.  The change itself was fine, but it looks like the behavior of the compiler changed between the last time the program was compiled and the current time.  The code in question was checking for equality between a packed decimal field and a zoned decimal field.  Unfortunately the zoned decimal field (for reasons TBD) contained only binary zeroes, rather than a valid zoned decimal value.  The code generated by our current V6.2 compiler is as follows:

PACK    162(6,R13),1416(10,R9) #                          WS-UNPACKED-BRANCH-ACCT-
CP      6(10,R4),162(6,R13)   #  CHR-XREF-APPL-ACCT                               
JNE     L0217                                                                     ​

We don't have access to the previous version of V6.2, but we do still have V5.2 hanging around.  Here is the code it generates:

MVC     166(10,R13),6(R4)     #  _VTS_1                  CHR-XREF-APPL-ACCT
OI      175(,R13),X'0F'       #  _VTS_1                                    
PACK    210(6,R13),1464(10,R8)                                             
OI      215(,R13),X'0F'       #  _VTS_2                                    
CP      166(10,R13),210(6,R13)                                             
JNE     L0216                                                              ​

And here is V4.2:

PACK  1112(6,13),3664(10,3)   TS2=0                             WS-UNPACKED-BRANCH-ACCT-NBR
CP    1198(10,2),1112(6,13)   CHR-XREF-APPL-ACCT                TS2=0                      
BC    7,1052(0,11)            GN=33(002BEE)                                                ​

So the behavior of V6.2 is now (generally) the same as V4.2, but different from V5.2 (and I assume the "older" V6.2).

As you can see, V6.2/4.2 simply packs the zoned decimal field and then compares the resulting packed decimal field to the other packed decimal field.  The V5.2 version, which I am assuming was how V6.2 was behaving until a (fairly) recent update, does a "sign fixup" on both fields prior to the compare.  Thus what was previously invalid data is fixed to be valid data prior to the compare operation, thus avoiding the data exception condition being generated.

We had a similar issue about a month ago so I knew pretty much immediately how to make a short term fix to resolve the issue.  And that was to add a check to make sure the zoned decimal field is numeric before doing the actual compare.  Of course the long-term fix is to make sure the zoned decimal field has a valid numeric value prior to doing the comparison.

It looks like the latest fixes for V6.2 and V6.3 have a new compiler option, INVDATA.  It's described here:  https://www.ibm.com/support/pages/apar/PH31500.

The APAR says that if you had NUMPROC(MIG) you should now use INVDATA(FORCENUMCMP,NOCLEANSIGN) (along with, I assume, NUMPROC(NOPFD)).  This will give us the behavior shown above for V6.2 and V4.2.  Would it be the case that we could specify INVDATA(FORCENUMCMP,CLEANSIGN) if we want to get the V5.2 behavior back?  It seems like we might want to do this, even though it's different than V4.2 behavior, because we've already converted over 1000 programs to V5.2 or V6.2, and we'd prefer to avoid unexpected production abends for "previously working" V5.2 (and older V6.2) code.

On a related note, I'm wondering if it might make sense to still use INVDATA(FORCENUMCMP,NOCLEANSIGN) for development compiles, with INVDATA(FORCENUMCMP,CLEANSIGN) only for production.  This way we could get data exceptions for bad data during development, but avoid them in production.  So hopefully we would find and fix invalid data situations prior to going in to production, but also avoid being hit by data exception abends in production.

------------------------------
Frank Swarbrick
------------------------------

Hi Frank,

You're correct. The ultimate goal of INVDATA (and previously, ZONEDATA) was to provide compatibility with COBOL 4, though we're only doing this for issues that our clients alert us to; a full COBOL 4 compatibility mode is infeasible. The initial ZONEDATA(MIG) release in the COBOL 5.1 timeframe was followed fairly shortly after by ZONEDATA(NOPFD), and initially, the only behavioural difference from either was whether compares would always be numeric (matching NUMPROC(MIG) in COBOL 4), or whether they might not be (matching NUMPROC(NOPFD) or NUMPROC(PFD) in COBOL 4). As time went on, we've added more compatibility to ZONEDATA in response to new behavioural issues due to invalid data raised by clients.

However, just as our client base has been accustomed to the COBOL 4 behaviour for invalid data, and COBOL 5.1 shook things up, many are already accustomed to the ZONEDATA(MIG) or ZONEDATA(NOPFD) behaviour they've seen so far, and for some, further changes are equally disruptive. Again, the end goal is COBOL 4 compatibility, so the big change last year about not cleaning sign codes under NUMPROC(MIG) is in line with that, but we understand how clients who have migrated to COBOL 5/6 and accepted new behaviour for some invalid data issues will find it hard to have the old behaviour restored.

So, exactly as you say, INVDATA(FORCENUMCMP,NOCLEANSIGN) provides the most compatibility with NUMPROC(MIG) in COBOL 4, and INVDATA(NOFORCENUMCMP,CLEANSIGN) provides the most compatibility with NUMPROC(NOPFD) or NUMPROC(PFD) in COBOL 4. But, for the clients who have adjusted to the sign codes being cleaned even under ZONEDATA(MIG), INVDATA(FORCENUMCMP,CLEANSIGN) will give that behaviour.

The one thing I wouldn't count on is that any particular INVDATA suboptions are more or less likely to give abends in general. INVDATA(NOCLEANSIGN) will give an abend for the case you're seeing, but since our goal is COBOL 4 compatibility, and not abends vs. no abends, you shouldn't rely on your INVDATA options to be a foolproof way to detect invalid data in development. I would instead recommend NUMCHECK for that, as it WILL check all relevant data items as they're read from and will abend (with the ABD suboption) when invalid data is encountered.

------------------------------
Mike Chase
Enterprise COBOL Developer
mike.chase@ca.ibm.com
------------------------------

Original Message:
Sent: Fri May 14, 2021 06:54 PM
From: Frank Swarbrick
Subject: New INVDATA compiler option

Our COBOL V4.2 compiler has NUMPROC(MIG) specified.  For V5.2 and now V6.2 (and 6.3) we have NUMPROC(NOPFD) and ZONEDATA(MIG).  Yesterday a program was implemented with a minor change, but caused major havoc.  The change itself was fine, but it looks like the behavior of the compiler changed between the last time the program was compiled and the current time.  The code in question was checking for equality between a packed decimal field and a zoned decimal field.  Unfortunately the zoned decimal field (for reasons TBD) contained only binary zeroes, rather than a valid zoned decimal value.  The code generated by our current V6.2 compiler is as follows:

PACK    162(6,R13),1416(10,R9) #                          WS-UNPACKED-BRANCH-ACCT-CP      6(10,R4),162(6,R13)   #  CHR-XREF-APPL-ACCT                               JNE     L0217                                                                     ​

We don't have access to the previous version of V6.2, but we do still have V5.2 hanging around.  Here is the code it generates:

MVC     166(10,R13),6(R4)     #  _VTS_1                  CHR-XREF-APPL-ACCTOI      175(,R13),X'0F'       #  _VTS_1                                    PACK    210(6,R13),1464(10,R8)                                             OI      215(,R13),X'0F'       #  _VTS_2                                    CP      166(10,R13),210(6,R13)                                             JNE     L0216                                                              ​

And here is V4.2:

PACK  1112(6,13),3664(10,3)   TS2=0                             WS-UNPACKED-BRANCH-ACCT-NBRCP    1198(10,2),1112(6,13)   CHR-XREF-APPL-ACCT                TS2=0                      BC    7,1052(0,11)            GN=33(002BEE)                                                ​

So the behavior of V6.2 is now (generally) the same as V4.2, but different from V5.2 (and I assume the "older" V6.2).

As you can see, V6.2/4.2 simply packs the zoned decimal field and then compares the resulting packed decimal field to the other packed decimal field.  The V5.2 version, which I am assuming was how V6.2 was behaving until a (fairly) recent update, does a "sign fixup" on both fields prior to the compare.  Thus what was previously invalid data is fixed to be valid data prior to the compare operation, thus avoiding the data exception condition being generated.

We had a similar issue about a month ago so I knew pretty much immediately how to make a short term fix to resolve the issue.  And that was to add a check to make sure the zoned decimal field is numeric before doing the actual compare.  Of course the long-term fix is to make sure the zoned decimal field has a valid numeric value prior to doing the comparison.

It looks like the latest fixes for V6.2 and V6.3 have a new compiler option, INVDATA.  It's described here:  https://www.ibm.com/support/pages/apar/PH31500.

The APAR says that if you had NUMPROC(MIG) you should now use INVDATA(FORCENUMCMP,NOCLEANSIGN) (along with, I assume, NUMPROC(NOPFD)).  This will give us the behavior shown above for V6.2 and V4.2.  Would it be the case that we could specify INVDATA(FORCENUMCMP,CLEANSIGN) if we want to get the V5.2 behavior back?  It seems like we might want to do this, even though it's different than V4.2 behavior, because we've already converted over 1000 programs to V5.2 or V6.2, and we'd prefer to avoid unexpected production abends for "previously working" V5.2 (and older V6.2) code.

On a related note, I'm wondering if it might make sense to still use INVDATA(FORCENUMCMP,NOCLEANSIGN) for development compiles, with INVDATA(FORCENUMCMP,CLEANSIGN) only for production.  This way we could get data exceptions for bad data during development, but avoid them in production.  So hopefully we would find and fix invalid data situations prior to going in to production, but also avoid being hit by data exception abends in production.

------------------------------
Frank Swarbrick
------------------------------

Thanks for all of the info, Mike.  I had forgotten about NUMCHECK until after I read this.  So now I am thinking maybe we go with INVDATA(FORCENUMCMP,CLEANSIGN) for both development and production, but use NUMCHECK as well in development (but not in production).  This is similar to what we do for SSRANGE.
Which brings me to a new request...  Using the MSG sub-option is nice because you can get all of the bad data situations reported in one run of the program, whereas with ABD you have to fix one, test it, fix the next one, test it, etc.  It seems to me, however, it would be very useful to have an option that "combines" the two behaviors.  Specifically, each individual issue would be reported as with MSG, but upon program termination the program could abend with a message indicating that there were some NUMCHECK warnings produced.
This would be useful because the warnings are easy to miss if you don't explicitly look for them.  Especially for online transactions, where there is no output you would generally look for.  If the program abended then the developer would know to look for the messages.
Any possibility of this?  Would be useful both for NUMCHECK and SSRANGE (and perhaps others I am not thinking of).

------------------------------
Frank Swarbrick
------------------------------

Original Message:
Sent: Fri May 14, 2021 07:17 PM
From: Mike Chase
Subject: New INVDATA compiler option

Hi Frank,

You're correct. The ultimate goal of INVDATA (and previously, ZONEDATA) was to provide compatibility with COBOL 4, though we're only doing this for issues that our clients alert us to; a full COBOL 4 compatibility mode is infeasible. The initial ZONEDATA(MIG) release in the COBOL 5.1 timeframe was followed fairly shortly after by ZONEDATA(NOPFD), and initially, the only behavioural difference from either was whether compares would always be numeric (matching NUMPROC(MIG) in COBOL 4), or whether they might not be (matching NUMPROC(NOPFD) or NUMPROC(PFD) in COBOL 4). As time went on, we've added more compatibility to ZONEDATA in response to new behavioural issues due to invalid data raised by clients.

However, just as our client base has been accustomed to the COBOL 4 behaviour for invalid data, and COBOL 5.1 shook things up, many are already accustomed to the ZONEDATA(MIG) or ZONEDATA(NOPFD) behaviour they've seen so far, and for some, further changes are equally disruptive. Again, the end goal is COBOL 4 compatibility, so the big change last year about not cleaning sign codes under NUMPROC(MIG) is in line with that, but we understand how clients who have migrated to COBOL 5/6 and accepted new behaviour for some invalid data issues will find it hard to have the old behaviour restored.

So, exactly as you say, INVDATA(FORCENUMCMP,NOCLEANSIGN) provides the most compatibility with NUMPROC(MIG) in COBOL 4, and INVDATA(NOFORCENUMCMP,CLEANSIGN) provides the most compatibility with NUMPROC(NOPFD) or NUMPROC(PFD) in COBOL 4. But, for the clients who have adjusted to the sign codes being cleaned even under ZONEDATA(MIG), INVDATA(FORCENUMCMP,CLEANSIGN) will give that behaviour.

The one thing I wouldn't count on is that any particular INVDATA suboptions are more or less likely to give abends in general. INVDATA(NOCLEANSIGN) will give an abend for the case you're seeing, but since our goal is COBOL 4 compatibility, and not abends vs. no abends, you shouldn't rely on your INVDATA options to be a foolproof way to detect invalid data in development. I would instead recommend NUMCHECK for that, as it WILL check all relevant data items as they're read from and will abend (with the ABD suboption) when invalid data is encountered.

------------------------------
Mike Chase
Enterprise COBOL Developer
mike.chase@ca.ibm.com

Original Message:
Sent: Fri May 14, 2021 06:54 PM
From: Frank Swarbrick
Subject: New INVDATA compiler option

Our COBOL V4.2 compiler has NUMPROC(MIG) specified.  For V5.2 and now V6.2 (and 6.3) we have NUMPROC(NOPFD) and ZONEDATA(MIG).  Yesterday a program was implemented with a minor change, but caused major havoc.  The change itself was fine, but it looks like the behavior of the compiler changed between the last time the program was compiled and the current time.  The code in question was checking for equality between a packed decimal field and a zoned decimal field.  Unfortunately the zoned decimal field (for reasons TBD) contained only binary zeroes, rather than a valid zoned decimal value.  The code generated by our current V6.2 compiler is as follows:

PACK    162(6,R13),1416(10,R9) #                          WS-UNPACKED-BRANCH-ACCT-CP      6(10,R4),162(6,R13)   #  CHR-XREF-APPL-ACCT                               JNE     L0217                                                                     ​

We don't have access to the previous version of V6.2, but we do still have V5.2 hanging around.  Here is the code it generates:

MVC     166(10,R13),6(R4)     #  _VTS_1                  CHR-XREF-APPL-ACCTOI      175(,R13),X'0F'       #  _VTS_1                                    PACK    210(6,R13),1464(10,R8)                                             OI      215(,R13),X'0F'       #  _VTS_2                                    CP      166(10,R13),210(6,R13)                                             JNE     L0216                                                              ​

And here is V4.2:

PACK  1112(6,13),3664(10,3)   TS2=0                             WS-UNPACKED-BRANCH-ACCT-NBRCP    1198(10,2),1112(6,13)   CHR-XREF-APPL-ACCT                TS2=0                      BC    7,1052(0,11)            GN=33(002BEE)                                                ​

So the behavior of V6.2 is now (generally) the same as V4.2, but different from V5.2 (and I assume the "older" V6.2).

As you can see, V6.2/4.2 simply packs the zoned decimal field and then compares the resulting packed decimal field to the other packed decimal field.  The V5.2 version, which I am assuming was how V6.2 was behaving until a (fairly) recent update, does a "sign fixup" on both fields prior to the compare.  Thus what was previously invalid data is fixed to be valid data prior to the compare operation, thus avoiding the data exception condition being generated.

We had a similar issue about a month ago so I knew pretty much immediately how to make a short term fix to resolve the issue.  And that was to add a check to make sure the zoned decimal field is numeric before doing the actual compare.  Of course the long-term fix is to make sure the zoned decimal field has a valid numeric value prior to doing the comparison.

It looks like the latest fixes for V6.2 and V6.3 have a new compiler option, INVDATA.  It's described here:  https://www.ibm.com/support/pages/apar/PH31500.

The APAR says that if you had NUMPROC(MIG) you should now use INVDATA(FORCENUMCMP,NOCLEANSIGN) (along with, I assume, NUMPROC(NOPFD)).  This will give us the behavior shown above for V6.2 and V4.2.  Would it be the case that we could specify INVDATA(FORCENUMCMP,CLEANSIGN) if we want to get the V5.2 behavior back?  It seems like we might want to do this, even though it's different than V4.2 behavior, because we've already converted over 1000 programs to V5.2 or V6.2, and we'd prefer to avoid unexpected production abends for "previously working" V5.2 (and older V6.2) code.

On a related note, I'm wondering if it might make sense to still use INVDATA(FORCENUMCMP,NOCLEANSIGN) for development compiles, with INVDATA(FORCENUMCMP,CLEANSIGN) only for production.  This way we could get data exceptions for bad data during development, but avoid them in production.  So hopefully we would find and fix invalid data situations prior to going in to production, but also avoid being hit by data exception abends in production.

------------------------------
Frank Swarbrick
------------------------------

You can open an RFE if you like and the team will discuss it. It's not impossible, but it'd require more bookkeeping and new suboptions (many clients wouldn't want that abend). It's not allowing anything that can't be done today, so I doubt it would be a high priority, but we'll be happy to consider it.

------------------------------
Mike Chase
Enterprise COBOL Developer
mike.chase@ca.ibm.com
------------------------------

Original Message:
Sent: Sat May 15, 2021 06:13 PM
From: Frank Swarbrick
Subject: New INVDATA compiler option

Thanks for all of the info, Mike.  I had forgotten about NUMCHECK until after I read this.  So now I am thinking maybe we go with INVDATA(FORCENUMCMP,CLEANSIGN) for both development and production, but use NUMCHECK as well in development (but not in production).  This is similar to what we do for SSRANGE.
Which brings me to a new request...  Using the MSG sub-option is nice because you can get all of the bad data situations reported in one run of the program, whereas with ABD you have to fix one, test it, fix the next one, test it, etc.  It seems to me, however, it would be very useful to have an option that "combines" the two behaviors.  Specifically, each individual issue would be reported as with MSG, but upon program termination the program could abend with a message indicating that there were some NUMCHECK warnings produced.
This would be useful because the warnings are easy to miss if you don't explicitly look for them.  Especially for online transactions, where there is no output you would generally look for.  If the program abended then the developer would know to look for the messages.
Any possibility of this?  Would be useful both for NUMCHECK and SSRANGE (and perhaps others I am not thinking of).

------------------------------
Frank Swarbrick

Original Message:
Sent: Fri May 14, 2021 07:17 PM
From: Mike Chase
Subject: New INVDATA compiler option

Hi Frank,

You're correct. The ultimate goal of INVDATA (and previously, ZONEDATA) was to provide compatibility with COBOL 4, though we're only doing this for issues that our clients alert us to; a full COBOL 4 compatibility mode is infeasible. The initial ZONEDATA(MIG) release in the COBOL 5.1 timeframe was followed fairly shortly after by ZONEDATA(NOPFD), and initially, the only behavioural difference from either was whether compares would always be numeric (matching NUMPROC(MIG) in COBOL 4), or whether they might not be (matching NUMPROC(NOPFD) or NUMPROC(PFD) in COBOL 4). As time went on, we've added more compatibility to ZONEDATA in response to new behavioural issues due to invalid data raised by clients.

However, just as our client base has been accustomed to the COBOL 4 behaviour for invalid data, and COBOL 5.1 shook things up, many are already accustomed to the ZONEDATA(MIG) or ZONEDATA(NOPFD) behaviour they've seen so far, and for some, further changes are equally disruptive. Again, the end goal is COBOL 4 compatibility, so the big change last year about not cleaning sign codes under NUMPROC(MIG) is in line with that, but we understand how clients who have migrated to COBOL 5/6 and accepted new behaviour for some invalid data issues will find it hard to have the old behaviour restored.

So, exactly as you say, INVDATA(FORCENUMCMP,NOCLEANSIGN) provides the most compatibility with NUMPROC(MIG) in COBOL 4, and INVDATA(NOFORCENUMCMP,CLEANSIGN) provides the most compatibility with NUMPROC(NOPFD) or NUMPROC(PFD) in COBOL 4. But, for the clients who have adjusted to the sign codes being cleaned even under ZONEDATA(MIG), INVDATA(FORCENUMCMP,CLEANSIGN) will give that behaviour.

The one thing I wouldn't count on is that any particular INVDATA suboptions are more or less likely to give abends in general. INVDATA(NOCLEANSIGN) will give an abend for the case you're seeing, but since our goal is COBOL 4 compatibility, and not abends vs. no abends, you shouldn't rely on your INVDATA options to be a foolproof way to detect invalid data in development. I would instead recommend NUMCHECK for that, as it WILL check all relevant data items as they're read from and will abend (with the ABD suboption) when invalid data is encountered.

------------------------------
Mike Chase
Enterprise COBOL Developer
mike.chase@ca.ibm.com

Original Message:
Sent: Fri May 14, 2021 06:54 PM
From: Frank Swarbrick
Subject: New INVDATA compiler option

Our COBOL V4.2 compiler has NUMPROC(MIG) specified.  For V5.2 and now V6.2 (and 6.3) we have NUMPROC(NOPFD) and ZONEDATA(MIG).  Yesterday a program was implemented with a minor change, but caused major havoc.  The change itself was fine, but it looks like the behavior of the compiler changed between the last time the program was compiled and the current time.  The code in question was checking for equality between a packed decimal field and a zoned decimal field.  Unfortunately the zoned decimal field (for reasons TBD) contained only binary zeroes, rather than a valid zoned decimal value.  The code generated by our current V6.2 compiler is as follows:

PACK    162(6,R13),1416(10,R9) #                          WS-UNPACKED-BRANCH-ACCT-CP      6(10,R4),162(6,R13)   #  CHR-XREF-APPL-ACCT                               JNE     L0217                                                                     ​

We don't have access to the previous version of V6.2, but we do still have V5.2 hanging around.  Here is the code it generates:

MVC     166(10,R13),6(R4)     #  _VTS_1                  CHR-XREF-APPL-ACCTOI      175(,R13),X'0F'       #  _VTS_1                                    PACK    210(6,R13),1464(10,R8)                                             OI      215(,R13),X'0F'       #  _VTS_2                                    CP      166(10,R13),210(6,R13)                                             JNE     L0216                                                              ​

And here is V4.2:

PACK  1112(6,13),3664(10,3)   TS2=0                             WS-UNPACKED-BRANCH-ACCT-NBRCP    1198(10,2),1112(6,13)   CHR-XREF-APPL-ACCT                TS2=0                      BC    7,1052(0,11)            GN=33(002BEE)                                                ​

So the behavior of V6.2 is now (generally) the same as V4.2, but different from V5.2 (and I assume the "older" V6.2).

As you can see, V6.2/4.2 simply packs the zoned decimal field and then compares the resulting packed decimal field to the other packed decimal field.  The V5.2 version, which I am assuming was how V6.2 was behaving until a (fairly) recent update, does a "sign fixup" on both fields prior to the compare.  Thus what was previously invalid data is fixed to be valid data prior to the compare operation, thus avoiding the data exception condition being generated.

We had a similar issue about a month ago so I knew pretty much immediately how to make a short term fix to resolve the issue.  And that was to add a check to make sure the zoned decimal field is numeric before doing the actual compare.  Of course the long-term fix is to make sure the zoned decimal field has a valid numeric value prior to doing the comparison.

It looks like the latest fixes for V6.2 and V6.3 have a new compiler option, INVDATA.  It's described here:  https://www.ibm.com/support/pages/apar/PH31500.

The APAR says that if you had NUMPROC(MIG) you should now use INVDATA(FORCENUMCMP,NOCLEANSIGN) (along with, I assume, NUMPROC(NOPFD)).  This will give us the behavior shown above for V6.2 and V4.2.  Would it be the case that we could specify INVDATA(FORCENUMCMP,CLEANSIGN) if we want to get the V5.2 behavior back?  It seems like we might want to do this, even though it's different than V4.2 behavior, because we've already converted over 1000 programs to V5.2 or V6.2, and we'd prefer to avoid unexpected production abends for "previously working" V5.2 (and older V6.2) code.

On a related note, I'm wondering if it might make sense to still use INVDATA(FORCENUMCMP,NOCLEANSIGN) for development compiles, with INVDATA(FORCENUMCMP,CLEANSIGN) only for production.  This way we could get data exceptions for bad data during development, but avoid them in production.  So hopefully we would find and fix invalid data situations prior to going in to production, but also avoid being hit by data exception abends in production.

------------------------------
Frank Swarbrick
------------------------------