Given the restriction that you should only compile once (unless you need to change the code, obviously) so that the exact same executable module is used both for all testing and in Production; and a general principle that operability is to be preferred to fastest possible (so, for example, producing formatted dumps and something that tells you the failing line of the COBOL program are essential in the event of an abend because of the reduction in Mean Time To Recover that this provides), what compiler options would you recommend with Enterprise COBOL 6.3, and why?

Obviously, some options are required for specific programs in order for them to behave as intended e.g. ARITH(EXTEND), or DYNAM/NODYNAM, because of the way they have been coded, but beyond that, would you aim to have the same core compiler options for all compiles, or have them on a case-by-case basis?

------------------------------
Charles Gaskell
------------------------------

My #1 recommendation is that you don't change compiler options when migrating between versions, and that you generally don't change compiler options without careful consideration. Some definitely change the behaviour of your programs.

I do recommend programs in the same application all use the same compiler options, and for many clients, the options are fixed site-wide anyway. The only real exception outside of cases you mentioned, where the code requires specific options, is OPTIMIZE. For clients who generally use OPT(2), if you find that specific programs are taking a long time to compile, dropping the OPT level to OPT(1) or even OPT(0) for those programs may help (though if compilation time is excessive, please open a Case). That said, some clients only use OPT(0) anyway as that can provide a better debugging experience in some non-IBM debugging tools.

I suggest using LIST instead of OFFSET, using ADATA, and for COBOL 6.4 users, using SMARTBIN. watsonx Code Assistant For z Code Optimization Advice (the Optimize step in wca4z) requires ADATA files and listings compiled with LIST to do its analysis; even if you don't currently intend to use wca4z Optimize, using options that support it will make things easier if you intend to use it in the future. A caveat: many clients use OFFSET instead of LIST, and I assume do so for good reason, as the output from LIST is more useful in troubleshooting. Perhaps some non-IBM tools require listings compiled with OFFSET? Definitely make sure it's safe to switch.

SMARTBIN stores additional data in a no-load segment along with the executable. This increases the size of your executables on disk but doesn't affect runtime memory usage or performance. That data is useful for programs that may use Automatic Binary Optimizer in the future, and z Code Optimization Advice may take advantage of it in the future as well. But again, this was only added in COBOL 6.4.

------------------------------
Mike Chase
Enterprise COBOL Developer
mike.chase@ca.ibm.com
------------------------------

Original Message:
Sent: Tue November 12, 2024 01:24 PM
From: Charles Gaskell
Subject: Recommended COBOL compiler options

Given the restriction that you should only compile once (unless you need to change the code, obviously) so that the exact same executable module is used both for all testing and in Production; and a general principle that operability is to be preferred to fastest possible (so, for example, producing formatted dumps and something that tells you the failing line of the COBOL program are essential in the event of an abend because of the reduction in Mean Time To Recover that this provides), what compiler options would you recommend with Enterprise COBOL 6.3, and why?

Obviously, some options are required for specific programs in order for them to behave as intended e.g. ARITH(EXTEND), or DYNAM/NODYNAM, because of the way they have been coded, but beyond that, would you aim to have the same core compiler options for all compiles, or have them on a case-by-case basis?

------------------------------
Charles Gaskell
------------------------------

My #1 recommendation is that you don't change compiler options when migrating between versions, and that you generally don't change compiler options without careful consideration. Some definitely change the behaviour of your programs.

I do recommend programs in the same application all use the same compiler options, and for many clients, the options are fixed site-wide anyway. The only real exception outside of cases you mentioned, where the code requires specific options, is OPTIMIZE. For clients who generally use OPT(2), if you find that specific programs are taking a long time to compile, dropping the OPT level to OPT(1) or even OPT(0) for those programs may help (though if compilation time is excessive, please open a Case). That said, some clients only use OPT(0) anyway as that can provide a better debugging experience in some non-IBM debugging tools.

I suggest using LIST instead of OFFSET, using ADATA, and for COBOL 6.4 users, using SMARTBIN. watsonx Code Assistant For z Code Optimization Advice (the Optimize step in wca4z) requires ADATA files and listings compiled with LIST to do its analysis; even if you don't currently intend to use wca4z Optimize, using options that support it will make things easier if you intend to use it in the future. A caveat: many clients use OFFSET instead of LIST, and I assume do so for good reason, as the output from LIST is more useful in troubleshooting. Perhaps some non-IBM tools require listings compiled with OFFSET? Definitely make sure it's safe to switch.

SMARTBIN stores additional data in a no-load segment along with the executable. This increases the size of your executables on disk but doesn't affect runtime memory usage or performance. That data is useful for programs that may use Automatic Binary Optimizer in the future, and z Code Optimization Advice may take advantage of it in the future as well. But again, this was only added in COBOL 6.4.

------------------------------
Mike Chase
Enterprise COBOL Developer
mike.chase@ca.ibm.com

Original Message:
Sent: Tue November 12, 2024 01:24 PM
From: Charles Gaskell
Subject: Recommended COBOL compiler options

Given the restriction that you should only compile once (unless you need to change the code, obviously) so that the exact same executable module is used both for all testing and in Production; and a general principle that operability is to be preferred to fastest possible (so, for example, producing formatted dumps and something that tells you the failing line of the COBOL program are essential in the event of an abend because of the reduction in Mean Time To Recover that this provides), what compiler options would you recommend with Enterprise COBOL 6.3, and why?

Obviously, some options are required for specific programs in order for them to behave as intended e.g. ARITH(EXTEND), or DYNAM/NODYNAM, because of the way they have been coded, but beyond that, would you aim to have the same core compiler options for all compiles, or have them on a case-by-case basis?

------------------------------
Charles Gaskell
------------------------------

Given the restriction that you should only compile once (unless you need to change the code, obviously) so that the exact same executable module is used both for all testing and in Production; and a general principle that operability is to be preferred to fastest possible (so, for example, producing formatted dumps and something that tells you the failing line of the COBOL program are essential in the event of an abend because of the reduction in Mean Time To Recover that this provides), what compiler options would you recommend with Enterprise COBOL 6.3, and why?

Obviously, some options are required for specific programs in order for them to behave as intended e.g. ARITH(EXTEND), or DYNAM/NODYNAM, because of the way they have been coded, but beyond that, would you aim to have the same core compiler options for all compiles, or have them on a case-by-case basis?

Compiling twice is an excellent choice!  OPT(0) for unit test, then OPT(2) for quality assurance test and production!  This method is extremely common, and many use SSRANGE and NUMCHECK in the dev/unit test compile as well, then turn those off for production.

Given the restriction that you should only compile once (unless you need to change the code, obviously) so that the exact same executable module is used both for all testing and in Production; and a general principle that operability is to be preferred to fastest possible (so, for example, producing formatted dumps and something that tells you the failing line of the COBOL program are essential in the event of an abend because of the reduction in Mean Time To Recover that this provides), what compiler options would you recommend with Enterprise COBOL 6.3, and why?

Obviously, some options are required for specific programs in order for them to behave as intended e.g. ARITH(EXTEND), or DYNAM/NODYNAM, because of the way they have been coded, but beyond that, would you aim to have the same core compiler options for all compiles, or have them on a case-by-case basis?

For those that recommend compiling twice, how do you ensure that the source used to do the second compile is the same as the source that you used to do the first compile? Even if you can guarantee that the main source hasn't changed (because you've stored it away in your SCM tool for example), I don't see how it's possible to ensure that Copybooks and DCLGENs which are pulled in haven't changed between the two compiles, potentially in ways that affect how the load load behaves, functionally. At what point in the Development Lifecycle is the second compile normally done - is it after all testing has been completed?

Also my understanding whilst options such as SSRANGE and NUMCHECK will pick up invalid subscript range settings and invalid data at run time, this can be very much dependent on the quality of the test data, and just because these are set in the original compile, the condition will only be caught if the test data causes it to be triggered, so a clean set of tests doesn't mean that you won't get problems when you run it in Production, using Production data - it could simply be that the testing data used don't include invalid data, or don't cause the subscript range to be exceeded. Therefore unless also kept on in the Production version, SSRANGE and NUMCHECK are only of limited use if only used in "testing" versions of the load module.

Compiling twice is an excellent choice!  OPT(0) for unit test, then OPT(2) for quality assurance test and production!  This method is extremely common, and many use SSRANGE and NUMCHECK in the dev/unit test compile as well, then turn those off for production.

Given the restriction that you should only compile once (unless you need to change the code, obviously) so that the exact same executable module is used both for all testing and in Production; and a general principle that operability is to be preferred to fastest possible (so, for example, producing formatted dumps and something that tells you the failing line of the COBOL program are essential in the event of an abend because of the reduction in Mean Time To Recover that this provides), what compiler options would you recommend with Enterprise COBOL 6.3, and why?

Obviously, some options are required for specific programs in order for them to behave as intended e.g. ARITH(EXTEND), or DYNAM/NODYNAM, because of the way they have been coded, but beyond that, would you aim to have the same core compiler options for all compiles, or have them on a case-by-case basis?

Charles,

  Wow, if people have to worry about their code staying safe in their SCM, they need a new SCM!

Of course the code will not change between compiles, unless someone changes it!  That would be

dumb of course.  Many COBOL users have changed to a 2-compile 2-test development process after

learning about it when migrating to COBOL V6.  It is a beautiful thing!  It allows a more robust development

environment and results in better code.  I don't think it is relevant to bring up the quality of testing data

here, since that has always been a challenge, more for some clients than others.  Regardless of how much

or how high-quality the data, it is still a good idea to develop with SSRANGE and NUMCHECK, and especially

with the new LSACHECK compiler option.  None of these help until tests are run.

  By the way, copybooks and DCLGENS are also protected by source control, so they won't change without users being notified

and aware.   I am surprised that you have not heard of this 2 compile development process, it is VERY common!

Tom

For those that recommend compiling twice, how do you ensure that the source used to do the second compile is the same as the source that you used to do the first compile? Even if you can guarantee that the main source hasn't changed (because you've stored it away in your SCM tool for example), I don't see how it's possible to ensure that Copybooks and DCLGENs which are pulled in haven't changed between the two compiles, potentially in ways that affect how the load load behaves, functionally. At what point in the Development Lifecycle is the second compile normally done - is it after all testing has been completed?

Also my understanding whilst options such as SSRANGE and NUMCHECK will pick up invalid subscript range settings and invalid data at run time, this can be very much dependent on the quality of the test data, and just because these are set in the original compile, the condition will only be caught if the test data causes it to be triggered, so a clean set of tests doesn't mean that you won't get problems when you run it in Production, using Production data - it could simply be that the testing data used don't include invalid data, or don't cause the subscript range to be exceeded. Therefore unless also kept on in the Production version, SSRANGE and NUMCHECK are only of limited use if only used in "testing" versions of the load module.

Compiling twice is an excellent choice!  OPT(0) for unit test, then OPT(2) for quality assurance test and production!  This method is extremely common, and many use SSRANGE and NUMCHECK in the dev/unit test compile as well, then turn those off for production.

Given the restriction that you should only compile once (unless you need to change the code, obviously) so that the exact same executable module is used both for all testing and in Production; and a general principle that operability is to be preferred to fastest possible (so, for example, producing formatted dumps and something that tells you the failing line of the COBOL program are essential in the event of an abend because of the reduction in Mean Time To Recover that this provides), what compiler options would you recommend with Enterprise COBOL 6.3, and why?

Obviously, some options are required for specific programs in order for them to behave as intended e.g. ARITH(EXTEND), or DYNAM/NODYNAM, because of the way they have been coded, but beyond that, would you aim to have the same core compiler options for all compiles, or have them on a case-by-case basis?

I didn't say that I hadn't heard of the 2-compiule development process, rather that where I work, we have a policy that what you test is exactly what you put live. A corollary of this is that if you need to recompile then  you either need to do sufficient retesting to prove that the program still functions the way that it did when you originally tested it, or that you sign off to accept the risk associated with not retesting it.

I agree that it's a very small risk, especially if you use a very well-established SCM (as we do), but that risk is still non-zero.

Our developers work in a series of shared environments containing multiple applications (continuous integration if you like) with copybooks typically representing the interfaces between different applications, each of which is being developed separately as well being tested to ensure interoperability. If application B is not allowed to start development of a copybook which they own but which is used by application A until application A has done their recompiles at the end of their testing, then this impacts on how fast application B can develop. If you have a principle that you only recompile when you have to (and recompilation triggers retesting) then as soon as Application A has compiled at the start of their testing, application B is free to start development of a new version of the copybook.

You also don't say what happens if your SCM tool flags up that something has changed between the "testing" compile and the "production-ready" compile, because of a legitimate change that another team has done. Do you force everything to roll back to the versions of copybooks, DCLGENs and linked-in artefacts, and fail the process if there are differences? Or force some level of retesting? Or another approach? Hope does it help if users are notified and aware that something has change between the first compile and the second compile? What do they do differently as a result of the notification?

You talk about a "2-compile, 2-test development process" What do you mean by "2-test"? Depending on the size of the development, one developers may well have at least 3 or 4 levels of 'functional' testing - Unit, Unit-regression, System, Integration, (Integrated) Regression, User Acceptance, each with a different focus. At which stage do you recommend doing the 2nd compile?

I'm not saying that it's not a development paradigm that can work. As you say, plenty of people use it, but I would like to understand better the pros and cons of the approach, as it's radically different to how we currently work, which is why I started my original question with "Given the restriction that you should only compile once". It's a bit like asking the question "how do I get from A to B by public transport?" and be told "you should use private car". It may well be a valid way to get from A to B, but it's not answering the question that was asked.

I've not come across the LSACHECK COBOL compiler option before. Is it brand new? Or a typo? Where can I find out more information about it?

Charles,

  Wow, if people have to worry about their code staying safe in their SCM, they need a new SCM!

Of course the code will not change between compiles, unless someone changes it!  That would be

dumb of course.  Many COBOL users have changed to a 2-compile 2-test development process after

learning about it when migrating to COBOL V6.  It is a beautiful thing!  It allows a more robust development

environment and results in better code.  I don't think it is relevant to bring up the quality of testing data

here, since that has always been a challenge, more for some clients than others.  Regardless of how much

or how high-quality the data, it is still a good idea to develop with SSRANGE and NUMCHECK, and especially

with the new LSACHECK compiler option.  None of these help until tests are run.

  By the way, copybooks and DCLGENS are also protected by source control, so they won't change without users being notified

and aware.   I am surprised that you have not heard of this 2 compile development process, it is VERY common!

Tom

For those that recommend compiling twice, how do you ensure that the source used to do the second compile is the same as the source that you used to do the first compile? Even if you can guarantee that the main source hasn't changed (because you've stored it away in your SCM tool for example), I don't see how it's possible to ensure that Copybooks and DCLGENs which are pulled in haven't changed between the two compiles, potentially in ways that affect how the load load behaves, functionally. At what point in the Development Lifecycle is the second compile normally done - is it after all testing has been completed?

Also my understanding whilst options such as SSRANGE and NUMCHECK will pick up invalid subscript range settings and invalid data at run time, this can be very much dependent on the quality of the test data, and just because these are set in the original compile, the condition will only be caught if the test data causes it to be triggered, so a clean set of tests doesn't mean that you won't get problems when you run it in Production, using Production data - it could simply be that the testing data used don't include invalid data, or don't cause the subscript range to be exceeded. Therefore unless also kept on in the Production version, SSRANGE and NUMCHECK are only of limited use if only used in "testing" versions of the load module.

Compiling twice is an excellent choice!  OPT(0) for unit test, then OPT(2) for quality assurance test and production!  This method is extremely common, and many use SSRANGE and NUMCHECK in the dev/unit test compile as well, then turn those off for production.

Given the restriction that you should only compile once (unless you need to change the code, obviously) so that the exact same executable module is used both for all testing and in Production; and a general principle that operability is to be preferred to fastest possible (so, for example, producing formatted dumps and something that tells you the failing line of the COBOL program are essential in the event of an abend because of the reduction in Mean Time To Recover that this provides), what compiler options would you recommend with Enterprise COBOL 6.3, and why?

Obviously, some options are required for specific programs in order for them to behave as intended e.g. ARITH(EXTEND), or DYNAM/NODYNAM, because of the way they have been coded, but beyond that, would you aim to have the same core compiler options for all compiles, or have them on a case-by-case basis?

Charles,

Not unlike you, I could not find LSACHECK in my April 2024 edition of the Programming Guide.  I did find it as an APAR with a CLOSED date of 21 November, which on my calendar is yesterday.

I read the description, and to me it is not dissimilar to PARMCHECK in that it would generate a lot of overhead that can be avoided by testing.  When PARMCHECK was introduced in COBOL 6 back in 2017 we tried it because it sounded like a great idea.   But in fact it only reports the problem, it does not prevent it.  For me both of the options,  MSG and ABD, have the same effect in practice because if a subroutine overran your storage, there was not much you could do about it.   And maybe I'm missing something, but you would be very fortunate if the offending parameter happens to be the last item in your Working-Storage and not in the middle.

This is a case where the two-compile method may be useful and I recommend it.  First, have the programmers test with SSRANGE and the other high-overhead options, and then when the problems are fixed, recompile without them.   Of course you need good test data.  I agree that the Integration Test version of the program is the one that should be moved up the line to Production.

I didn't say that I hadn't heard of the 2-compiule development process, rather that where I work, we have a policy that what you test is exactly what you put live. A corollary of this is that if you need to recompile then  you either need to do sufficient retesting to prove that the program still functions the way that it did when you originally tested it, or that you sign off to accept the risk associated with not retesting it.

I agree that it's a very small risk, especially if you use a very well-established SCM (as we do), but that risk is still non-zero.

Our developers work in a series of shared environments containing multiple applications (continuous integration if you like) with copybooks typically representing the interfaces between different applications, each of which is being developed separately as well being tested to ensure interoperability. If application B is not allowed to start development of a copybook which they own but which is used by application A until application A has done their recompiles at the end of their testing, then this impacts on how fast application B can develop. If you have a principle that you only recompile when you have to (and recompilation triggers retesting) then as soon as Application A has compiled at the start of their testing, application B is free to start development of a new version of the copybook.

You also don't say what happens if your SCM tool flags up that something has changed between the "testing" compile and the "production-ready" compile, because of a legitimate change that another team has done. Do you force everything to roll back to the versions of copybooks, DCLGENs and linked-in artefacts, and fail the process if there are differences? Or force some level of retesting? Or another approach? Hope does it help if users are notified and aware that something has change between the first compile and the second compile? What do they do differently as a result of the notification?

You talk about a "2-compile, 2-test development process" What do you mean by "2-test"? Depending on the size of the development, one developers may well have at least 3 or 4 levels of 'functional' testing - Unit, Unit-regression, System, Integration, (Integrated) Regression, User Acceptance, each with a different focus. At which stage do you recommend doing the 2nd compile?

I'm not saying that it's not a development paradigm that can work. As you say, plenty of people use it, but I would like to understand better the pros and cons of the approach, as it's radically different to how we currently work, which is why I started my original question with "Given the restriction that you should only compile once". It's a bit like asking the question "how do I get from A to B by public transport?" and be told "you should use private car". It may well be a valid way to get from A to B, but it's not answering the question that was asked.

I've not come across the LSACHECK COBOL compiler option before. Is it brand new? Or a typo? Where can I find out more information about it?

Charles,

  Wow, if people have to worry about their code staying safe in their SCM, they need a new SCM!

Of course the code will not change between compiles, unless someone changes it!  That would be

dumb of course.  Many COBOL users have changed to a 2-compile 2-test development process after

learning about it when migrating to COBOL V6.  It is a beautiful thing!  It allows a more robust development

environment and results in better code.  I don't think it is relevant to bring up the quality of testing data

here, since that has always been a challenge, more for some clients than others.  Regardless of how much

or how high-quality the data, it is still a good idea to develop with SSRANGE and NUMCHECK, and especially

with the new LSACHECK compiler option.  None of these help until tests are run.

  By the way, copybooks and DCLGENS are also protected by source control, so they won't change without users being notified

and aware.   I am surprised that you have not heard of this 2 compile development process, it is VERY common!

Tom

For those that recommend compiling twice, how do you ensure that the source used to do the second compile is the same as the source that you used to do the first compile? Even if you can guarantee that the main source hasn't changed (because you've stored it away in your SCM tool for example), I don't see how it's possible to ensure that Copybooks and DCLGENs which are pulled in haven't changed between the two compiles, potentially in ways that affect how the load load behaves, functionally. At what point in the Development Lifecycle is the second compile normally done - is it after all testing has been completed?

Also my understanding whilst options such as SSRANGE and NUMCHECK will pick up invalid subscript range settings and invalid data at run time, this can be very much dependent on the quality of the test data, and just because these are set in the original compile, the condition will only be caught if the test data causes it to be triggered, so a clean set of tests doesn't mean that you won't get problems when you run it in Production, using Production data - it could simply be that the testing data used don't include invalid data, or don't cause the subscript range to be exceeded. Therefore unless also kept on in the Production version, SSRANGE and NUMCHECK are only of limited use if only used in "testing" versions of the load module.

Compiling twice is an excellent choice!  OPT(0) for unit test, then OPT(2) for quality assurance test and production!  This method is extremely common, and many use SSRANGE and NUMCHECK in the dev/unit test compile as well, then turn those off for production.

Given the restriction that you should only compile once (unless you need to change the code, obviously) so that the exact same executable module is used both for all testing and in Production; and a general principle that operability is to be preferred to fastest possible (so, for example, producing formatted dumps and something that tells you the failing line of the COBOL program are essential in the event of an abend because of the reduction in Mean Time To Recover that this provides), what compiler options would you recommend with Enterprise COBOL 6.3, and why?

Obviously, some options are required for specific programs in order for them to behave as intended e.g. ARITH(EXTEND), or DYNAM/NODYNAM, because of the way they have been coded, but beyond that, would you aim to have the same core compiler options for all compiles, or have them on a case-by-case basis?

Compiling twice is an excellent choice!  OPT(0) for unit test, then OPT(2) for quality assurance test and production!  This method is extremely common, and many use SSRANGE and NUMCHECK in the dev/unit test compile as well, then turn those off for production.

Given the restriction that you should only compile once (unless you need to change the code, obviously) so that the exact same executable module is used both for all testing and in Production; and a general principle that operability is to be preferred to fastest possible (so, for example, producing formatted dumps and something that tells you the failing line of the COBOL program are essential in the event of an abend because of the reduction in Mean Time To Recover that this provides), what compiler options would you recommend with Enterprise COBOL 6.3, and why?

Obviously, some options are required for specific programs in order for them to behave as intended e.g. ARITH(EXTEND), or DYNAM/NODYNAM, because of the way they have been coded, but beyond that, would you aim to have the same core compiler options for all compiles, or have them on a case-by-case basis?

What type of programs expected to have significant improvements between OPT(2) and OPT(1)? Is there any risk to have different runtime results changing from OPT(0 or 1) and OPT(2)? 

Compiling twice is an excellent choice!  OPT(0) for unit test, then OPT(2) for quality assurance test and production!  This method is extremely common, and many use SSRANGE and NUMCHECK in the dev/unit test compile as well, then turn those off for production.

Given the restriction that you should only compile once (unless you need to change the code, obviously) so that the exact same executable module is used both for all testing and in Production; and a general principle that operability is to be preferred to fastest possible (so, for example, producing formatted dumps and something that tells you the failing line of the COBOL program are essential in the event of an abend because of the reduction in Mean Time To Recover that this provides), what compiler options would you recommend with Enterprise COBOL 6.3, and why?

Obviously, some options are required for specific programs in order for them to behave as intended e.g. ARITH(EXTEND), or DYNAM/NODYNAM, because of the way they have been coded, but beyond that, would you aim to have the same core compiler options for all compiles, or have them on a case-by-case basis?

What type of programs expected to have significant improvements between OPT(2) and OPT(1)? Is there any risk to have different runtime results changing from OPT(0 or 1) and OPT(2)? 

Compiling twice is an excellent choice!  OPT(0) for unit test, then OPT(2) for quality assurance test and production!  This method is extremely common, and many use SSRANGE and NUMCHECK in the dev/unit test compile as well, then turn those off for production.

Given the restriction that you should only compile once (unless you need to change the code, obviously) so that the exact same executable module is used both for all testing and in Production; and a general principle that operability is to be preferred to fastest possible (so, for example, producing formatted dumps and something that tells you the failing line of the COBOL program are essential in the event of an abend because of the reduction in Mean Time To Recover that this provides), what compiler options would you recommend with Enterprise COBOL 6.3, and why?

Obviously, some options are required for specific programs in order for them to behave as intended e.g. ARITH(EXTEND), or DYNAM/NODYNAM, because of the way they have been coded, but beyond that, would you aim to have the same core compiler options for all compiles, or have them on a case-by-case basis?

Hi,

If I may offer my opinion on this subject.

The problem with level 2 optimization is that the binary code can be reorganized beyond the boundary of the original source COBOL instruction: this makes debugging complicated.
We use a debugger that doesn't require any specific compilation options (just LIST or OFFSET, no use of the TEST option), because it works from the compilation listing (it can also work with the dwarf information included in the loadmodule).
When the code is compiled using level 2 optimization, the execution in step-by-step mode is very surprising: the instructions are apparently not executed in order, there may be returns to instructions that have already been executed... so the program logic becomes very difficult to follow.
This can also be misleading during abends and dump captures: there is no guarantee that the COBOL instruction in problem is actually the one that created the problem; the values ​​of indices or indexes to access memory tables view at dump level may not be the values ​​actually used, (the true value is in a register of CPU).
Program profiling (Application Performance Analyzer) can also mask the instructions responsible for CPU consumption.
For these reasons, we have chosen to limit ourselves to optimization level 1, which allows us to debug programs delivered to production (and reproduce problems with the same binaries).
Note that for the program debugging phase, we compile without optimization, and as soon as this phase is completed, we recompile in optimization level 1. It is the result of this compilation that goes through all the qualification stages before the final production release stage, without recompilation (what has been qualified is what is delivered to production).

Furthermore, when profiling program execution, we see that CPU consumption related to program code is rather limited because a large portion of CPU consumption comes from middleware (CICS, Db2, etc.), LE/COBOL runtimes, and the system itself.
Therefore, a level 1 or level 2 compilation will have little impact on total consumption, except in specific cases of intensive calculations, and perhaps intensive use of memory tables. In this case, the main focus should be on COBOL source code review by optimizing the algorithms used, using formats adapted to the nature of the operations (packed decimal for arithmetic operations while avoiding mixing the number of digits, binary for memory addresses, using indexes rather than subscripts, etc.). In other words: think about how to code rather than relying on the optimizations provided by the compiler.

On the other hand, using optimization level 1, compared to no optimization, provides a real and measurable benefit, particularly with PERFORM inlining, which is always active, even with the NOINLINE option, on single PERFORMs (Paragraphs or Sections called from a single location in the program). This allows you to benefit from code caching at the processor level and avoids memory page reloads (in my opinion, more so than the argument of avoiding saving return addresses and avoiding jumps).

We leveraged Level 1 optimization in the development of a COBOL framework (a set of technical Copybooks) that breaks down resource access into logical and physical levels:
- the program calls a logical level, which is agnostic to the resource type and reusable for all resource types,
- the logical level calls the physical level, which is dependent on the resource type (file, Db2, etc.), but is not concerned with the access logic,
- in the case of Db2, the physical level calls the SQL statement contained in the program, while the logical and physical levels are externalized into Copybooks for shared use.
This facilitates (and secures, standardizes) program writing, and the program (the developer) only needs to worry about its processing logic and delegates all technical implementation to the framework. So a resource access can be broken down into 3 levels of PERFORM, which might seem problematic from a performance point of view, but thanks to PERFORM inlining all this code is grouped together at compile time and behaves as if everything had been coded as a block.

What type of programs expected to have significant improvements between OPT(2) and OPT(1)? Is there any risk to have different runtime results changing from OPT(0 or 1) and OPT(2)? 

Compiling twice is an excellent choice!  OPT(0) for unit test, then OPT(2) for quality assurance test and production!  This method is extremely common, and many use SSRANGE and NUMCHECK in the dev/unit test compile as well, then turn those off for production.

Given the restriction that you should only compile once (unless you need to change the code, obviously) so that the exact same executable module is used both for all testing and in Production; and a general principle that operability is to be preferred to fastest possible (so, for example, producing formatted dumps and something that tells you the failing line of the COBOL program are essential in the event of an abend because of the reduction in Mean Time To Recover that this provides), what compiler options would you recommend with Enterprise COBOL 6.3, and why?

Obviously, some options are required for specific programs in order for them to behave as intended e.g. ARITH(EXTEND), or DYNAM/NODYNAM, because of the way they have been coded, but beyond that, would you aim to have the same core compiler options for all compiles, or have them on a case-by-case basis?

Given the restriction that you should only compile once (unless you need to change the code, obviously) so that the exact same executable module is used both for all testing and in Production; and a general principle that operability is to be preferred to fastest possible (so, for example, producing formatted dumps and something that tells you the failing line of the COBOL program are essential in the event of an abend because of the reduction in Mean Time To Recover that this provides), what compiler options would you recommend with Enterprise COBOL 6.3, and why?

Obviously, some options are required for specific programs in order for them to behave as intended e.g. ARITH(EXTEND), or DYNAM/NODYNAM, because of the way they have been coded, but beyond that, would you aim to have the same core compiler options for all compiles, or have them on a case-by-case basis?