Technical background:
The webMethods SAP adapter uses the SAP JCo library as underlying middleware to connect to SAP ECC and S4/HANA systems.
The JCo library has been designed as a closed "black box" with an API which does not allow the host application (here: WmSAP) to handle 'physical' SAP connections itself, including the start/stop and connection error handling. Any such operations can only be conducted within the JCo library - and this general limitation cannot be defeated.
Low-level JCo connection errors cause the interruption of WmSAP client and listener connections, which may be in some cases be repairable and in other cases not. The following sections describe how to identify and handle such errors.
Which SAP JCo version to use:
JCo 3.0.x is out of support and has been replaced by version 3.1. Unfortunately, the first releases 3.1.2 - 3.1.11 show connection issues, listener start/stop problems and may fail with multi-threaded outbound qRFC/tRFC calls, which includes outbound IDoc operations.
JCo 3.1.12 resolves these issues, and it is therefore strongly recommended to upgrade any older version to 3.1.12 or higher. WmSAP 10.1 Fix14 (and higher) will require 3.1.11 (or higher) and will not start with previous JCo versions.
Typical JCo connection error messages:
The following typical error messages in the WmSAP logs indicate the occurrence of SAP JCo or CPIC low-level connection errors:
"Error in RFC Listener ... with status "DEAD" and connection..."
"Error in RFC Listener ... with status "ALIVE" and connection..."
"ERROR partner '...' not reached"
"ERROR connection to partner '...' broken"
"ERRNO TEXT Connection reset by peer"
"ERRNO TEXT Connection refused"
"ERRNO TEXT Connection timed out"
"ERROR no conversation found with id"
"no SAP ErrInfo available"
"Invalid rfcHandle==null encountered in listen method"
"Failed to register JCoServer connection: Connect to SAP gateway failed"
"JCO_ERROR_TIMEOUT - Timeout occurred while logging on to an ABAP system"
General advice for JCo connection errors:
Some connections errors, especially time-out related errors, may be resolved by increasing the CPIC keep-alive and pong timeout values. This can be achieved by setting the following watt property values in the Integration Server Admin UI > Settings > Extended:
watt.sap.rfc.cpic.ka.period=30
watt.sap.rfc.cpic.ka.timeout=800
watt.sap.rfc.cpic.pong.timeout=120
The Integration Server needs to be restarted after the values have been set.
How to handle specific JCo error messages:
- Error message: "This is caused by either
a) an erroneous server configuration”
b) the backend system has been shutdown”, or
c) network problems."
If the WmSAP log or JCo trace files contain such error messages, then proceed as indicated by the message text and verify the following points:
a) the WmSAP RFC connection attributes are still correct
b) the SAP backend system and SAP Gateway instances are still up & running
c) the network and firewalls between WmSAP and the connected SAP systems work as expected - for this please see the SAP Network FAQ section below
- Error message: "no SAP ErrInfo available"
The "no SAP ErrInfo available" error usually indicates a low-level network issue in which the connection has been interrupted without an error message from the TCP/IP stack or CPIC layer or the connection partner, so that the JCo library cannot determine the root cause.
- Error message: "Connection timed out"
If the connected SAP system is not down, then "Connection timed out" errors are almost always caused by network issues. This error message comes from the TCP/IP stack of the OS and can be caused by missing TCP/IP packages.
SAP Network FAQ and Tools:
This link to SAP's Network Interface FAQ provides hints regarding SAP network and SAP Gateway issues, analysis and testing with the NIPING tool:
https://wiki.scn.sap.com/wiki/display/SI/FAQ+Network+Interface
Microsoft blog entry describing the usage of NIPING:
https://blogs.msdn.microsoft.com/saponsqlserver/2011/04/26/niping-a-useful-tool-from-sap/
The FAQ and NIPING tool should be used by the network administration team to identify the root cause in the network(s) and firewall(s) between the WmSAP instance and the SAP backend systems and Gateway.
Cookbook: How to proceed with SAP JCo/CPIC connection errors:
1. Scan the WmSAP logs for typical JCo low-level connection error messages (see examples above).
2. Check if the affected SAP Gateway and ECC systems are running and accessible, and that the connection errors have not been caused by downtimes of the SAP systems.
3. Check if increasing the keep-alive and pong parameters described in section “General advice for JCo connection errors” resolves the issue.
4. If there were no SAP downtimes, the network team needs to check if there are network problems. For this, download the NIPING tool and follow the steps and procedures in the SAP Network FAQ and the Microsoft NIPING blog.
5. If the previous steps did not resolve the issue, then a new SAP support ticket should be created for the BC_MID_CON_JCO component.
IMPORTANT: SAP support requires the latest JCo version to be installed, otherwise the ticket will not be processed.
Requirements: A detailed error description and the detailed WmSAP logs and JCo trace files covering the incident need to be attached to the ticket. For this, the log level of the WmSAP JCo log facility must be set to 'Trace' and the 'RFC Trace' connection attribute has to be enabled for all affected RFC connections and listeners:
The JCo trace and error files can be found in <IS instance>\packages\WmSAP\logs.
IMPORTANT: Setting the log level of the JCo log facility to 'Trace' may increase the memory consumption significantly.
There have been cases of up to 30-40% increase in memory consumption, causing memory issues and even OOM exceptions, so this setting needs to be handled carefully in production environments, and may require an increase of the IS Java max heap size setting.
6. In some cases, SAP support may also ask for the CPIC trace files. To enable the CPIC traces, you need to set the IS configuration switch "watt.sap.debug.CPICtrace=3" in the IS Admin UI Extended, and then change the WmSAP log level once – no restart is required, the log level change will trigger the creation of CPIC trace files.
The CPIC trace files can also be found in <IS instance>\packages\WmSAP\logs.
Why SAP support tickets need to be created by the WmSAP customer and not by IBM/webMethods:
IMPORTANT: SAP support hands out hot fixes only to customers who have created a related support ticket. It is therefore mandatory for WmSAP customers now to create dedicated SAP support tickets - otherwise SAP hot fixes cannot be received. SAP support also does not accept support tickets which have been created by 3rd parties (like IBM/webMethods) on behalf of SAP customers.
Beside of this, it is best practice and recommended approach that WmSAP customers open support tickets for their issues directly with SAP, because SAP support will need data like the problem description, JCo log and trace files, SAP Gateway traces etc. directly from the servers of the customer, and sometimes SAP support also needs a remote connection to the customer's SAP systems.
To accelerate the solution of the issue, it is therefore recommended that WmSAP customers open their support tickets directly with SAP.
SAP Notes and Support Links:
Connection Termination: https://help.sap.com/saphelp_snc700_ehp01/helpdata/en/48/6b5bf4b74c07bee10000000a42189d/content.htm?no_cache=true
Initializing connection to SAP painfully slow:
https://answers.sap.com/questions/1541101/initializing-connection-to-sap-painfully-slow.html
WSAECONNRESET Connection reset by peer: https://answers.sap.com/questions/6884846/wsaeconnreset-connection-reset-by-peer.html
Gateway connections remain open because of firewall: https://wiki.scn.sap.com/wiki/display/ABAPConn/Gateway+connections+remain+open+because+of+firewall