Dears,
Recently I upgraded ACE 11.0.0.20 to 11.0.0.22. As usual after the upgrade I performed a number of tests, including HTTPS calls. All worked fine. The next day some .Net and Java clients reported SSL handshakes are failing. Postman and SAP work without any problem, however for some clients SSL handshake messages became too big. Below is the exception I got from one ACE application calling another application in the same integration server.
javax.net.ssl.SSLProtocolException: The size of the handshake message (90274) exceeds the maximum allowed size (32768)
I restarted the integration server from 11.0.0.20 console and everything works fine again. I enabled TLS Trace and compared SSL handshakes produced by both 11.0.0.20 and 11.0.0.22 versions. Indeed the latter is quite big. TLS 1.2 Handshake packet containts a certificate list with the same certificate listed over 100 times (the serial number is identical). It is the self-signed certificate I configured myself using sha512WithRSAEncryption algorithm and 2048 bit public-key. The TLS Trace log from 11.0.0.20 has the same certificate listed only once.
I noticed 11.0.0.20 uses OpenSSL 1.1.1q 5 Jul 2022 while 11.0.0.22 uses OpenSSL 3.0.9+quic 30 May 2023.
Both TLS 1.2 and 1.3 are enabled for integration server.
I would be glad if anybody could give me a hint where the issue could be, I'm out of ideas.
------------------------------
Best regards,
Patrick
------------------------------