This is because the cipher suites which uses RSA key exchange are disabled by default
since 8.5.5.24 and 9.0.5.15.
This change is also documented as a known issue
https://www.ibm.com/support/pages/node/6965774#notice
```
9.0.5.15
IBM HTTP Server: SSL ciphers using RSA key exchange are no longer enabled by default.
```
This change was introduced by the APAR below.
https://www.ibm.com/support/pages/apar/PH51473
It's likely that the client supports the cipher suites using RSA key exchange only.
So as a tentative measure, you can re-enable the RSA key with the way described in PH51473.
By increasing IHS loglevel, we can see what cipher suites are enabled for each protocol,
what protocol is being used, what cipher list is sent by the client as client_hello
and what cipher is finally used.
Here is the output when the client intentionally uses the cipher using RSA key exchange.
# openssl ciphers -v | grep Kx=RSA
AES256-GCM-SHA384 TLSv1.2 Kx=RSA Au=RSA Enc=AESGCM(256) Mac=AEAD
:
# openssl s_client -connect localhost:443 -tls1_2 -cipher AES256-GCM-SHA384
9.0.5.14 output:
[Tue Sep 26 17:20:03.007664 2023] [ibm_ssl:debug] [pid 300616:tid 139645610354432] [client 127.0.0.1:46594] [7f01b8002ba0] [300616] SSL handshake initiated [127.0.0.1:46594 -> 127.0.0.1:443] fd 11 userdata 7f01c6ffdcf0
[Tue Sep 26 17:20:03.007873 2023] [ibm_ssl:debug] [pid 300616:tid 139645610354432] mod_ibm_ssl.c(1407): About to handshake: SSLV2 not enabled, SSLV3 not enabled, TLSv10 not enabled, TLSv11 not enabled, TLSv12 ciphers='TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA256,TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA', TLSv13 ciphers='TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256', FIPS is disabled
:
[Tue Sep 26 17:20:03.016811 2023] [ibm_ssl:trace2] [pid 300616:tid 139645610354432] [client 127.0.0.1:46594] <client_hello>
:
[Tue Sep 26 17:20:03.016861 2023] [ibm_ssl:trace2] [pid 300616:tid 139645610354432] [client 127.0.0.1:46594] TLSV12
:
[Tue Sep 26 17:20:03.016994 2023] [ibm_ssl:trace2] [pid 300616:tid 139645610354432] [client 127.0.0.1:46594] cipher_suites
[Tue Sep 26 17:20:03.017002 2023] [ibm_ssl:trace2] [pid 300616:tid 139645610354432] [client 127.0.0.1:46594] Length: 04
[Tue Sep 26 17:20:03.017009 2023] [ibm_ssl:trace2] [pid 300616:tid 139645610354432] [client 127.0.0.1:46594] 00 9D 00 FF ....
[Tue Sep 26 17:20:03.017023 2023] [ibm_ssl:trace2] [pid 300616:tid 139645610354432] [client 127.0.0.1:46594] tls_rsa_with_aes_256_gcm_sha384,tls_ri_scsv
:
[Tue Sep 26 17:20:03.017435 2023] [ibm_ssl:info] [pid 300616:tid 139645610354432] [client 127.0.0.1:46594] [7f01b8002ba0] [300616] SSL2001I: SSL handshake complete, Session ID: HgD6bu1LfvUEqiAGO6NmfJYQVkD2KMcjmukFmxBf4vo= (new) [127.0.0.1:46594 -> 127.0.0.1:443] (TLSV12,TLS_RSA_WITH_AES_256_GCM_SHA384) 7ms
9.0.5.16 output:
As you see, TLS_RSA_xxxx are no longer listed in the available cipher list and the SSL handshake was not completed.
[Tue Sep 26 17:41:02.002188 2023] [ibm_ssl:debug] [pid 302515:tid 140633584432896] [client 127.0.0.1:49088] [7fe7c0002ba0] [302515] SSL handshake initiated [127.0.0.1:49088 -> 127.0.0.1:443] fd 12 userdata 7fe7ced7ece0
[Tue Sep 26 17:41:02.002481 2023] [ibm_ssl:debug] [pid 302515:tid 140633584432896] mod_ibm_ssl.c(1455): About to handshake: SSLV2 not enabled, SSLV3 not enabled, TLSv10 not enabled, TLSv11 not enabled, TLSv12 ciphers='TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA', TLSv13 ciphers='TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256', FIPS is disabled
:
[Tue Sep 26 17:41:02.005427 2023] [ibm_ssl:warn] [pid 302515:tid 140633584432896] [client 127.0.0.1:49088] [7fe7c0002ba0] [302515] SSL0222W: SSL Handshake Failed, No ciphers specified (no shared ciphers or no shared protocols).[127.0.0.1:49088 -> 127.0.0.1:443] [2 ms]
:
[Tue Sep 26 17:41:02.005754 2023] [ibm_ssl:debug] [pid 302515:tid 140633584432896] [client 127.0.0.1:49088] <client_hello>
:
[Tue Sep 26 17:41:02.005784 2023] [ibm_ssl:debug] [pid 302515:tid 140633584432896] [client 127.0.0.1:49088] TLSV12
:
[Tue Sep 26 17:41:02.005825 2023] [ibm_ssl:debug] [pid 302515:tid 140633584432896] [client 127.0.0.1:49088] cipher_suites
[Tue Sep 26 17:41:02.005828 2023] [ibm_ssl:debug] [pid 302515:tid 140633584432896] [client 127.0.0.1:49088] Length: 04
[Tue Sep 26 17:41:02.005832 2023] [ibm_ssl:debug] [pid 302515:tid 140633584432896] [client 127.0.0.1:49088] 00 9D 00 FF ....
[Tue Sep 26 17:41:02.005836 2023] [ibm_ssl:debug] [pid 302515:tid 140633584432896] [client 127.0.0.1:49088] tls_rsa_with_aes_256_gcm_sha384,tls_ri_scsv
:
[Tue Sep 26 17:41:02.006034 2023] [ibm_ssl:debug] [pid 302515:tid 140633584432896] [client 127.0.0.1:49088] [7fe7c0002ba0] [302515] gsk_secure_close rc [0] [127.0.0.1:49088 -> 127.0.0.1:443] 3ms