Db2

Hello,

I face a very strange issues, running DB2 11.5.8.0 on Debian Buster virtualized with Proxmox.

We have several hypervisors, several of them are AMD EPYC 7371 based and when I CREATE DATABASE on a virtual machine running on one of these everything runs smoothly.

A new AMD EPYC 7343 hypervisor is now available, and when I migrate the exact same virtual machine on it, "CREATE DATABASE"  hangs with two db2sysc hitting 100%. In db2diag, nothing happens after "DB2SPCAT Views recreated", while the next step on the other hypervisors is "FUNCTION: DB2 Common, Cryptography, cryptContextRealInit, probe:2716".

I checked the AESNI with openssl and it works fine in both contexts.

I also tried to limit the CPU flags to the very basic set without more success.

CREATE DATABASE is not only symptom but as it is can easily be reproduced I use it for debugging.

I'm a bit lost here with the next steps for debugging, any help would be greatly appreciated.

Thank you

Patrick

Hello,

I face a very strange issues, running DB2 11.5.8.0 on Debian Buster virtualized with Proxmox.

We have several hypervisors, several of them are AMD EPYC 7371 based and when I CREATE DATABASE on a virtual machine running on one of these everything runs smoothly.

A new AMD EPYC 7343 hypervisor is now available, and when I migrate the exact same virtual machine on it, "CREATE DATABASE"  hangs with two db2sysc hitting 100%. In db2diag, nothing happens after "DB2SPCAT Views recreated", while the next step on the other hypervisors is "FUNCTION: DB2 Common, Cryptography, cryptContextRealInit, probe:2716".

I checked the AESNI with openssl and it works fine in both contexts.

I also tried to limit the CPU flags to the very basic set without more success.

CREATE DATABASE is not only symptom but as it is can easily be reproduced I use it for debugging.

I'm a bit lost here with the next steps for debugging, any help would be greatly appreciated.

Thank you

Patrick

Hi!

As Ian stated - if you consider "production level" Db2 installation - you are strongly advised to follow and adhere to system requirements for running Db2.

For my personal amusement I set up Debian (Debian 12) installation in Oracle Virtual Box VM and I installed Db2 V11.5.8.
One of the red lights should grab your attention was message from db2prereqcheck:



However - if you know Db2 prerequisites and how to address them - you may complete installation:


As you can see - SAMPLE database was created and is accessible.

I would suspect your virtualizer as guilty party.

I would not run production workload on my Debian distribution in Virtual Box though.

Hello,

I face a very strange issues, running DB2 11.5.8.0 on Debian Buster virtualized with Proxmox.

We have several hypervisors, several of them are AMD EPYC 7371 based and when I CREATE DATABASE on a virtual machine running on one of these everything runs smoothly.

A new AMD EPYC 7343 hypervisor is now available, and when I migrate the exact same virtual machine on it, "CREATE DATABASE"  hangs with two db2sysc hitting 100%. In db2diag, nothing happens after "DB2SPCAT Views recreated", while the next step on the other hypervisors is "FUNCTION: DB2 Common, Cryptography, cryptContextRealInit, probe:2716".

I checked the AESNI with openssl and it works fine in both contexts.

I also tried to limit the CPU flags to the very basic set without more success.

CREATE DATABASE is not only symptom but as it is can easily be reproduced I use it for debugging.

I'm a bit lost here with the next steps for debugging, any help would be greatly appreciated.

Thank you

Patrick

Hello,

Thanks for your answers, we tried with Ubuntu 20 on the same hypervisor and did not face the issue.

I will follow your advice and switch to a supported OS.

Thank a lot for your replies.

Patrick

Hi!

As Ian stated - if you consider "production level" Db2 installation - you are strongly advised to follow and adhere to system requirements for running Db2.

For my personal amusement I set up Debian (Debian 12) installation in Oracle Virtual Box VM and I installed Db2 V11.5.8.
One of the red lights should grab your attention was message from db2prereqcheck:



However - if you know Db2 prerequisites and how to address them - you may complete installation:


As you can see - SAMPLE database was created and is accessible.

I would suspect your virtualizer as guilty party.

I would not run production workload on my Debian distribution in Virtual Box though.

Hello,

I face a very strange issues, running DB2 11.5.8.0 on Debian Buster virtualized with Proxmox.

We have several hypervisors, several of them are AMD EPYC 7371 based and when I CREATE DATABASE on a virtual machine running on one of these everything runs smoothly.

A new AMD EPYC 7343 hypervisor is now available, and when I migrate the exact same virtual machine on it, "CREATE DATABASE"  hangs with two db2sysc hitting 100%. In db2diag, nothing happens after "DB2SPCAT Views recreated", while the next step on the other hypervisors is "FUNCTION: DB2 Common, Cryptography, cryptContextRealInit, probe:2716".

I checked the AESNI with openssl and it works fine in both contexts.

I also tried to limit the CPU flags to the very basic set without more success.

CREATE DATABASE is not only symptom but as it is can easily be reproduced I use it for debugging.

I'm a bit lost here with the next steps for debugging, any help would be greatly appreciated.

Thank you

Patrick

Hello,

I face a very strange issues, running DB2 11.5.8.0 on Debian Buster virtualized with Proxmox.

We have several hypervisors, several of them are AMD EPYC 7371 based and when I CREATE DATABASE on a virtual machine running on one of these everything runs smoothly.

A new AMD EPYC 7343 hypervisor is now available, and when I migrate the exact same virtual machine on it, "CREATE DATABASE"  hangs with two db2sysc hitting 100%. In db2diag, nothing happens after "DB2SPCAT Views recreated", while the next step on the other hypervisors is "FUNCTION: DB2 Common, Cryptography, cryptContextRealInit, probe:2716".

I checked the AESNI with openssl and it works fine in both contexts.

I also tried to limit the CPU flags to the very basic set without more success.

CREATE DATABASE is not only symptom but as it is can easily be reproduced I use it for debugging.

I'm a bit lost here with the next steps for debugging, any help would be greatly appreciated.

Thank you

Patrick

Did anyone figure yout why this is happening?

If have the same issue with DB2 11.5.9 on bare metal tested under ubuntu 20.04, 22.04, 24.04.

It is successful in 4 / 10 test run, but fails in 6 / 10 test runs.

We are now completly puzzled how to proceed.

Hello,

I face a very strange issues, running DB2 11.5.8.0 on Debian Buster virtualized with Proxmox.

We have several hypervisors, several of them are AMD EPYC 7371 based and when I CREATE DATABASE on a virtual machine running on one of these everything runs smoothly.

A new AMD EPYC 7343 hypervisor is now available, and when I migrate the exact same virtual machine on it, "CREATE DATABASE"  hangs with two db2sysc hitting 100%. In db2diag, nothing happens after "DB2SPCAT Views recreated", while the next step on the other hypervisors is "FUNCTION: DB2 Common, Cryptography, cryptContextRealInit, probe:2716".

I checked the AESNI with openssl and it works fine in both contexts.

I also tried to limit the CPU flags to the very basic set without more success.

CREATE DATABASE is not only symptom but as it is can easily be reproduced I use it for debugging.

I'm a bit lost here with the next steps for debugging, any help would be greatly appreciated.

Thank you

Patrick

What is the CPU on the bare metal box(es) that experience the Db2 hangs?

If the CPU is from AMD EPYC family, study this document https://www.ibm.com/support/pages/db2-may-hang-while-processing-encryption-related-functions-ibm-gskit-library-some-amd-epyc-processor

If the CPU is not from AMD, you might want to upgrade the GSKIT anyway (the latest build might not be incorporated in your specific fixlevel of 11.5.9)  , and then retry, contact IBM support for details.

------------------------------
Mike O'Reilly

Original Message:
Sent: Wed August 28, 2024 01:57 AM
From: Nikolai Stein
Subject: CREATE DATABASE hangs when vm runs on one hypervisor

Did anyone figure yout why this is happening?

If have the same issue with DB2 11.5.9 on bare metal tested under ubuntu 20.04, 22.04, 24.04.

It is successful in 4 / 10 test run, but fails in 6 / 10 test runs.

We are now completly puzzled how to proceed.

------------------------------
Nikolai Stein
General Manager
REQUISIS GmbH
Berlin
+49-30-536506-711

Original Message:
Sent: Mon October 30, 2023 06:25 AM
From: Patrick Ambuhl
Subject: CREATE DATABASE hangs when vm runs on one hypervisor

Hello,

I face a very strange issues, running DB2 11.5.8.0 on Debian Buster virtualized with Proxmox.

We have several hypervisors, several of them are AMD EPYC 7371 based and when I CREATE DATABASE on a virtual machine running on one of these everything runs smoothly.

A new AMD EPYC 7343 hypervisor is now available, and when I migrate the exact same virtual machine on it, "CREATE DATABASE"  hangs with two db2sysc hitting 100%. In db2diag, nothing happens after "DB2SPCAT Views recreated", while the next step on the other hypervisors is "FUNCTION: DB2 Common, Cryptography, cryptContextRealInit, probe:2716".

I checked the AESNI with openssl and it works fine in both contexts.

I also tried to limit the CPU flags to the very basic set without more success.

CREATE DATABASE is not only symptom but as it is can easily be reproduced I use it for debugging.

I'm a bit lost here with the next steps for debugging, any help would be greatly appreciated.

Thank you

Patrick

------------------------------
Patrick Ambuhl
------------------------------

What is the CPU on the bare metal box(es) that experience the Db2 hangs?

If the CPU is from AMD EPYC family, study this document https://www.ibm.com/support/pages/db2-may-hang-while-processing-encryption-related-functions-ibm-gskit-library-some-amd-epyc-processor

If the CPU is not from AMD, you might want to upgrade the GSKIT anyway (the latest build might not be incorporated in your specific fixlevel of 11.5.9)  , and then retry, contact IBM support for details.

Did anyone figure yout why this is happening?

If have the same issue with DB2 11.5.9 on bare metal tested under ubuntu 20.04, 22.04, 24.04.

It is successful in 4 / 10 test run, but fails in 6 / 10 test runs.

We are now completly puzzled how to proceed.

Hello,

I face a very strange issues, running DB2 11.5.8.0 on Debian Buster virtualized with Proxmox.

We have several hypervisors, several of them are AMD EPYC 7371 based and when I CREATE DATABASE on a virtual machine running on one of these everything runs smoothly.

A new AMD EPYC 7343 hypervisor is now available, and when I migrate the exact same virtual machine on it, "CREATE DATABASE"  hangs with two db2sysc hitting 100%. In db2diag, nothing happens after "DB2SPCAT Views recreated", while the next step on the other hypervisors is "FUNCTION: DB2 Common, Cryptography, cryptContextRealInit, probe:2716".

I checked the AESNI with openssl and it works fine in both contexts.

I also tried to limit the CPU flags to the very basic set without more success.

CREATE DATABASE is not only symptom but as it is can easily be reproduced I use it for debugging.

I'm a bit lost here with the next steps for debugging, any help would be greatly appreciated.

Thank you

Patrick

I confirm that the ICC_SHIFT=3 works for us, we still have some tests to run but everything seems running well at the DBServer level. However we face a similar issue on client side servers with AMD EPYC CPUs  using the DB2 client, is there a setting like the ICC_SHIFT for the client stack (PHP8) in our case.

Thanks a lot for the server solution !

Patrick

What is the CPU on the bare metal box(es) that experience the Db2 hangs?

If the CPU is from AMD EPYC family, study this document https://www.ibm.com/support/pages/db2-may-hang-while-processing-encryption-related-functions-ibm-gskit-library-some-amd-epyc-processor

If the CPU is not from AMD, you might want to upgrade the GSKIT anyway (the latest build might not be incorporated in your specific fixlevel of 11.5.9)  , and then retry, contact IBM support for details.

Did anyone figure yout why this is happening?

If have the same issue with DB2 11.5.9 on bare metal tested under ubuntu 20.04, 22.04, 24.04.

It is successful in 4 / 10 test run, but fails in 6 / 10 test runs.

We are now completly puzzled how to proceed.

Hello,

I face a very strange issues, running DB2 11.5.8.0 on Debian Buster virtualized with Proxmox.

We have several hypervisors, several of them are AMD EPYC 7371 based and when I CREATE DATABASE on a virtual machine running on one of these everything runs smoothly.

A new AMD EPYC 7343 hypervisor is now available, and when I migrate the exact same virtual machine on it, "CREATE DATABASE"  hangs with two db2sysc hitting 100%. In db2diag, nothing happens after "DB2SPCAT Views recreated", while the next step on the other hypervisors is "FUNCTION: DB2 Common, Cryptography, cryptContextRealInit, probe:2716".

I checked the AESNI with openssl and it works fine in both contexts.

I also tried to limit the CPU flags to the very basic set without more success.

CREATE DATABASE is not only symptom but as it is can easily be reproduced I use it for debugging.

I'm a bit lost here with the next steps for debugging, any help would be greatly appreciated.

Thank you

Patrick

The workaround (of setting ICC_SHIFT=3 in the config files for GSKit) is applicable anywhere that the GSKit runs, and that includes on the Db2-clients as well as on Db2-servers .  The GsKit is the component that causes the hang on specific CPUs, and it is this component that the workaround targets, and GSKit libraries are supplied with many client and server products from IBM , it is pervasive.

For example, if you are using php_ibm_db2 that is a wrapper for IBM's CLIDRIVER which includes the encryption algorithms of GSKit, although the directory paths will differ from those on the Db2-server. Just search the client filesystem for the two icclib directories, and adjust the ICCSIGN.TXT appropriately in both directories.

However it is also possible to wait for IBM to re-issue a new build of clidriver that includes the updated GSKit that contains a code fix to solve the issue and in this case you do not then need to apply that specific workaround, although of course you will need to service (i.e. apply the fixed clidriver, or uninstall and re-install the php_ibm_db2 to get the latest clidriver downloaded from IBM automatically) your client environments. 

It is wise to continually service clidriver in your estate anyway, to get the relevant security and stability fixes that arrive from time to time.

I confirm that the ICC_SHIFT=3 works for us, we still have some tests to run but everything seems running well at the DBServer level. However we face a similar issue on client side servers with AMD EPYC CPUs  using the DB2 client, is there a setting like the ICC_SHIFT for the client stack (PHP8) in our case.

Thanks a lot for the server solution !

Patrick

What is the CPU on the bare metal box(es) that experience the Db2 hangs?

If the CPU is from AMD EPYC family, study this document https://www.ibm.com/support/pages/db2-may-hang-while-processing-encryption-related-functions-ibm-gskit-library-some-amd-epyc-processor

If the CPU is not from AMD, you might want to upgrade the GSKIT anyway (the latest build might not be incorporated in your specific fixlevel of 11.5.9)  , and then retry, contact IBM support for details.

Did anyone figure yout why this is happening?

If have the same issue with DB2 11.5.9 on bare metal tested under ubuntu 20.04, 22.04, 24.04.

It is successful in 4 / 10 test run, but fails in 6 / 10 test runs.

We are now completly puzzled how to proceed.

Hello,

I face a very strange issues, running DB2 11.5.8.0 on Debian Buster virtualized with Proxmox.

We have several hypervisors, several of them are AMD EPYC 7371 based and when I CREATE DATABASE on a virtual machine running on one of these everything runs smoothly.

A new AMD EPYC 7343 hypervisor is now available, and when I migrate the exact same virtual machine on it, "CREATE DATABASE"  hangs with two db2sysc hitting 100%. In db2diag, nothing happens after "DB2SPCAT Views recreated", while the next step on the other hypervisors is "FUNCTION: DB2 Common, Cryptography, cryptContextRealInit, probe:2716".

I checked the AESNI with openssl and it works fine in both contexts.

I also tried to limit the CPU flags to the very basic set without more success.

CREATE DATABASE is not only symptom but as it is can easily be reproduced I use it for debugging.

I'm a bit lost here with the next steps for debugging, any help would be greatly appreciated.

Thank you

Patrick