Db2 On Premise and Cloud

Expand all | Collapse all

DB2 HADR issue in Docker image

  • 1.  DB2 HADR issue in Docker image

    Posted Thu December 06, 2018 12:19 AM
      |   view attached
    Hi, I downloaded the DB2 Helm chart from github and db2_developer_c image from docker store. Then I tried to install DB2 Helm chart for HADR setup. The installation seems successful. 2 DB2 pods are created and instance and database are created. But check the standby pod log, it shows error in hadr setting:
    ==================================================
    (*) HADR setup phase for instance db2inst1 and database FTMDB...
    (*) Setting up as a STANDBY Server ...
    (!!!) There was an error while executing setup on STANDBY ...
    Please check /tmp/setup_hadr.log for more information.
    (!!!) There was a failure during HADR setup...
    ==================================================
    And trace into setup_hadr.log, it shows:
    ERROR: The database backup file was not found under /hadr'
    However I check this path "/hadr", there is already a db backup file.

    After reinstalling db2 helm charts several times, found that this problem happens in about 50% installations. Don't know what is the root cause that standby pod cannot find backup file.

    The pod log and setup_hadr.log are attached.(logs.tar.gz)
    Not sure if anyone also encounter this problem, thanks.

    ------------------------------
    Boris Chang
    ------------------------------

    Attachment(s)

    gz
    logs.tar.gz   26K 1 version


  • 2.  RE: DB2 HADR issue in Docker image

    Posted Wed December 12, 2018 01:06 PM
    Hi @Boris Chang,

    Can you describe the environment you're running on  - what Kubernetes (IBM Cloud Private, Redhat OpenShift, etc ) platform and version are you using for the chart install?
    Describe the persistent volumes you are using as well - storageclass, type of storage. The /hadr volume needs to be a shared volume across the two pods - NFS or GlusterFS - hostpath will not work.
    Also, if you are running the install on a command line, can you share your helm install command and parameters?


    ------------------------------
    IRINA DELIDJAKOVA
    ------------------------------



  • 3.  RE: DB2 HADR issue in Docker image

    Posted Thu December 13, 2018 01:24 AM
      |   view attached
    Hi @IRINA DELIDJAKOVA, thanks for reply.
    We used IBM Cloud Private to install chart. We tried the ICP admin console and helm command line to install chart. Both have same result. The helm install command is: "helm install ibm-db2oltp-dev -n db2 --tls --namespace db2​". The folder "ibm-db2oltp-dev" contains the db2 helm charts stuff and customized values.yaml. This values.yaml is attached as well(values.tar.gz).
    About the persistent volume, we used GPFS to share the path across through our worker nodes. So primary and standby pods exactly can access the same volume.
    The thing that makes me confused is: after the HADR setup failure, I attach into the primary and standby pod and check the /hadr path, there is exactly a db2 backup file. Don't know why the hadr script cannot find the backup file.

    BTW, after the hadr setting failure, I tried to attach into primary and standby pod, and run the setup_hadr.sh manually, it can work and setup hadr config well. The command I used is: "kubectl exec -it <standby pod name> -- /var/db2_setup/lib/setup_hadr.sh -standby -use_backup -database FTMDB -instance db2inst1 -remote <primary_hostname>:<primary_ipaddr>".
    Then tried to start hadr: "kubectl exec -it <standby pod name> -- /var/db2_setup/lib/manage_hadr.sh -start_as standby -database FTMDB -instance db2inst1" and "kubectl exec -it <primary pod name> -- /var/db2_setup/lib/manage_hadr.sh -start_as primary -database FTMDB -instance db2inst1".
    Then finally the HADR can go back to work, after these manual steps.

    ------------------------------
    Boris Chang
    ------------------------------

    Attachment(s)

    gz
    values.tar.gz   863B 1 version