Maximo

 View Only
  • 1.  Error with Ansible masdevops - Db2 pod c-db2w-shared-restore-morph keeps crashing

    Posted Mon July 03, 2023 11:12 AM
      |   view attached

    I am installing MAS Core + Manage using Ansible masdevops collection.

    Core installation goes well, but the following issue occured the last (and only) 3 times I tried to add Manage

    When at the step to set up the db2 cluster instance, the c-db2w-shared-restore-morph pod fails

    It will fail 3 times in total, each time spawning a new pod, then it stops trying.

    Looking at the pod logs itself I don't see much that looks like an error except it fails at "post restore morph"

    + sudo touch /mnt/bludata0/support/db2u_post_restoremorph.logs
    + sudo chmod 666 /mnt/bludata0/support/db2u_post_restoremorph.logs
    + exec
    + return 1
    + exit 1
    command terminated with exit code 1
    + exit 1

    From then on the FormationStatus of the db2w-shared db2ucluster stays Progressing forever

    I attached the log file from the first occurence of failing pod



    ------------------------------
    Francois Beliveau
    ------------------------------


  • 2.  RE: Error with Ansible masdevops - Db2 pod c-db2w-shared-restore-morph keeps crashing

    IBM Champion
    Posted Tue July 04, 2023 02:16 AM

    What is Your config for storage? I would assume that there is something missing there, since /mnt/blumeta0 in the c-db2w-shared-restore-morph is bound to PVC c-db2w-shared-meta

    Does this PVC binds? Do You have RWX capable StorageClass? Did You set the variables for DB2 playbook?



    ------------------------------
    Witold Wierzchowski
    ------------------------------



  • 3.  RE: Error with Ansible masdevops - Db2 pod c-db2w-shared-restore-morph keeps crashing

    Posted Tue July 04, 2023 10:49 AM

    This is a single-node cluster so I am using LVM Storage operator that is using extra disks. All PVCs are bound, Db2 and others

    But I realise now I only defined storage class for meta and data, not for logs and backup

    Will give it a try right about now and come update



    ------------------------------
    Francois Beliveau
    ------------------------------



  • 4.  RE: Error with Ansible masdevops - Db2 pod c-db2w-shared-restore-morph keeps crashing

    Posted Tue July 04, 2023 11:14 AM

    OK so I did forget the storage class for logs and backup

    Now I see the backup claim has been created but cannot be bound due to RWX policy

    It seems the LVM operator as per default config does not support ReadWriteMany?

    failed to provision volume with StorageClass "odf-lvm-vg1": rpc error: code = InvalidArgument desc = unsupported access mode: MULTI_NODE_MULTI_WRITER

    This is a Single-Node test environment i am trying to setup. I need to search to see if I will be able to have LVM operator support ReadWriteMany I guess

    Just to give it a try I tempered with the PVC (deleted it and recreated it with ReadWriteOnce) just to see how far it can go



    ------------------------------
    Francois Beliveau
    ------------------------------



  • 5.  RE: Error with Ansible masdevops - Db2 pod c-db2w-shared-restore-morph keeps crashing

    IBM Champion
    Posted Tue July 04, 2023 11:32 AM

    Take a look here https://ibm-mas.github.io/ansible-devops/roles/db2/#role-variables-storage

    You can export storage access mode for each DB2 PVC with

    export DB2_xxxx_STORAGE_ACCESSMODE=ReadWriteOnce

    where xxxx is either data, meta, logs etc.

    You need to set those vars before running install Manage playbook.



    ------------------------------
    Witold Wierzchowski
    ------------------------------



  • 6.  RE: Error with Ansible masdevops - Db2 pod c-db2w-shared-restore-morph keeps crashing

    Posted Wed July 05, 2023 08:47 AM

    No need to worry about RWX for SNO. RWO of LVM works. You also don't have to define separate storage for DB2 (that's a design choice better suited for a full cluster).  Regarding your original Db2 morph issue. I think there's a bug out there with DB2 container software. The culprit appears to be Instance Memory. Try to increase the memory (Request/Limit) in db2ucluster and see if that does the trick. You may not have enough memory available since you're on SNO. I would run the DB2 server the old fashioned way, outside of OCP.



    ------------------------------
    Arif Ali
    ------------------------------



  • 7.  RE: Error with Ansible masdevops - Db2 pod c-db2w-shared-restore-morph keeps crashing

    Posted Wed July 05, 2023 09:47 AM

    Increasing the memory did the trick. And I know the best way would be to have the DB2 instance outside of the cluster but my mandate right now is to make Manage fit in a 16 vCPU 64GB OpenShift cluster

    I managed to get past by the db2 cluster creation by increasing memory settings, but now something else is crashing... still one issue fixed, some more to go i guess



    ------------------------------
    Francois Beliveau
    ------------------------------



  • 8.  RE: Error with Ansible masdevops - Db2 pod c-db2w-shared-restore-morph keeps crashing

    IBM Champion
    Posted Tue July 23, 2024 05:21 PM

    Old post I realize, but I am running into the same issue on a SNO environment.  Did you ever find a solution?



    ------------------------------
    Jason VenHuizen
    https://sharptree.io
    https://opqo.ioJason VenHuizen
    https://sharptree.io
    https://opqo.io
    ------------------------------