Cloud Pak for Data

Cloud Pak for Data

Come for answers. Stay for best practices. All we’re missing is you.

 View Only

  • 1.  The Problem After the Uninstallation

    Posted Mon May 11, 2020 09:52 PM
    Hello,

    I apologize my continuous questions.
    I uninstalled Watson Discovery 2.1.1 following the instruction https://www.ibm.com/support/knowledgecenter/SSQNUZ_2.5.0/cpd/svc/watson/discovery-uninstall.html.
    Although the uninstallation run successfully, but some pods got problems.

    One of them is elastic-master2.
    ----
    Warning FailedMount 1m kubelet, wkr04 Unable to mount volumes for pod "elasticsearch-master-2_icp4d(87ef8253-93f1-11ea-8d99-42010a920005)": timeout expired waiting for volumes to attach or mount for pod "icp4d"/"elasticsearch-master-2". list of unmounted volumes=[backups-pv]. list of unattached volumes=[elasticsearch-master elasticsearch-master-cert esconfig backups-pv default-token-zl8kg]
    Warning FailedMount 1m (x2 over 2m) kubelet, wkr04 MountVolume.SetUp failed for volume "pvc-ebe54b0a-40dd-11ea-8dad-42010a920004" : rpc error: code = Internal desc = failed to attach volume: All replica nodes are down
    ----

    Could you tell me about this problem and cause in detail?
    What kind of problem is disturbing the run?

    Kind regards,
    Chris

    ------------------------------
    Chris
    ------------------------------

    #CloudPakforDataGroup


  • 2.  RE: The Problem After the Uninstallation

    Posted Tue May 12, 2020 06:14 AM

    Hi,

    Could you please describe the pod ? What storage are you using (Portworx) ?

    Is the pod part of the WD deployment ?

    Thanks



    ------------------------------
    TOMASZ HANUSIAK
    ------------------------------

    Original Message


  • 3.  RE: The Problem After the Uninstallation

    Posted Wed May 13, 2020 10:24 PM
      |   view attached
    Hello,

    Thank you  for your cooperation.
    I use Portworx as storage.

    # oc get po |grep -v 1/1 |grep -v 2/2 |grep -v 3/3
    NAME READY STATUS RESTARTS AGE
    asset-files-api-56648497d-mg57k 0/1 ContainerCreating 0 1d
    ax-cdsx-notebooks-job-manager-deploy-694f455cc5-27cjz 0/1 Running 48 1d
    cognos-ibm-cde-prod-proxy-f5468595f-6j5xn 0/2 Init:0/1 1 1d
    cognos-ibm-cde-prod-proxy-f5468595f-z4qjz 0/2 Init:0/1 1 1d
    cognos-ibm-cde-prod-redis-84b8b6559c-dh2k8 0/1 ContainerCreating 0 1d
    ds-engine-compute-0 0/1 ContainerCreating 0 1d
    elasticsearch-master-0 0/2 ContainerCreating 0 1d
    elasticsearch-master-1 0/2 ContainerCreating 0 1d
    elasticsearch-master-2 0/2 ContainerCreating 0 1d
    ia-analysis-7c54785947-x8rv8 0/1 Init:0/1 0 1d
    is-en-conductor-0 0/1 Init:0/2 0 1d
    is-engine-compute-0 0/1 ContainerCreating 0 1d
    ngp-projects-api-79d5c59bb8-pqgm2 0/1 CrashLoopBackOff 45 1d
    omag-69bf78fc87-4lcc6 0/1 Init:0/1 0 1d
    portal-notifications-6447d59c84-lqwhk 0/1 CrashLoopBackOff 49 1d
    rabbitmq-ha-0 0/1 Init:0/1 0 1d
    utils-api-6656ccb868-5spdg 0/1 ContainerCreating 0 1d
    wdp-activities-56d99cb5d4-psnn7 0/1 CrashLoopBackOff 42 1d
    wdp-couchdb-0 0/1 Init:0/1 0 1d
    wdp-couchdb-1 0/1 Init:0/1 0 1d
    wdp-couchdb-2 0/1 Running 1 1d
    wdp-profiling-fd57596b9-wthls 0/1 Running 10 1d
    wkc-search-6d57f4bf84-r2pdw 0/1 Init:CrashLoopBackOff 51 1d
    zen-metastoredb-2 0/1 Init:0/2 0 1d

    I extracted several outputs from the pods above. Please have a look at the file attached.
    Most pods show the same results that they cannot mount volume pvc.

    Regards,
    Chris

    ------------------------------
    Chris
    ------------------------------

    Attachment(s)

    txt
    pod_error_afterwduninstall.txt   23 KB 1 version
    Original Message


  • 4.  RE: The Problem After the Uninstallation

    Posted Thu May 14, 2020 04:44 AM

    Hi,

    Check pods across the cluster:
    kubectl get po --no-headers --all-namespaces -o wide| grep -Ev '([[:digit:]])/\1.*R' | grep -v 'Completed'

    on nodes with Portworx, try running
    pxctl status

    you can also check individual Portworx volumes, for example:
    pxctl volume list

    pxctl volume inspect pvc-2c9c1053-d49b-11e9-bb4a-00163e01c62f

    Thanks



    ------------------------------
    TOMASZ HANUSIAK
    ------------------------------

    Original Message


  • 5.  RE: The Problem After the Uninstallation

    Posted Thu May 14, 2020 10:03 PM
    Edited by System Admin Fri January 20, 2023 04:49 PM
    Hello,

    Thank you for your reply.
    My machine does not have pxctl command, I use portworx though.

    And also, I got this message(I tried the command from https://www.ibm.com/support/knowledgecenter/SSQNUZ_2.5.0/cpd/install/portworx-install.html).
    # PX_POD=$(kubectl get pods -l name=portworx -n kube-system -o jsonpath='{.items[0].metadata.name}')
    # kubectl exec $PX_POD -n kube-system -- /opt/pwx/bin/pxctl status
    error: unable to upgrade connection: container not found ("portworx")

    Doesn't portworx go well?

    Regards,
    Chris

    ------------------------------
    Chris
    ------------------------------

    Original Message


  • 6.  RE: The Problem After the Uninstallation

    Posted Fri May 15, 2020 04:42 AM

    Hi,

    What do you get out of:

    kubectl get pods -l name=portworx -n kube-system

    or even
    kubectl get po --all-namespaces | grep portworx

    Thanks



    ------------------------------
    TOMASZ HANUSIAK
    ------------------------------

    Original Message