AIOps

Expand all | Collapse all

topology-merge and topology-search pods in CrashLoopBackoff state

  • 1.  topology-merge and topology-search pods in CrashLoopBackoff state

    Posted Fri April 02, 2021 02:45 PM
    Watson AIOps 2.0 Event Manager installed on OpenShift 4.7.

    The topology-search and topology-merge pods keep restarting and can be seen in a CrashLoopBackoff state. Why I found this is when I go to Topology from NOI, I get an error:

    (Critical Failed to connect to the Search Service. The application configuration may be incorrect, or the ASM backend services may not be running.)

    That's why I looked at the status of the pods. These two pods have restarted 47 times since I installed NOI 2 hours ago.

    Any pointers? Running 'oc describe pod <pod_name>' doesn't show any errors. There is a warning that neither pod can get to the healtcheck URL, but that's the only error.

    ------------------------------
    Frank Tate
    Gulfsoft Consulting
    ------------------------------


  • 2.  RE: topology-merge and topology-search pods in CrashLoopBackoff state

    Posted Mon April 05, 2021 07:10 AM
    @john postoyko Thanks for your attention on this.​

    ------------------------------
    VEERAMANI NAMBI
    Offering Manager, GoToMarket - Communities
    IBM
    ------------------------------



  • 3.  RE: topology-merge and topology-search pods in CrashLoopBackoff state

    Posted Tue April 06, 2021 09:09 AM

    Hi Frank,

    usually its because the *topology-topology* pod is not started or unavailable..

    anything in the logs ?



    ------------------------------
    mario schuerewegen
    IBM
    Brussels
    (3249) 528-1079
    ------------------------------



  • 4.  RE: topology-merge and topology-search pods in CrashLoopBackoff state

    Posted Tue April 06, 2021 09:34 AM
    Thanks for the reply, Mario. It turns out that the OpenShift version is the problem. I have reinstalled at OpenShift 4.6.17 and now topology search works (and the topology-search pod stays alive). However, it looks like I should have installed OpenShift 4.6.16 to avoid another problem. Thanks again for the reply.

    ------------------------------
    Frank Tate
    Gulfsoft Consulting
    ------------------------------



  • 5.  RE: topology-merge and topology-search pods in CrashLoopBackoff state

    Posted Tue April 06, 2021 12:01 PM
    Edited by Frank Tate Tue April 06, 2021 12:06 PM
    OpenShift 4.6.16 doesn't work as well as 4.6.17. Topology search fails (and the pod is in CrashLoopBackoff). Is there a known good version of OpenShift that I can install on that will allow all the pods to run correctly when run on vSphere 6.7 with Installer Provisioned Infrastructure?

    So far I've tried the following OpenShift versions:

    4.6.16 - topology search doesn't work (there's no healthcron pod, so I'm assuming it worked)
    4.6.17 - topology search works, but the healthcron pod can't be created
    4.6.23 - too unstable to even install Watson AI Ops
    4.7- topology search doesn't work



    ------------------------------
    Frank Tate
    Gulfsoft Consulting
    ------------------------------