Containers, Kubernetes, OpenShift on Power

Hi

I'm reaching out for assistance with an issue I encountered during the installation of a 3-node OpenShift 4.17.11 cluster on an IBM Power 9 Server.

The installation process seems to proceed normally up to the Finalizing Stage, but then it takes an unusually long time and eventually fails, resulting in an error. I attempted a fresh reinstallation, but the issue persists, ending with the same outcome.
Below is the screenshot of cluster events

Hey Anup, I think you have a problem with a load balancer. I don't think you have everything setup. Authentication and Console depend on the ingress being fully setup. If you have more information on these `oc get co ingress` - it should help tell us what is going on. Thanks, Paul

------------------------------
PAUL BASTIDE
Senior Software Engineer
IBM

Original Message:
Sent: Fri January 17, 2025 09:57 PM
From: Anup Regmi
Subject: Error during OpenShift Installation in Power 9 Server

Hi

I'm reaching out for assistance with an issue I encountered during the installation of a 3-node OpenShift 4.17.11 cluster on an IBM Power 9 Server.

The installation process seems to proceed normally up to the Finalizing Stage, but then it takes an unusually long time and eventually fails, resulting in an error. I attempted a fresh reinstallation, but the issue persists, ending with the same outcome.
Below is the screenshot of cluster events

------------------------------
Anup Regmi
------------------------------

Hi Paul, Thanks for your response.

Looks like that is the issue, I was installing using assisted installation and thought RedHat will take care of those things.
Below is the status and cause of failure:

The "default" ingress controller reports Degraded=True: DegradedConditions: One or more other status conditions indicate a degraded state: CanaryChecksSucceeding=False (CanaryChecksRepetitiveFailures: Canary route checks for the default ingress controller are failing. Last 2 error messages:
      error sending canary HTTP Request: Timeout: Get "https://canary-openshift-ingress-canary.apps.ppcocp.thakralone.lab": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
      error sending canary HTTP request: DNS error: Get "https://canary-openshift-ingress-canary.apps.ppcocp.thakralone.lab": dial tcp: lookup canary-openshift-ingress-canary.apps.ppcocp.thakralone.lab on 172.30.0.10:53: no such host (x5932 over 98h52m4s))

While checking status of the route mentioned above, got responses as below:

Can you guide on how we can fix this error?

------------------------------
Anup Regmi


Original Message:
Sent: Thu January 23, 2025 09:41 AM
From: PAUL BASTIDE
Subject: Error during OpenShift Installation in Power 9 Server

Hey Anup, I think you have a problem with a load balancer. I don't think you have everything setup. Authentication and Console depend on the ingress being fully setup. If you have more information on these `oc get co ingress` - it should help tell us what is going on. Thanks, Paul

------------------------------
PAUL BASTIDE
Senior Software Engineer
IBM

Original Message:
Sent: Fri January 17, 2025 09:57 PM
From: Anup Regmi
Subject: Error during OpenShift Installation in Power 9 Server

Hi

I'm reaching out for assistance with an issue I encountered during the installation of a 3-node OpenShift 4.17.11 cluster on an IBM Power 9 Server.

The installation process seems to proceed normally up to the Finalizing Stage, but then it takes an unusually long time and eventually fails, resulting in an error. I attempted a fresh reinstallation, but the issue persists, ending with the same outcome.
Below is the screenshot of cluster events

------------------------------
Anup Regmi
------------------------------

Hi Paul, Thanks for your response.

Looks like that is the issue, I was installing using assisted installation and thought RedHat will take care of those things.
Below is the status and cause of failure:

The "default" ingress controller reports Degraded=True: DegradedConditions: One or more other status conditions indicate a degraded state: CanaryChecksSucceeding=False (CanaryChecksRepetitiveFailures: Canary route checks for the default ingress controller are failing. Last 2 error messages:
      error sending canary HTTP Request: Timeout: Get "https://canary-openshift-ingress-canary.apps.ppcocp.thakralone.lab": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
      error sending canary HTTP request: DNS error: Get "https://canary-openshift-ingress-canary.apps.ppcocp.thakralone.lab": dial tcp: lookup canary-openshift-ingress-canary.apps.ppcocp.thakralone.lab on 172.30.0.10:53: no such host (x5932 over 98h52m4s))

While checking status of the route mentioned above, got responses as below:

Can you guide on how we can fix this error?

------------------------------
Anup Regmi


Original Message:
Sent: Thu January 23, 2025 09:41 AM
From: PAUL BASTIDE
Subject: Error during OpenShift Installation in Power 9 Server

Hey Anup, I think you have a problem with a load balancer. I don't think you have everything setup. Authentication and Console depend on the ingress being fully setup. If you have more information on these `oc get co ingress` - it should help tell us what is going on. Thanks, Paul

------------------------------
PAUL BASTIDE
Senior Software Engineer
IBM

Original Message:
Sent: Fri January 17, 2025 09:57 PM
From: Anup Regmi
Subject: Error during OpenShift Installation in Power 9 Server

Hi

I'm reaching out for assistance with an issue I encountered during the installation of a 3-node OpenShift 4.17.11 cluster on an IBM Power 9 Server.

The installation process seems to proceed normally up to the Finalizing Stage, but then it takes an unusually long time and eventually fails, resulting in an error. I attempted a fresh reinstallation, but the issue persists, ending with the same outcome.
Below is the screenshot of cluster events

------------------------------
Anup Regmi
------------------------------

Hi Anup,

The Assisted install for power does not support Cluster managed network, only User Managed network works, so the bastion is required to be setup for OCP installation. Have you do that before installation with Assisted installer? Also what power env do you use, HMC only or PowerVC?  You can reference the doc Paul referenced above for bastion setup.

Thanks.

C. Zhang

Hi Paul, Thanks for your response.

Looks like that is the issue, I was installing using assisted installation and thought RedHat will take care of those things.
Below is the status and cause of failure:

The "default" ingress controller reports Degraded=True: DegradedConditions: One or more other status conditions indicate a degraded state: CanaryChecksSucceeding=False (CanaryChecksRepetitiveFailures: Canary route checks for the default ingress controller are failing. Last 2 error messages:
      error sending canary HTTP Request: Timeout: Get "https://canary-openshift-ingress-canary.apps.ppcocp.thakralone.lab": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
      error sending canary HTTP request: DNS error: Get "https://canary-openshift-ingress-canary.apps.ppcocp.thakralone.lab": dial tcp: lookup canary-openshift-ingress-canary.apps.ppcocp.thakralone.lab on 172.30.0.10:53: no such host (x5932 over 98h52m4s))

While checking status of the route mentioned above, got responses as below:

Can you guide on how we can fix this error?

------------------------------
Anup Regmi


Original Message:
Sent: Thu January 23, 2025 09:41 AM
From: PAUL BASTIDE
Subject: Error during OpenShift Installation in Power 9 Server

Hey Anup, I think you have a problem with a load balancer. I don't think you have everything setup. Authentication and Console depend on the ingress being fully setup. If you have more information on these `oc get co ingress` - it should help tell us what is going on. Thanks, Paul

------------------------------
PAUL BASTIDE
Senior Software Engineer
IBM

Original Message:
Sent: Fri January 17, 2025 09:57 PM
From: Anup Regmi
Subject: Error during OpenShift Installation in Power 9 Server

Hi

I'm reaching out for assistance with an issue I encountered during the installation of a 3-node OpenShift 4.17.11 cluster on an IBM Power 9 Server.

The installation process seems to proceed normally up to the Finalizing Stage, but then it takes an unusually long time and eventually fails, resulting in an error. I attempted a fresh reinstallation, but the issue persists, ending with the same outcome.
Below is the screenshot of cluster events

------------------------------
Anup Regmi
------------------------------

Hi

I have HMC only environment, I will try installing as per Paul's suggestion. Hope SNO is supported as well.

Thanks for your response.

Hi Anup,

The Assisted install for power does not support Cluster managed network, only User Managed network works, so the bastion is required to be setup for OCP installation. Have you do that before installation with Assisted installer? Also what power env do you use, HMC only or PowerVC?  You can reference the doc Paul referenced above for bastion setup.

Thanks.

C. Zhang

Hi Paul, Thanks for your response.

Looks like that is the issue, I was installing using assisted installation and thought RedHat will take care of those things.
Below is the status and cause of failure:

The "default" ingress controller reports Degraded=True: DegradedConditions: One or more other status conditions indicate a degraded state: CanaryChecksSucceeding=False (CanaryChecksRepetitiveFailures: Canary route checks for the default ingress controller are failing. Last 2 error messages:
      error sending canary HTTP Request: Timeout: Get "https://canary-openshift-ingress-canary.apps.ppcocp.thakralone.lab": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
      error sending canary HTTP request: DNS error: Get "https://canary-openshift-ingress-canary.apps.ppcocp.thakralone.lab": dial tcp: lookup canary-openshift-ingress-canary.apps.ppcocp.thakralone.lab on 172.30.0.10:53: no such host (x5932 over 98h52m4s))

While checking status of the route mentioned above, got responses as below:

Can you guide on how we can fix this error?

------------------------------
Anup Regmi


Original Message:
Sent: Thu January 23, 2025 09:41 AM
From: PAUL BASTIDE
Subject: Error during OpenShift Installation in Power 9 Server

Hey Anup, I think you have a problem with a load balancer. I don't think you have everything setup. Authentication and Console depend on the ingress being fully setup. If you have more information on these `oc get co ingress` - it should help tell us what is going on. Thanks, Paul

------------------------------
PAUL BASTIDE
Senior Software Engineer
IBM

Original Message:
Sent: Fri January 17, 2025 09:57 PM
From: Anup Regmi
Subject: Error during OpenShift Installation in Power 9 Server

Hi

I'm reaching out for assistance with an issue I encountered during the installation of a 3-node OpenShift 4.17.11 cluster on an IBM Power 9 Server.

The installation process seems to proceed normally up to the Finalizing Stage, but then it takes an unusually long time and eventually fails, resulting in an error. I attempted a fresh reinstallation, but the issue persists, ending with the same outcome.
Below is the screenshot of cluster events

------------------------------
Anup Regmi
------------------------------

Hi

I have HMC only environment, I will try installing as per Paul's suggestion. Hope SNO is supported as well.

Thanks for your response.

Hi Anup,

The Assisted install for power does not support Cluster managed network, only User Managed network works, so the bastion is required to be setup for OCP installation. Have you do that before installation with Assisted installer? Also what power env do you use, HMC only or PowerVC?  You can reference the doc Paul referenced above for bastion setup.

Thanks.

C. Zhang

Hi Paul, Thanks for your response.

Looks like that is the issue, I was installing using assisted installation and thought RedHat will take care of those things.
Below is the status and cause of failure:

The "default" ingress controller reports Degraded=True: DegradedConditions: One or more other status conditions indicate a degraded state: CanaryChecksSucceeding=False (CanaryChecksRepetitiveFailures: Canary route checks for the default ingress controller are failing. Last 2 error messages:
      error sending canary HTTP Request: Timeout: Get "https://canary-openshift-ingress-canary.apps.ppcocp.thakralone.lab": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
      error sending canary HTTP request: DNS error: Get "https://canary-openshift-ingress-canary.apps.ppcocp.thakralone.lab": dial tcp: lookup canary-openshift-ingress-canary.apps.ppcocp.thakralone.lab on 172.30.0.10:53: no such host (x5932 over 98h52m4s))

While checking status of the route mentioned above, got responses as below:

Can you guide on how we can fix this error?

------------------------------
Anup Regmi


Original Message:
Sent: Thu January 23, 2025 09:41 AM
From: PAUL BASTIDE
Subject: Error during OpenShift Installation in Power 9 Server

Hey Anup, I think you have a problem with a load balancer. I don't think you have everything setup. Authentication and Console depend on the ingress being fully setup. If you have more information on these `oc get co ingress` - it should help tell us what is going on. Thanks, Paul

------------------------------
PAUL BASTIDE
Senior Software Engineer
IBM

Original Message:
Sent: Fri January 17, 2025 09:57 PM
From: Anup Regmi
Subject: Error during OpenShift Installation in Power 9 Server

Hi

I'm reaching out for assistance with an issue I encountered during the installation of a 3-node OpenShift 4.17.11 cluster on an IBM Power 9 Server.

The installation process seems to proceed normally up to the Finalizing Stage, but then it takes an unusually long time and eventually fails, resulting in an error. I attempted a fresh reinstallation, but the issue persists, ending with the same outcome.
Below is the screenshot of cluster events

------------------------------
Anup Regmi
------------------------------

Hi Paul, Thanks for your response.

Looks like that is the issue, I was installing using assisted installation and thought RedHat will take care of those things.
Below is the status and cause of failure:

The "default" ingress controller reports Degraded=True: DegradedConditions: One or more other status conditions indicate a degraded state: CanaryChecksSucceeding=False (CanaryChecksRepetitiveFailures: Canary route checks for the default ingress controller are failing. Last 2 error messages:
      error sending canary HTTP Request: Timeout: Get "https://canary-openshift-ingress-canary.apps.ppcocp.thakralone.lab": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
      error sending canary HTTP request: DNS error: Get "https://canary-openshift-ingress-canary.apps.ppcocp.thakralone.lab": dial tcp: lookup canary-openshift-ingress-canary.apps.ppcocp.thakralone.lab on 172.30.0.10:53: no such host (x5932 over 98h52m4s))

While checking status of the route mentioned above, got responses as below:

Can you guide on how we can fix this error?

------------------------------
Anup Regmi


Original Message:
Sent: Thu January 23, 2025 09:41 AM
From: PAUL BASTIDE
Subject: Error during OpenShift Installation in Power 9 Server

Hey Anup, I think you have a problem with a load balancer. I don't think you have everything setup. Authentication and Console depend on the ingress being fully setup. If you have more information on these `oc get co ingress` - it should help tell us what is going on. Thanks, Paul

------------------------------
PAUL BASTIDE
Senior Software Engineer
IBM

Original Message:
Sent: Fri January 17, 2025 09:57 PM
From: Anup Regmi
Subject: Error during OpenShift Installation in Power 9 Server

Hi

I'm reaching out for assistance with an issue I encountered during the installation of a 3-node OpenShift 4.17.11 cluster on an IBM Power 9 Server.

The installation process seems to proceed normally up to the Finalizing Stage, but then it takes an unusually long time and eventually fails, resulting in an error. I attempted a fresh reinstallation, but the issue persists, ending with the same outcome.
Below is the screenshot of cluster events

------------------------------
Anup Regmi
------------------------------