IBM Security Verify

Hi ,

Running both ISAM and ISVA from the latest Helm Charts in a Kubernetes cluster with 2 worker nodes.

we have a case open with IBM support were we reported a compatibility issue between ISAM 9.0.7 and ISVA 10, when trying to access IBM ICN.
It worked with ISAM but not with ISVA.

I've spent some (read "a lot of") time making sure that the environments and test case for both ISAM and ISVA are the same.
They are now deployed in seperate namespaces in the same Kubernetes Cluster accessing (junctioning to) the same ICN backend.
In doing this, I now can say that ISAM and ISVA both can access ICN with expected results and both ISAM and ISVA can fail to do so in what I hope is a consistent manner.

If I log in to the reverse proxy (for either ISAM or ISVA) directly on the nodeport it is assigned by the Kubernetes deploy, everything works as expected :-), i.e nodeIP:nodeport/junctiontoICN connects and responds correctly for ICN tasks.

But if I add an ingress to the Kubernetes cluster (mainly to allow for hostname routing/virtual hosts) the connection from the reverse proxy to ICN causes a continuous reload loop of the ICN desktop. It looks like ICN runs through all the steps for building the desktop and once it's done repeats the same operation over and over.

The ingress (annotated with 'nginx.ingress.kubernetes.io/backend-protocol: HTTPS' )  is simple enough, expose a hostname listening on port 80, redirect that that traffic to the reverse proxys service port:

spec:
  rules:
  - host: some.internal.host.name
  http:
    paths:
    - backend:
      serviceName: verify-access-isvawrp-webseal
      servicePort: 443
    pathType: ImplementationSpecific



Not being a Kubernetes specialist, what  am I missing/misunderstanding?


Thanks in advance
Anders

------------------------------
Anders Domeij
CGI Sweden AB
------------------------------

Hi Anders,

If you have a working configuration and a failing configuration, I would hope that you'd be able to see some difference in a browser trace between the two that would point in the right direction.  I would want to focus especially in cookies being set and returned - and on the hostname/port in any re-directs that are sent.  My suspicion would be that the change in protocol and/or hostname is causing state to be lost in either WebSEAL or the backend service.

Looking at your ingress configuration, am I right in thinking that you are connecting with the client on port 80 (non-secure) and then having it forward to WebSEAL on port 443 (secure)?  If that's the case then I'm not surprised if things don't work as expected.  Cookies set by backend and WebSEAL are likely to have the secure flag set which means the browser won't return them on the non-secure connection it has to the ingress.

This is the ingress configuration I have used - which makes both client->ingress and ingress->WebSEAL secure:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: verify-access-isvawrp-webseal
  annotations:
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
    nginx.ingress.kubernetes.io/affinity: "cookie"
spec:
  tls:
  - hosts:
    - www.iamlab.ibm.com
  rules:
  - host: www.iamlab.ibm.com
    http:
      paths:
      - path: /
        backend:
          serviceName: verify-access-isvawrp-webseal
          servicePort: 443

Jon.

------------------------------
Jon Harry
Consulting IT Security Specialist
IBM
------------------------------

Original Message:
Sent: Mon October 05, 2020 11:48 AM
From: Anders Domeij
Subject: Kubernetes -- Reverse Proxy and Ingress oddities

Hi ,

Running both ISAM and ISVA from the latest Helm Charts in a Kubernetes cluster with 2 worker nodes.

we have a case open with IBM support were we reported a compatibility issue between ISAM 9.0.7 and ISVA 10, when trying to access IBM ICN.
It worked with ISAM but not with ISVA.

I've spent some (read "a lot of") time making sure that the environments and test case for both ISAM and ISVA are the same.
They are now deployed in seperate namespaces in the same Kubernetes Cluster accessing (junctioning to) the same ICN backend.
In doing this, I now can say that ISAM and ISVA both can access ICN with expected results and both ISAM and ISVA can fail to do so in what I hope is a consistent manner.

If I log in to the reverse proxy (for either ISAM or ISVA) directly on the nodeport it is assigned by the Kubernetes deploy, everything works as expected :-), i.e nodeIP:nodeport/junctiontoICN connects and responds correctly for ICN tasks.

But if I add an ingress to the Kubernetes cluster (mainly to allow for hostname routing/virtual hosts) the connection from the reverse proxy to ICN causes a continuous reload loop of the ICN desktop. It looks like ICN runs through all the steps for building the desktop and once it's done repeats the same operation over and over.

The ingress (annotated with 'nginx.ingress.kubernetes.io/backend-protocol: HTTPS' )  is simple enough, expose a hostname listening on port 80, redirect that that traffic to the reverse proxys service port:

spec:
  rules:
  - host: some.internal.host.name
  http:
    paths:
    - backend:
      serviceName: verify-access-isvawrp-webseal
      servicePort: 443
    pathType: ImplementationSpecific



Not being a Kubernetes specialist, what  am I missing/misunderstanding?


Thanks in advance
Anders

------------------------------
Anders Domeij
CGI Sweden AB
------------------------------

Thanks Jon

Is it OK for you to PM me?
I have some questions regarding this that I'm not at liberty st share in a community.

Thanks

------------------------------
Anders Domeij
CGI Sweden AB
------------------------------

Original Message:
Sent: Mon October 05, 2020 12:44 PM
From: Jon Harry
Subject: Kubernetes -- Reverse Proxy and Ingress oddities

Hi Anders,

If you have a working configuration and a failing configuration, I would hope that you'd be able to see some difference in a browser trace between the two that would point in the right direction.  I would want to focus especially in cookies being set and returned - and on the hostname/port in any re-directs that are sent.  My suspicion would be that the change in protocol and/or hostname is causing state to be lost in either WebSEAL or the backend service.

Looking at your ingress configuration, am I right in thinking that you are connecting with the client on port 80 (non-secure) and then having it forward to WebSEAL on port 443 (secure)?  If that's the case then I'm not surprised if things don't work as expected.  Cookies set by backend and WebSEAL are likely to have the secure flag set which means the browser won't return them on the non-secure connection it has to the ingress.

This is the ingress configuration I have used - which makes both client->ingress and ingress->WebSEAL secure:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: verify-access-isvawrp-webseal
  annotations:
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
    nginx.ingress.kubernetes.io/affinity: "cookie"
spec:
  tls:
  - hosts:
    - www.iamlab.ibm.com
  rules:
  - host: www.iamlab.ibm.com
    http:
      paths:
      - path: /
        backend:
          serviceName: verify-access-isvawrp-webseal
          servicePort: 443

Jon.

------------------------------
Jon Harry
Consulting IT Security Specialist
IBM

Original Message:
Sent: Mon October 05, 2020 11:48 AM
From: Anders Domeij
Subject: Kubernetes -- Reverse Proxy and Ingress oddities

Hi ,

Running both ISAM and ISVA from the latest Helm Charts in a Kubernetes cluster with 2 worker nodes.

we have a case open with IBM support were we reported a compatibility issue between ISAM 9.0.7 and ISVA 10, when trying to access IBM ICN.
It worked with ISAM but not with ISVA.

I've spent some (read "a lot of") time making sure that the environments and test case for both ISAM and ISVA are the same.
They are now deployed in seperate namespaces in the same Kubernetes Cluster accessing (junctioning to) the same ICN backend.
In doing this, I now can say that ISAM and ISVA both can access ICN with expected results and both ISAM and ISVA can fail to do so in what I hope is a consistent manner.

If I log in to the reverse proxy (for either ISAM or ISVA) directly on the nodeport it is assigned by the Kubernetes deploy, everything works as expected :-), i.e nodeIP:nodeport/junctiontoICN connects and responds correctly for ICN tasks.

But if I add an ingress to the Kubernetes cluster (mainly to allow for hostname routing/virtual hosts) the connection from the reverse proxy to ICN causes a continuous reload loop of the ICN desktop. It looks like ICN runs through all the steps for building the desktop and once it's done repeats the same operation over and over.

The ingress (annotated with 'nginx.ingress.kubernetes.io/backend-protocol: HTTPS' )  is simple enough, expose a hostname listening on port 80, redirect that that traffic to the reverse proxys service port:

spec:
  rules:
  - host: some.internal.host.name
  http:
    paths:
    - backend:
      serviceName: verify-access-isvawrp-webseal
      servicePort: 443
    pathType: ImplementationSpecific



Not being a Kubernetes specialist, what  am I missing/misunderstanding?


Thanks in advance
Anders

------------------------------
Anders Domeij
CGI Sweden AB
------------------------------

Hello Anders,
Not sure if you are using a K8s cluster on a public cloud provider or on bare metal ?
You will need to use HTTPS from the client to the loadbalancer and from the load balancer to the service object (WebSEAL).
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
    nginx.ingress.kubernetes.io/affinity: "cookie"
The first annotation is to guarantee HTTPS traffic between the "NGINX " application load balancer and the service
The latter annotation is to guarantee stickiness to a specific POD via the service . Otherwise your request will go from one pod to another pod and you will have to reauthenticate.
Normally you also will need a certificate on the first (TLS) termination point

Kind regards,
Serge Vereecke

------------------------------
Serge Vereecke
------------------------------

Original Message:
Sent: Wed October 07, 2020 02:44 AM
From: Anders Domeij
Subject: Kubernetes -- Reverse Proxy and Ingress oddities

Thanks Jon

Is it OK for you to PM me?
I have some questions regarding this that I'm not at liberty st share in a community.

Thanks

------------------------------
Anders Domeij
CGI Sweden AB

Original Message:
Sent: Mon October 05, 2020 12:44 PM
From: Jon Harry
Subject: Kubernetes -- Reverse Proxy and Ingress oddities

Hi Anders,

If you have a working configuration and a failing configuration, I would hope that you'd be able to see some difference in a browser trace between the two that would point in the right direction.  I would want to focus especially in cookies being set and returned - and on the hostname/port in any re-directs that are sent.  My suspicion would be that the change in protocol and/or hostname is causing state to be lost in either WebSEAL or the backend service.

Looking at your ingress configuration, am I right in thinking that you are connecting with the client on port 80 (non-secure) and then having it forward to WebSEAL on port 443 (secure)?  If that's the case then I'm not surprised if things don't work as expected.  Cookies set by backend and WebSEAL are likely to have the secure flag set which means the browser won't return them on the non-secure connection it has to the ingress.

This is the ingress configuration I have used - which makes both client->ingress and ingress->WebSEAL secure:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: verify-access-isvawrp-webseal
  annotations:
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
    nginx.ingress.kubernetes.io/affinity: "cookie"
spec:
  tls:
  - hosts:
    - www.iamlab.ibm.com
  rules:
  - host: www.iamlab.ibm.com
    http:
      paths:
      - path: /
        backend:
          serviceName: verify-access-isvawrp-webseal
          servicePort: 443

Jon.

------------------------------
Jon Harry
Consulting IT Security Specialist
IBM

Original Message:
Sent: Mon October 05, 2020 11:48 AM
From: Anders Domeij
Subject: Kubernetes -- Reverse Proxy and Ingress oddities

Hi ,

Running both ISAM and ISVA from the latest Helm Charts in a Kubernetes cluster with 2 worker nodes.

we have a case open with IBM support were we reported a compatibility issue between ISAM 9.0.7 and ISVA 10, when trying to access IBM ICN.
It worked with ISAM but not with ISVA.

I've spent some (read "a lot of") time making sure that the environments and test case for both ISAM and ISVA are the same.
They are now deployed in seperate namespaces in the same Kubernetes Cluster accessing (junctioning to) the same ICN backend.
In doing this, I now can say that ISAM and ISVA both can access ICN with expected results and both ISAM and ISVA can fail to do so in what I hope is a consistent manner.

If I log in to the reverse proxy (for either ISAM or ISVA) directly on the nodeport it is assigned by the Kubernetes deploy, everything works as expected :-), i.e nodeIP:nodeport/junctiontoICN connects and responds correctly for ICN tasks.

But if I add an ingress to the Kubernetes cluster (mainly to allow for hostname routing/virtual hosts) the connection from the reverse proxy to ICN causes a continuous reload loop of the ICN desktop. It looks like ICN runs through all the steps for building the desktop and once it's done repeats the same operation over and over.

The ingress (annotated with 'nginx.ingress.kubernetes.io/backend-protocol: HTTPS' )  is simple enough, expose a hostname listening on port 80, redirect that that traffic to the reverse proxys service port:

spec:
  rules:
  - host: some.internal.host.name
  http:
    paths:
    - backend:
      serviceName: verify-access-isvawrp-webseal
      servicePort: 443
    pathType: ImplementationSpecific



Not being a Kubernetes specialist, what  am I missing/misunderstanding?


Thanks in advance
Anders

------------------------------
Anders Domeij
CGI Sweden AB
------------------------------