API Connect

 View Only

IBM API Connect: How to use Load Balancers alongside Developer Portal's Security Modules.

By Reece Oliver posted Thu April 06, 2023 11:22 AM

  

Introduction

NOTE: PLEASE MAKE SURE YOUR LOAD BALANCER IS CORRECTLY CONFIGURED TO FORWARD THE CLIENTS IPS OTHERWISE THIS TUTORIAL WILL NOT WORK. THIS IS NOT SOMETHING WE CAN ASSIST ON AND IT'S DOWN TO YOU TO LOOK AT THE DOCUMENTATION OF YOUR LOAD BALANCER TO ENABLE THIS.

Having multiple load balancers and reverse proxies in your upstream network can cause them to be banned instead of the client IP if the perimeter security module is triggered on the Developer Portal. In 10.0.5.2 and 10.0.1.9 and above we have added Trusted Reverse Proxies which allows a Drupal module to control reverse proxies list making sure the correct IP gets banned.

X-Forwarded-For: <client>, <proxy1>, <proxy2> the first ip should be your client-ip and this is the ip that will be banned, this blog will help with some of the common ways you should setup your network to facilitate this.

In versions older than those, you will have to add all the Load Balancers IPs manually using the Portal Admin CLI after configuring the ingress to forward all IPs https://www.ibm.com/docs/en/api-connect/10.0.5.x_lts?topic=allowlist-apic-service-ip-allowlistadd

VMware / OVA Steps

Configure NGINX Ingress to Forward IPs

First you need to ssh into your Portal OVA

Next edit the configmap kubectl edit configmap -n kube-system ingress-nginx-ingress-controller and add the following options to the data object.

Setting these options makes the ingress controller pass through the inbound X-Forwarded-For header instead of overwriting them. As Drupal needs to know the full chain of IPs so it know what client IP to block.

Enabling Trusted Reverse Proxies

To enable this feature we need to add a environment variable to the Portal Cluster. First you will need to create an extra values yaml file that contains the following

spec:
  template:
  - containers:
    - env:
       - name: ENABLE_TRUSTED_REVERSE_PROXY
         value: "true"
      name: admin
    name: www

Next set the extra-values in the project directory to point to this file.

apicup subsys set <portal_subsystem_name> extra-values-file extravalues.yaml

Finally deploy the changes to the Portal Cluster by running

apicup subsys install <portal_subsystem_name>

Kubernetes Steps

Configure NGINX Ingress to Forward IPs

Locate the config map for your NGINX ingress controller. In this example we have installed ours into kube-system yours maybe installed elsewhere.

Next edit the configmap kubectl edit configmap -n kube-system ingress-controller-ingress-nginx-controller and add the following options to the data object.

Setting these options makes the ingress controller pass through the inbound X-Forwarded-For header instead of overwriting them. As Drupal needs to know the full chain of IPs so it know what client IP to block.

Enabling Trusted Reverse Proxies

To enable this feature we need to add a environment variable to the Portal Cluster. First you will need to get the name of you portal cluster by running kubectl get ptl

No that you have the name of your Portal cluster you need to edit it and add the following to the spec object. kubectl edit ptl portal

  template:
  - containers:
    - env:
       - name: ENABLE_TRUSTED_REVERSE_PROXY
         value: "true"
       name: admin
    name: www

Openshift (OCP / CP4i) Steps

Infrastructure Node running haproxy located infront of an APIC Instance and NGINX Ingress

We have noticed that this setup is fairly common as well so here are some tips on how to correctly setup haProxy so that it forwards all IP addresses. In the example we will be using an OCP cluster but same can apply for CP4I.

Our haProxy is configured in tcp mode with the following config:

frontend ingress-https
        bind *:443
        default_backend ingress-https
        mode tcp
        option tcplog

backend ingress-https
        balance source
        mode tcp
        server master0 <redacted-ip>:443 check 
        server master1 <redacted-ip>:443 check 
        server master2 <redacted-ip>443 check 
        server worker0 <redacted-ip>:443 check 
        server worker1 <redacted-ip>:443 check 
        server worker2 <redacted-ip>:443 check 
        server worker3 <redacted-ip>:443 check 
        server worker4 <redacted-ip>:443 check 
        server worker5 <redacted-ip>:443 check 
        server worker6 <redacted-ip>:443 check 

As documented here - How to manage IP security in the Developer Portal, you can get the client ip passed through from the external haProxy using the send-proxy directive 'provided both the load balancer and the ingress controller are compatible with the proxy protocol, and have the protocol enabled'. In the case of our OCP cluster the ingress controller doesn't have the PROXY protocol enabled by default.

First modify the external haProxy config to add send-proxy on to the end of the server lines:


Once you have saved your changes you will need to reload haProxy `systemctl reload haproxy`

Next modify the ingress controller definition with the following block:

First set your project to location of the ingress controller our is located in oc project openshift-ingress-operator

Then edit the Ingress Controller by running oc edit IngressController and adding the following snippet to the spec object

  endpointPublishingStrategy:
    hostNetwork:
      protocol: PROXY
    type: HostNetwork

Enabling Trusted Reverse Proxies

To enable this feature we need to add a environment variable to the Portal Cluster. First you will need to get the name of you portal cluster by running oc get ptl or if you are using the top level CR oc get apiconnectcluster

No that you have the name of your Portal cluster you need to edit it and add the following to the spec object. oc edit ptl <portal-name> or if you are using top level like me oc edit apiconnectcluster production.

Clearing Banned IPs

If you have accidentally banned your load balancer IPs and need to clear the bans. Then you can use the Portal Admin CLI command security:clear-bans documented at https://www.ibm.com/docs/en/api-connect/10.0.5.x_lts?topic=security-apic-securityclear-bans

Conclusion

Once you have saved the changes and added the template overrides the www pods should restart with the environment variables set on them and the Trusted Reverse Proxy feature should now be enabled. Testing the perimeter module now you should see the Client IP being banned and not your load balancer’s.

There are some warnings that come with enabling Trusted Reverse Proxy - If clients are able to spoof the x-forwarded-for header, IP bans and other such features on your site could be circumvented. Only enable if you know and understand your upstream network.


#APIConnect #developerportal #portal #drupal

0 comments
94 views

Permalink