DataPower horizontal pod autoscaling with MicroK8s

By Pedro Busko posted Fri June 04, 2021 01:48 PM


The purpose of this post is to describe the steps of a lightweight Autoscaling DataPower Pods scenario, running in a laptop for example, the instructions are not Production ready recommendations and are solely for demo and POT purposes.

The Kubernetes implementation used was MicroK8s.

Hardware used

Apple Mac Mini (Late 2012)
Processor: 2.6Ghz Quad-Core Intel Core i7
Memory: 16GB RAM

Install MicroK8s

As a first step, let’s install MicroK8s on your machine by running the following commands:

Install Homebrew

Because we are doing this on a Mac, first thing we need is to install Homebrew

/bin/bash -c "$(curl -fsSL"​

Install MicroK8s

Now let's install MicroK8s:

brew install ubuntu/microk8s/microk8s

microk8s install

microk8s status --wait-ready​


There are scenarios where the Microk8s installation command fails, this is because for some reason, in some cases macOS’s built-in DNS server is broken.

If that happens, we need some additional steps to finish MicroK8s installation. First let’s configure DNS inside the VM instance to use an external working DNS server.

Access the VM:

multipass shell microk8s-vm

Then let’s append the DNS server to resolv.conf, we will use which is a free DNS service provided by CloudFlare. From inside the VM:

sudo vi /etc/resolv.conf

Append the following line:


ATTENTION - resolv.conf file will be rewritten at logon time. You need to do this workaround every time you logon or keep the session open, an alternative is to use a custom cloud-init to set /etc/resolv.conf for you on first boot

Next step, is to install MicroK8s directly inside the VM:

sudo snap install microk8s --classic --channel=latest/stable
sudo iptables -P FORWARD ACCEPT

sudo usermod -a -G microk8s $USER
sudo chown -f -R $USER ~/.kube

Now on the host machine from a new terminal window, check MicroK8s status:

microk8s status --wait-ready

Enable MicroK8s add-ons

MicroK8s is a lightweight Kubernetes implementation, and only install the bare minimum components, to have the additional capabilities installed, we need to enable the add-ons. Run the following commands to enable add-ons:

microk8s enable dashboard 
microk8s enable dns
microk8s enable ha-cluster
microk8s enable helm3
microk8s enable ingress
microk8s enable metrics-server
microk8s enable registry
microk8s enable storage

Changing Multipass instance memory and CPU

Since we are running this on a Mac, MicroK8s requires Multipass, a VM system for running
Ubuntu and other packages required by MicroK8s. The default VM created on the installation is not enough and we need to change its memory and CPU. Here we used 4 CPUs and 12GB of RAM

# stop multipassd
sudo launchctl unload /Library/LaunchDaemons/com.canonical.multipassd.plist

# edit the values
sudo vi /var/root/Library/Application\ Support/multipassd/multipassd-vm-instances.json​

"primary": {
"deleted": false,
"disk_space": "5368709120",
"mac_addr": "52:54:00:5c:f4:14",
"mem_size": "12884901888",
"metadata": {
"mounts": [
"num_cores": 4,
"ssh_username": "ubuntu",
"state": 0

# start multipassd again
sudo launchctl load /Library/LaunchDaemons/com.canonical.multipassd.plist​

Install Helm

Because we are using a lightweight Kubernetes implementation, we will use Helm to install the DataPower operator that will be used later. 

brew install helm​

Install DataPower operator

Here are the steps to install DataPower operator:

First you download 1.3.1 release.

Unpack it

cd charts/stable/datapower-operator

helm install datapower-operator .

Create DataPowerService operand

For testing purpose, we will be using an XML Firewall that is simply a passthrough to a public API, on this case hosted on, the XML FW is configured on the xmlfirewall.cfg file.

%if% available "xmlfirewall"

xmlfirewall "XMLFirewall"
  local-address "" "2048"
  summary "an example XML Firewall Service"
  priority normal
  default-param-namespace ""
  query-param-namespace ""
  no force-policy-exec 
  monitor-processing-policy terminate-at-first-throttle
  debugger-type internal
  debug-history 25
  remote-address "" "80"
  xml-manager default
  stylesheet-policy default
  max-message-size 0
  request-type json
  response-type unprocessed
  request-attachments strip
  response-attachments strip
  root-part-not-first-action process-in-order
  front-attachment-format dynamic
  back-attachment-format dynamic
  delay-errors-duration 1000
  soap-schema-url "store:///schemas/soap-envelope.xsd"
  wsdl-response-policy off
  no firewall-parser-limits 
  bytes-scanned 4194304
  element-depth 512
  attribute-count 128
  max-node-size 33554432
  max-prefixes 1024
  max-namespaces 1024
  max-local-names 60000
  attachment-byte-count 2000000000
  attachment-package-byte-count 0
  external-references forbid
  credential-charset protocol
  ssl-config-type proxy

http "XMLFirewall" xmlfirewall
  http-timeout 120
  persistent-timeout 180
  host-rewriting "on"
  no silence-warning 
  no compression 
  no include-response-type-encoding
  no always-show-errors
  no disallow-get
  no disallow-empty-reply
  client-address "X-Client-IP"
  http-global-tranID-label "X-Global-Transaction-ID"
  httpproxy-port 800
  version "HTTP/1.1" "HTTP/1.1"
  no chunked-uploads 


And for the DataPower operand custom resource, a YAML file (my_datapowerservice_cr.yaml) is used. This is where the DataPower CPU, Memory, and Autoscaling is configured.

For testing purposes, we are setting up the DataPower resources with the bare minimal supported, 1 initial replica with 1 CPU and 4Gi RAM. And Horizontal Pod Autoscaling with a 10% target CPU utilization and max of 2 replicas.

kind: DataPowerService
  name: mydatapower
    accept: true
    use: developers-limited
    license: L-RJON-BYDQZF
      cpu: 1
      memory: 4Gi
      memory: 4Gi
    method: HPA
      targetCPUUtilizationPercentage: 10
      targetMemoryUtilizationPercentage: 0
      maxReplicas: 2
      minReplicas: 1
  - name: "test-domain"
      - "xmlfirewall-config"
  version: 10.0-cd
  replicas: 1
  - name: admin
    passwordSecret: admin-credentials
    accessLevel: privileged​

These are the commands to create all the objects necessary for the DataPower operand:

microk8s kubectl create secret generic admin-credentials --from-literal=password=admin

microk8s kubectl create configmap xmlfirewall-config --from-file=xmlfirewall.cfg

microk8s kubectl apply -f my_datapowerservice_cr.yaml​

Create Service

There are multiple ways to expose a service to external network requests, the simplest way is using NodePort. NodePort is a configuration setting you declare in a service’s YAML. Set the service spec’s type to NodePort. Then, Kubernetes will allocate a specific port on each Node to that service, and any request to your cluster on that port gets forwarded to the service.

On this case we created a service called xmlfirewall, defined on xmlfirewall_service.yml. Port 2048 is the one used by the XML Firewall.

apiVersion: v1
kind: Service
  name: xmlfirewall
  type: NodePort
  selector: datapower default-mydatapower
  - protocol: TCP
    port: 2048
    targetPort: 2048
    name: xmlfirewall​

These are the commands to create the Service that will be exposed using NodePort:

microk8s kubectl apply -f xmlfirewall_service.yml

microk8s kubectl get service​

You should write down the node port, you will need later.

Next step is to get the Multipass VM IP address that can be accessed from the host machine, your laptop for example.

multipass list

Write down the IP address that belongs to the same network as the host machine.

The service endpoint will be http://<<ip address>>:<<node port>>

Load Test

For the Load Test, we will use SOAP UI and the LoadTEST feature. Here are the steps:

Download and Install SOAP UI

Create new REST Request

Generate new Test Suite with a Load Test

Edit the Load Test for the desired settings

Run the Test Suite and observe the pod scaling on the Dashboard or on CLI


microk8s kubectl top pods

Scale up

Scale down


Scale up

Scale down

I'd like to thank @Bruno R Neves and@Aidan Harbison for the help troubleshooting and reviewing this material.


Troubleshooting “Error: Kubernetes cluster unreachable”

mkdir /etc/microk8s
microk8s.config > /etc/microk8s/microk8s.conf
export KUBECONFIG=/etc/microk8s/microk8s.conf​

Error: .microk8s/config: no such file or directory

To fix this error, do the following steps:
cd $HOME
mkdir .microk8s
cd .microk8s
microk8s config > config​

How to access the Multipass VM?

multipass shell microk8s-vm​

How to access Kubernetes dashboard?

Simplest way is with the following command:

microk8s dashboard-proxy​

#write down the token

If that fails, there is a second alternative:

token=$(microk8s kubectl -n kube-system get secret | grep default-token | cut -d " " -f1)
microk8s kubectl -n kube-system describe secret $token​

#write down the token

microk8s kubectl port-forward -n kube-system service/kubernetes-dashboard 10443:443​

How to access DataPower WebGUI?

microk8s kubectl attach mydatapower-0 --stdin --tty

idg# co
idg(config)# web-mgmt
idg(config web-mgmt)# admin-state enabled
idg(config web-mgmt)# exit

microk8s kubectl port-forward mydatapower-0 9090:9090​

How to start and stop MicroK8s?

microk8s start

microK8s stop​




Thu June 10, 2021 08:01 PM

Thank you @Pedro Busko for an easy to follow, step by step guide!