Instana

 View Only

Using Instana to observe a local Kubernetes cluster with kind and podman on a Mac

By Isabell Sippli posted Fri October 21, 2022 11:28 AM

  
I wanted to play around with some features in Instana (go here for your own free 14 day trial), and for that, I wanted a local, simple setup.
I also had not been using a local Kubernetes setup in a while, so I decided to throw that in the mix, too.

When setting up a local Kubernetes cluster, you have several options. To name a few:
  1. MicroK8s
  2. Minikube
  3. kind

I tried all three, in the order above.
What worked for me best was kind, and I will use it for the remainder of this article.

Please note that environment I used is for test/demo purposes only. Kind should not be used for production purposes ("kind was primarily designed for testing Kubernetes itself, but may be used for local development or CI." source)

My setup
  • My MacBook Pro 2021, with an M1 chip. Note: This is an ARM architecture, which does make a difference for setting up some of the local Kubernetes infrastructure, e.g. hyperkit is not supported on ARM at the time of writing this article. A colleague of mine tried it on an i7 Mac though and it worked as well
  • Podman version 4.2.1
  • Kind version 0.16.0
Installing the Kubernetes cluster
Install podman
Following these instructions:
brew install podman
I then created a VM with more resources than the default (to make it perform well) and a custom short name (due to this issue).
podman machine init --cpus 6 --memory 12288 --disk-size 50 foo
podman machine start foo
podman system connection default foo
Install KIND

Following the instructions to install kind and setup a cluster with rootless podman - no changes needed from what was described in the related article:

brew install kind 
$ KIND_EXPERIMENTAL_PROVIDER=podman kind create cluster

Once this is complete, you can play around with your freshly deployed k8s cluster. You might have to install kubectl, depending on what is already available on your machine.

isabell$ kubectl cluster-info
Kubernetes control plane is running at https://127.0.0.1:61245
CoreDNS is running at https://127.0.0.1:61245/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'..

isabell$ kubectl get all
NAME READY STATUS RESTARTS AGE
pod/dnsutils 1/1 Running 46 (53m ago) 46h

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 46h
Installing the Instana agent into the cluster

I went to the "Agents" page of my Instana SaaS instance (More > Agents > Install Instana Agents), and selected YAML based install

Image showing the screen to Install Instana Agent



I really like the visual guidance towards adding all the required fields.
I then downloaded the yaml (my-agent.yaml)
Before I could deploy it to my cluster, I had to make one minor tweak. In the yaml file, I set one additional property to define initial and max JVM heap size for the Agent.

 containers:
- name: instana-agent
image: "icr.io/instana/agent:latest"
imagePullPolicy: Always
env:
- name: INSTANA_AGENT_LEADER_ELECTOR_PORT
value: "42655"
- name: INSTANA_ZONE
value: "isabell-zone"
[...]
- name: JAVA_OPTS
value: "-Xms150m -Xmx512m"
Why did I need that? The out of the box memory calculator does not work on my configuration, hence the agent's maximum heap was set to 160M, which led to an out-of-memory exception, as the agent needed more memory on my particular instance. I liked that the agent told me this explicitly:
isabell$ kubectl logs instana-agent-hrfm6 -n instana-agent
Defaulted container "instana-agent" out of: instana-agent, leader-elector
Docker socket not found at /var/run/docker.sock
Starting Instana Agent ...
Found 'JAVA_OPTS' settings: '-XX:+ExitOnOutOfMemoryError '.
Skipping the memory calculator: cannot retrieve the maximum amount of memory to be used by the agent from /sys/fs/cgroup/memory/memory.limit_in_bytes .
The memory calculator has not been used, and neither the 'JAVA_MAX_MEM' nor 'JAVA_OPTS' environment variables specify a maximum heap setting; setting the maximum heap to 160m.

Eventually, I could deploy and start my agent with:

isabell$ kubectl apply -f my-agent.yaml
After a while, the agent shows up in the list of available agents in Instana:
Image showing how the Kubernetes host agent shows up in the Instana UI
and the cluster shows up as a new K8s cluster under "Infrastructure":
Image showing local K8s cluster being observed by Instana

This is now a really nice foundation for a fully local setup to be observed by Instana.
Let me know if this worked for you, and share your experience please!
3 comments
88 views

Permalink

Comments

Fri December 06, 2024 11:30 AM

Have you tried to instrument with mode than 1 node? Instana doesn't show the other nodes.

Wed November 02, 2022 11:10 AM

Hi Isabell, 

Thanks for this blog. I am going through it.
I think it may help to add a link on how to get access Instana Free trial. 

Cheers

Salil

Wed October 26, 2022 09:50 AM

Hi Isabell,

Thanks for running the agent and taking the time to write up this guide!
The memory issue you ran into is a change in Docker Desktop which the agent doesn't support yet.
As a different workaround, you may set 

"deprecatedCgroupv1": true,

in ~/Library/Group\ Containers/group.com.docker/settings.json.
This will re-enable cgroup V1 support in Docker Desktop and "fix" the agent's memory calculation.

I will take this to the team and we will provide a fix for this in a future release.

Best
Henning