Instana U

 View Only

How to Deploy Instana Synthetic PoP in an Air-Gapped Environment

By Hai Jun Xu posted Wed August 16, 2023 01:32 AM

  

Co-Authors

Hai Jun Xu (xuhjbj@cn.ibm.com) - Developer for Instana Synthetic PoP

Rong Zhu Shang(shangrz@cn.ibm.com) - Developer for Instana Synthetic PoP

Rui Yin (yinrui@cn.ibm.com) - Tech lead for Instana Synthetic

What is Synthetic Monitoring?

Instana Synthetic monitoring, also known as proactive monitoring, can simulate actions that an end-user takes on an application from different locations. , and continuously monitors at a specific interval for performance characteristics such as availability and response time. Synthetic monitoring is built on Instana, and it works seamlessly with other Instana capabilities. With the Synthetic monitoring features, you can create Synthetic tests to monitor an application.

What is Synthetic PoP?


Instana Synthetic PoP (Point of Presence) is the agent in which the Synthetic tests are run. Synthetic PoP can be deployed by using the Helm charts. 

An air-gapped environment is a network environment that does not have internet access.

This blog covers deploying an Instana Synthetic PoP in an air gapped environment.

Deploying Synthetic PoP in an Air-Gapped Environment

Before you deploy Synthetic PoP, ensure that following prerequisites are met:

·        Two Linux virtual machines with the following configuration:

  • VM A:

      operation system: Ubuntu 20.04

      host name: airgap1.fyre.ibm.com

      The VM must have internet access.

  • VM B:

       Operation system: Ubuntu 22.04

      Host name: testversion1.fyre.ibm.com

      The VM is the target host with no internet connectivity.

Make sure the following tools are installed on both VMs:

  • Docker:

You can use the following command to install Docker 20.10 on the main Linux distributions such as SUSE Linux Enterprise or Ubuntu:

 curl https://releases.rancher.com/install-docker/20.10.sh | sh

  •      Helm:

https://helm.sh/

  •        Synthetic PoP support different Kubernetes distributions such as ocp, minkube, and microk8s. Microk8s is used as an example in the document. Microk8s:  https://microk8s.io/

References for installing tools on VM B:

·        Installing MicroK8s Offline or in an air gapped environment

·        Installing Helm from the Binary releases

root@testversion1:~# tar -zxvf helm-v3.12.2-linux-amd64.tar.gz

linux-amd64/ linux-amd64/helm linux-amd64/LICENSE

root@testversion1:~# mv linux-amd64/helm /usr/local/bin/helm

root@testversion1:~# helm version

version.BuildInfo{Version:"v3.12.2", GitCommit:"1e210a2c8cc5117d1055bfaa5d40f51bbc2e345e", GitTreeState:"clean", GoVersion:"go1.20.5"}

Complete the following steps to deploy an Instana Synthetic PoP in an air-gapped environment:

     1.  Prepare and get access to the instana docker image registry from VM A: Manually pull all the synthetic pop images by using the username and password. The username is ”_”. Use download key from VM A as the password.

root@airgap1:~/airgap/images# docker login containers.instana.io -u _ -p <downloadkey>

        2.  Download the necessary images: (Here are using the image.tag from the default values in pop helm chart , different engines with different values.)

docker pull containers.instana.io/synthetic/synthetic-pop-controller:1.264.0

docker save containers.instana.io/synthetic/synthetic-pop-controller:1.264.0 > synthetic-pop-controller.tar

docker pull containers.instana.io/synthetic/synthetic-playback-http:1.264.0

docker save  containers.instana.io/synthetic/synthetic-playback-http:1.264.0 > synthetic-playback-http.tar

docker pull containers.instana.io/synthetic/synthetic-playback-javascript:1.264.0

docker save containers.instana.io/synthetic/synthetic-playback-javascript:1.264.0 > synthetic-playback-javascript.tar

docker pull containers.instana.io/synthetic/synthetic-playback-browserscript:1.264.0

docker save containers.instana.io/synthetic/synthetic-playback-browserscript:1.264.0 > synthetic-playback-browserscript.tar

docker pull containers.instana.io/synthetic/redis:1.263.0

docker save containers.instana.io/synthetic/redis:1.263.0 > redis.tar

      3. Check whether the following files are available in the images directory:

root@testversion1:~/airgap/images#

-rw-r--r-- 1 root root 851139584 Jul 11 02:54 pop-controller.tar

-rw-r--r-- 1 root root 259670528 Jul 11 02:58 redis.tar

-rw-r--r-- 1 root root 797607424 Jul 11 02:56 synthetic-playback-http.tar

-rw-r--r-- 1 root root 797607424 Jul 11 02:57 synthetic-playback-javascript.tar

4. Archive the image directory:

tar -cvf  data.tar ~/airgap/images

5. Copy the tar file to the local registry of the VM B (host machine):

The tar file is around 2.6 GB.

6. Copy the images on VM B, move the data.tar file and release the images:

cd /root/airgap/

tar -xvf data.tar

 

Skip step 7, if you have a local registry.

7.  Create a registry server on VM B:

a.  Obtain a certificate authority:

In a production environment, you must obtain a certificate from a CA. In a test or development environment, you can generate your own CA. To generate a CA, complete the following steps:

 i.   Generate a Certificate Authority Certificate:

          1).     Generate a CA certificate private key:

openssl genrsa -out ca.key 4096

         2).    Generate the CA certificate:

Adapt the values in the -subj option to reflect your organization. use the FQDN to connect registry host, specify it as the common name (CN) attribute.

openssl req -x509 -new -nodes -sha512 -days 3650 \

 -subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=testversion1.fyre.ibm.com" \

 -key ca.key \

 -out ca.crt

 ii.     Generate a server certificate:

                                     1).     Generate a CA certificate private key:

openssl genrsa -out testversion1.fyre.ibm.com.key 4096

                    2).   Generate the CA certificate:

 openssl req -sha512 -new \

    -subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=testversion1.fyre.ibm.com" \

    -key testversion1.fyre.ibm.com.key \

    -out testversion1.fyre.ibm.com.csr

                    3). Generate an x509 v3 extension file:

cat > v3.ext <<-EOF

authorityKeyIdentifier=keyid,issuer

basicConstraints=CA:FALSE

keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment

extendedKeyUsage = serverAuth

subjectAltName = @alt_names

[alt_names]

DNS.1=testversion1.fyre.ibm.com

DNS.2=testversion1.fyre.ibm

DNS.3=testversion1.fyre.ibm.com

EOF

Use the v3.ext file to generate a certificate for registry host. CRS and CRT file names are same with the registry host name.

openssl x509 -req -sha512 -days 3650 \

    -extfile v3.ext \

    -CA ca.crt -CAkey ca.key -CAcreateserial \

    -in testversion1.fyre.ibm.com.csr \

    -out testversion1.fyre.ibm.com.crt

b. Provide the certificates to Registry and Docker:

When using authentication, some versions of Docker also require you to trust the certificate at the operating system level.

openssl s_client -showcerts -connect testversion1.fyre.ibm.com:5000 </dev/null 2>/dev/null|openssl x509 -outform PEM >mycertfile.pem

cp mycertfile.pem /usr/local/share/ca-certificates/testversion1.fyre.ibm.com.crt

update-ca-certificates

8.  Restrict access:

The simplest way to achieve access restriction is through basic authentication. It is very similar to the basic authentication mechanism on the other web server. The following example shows storing the secrets with htpasswd by using native basic authentication.

Create a password file with one entry for the user “testuser”, with the password “testpassword”:

mkdir auth

docker run \

  --entrypoint htpasswd \

  httpd:2 -Bbn testuser testpassword > auth/htpasswd

9.  Check the prepared items:

root@testversion1:~/airgap#

auth\

     htpasswd

mycerts\

    ca.crt  ca.key  

    testversion1.fyre.ibm.com.crt  

    testversion1.fyre.ibm.com.key  

10. Start the registry server:

docker run -d \

  --restart=always \

  --name registry \

  -v "$(pwd)"/auth:/auth \

  -e "REGISTRY_AUTH=htpasswd" \

  -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" \

  -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \

  -v "$(pwd)"/mycerts:/certs \

  -e REGISTRY_HTTP_ADDR=0.0.0.0:5000 \

  -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/testversion1.fyre.ibm.com.crt \

  -e REGISTRY_HTTP_TLS_KEY=/certs/testversion1.fyre.ibm.com.key \

  -p 5000:5000 \

  registry:2

11. Check the registry server:

root@testversion1:~# docker ps

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES

34eadb5814f9        registry:2          "/entrypoint.sh /etc…"   2 days ago          Up 2 days           0.0.0.0:5000->5000/tcp   registry

12. Load the images to the local registry:

      Check whether Docker clients can pull and push to your registry by using its external address, run the following commands:

docker load < synthetic-pop-controller.tar

docker tag containers.instana.io/synthetic/synthetic-pop-controller:1.264.0 localhost:5000/synthetic/synthetic-pop-controller:1.264.0

docker push localhost:5000/synthetic/synthetic-pop-controller:1.264.0

docker load < synthetic-playback-http.tar

docker tag containers.instana.io/synthetic/synthetic-playback-http:1.264.0 localhost:5000/synthetic/synthetic-playback-http:1.264.0

docker push localhost:5000/synthetic/synthetic-playback-http:1.264.0

docker load < synthetic-playback-javascript.tar

docker tag containers.instana.io/synthetic/synthetic-playback-javascript:1.264.0 localhost:5000/synthetic/synthetic-playback-javascript:1.264.0

docker push localhost:5000/synthetic/synthetic-playback-javascript:1.264.0

docker load < synthetic-playback-browserscript.tar

docker tag containers.instana.io/synthetic/synthetic-playback-browserscript:1.264.0 localhost:5000/synthetic/synthetic-playback-browserscript:1.264.0

docker push localhost:5000/synthetic/synthetic-playback-browserscript:1.264.0

docker load < redis.tar

docker tag containers.instana.io/synthetic/redis:1.263.0 localhost:5000/synthetic/redis:1.263.0

docker push localhost:5000/synthetic/redis:1.263.0

13. Check local registry images:

curl -u testuser:testpassword -X GET https://testversion1.fyre.ibm.com:5000/v2/_catalog

{"repositories":["synthetic/redis","synthetic/synthetic-playback-browserscript","synthetic/synthetic-playback-http","synthetic/synthetic-playback-javascript","synthetic/synthetic-pop-controller"]}

14. Install PoP:

          a.  Download helm charts, click the https://artifacthub.io/packages/helm/instana/synthetic-pop?modal=install link in helm on VM A and then copy the charts into VM B.

The following is an example of the helm chart:

root@testversion1:~/airgap# tar -zxvf synthetic-pop-1.1.4.tgz

root@testversion1:~/airgap# ls

synthetic-pop synthetic-pop-1.0.14.tgz

  b.  Create secret:

Create a secret in with namespace synap which is used to access the local docker registry. Set the right credentials for docker username and docker password:

kubectl create secret docker-registry my-secret \

  --docker-server=testversion1.fyre.ibm.com:5000 \

  --docker-username=testuser \

  --docker-password=testpassword \

  --docker-email=mymail@example.com \

  --namespace=synap

  c.   Run the helm command:

Run the helm command to deploy Synthetic PoP on namespace synap. You can set the imagePath and downloadSecret with Synthetic PoP helm chart.

The following example shows the default values in the values.yaml file:

# Image common path, for instana production, it is "containers.instana.io/synthetic"

imagePath: "containers.instana.io/synthetic"

# Required, either downloadKey or downloadSecret must be provided to pull images

# The string of download key, this value can be found by clicking on "Deploy a PoP" within the Instana Synthetic Monitoring user interface page.

downloadKey: ""

# The name of secret of kubernetes.io/dockerconfigjson type used to pull images from the image registry.

downloadSecret: "instana-io"

Because VM B is an AirGap environment with local docker registry, enter the values shown in the following example:

The value for downloadSecret is my-secret created in the previous step;

The local docker registry server is "testversion1.fyre.ibm.com:5000". Change the value "containers.instana.io" to "testversion1.fyre.ibm.com:5000".

root@testversion1:~/airgap#helm install synthetic-pop \

  ./synthetic-pop \

    --namespace synap \

    --create-namespace \

    --set controller.location="R254;R254;China;Beijing;39.54;116.23;R246Double on Microk8s" \

    --set controller.instanaKey="*****" \

    --set controller.instanaSyntheticEndpoint="https://synsfbd1.fyre.ibm.com/synthetics" \

    --set redis.tls.enabled=false \

    --set redis.password="a1fc5d01bcbb" \

    --set imagePath="testversion1.fyre.ibm.com:5000/synthetic" \

    --set downloadSecret="my-secret" 

 

15.   Check whether Instana Synthetic PoP is deployed successfully, run the following commands: 

root@testversion1:~/airgap# kubectl get pods -n synap

NAME                                                         READY         STATUS       RESTARTS     AGE

synthetic-pop-redis-66c6c6f9b-p7s5l                          1/1           Running        0          4d3h

synthetic-pop-javascript-playback-engine-849bb8bb6d-wxbt7    1/1           Running        0          4d3h

synthetic-pop-http-playback-engine-7bcc8b5c76-bxz4d          1/1           Running        0          4d3h

synthetic-pop-controller-6b57859f58-6vbdc                    1/1           Running        0          162m

synthetic-pop-browserscript-playback-engine-595f45594c-vjg25 1/1           Running        0          4d7m

When you complete the procedure to deploy Synthetic PoP in an air-gapped environment, the operational efficiency improves significantly.   


#synthetic-pop

Permalink