Updated Instana plug-in for OpenTelemetry
Co-Authors
Zhen Gao (gaozhenz@cn.ibm.com) - Developer for Instana OTEL
Zhi Li (bjlizhi@cn.ibm.com) - Developer for Instana OTEL
Rui Liu (liurui@cn.ibm.com) - Tech lead for Instana OTEL
OpenTelemetry-based data collectors work with prebuilt generic sensors in Instana to inject the monitoring data.
The ehnacements on the Instan OpenTelemetry include:
· Improved handling and display of complex metrics with histograms: Histograms are used to show complex metrices. Compared to other types of diagrams, histograms has several advantages. Histograms such as bar charts can present highs, lows, concentration, and trends within the data to indicate common value ranges or concentration within specific intervals. Histograms helps to swiftly identify outliers or anomalies within the data, as they might appear as extreme values in the distribution.
· Attributes of metrics are retained: All types of data are retained with different dimensions.
- Better equipped to manage the dependencies of the apps: The generic sensor and entity plug-in for OpenTelemetry can handle the dependencies of every app that runs in containers.
Design
Histograms:
- Both DELTA mode and CUMULATIVE mode are supported.
- Real-time average value is displayed in the built-in dashboard.
Note: If data goes through otlp-acceptor at backend, the real-time average value for CUMULATIVE mode is lost as date is not supported at backend.
- The _max and _min data points are displayed.
Attributes:
- All OpenTelemetry attributes for data points are retained.
- Resource attributes for service.name, pid, container.id, host.name are retained as tags. All other tags for resource attributes are discarded because too many tags might severely decrease the user experience of custom dashboards.
Dependency with entities:
- For in-container apps, the container.id parameter is identified and dependency is established with the corresponding container entity;
- The dependency between Generic OpenTelemetry entity and OTelProcess entity is optimized.
Sample payload of generic OpenTelemetry plug-in
A sample payload of Generic OpenTelemetry plug-in is shown in the following snippet:
{
"pid": "1234",
"remote": "28322@d4d7c18968f34007a04b815b27582c62",
"resource":{
"service.name":"my-service",
"pid": "1234",
"hostname": "dk-liunoone.fyre.ibm.com"
"container.id": "90202a241f1bc13f96068735ddf4ccbd7b8af7c44c9e2210d835a44c58f39eb4",
"sdk.opentelemetry.type": "sensor-sdk:dameng"
},
"metrics": {
"sums": {
"sdk.logs/db.client.connections.idle.min{pool.name=\"HikariPool-1\"}": 10
},
"gauges": {
"hikaricp/queueSize{logRecordProcessorType=\"BatchLogRecordProcessor\",}": 0
},
"histograms": {
"hikaricp/db.client.connections.create_time_count{pool.name=\"HikariPool-1\", }": 59,
"hikaricp/db.client.connections.create_time_sum{pool.name=\"HikariPool-1\", }": 55.000000,
"hikaricp/db.client.connections.create_time_min{pool.name=\"HikariPool-1\", }": 0.000000,
"hikaricp/db.client.connections.create_time_max{pool.name=\"HikariPool-1\", }": 8.000000,
"hikaricp/db.client.connections.create_time_mean{pool.name=\"HikariPool-1\", }": 3.800000,
"hikaricp/db.client.connections.create_time_bucket{pool.name=\"HikariPool-1\", le=\"0.0\",}":23.0,
"hikaricp/db.client.connections.create_time_bucket{pool.name=\"HikariPool-1\", le=\"5.0\",}":35.0,
"hikaricp/db.client.connections.create_time_bucket{pool.name=\"HikariPool-1\", le=\"10.0\",}":1.0,
"hikaricp/db.client.connections.create_time_bucket{pool.name=\"HikariPool-1\", le=\"25.0\",}":0.0,
"hikaricp/db.client.connections.create_time_bucket{pool.name=\"HikariPool-1\", le=\"50.0\",}":0.0,
"hikaricp/db.client.connections.create_time_bucket{pool.name=\"HikariPool-1\", le=\"75.0\",}":0.0,
"hikaricp/db.client.connections.create_time_bucket{pool.name=\"HikariPool-1\", le=\"100.0\",}":0.0,
"hikaricp/db.client.connections.create_time_bucket{pool.name=\"HikariPool-1\", le=\"250.0\",}":0.0,
"hikaricp/db.client.connections.create_time_bucket{pool.name=\"HikariPool-1\", le=\"500.0\",}":0.0,
"hikaricp/db.client.connections.create_time_bucket{pool.name=\"HikariPool-1\", le=\"750.0\",}":0.0,
"hikaricp/db.client.connections.create_time_bucket{pool.name=\"HikariPool-1\", le=\"1000.0\",}":0.0,
"hikaricp/db.client.connections.create_time_bucket{pool.name=\"HikariPool-1\", le=\"2500.0\",}":0.0,
"hikaricp/db.client.connections.create_time_bucket{pool.name=\"HikariPool-1\", le=\"5000.0\",}":0.0,
"hikaricp/db.client.connections.create_time_bucket{pool.name=\"HikariPool-1\", le=\"7500.0\",}":0.0,
"hikaricp/db.client.connections.create_time_bucket{pool.name=\"HikariPool-1\", le=\"10000.0\",}":0.0,
"hikaricp/db.client.connections.create_time_bucket{pool.name=\"HikariPool-1\", le=\"+Inf\",}":0.0
}
}
}
Deploying OpenTelemetry Demo in OCP
To deploy OpenTelemetry Demo in OCP, complete the following steps:
- Log on to OpenShift CLI (oc).
- Create the namespace otel-demo:
kubectl create namespace otel-demo
kubectl project otel-demo
- Install the otel-demo application by using Helm:
helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
- Install helm chart with the release name my-otel-demo:
helm install my-otel-demo open-telemetry/opentelemetry-demo
- Expose services using kubectl port-forward:
kubectl port-forward svc/my-otel-demo-frontendproxy 8080:8080
- Create route for services for my-otel-demo:
my-otel-demo-frontendproxy
my-otel-demo-grafana
my-otel-demo-jaeger-query
A route that creates my-otel-demo-frontendproxy is shown in the following example
Deploying Instana agent on OCP
To deploy Instana agent on Red Hat OpenShift Container Platform:
helm install instana-agent \
--repo https://agents.instana.io/helm \
--namespace instana-agent \
--create-namespace \
--set openshift=true \
--set agent.key=*************** \
--set agent.downloadKey=*********** \
--set agent.endpointHost=ingress-magenta-saas.instana.rocks \
--set agent.endpointPort=443 \
--set cluster.name='cluster101' \
--set zone.name='zone101' \
instana-agent
Generic OpenTelemetry plug-in for OCP
Custom dashboard page
Histogram
Custom tags for gauge and histogram metric
Generic OpenTelemetry entity plug-in page
Stack infrastructure of a generic OpenTelemetry entity
Dependency tree view of a generic OpenTelemetry entity which runs in a CRI-O container.
Navigation for a generic OpenTelemetry entity which runs on a CRI-O container
Trace of a call from and to OpenTelemetry services page
Analytics call traces
