Instana is a fully automated market-leading Application Performance Management (APM) solution with extensive support for middlewares, services, applications and environments monitoring. One of the most important customer requirements about Instana integration is how to retrieve data from Instana, and use the data in other systems, such as business monitoring systems, KPI systems, Kanban systems, etc.
So, the first question is: How many types of data can you get from Instana ? Generally speaking, you can get metrics, events, traces, topologies, physical or logical resources monitored by Instana, and Instana settings.
The second question is: How can you access the data ? If you want to get the data from self-hosted Instana, you can read Instana databases or Kafka directly. It is very efficient, but there are no official documents and official support for how to access and use the data. If you want to get the event data, you can use WebHook as a listener. If you want to get the SaaS-based Instana data, you can't access the data servers directly, but you can use WebHook instead. Note that the event WebHook must be a publicly accessible endpoint. Finally, one way that will always work is to use the Instana back-end REST API.
Using WebHook to receive events
WebHook is a recommended way to get real-time events from Instana. Two steps are needed. Firstly, configure the WebHook Alert Channel. On the Instana user interface, go to Settings > Team Settings > Events & Alerts >Alert Channels, and click Add Alert Channel and select Generic Webhook:
Secondly, configure Instana Alert to use the WebHook Alert Channel for all events or only for subscribed events. On the Instana user interface, go to Settings > Team Settings > Events & Alerts > Alerts, and click New Alert.
The events are received as a HTTP POST to the configured URLs (HTTP or HTTPS). Here is a typical event received by the WebHook:
{
"issue": {
"id": "f1-QhQw_Qc6jOnlSKdj54w",
"type": "presence",
"start": 1641301305000,
"text": "offline",
"description": "CRI-O Container on Host worker2.cluster1.mycompany.com",
"customZone": "not available",
"availabilityZone": "not available",
"zone": "not available",
"fqdn": "worker2.cluster1.mycompany.com",
"entity": "CRI-O Container",
"entityLabel": "camel-k-operator (default/camel-k-operator-55694d5487-hf8n8)",
"tags": "",
"container": "not available",
"service": "not available",
"containerNames": []
}
}
Note: If you want to get the SaaS-based Instana data, the event WebHook must be a publicly accessible endpoint. If the WebHook stops working for a period of time, the events within that period of time are lost. You need to use Instana back-end REST API to read those lost alerts.
Using Instana back-end REST API to read data
Instana back-end API is the simplest way to access all kinds of Instana data with (see official support documents). It can also be used to query and change Instana configurations.
Instana back-end REST API includes the following contents:
- API for infrastructure monitoring
- API for website monitoring
- API for application monitoring
- APIs for events
- API for settings
Instana API follows the OpenAPI v3 (previously known as Swagger) format. If you are using the latest release version of Instana, you can use the Swagger user interface to view Instana APIs. Open Swagger Editor with your web browser, and import the URL https://instana.github.io/openapi/openapi.yaml. Click to hide the error messages, which are not required on top of the page, and then you can use the rich Swagger user interface to view Instana back-end REST API.
Setting up API tokensTo use Instana API, An API token is required. Security tokens can be created on the Instana user interface. Click the Settings icon, click Team Settings > API Tokens, and then click Add API Token to create a new API token.
Set appropriate permissions for the API token in accordance with your requirements to use Instana Back-end API.
Here is a curl command to call a simple Instana back-end service, which is used to list all supported plug-ins of infrastructure monitoring:
curl -ik -H "Authorization: apiToken $TOKEN" "https://$ENDPOINT/api/infrastructure-monitoring/catalog/plugins?pretty"
Here are typical settings for the TOKEN and ENDPOINT environment variables for a public Instana server:
ENDPOINT=xxxx.instana.io
TOKEN=xxxxxxxx
Here are typical settings for TOKEN and ENDPOINT environment variables for a self-hosted Instana server:
ENDPOINT=xxxx.yyyy.com
TOKEN=xxxxxxxx
Tips on API selection and usages
Most Instana back-end REST APIs are straightforward, but there are still some general considerations:
- Time window or Time frame is a widely-used API parameter for queries of all kinds of data points. For Infrastructure, it means the snapshots that have been active within the time window. For Instana Application Perspectives and Instana Services, it means the calls occurred within the time window. The default value is 10 minutes. The Time window or Time frame usually has big impact on the returned data of the API. The parameter is measured in milliseconds.
- Sometimes there might be a big volume of returned data when you call an Instana API. Pagination can be used in Instana API. Usually you can use the following parameters in the query string to specify the settings for pagination:
- pageSize: Set the number of items thatyou want to return with one query. The default value is 50. But you can set pageSize to any positive integer.
- page: Set the total number of pages that you want to retrieve. The default value is 1.
In the returned payload of API call, you can also find related settings of pagination. For example:
{
"items" : [ {
"id" : "LktRgmy3S9y-UPOTjSBxEg",
"label" : "test",
"boundaryScope" : "ALL",
"entityType" : "APPLICATION",
"_links" : {
}
}, {
"id" : "ih6xwuAcSBGcbWuaIfb80w",
"label" : "myapp",
"boundaryScope" : "ALL",
"entityType" : "APPLICATION",
"_links" : {
}
}, {
"id" : "oC-1JTP8TT6lzpqQcI9v-Q",
"label" : "app-ns2",
"boundaryScope" : "ALL",
"entityType" : "APPLICATION",
"_links" : {
}
}, {
"id" : "GqhFi0VGQ2KcdY2uHt7aVQ",
"label" : "app-ns1",
"boundaryScope" : "ALL",
"entityType" : "APPLICATION",
"_links" : {
}
}, {
"id" : "sUmo0BGyR7ecm52tre8JnQ",
"label" : "MyJavaApps",
"boundaryScope" : "ALL",
"entityType" : "APPLICATION",
"_links" : {
}
}, {
"id" : "nmSARzgJQpmm7RyT-F2tEg",
"label" : "All Services",
"boundaryScope" : "INBOUND",
"entityType" : "APPLICATION",
"_links" : {
}
} ],
"page" : 1,
"pageSize" : 50,
"totalHits" : 6,
"_links" : {
}
}
You can find settings of page and pageSize in the payload. The value of totalHits means the total number of items, which can be larger than the value of pageSize.
- There is a special type of pagination in Instana back-end REST API, which is called CursorPagination. The following parameters can be used for CursorPagination:
ingestionTime integer <int64>
offset integer <int32>
retrievalSize integer <int32> [ 1 .. 200 ]
Note that the range of values of retrievalSize is 1-200. You can't use a integer, of which the value is more than 200.
- Instana continues to improve the API for topology.
- There is another useful API to get the list of snapshots of Infrastructure entities (plugins) for specified Application Perspective or Instana Service. Here is a typical usage example:
curl -ik -H "Authorization: apiToken $TOKEN" "https://$ENDPOINT/api/infrastructure-monitoring/snapshots?query=entity.service.id:a9bb2fe83b3a1ff2bcaef030dcf01173c1e49c83+AND+entity.application.id:0t-OG5LhSoSGbeQxVkjhVw&windowSize=6000000"
Rate limit
The rate limit of Instana API must be considered if you plan to use Instana API heavily. There is a rate limit applied to common API usage. Up to 5,000 calls per hour can be made. How many remaining calls can be made is described in the following HTTP headers, which are part of the responses of the API server.
X-RateLimit-Limit: Shows the maximum number of calls that might be executed per hour.
X-RateLimit-Remaining: How many calls might still be executed within the current hour.
X-RateLimit-Reset: Time when the remaining calls will be reset to the limit. For compatibility with other rate limited APIs, this date is not the date in milliseconds, but in seconds since 1970-01-01T00:00:00+00:00.
There is more rate limits for infrastructure metrics. The rate limit is 1000 calls per hour if snapshot IDs are used, while the rate limit is 500 calls per hour if a Dynamic Focus query is used. Another limitation is that Instana can only return 600 data points per call, so user need to choose a proper time window and "rollup" parameter.
References