Authored by @Ying Mo, @Paras Kampasi @Roger Kuo @Yanwei Li
Background
Instana is a highly versatile tool, capable of observing a wide range of systems and technologies efficiently. However, users may have experienced some lack of ease of use when attempting to monitor some new systems or technologies that are not currently supported by Instana. Instead of being able to configure and extend Instana's observation capabilities independently, users may need to request for the new capabilities to be added. Such additions may include necessary code changes, e.g.: a new sensor to collect observability data, backend code to process the data sent from the sensor, and UI code to define how to visualize the data. After our team adds the feature, users may need to wait a few weeks or months for the next Instana release before they can adopt the new observation capabilities.
Fortunately, with the advent of OpenTelemetry, users now have the chance to improve the agility of the process. OpenTelemetry offers extensive support for collecting data from a wide range of systems and technologies. By leveraging the fruitful assets and contributions from the OpenTelemetry community, users can now integrate new observability functionalities independently without needing to wait for the next Instana releases. OpenTelemetry empowers Instana users to swiftly adapt to their evolving observability needs and enhances the overall flexibility of the Instana observability solution.
However, using OpenTelemetry only solves half of the problem. By default, Instana treats all metrics collected by OpenTelemetry as custom metrics. Due to the unknown semantics of the newly collected telemetry data, Instana cannot automatically process and visualize the data. To resolve this issue, Instana allows users to define how to visualize the new telemetry data by creating custom dashboards. With the combination of OpenTelemetry and Instana custom dashboards, users can both effectively take control of the data visualization process in Instana and contribute back to Instana by introducing new observation capabilities for external systems and technologies.
The Instana team is aware of the importance of this idea and has been continuously striving to improve the product. We want to help our users have a better experience with extending Instana's observation capabilities to support new systems and technologies. This blog aims to share some of our recent updates in this area as well as some current methods and best practices for this process.
Explore the public dashboards
Regular Instana users may find some difficulty in defining a custom dashboard from scratch, as it may require more familiarity of the feature as well as some deeper knowledge of the systems or technologies being used. If such is the case, the best way to start would be to search for and explore some existing dashboards that may fit similar needs.
Currently, the best place to search for and explore reusable dashboards is at https://www.npmjs.com/. The Instana team has been working on tooling support to help users bundle dashboards together as a cohesive integration package and publish it for public use by other Instana users.
Users can browse through existing packages to search for relevant integration packages. After selecting a package, users are redirected to a landing page where they can find comprehensive details about the package, such as version history, documentation, and other additional resources and information.
Download and import the dashboards
With publicly available integration packages, users can now download and import custom dashboards into their existing Instana environments. This will immediately extend the observation capability of the environment to support the specified new systems or technologies.
The CLI for integration package management
The Instana team has created a command line tool, called Instana CLI for Integration Package Management, to support the downloading and importing process. In other words, this tool can be used to download bundled packages onto local machines and then import them into existing Instana environments.
The latest version of the tool can be found at: https://github.com/instana/observability-as-code/releases. Different binaries of the tool for different platforms can be found on the release page. Download the matching binary for a specific platform and rename it to stanctl-integration. The tool should now be able to be run anywhere on the local machine.
Entering stanctl-integration in the command line will return helpful information and the available commands supported by this tool:
$ stanctl-integration
Usage: stanctl-integration <command> <options>
Commands:
stanctl-integration download Download an integration package
stanctl-integration import Import an integration package into an environment
stanctl-integration export Export integration elements from an environment
stanctl-integration init Initialize a new integration package
stanctl-integration publish Publish the local integration package
Options:
--version Show version number [boolean]
-h, --help Show help [boolean]
You need at least one command before moving on
Download and import dashboards using CLI
Download the dashboards
To download an integration package with the tool, the package name must be specified. For example, the following command will download a package named @instana-integration/go:
$ stanctl-integration download --package @instana-integration/go
By default, the tool will create a sub-directory in the current working directory to store all the downloaded integration packages. This sub-directory will act as the root directory for the downloads. Using the --location option, users can define a specific directory as the root directory. For example, the following command will download the package under the root directory ./my-packages:
$ stanctl-integration download --package @instana-integration/go --location ./my-packages
Import the dashboards
Once an integration package is downloaded, the CLI tool can be used to import the package into any existing Instana environment. Users will need to provide an Instana server address and API token. An API token is required because the tool will make a series of Instana API calls to complete the import process. For example:
$ stanctl-integration import --package @instana-integration/go \
--server $INSTANA_SERVER \
--token $API_TOKEN
Again, users can specify the root directory for the downloads by using the --location option. For example:
$ stanctl-integration import --package @instana-integration/go --location ./my-packages \
--server $INSTANA_SERVER \
--token $API_TOKEN
To successfully import packages, the provided API token must have the Sharing custom dashboards publicly with all users and API tokens permission enabled. To confirm this, navigate through Settings -> Team Settings -> API Tokens from the left-hand side navigation menu. In the API Tokens page, select the token being used. In the Edit API Token dialog, find the Owner Permissions section and make sure the Sharing custom dashboards publicly with all users and API tokens item is enabled.
Specify parameter values for the dashboards
Some of the custom dashboards within a package may include extra parameters. This is caused when certain data values are not defined when creating the dashboard. The package’s homepage will include information for any such existing parameters. When importing such dashboards, the --set option can be used to specify the actual values of the parameters. For example:
$ stanctl-integration import --package @instana-integration/go \
--server $INSTANA_SERVER \
--token $API_TOKEN \
--set key1=value1 \
--set key2=value2
Conclusion
IBM Instana is highly versatile and capable of observing a wide range of systems and technologies efficiently. However, adding observation capabilities for unsupported and new systems and technologies requires a tedious process and waiting period.
By leveraging OpenTelemetry, users can now circumvent this process by using existing OpenTelemetry collector implementations from the community or building their own. With the help of Instana’s custom dashboards, users can also define how collected data is visualized. This allows users to independently extend Instana’s observation capabilities to support new systems or technologies.
The Instana team has been continuously striving to improve the product to help customers gain a better experience with this requirement. Recent updates include the ability to bundle custom dashboards as sharable integration packages that can then be published and publicly used by other users. The team has also provided tooling support for direct downloading and importing of the packages.
The next blog post will discuss how to make custom dashboards publicly sharable, thereby contributing to the whole Instana community and ecosystem.
#CustomDashboards
#General
#OpenTelemetry
#Integration
#Tutorial