Use IBM Storage Insights REST APIs to integrate IBM Storage observability with Grafana
Acknowledgment: Ramakrishna Vadla, Sandeep Patil, Iti Srijan, Naveen Dasarathan, Srividhya Kalyanasundaram, Manan Bhatt, Syamlal Krishnan
Integrating IBM Storage Insights with Grafana enables real-time monitoring and insightful visualizations of your storage environment. In this guide, we’ll walk you through each step, from configuring IBM Storage Insights as a data source in Grafana to creating tailored dashboards for monitoring performance, capacity, and health metrics.
1. Introduction to IBM Storage Insights and Grafana
IBM Storage Insights offers a suite of REST APIs that allow seamless integration of storage system data—such as configuration, performance, and health metrics—into a variety of external applications. These APIs support building an interconnected ecosystem where storage insights can enhance the functionality of operational dashboards, incident management systems, and even security platforms.
Many enterprise organizations rely on a variety of vendor-specific observability tools, yet Grafana stands out as the preferred solution for an integrated operations dashboard. Its flexibility and compatibility make it the ideal choice for creating a unified, single-pane view across multi-vendor environments. With the IBM Storage Insights REST APIs and a pre-configured Grafana dashboard template, customers can effortlessly integrate monitoring of IBM storage, switches, and servers into a Grafana-based operational dashboard.
The diagram below illustrates the high-level integration workflow between IBM Storage Insights and the Grafana Dashboard.
2. Prerequisites for Grafana Integration
Before setting up the integration, ensure the following prerequisites are in place:
- Grafana Installation: Make sure you have Grafana installed, preferably version 11.1.0 or higher for optimal compatibility.
- Infinity Plugin: The Infinity plugin is essential for connecting REST API data to Grafana.
- IBM Storage Insights Access: You’ll need API access from IBM Storage Insights, which includes generating an API key for authentication. With these prerequisites in place, the integration process becomes streamlined, allowing you to focus on building valuable dashboards.
3. How to Access the IBM Storage Insights Open-Source Grafana Dashboard
IBM has developed an open-source Grafana dashboard specifically for IBM Storage Insights, making it easy to visualize and monitor key storage metrics out-of-the-box. This dashboard, available on GitHub, serves as a robust foundation for displaying critical data such as storage health, performance, and capacity. By utilizing this pre-built dashboard, users save time on initial configuration and gain access to an expertly crafted tool that can be customized to suit unique monitoring needs.
The IBM Storage Insights Grafana dashboard provides a single-pane-of-glass view of the entire fleet of storage systems within a data center. Additionally, users can drill down into specific storage systems for a more granular view, gaining insights into detailed metrics.
Key Panels Available
- Tenant View: Provides a comprehensive view across the entire IBM Storage Insights tenant, offering insights into::
o Number of block storage systems, their health conditions, and capacity utilization.
o Capacity information of volumes mapped to a host as well as those not mapped.
o A summary of security alerts and total alerts categorized by criticality.
- Storage System View: Provides an in-depth view of a selected storage system, offering insights into:
o Alerts and ransomware notifications, with counts and breakdowns by criticality.
o Capacity metrics, showing details for volumes mapped to a host as well as those not mapped.
o Performance and sustainability metrics, including temperature readings and system throughput.
o Internal resources, such as counts of volumes, pools, and ports.
Below is a screenshot of the IBM Storage Insights Grafana Dashboard, giving a preview of the pre-configured layout and the types of insights available
This comprehensive layout not only provides a holistic view of your storage environment but is also customizable. For example, you can adjust the panel settings, add new visualizations, or modify the queries to display additional metrics specific to your operational goals.
4. Understanding Data Sources and Creating One
In Grafana, a data source acts as the backend connector that facilitates data flow from IBM Storage Insights to your dashboard. Setting up a dedicated data source for IBM Storage Insights is essential for accessing storage metrics, performance data, and alerts within Grafana. This process involves two key steps: first, fetching an API key from IBM Storage Insights, and then using this key to create the data source in Grafana.
4.1 Fetching the API Key from IBM Storage Insights
To enable secure communication between IBM Storage Insights and Grafana, an API key is required. This key will authenticate your requests and ensure secure data access from IBM Storage Insights. Below procedure details on fetching the API Key from Storage Insights
- Log into IBM Storage Insights:
- Access your IBM Storage Insights account and ensure you have the administrator privileges.
- Navigate to User Access Management:
-
- Within the IBM Storage Insights interface, Hover on Configuration and select User Access Management.
- Click on 3 dots at the end and select Create REST API Key option.
- Specify the time period on how long this Key needs to be valid and Click on Create Button.
- Click on Copy Key Option
4.2 Creating a Data Source in Grafana Using the API Key
In this step, you’ll configure the IBM Storage Insights data source in Grafana using the API key retrieved earlier. This setup enables Grafana to pull storage metrics, performance data, and alerts directly from IBM Storage Insights.
- Open Data Source Configuration in Grafana:
- In your Grafana instance, go to Configuration > Data Sources.
- Click Add data source and select the Infinity Plugin from the list of options.
- In the Authentication Tab, Click on API Key Value Pair
- Enter the API Endpoint and Authentication Details:
- In the Infinity Plugin Authentication Auth Type enter the text x-api-key in the Key field
- Enter the API Key fetched from Storage Insights in the Value field.
- Enter the text https://insights.ibm.com in the Allowed Hosts field and click on Add button.
- Save and Test the Data Source:
- Once the API endpoint and authentication are configured, click Save & Test to confirm the connection.
- A successful test will confirm that Grafana can retrieve data from IBM Storage Insights, enabling you to start building dashboards.
By completing these steps, you’ve set up a secure and reliable data connection between IBM Storage Insights and Grafana, making it easy to visualize and monitor your storage data.
5. Quickly Importing the IBM Storage Insights Dashboard
To get started quickly, IBM Storage Insights’ open-source dashboard comes with a pre-configured template. To import:
- Download the JSON file for the dashboard from theGitHub repository.
- In Grafana, Navigate to Menu > Dashboard > Click on New Button > Import.
- Click on “Upload dashboard JSON file” for the pre-configured dashboard.
- Select the data source created as per step 2.
- Once the dashboard is launched, enter the SI Tenant ID in the TenantID text box. (Tenant ID info can be found from the SI UI URL. If your SI URL is https://insights.ibm.com/gui/XXXX-XXX-XXX-XXX, then TenantID is XXXX-XXX-XXX-XXX)
- Click on the Save Dashboard button icon.
- Enable "Save current variable values as dashboard default", this enables you to save the TenantID in the dashboard.
- Click on Save.
6. Creating Custom Panels
If you need additional panels tailored to your specific monitoring requirements, you can create custom panels within the existing Grafana dashboard.
- Building a New Panel:
- When creating a new panel, first select the Data Source that’s currently configured for IBM Storage Insights.
- Enter the specific REST API URL relevant to the data you want to display.
- Ensure the headers include x-api-token as the key, with ${APIToken} as the value for secure authentication.
- Adding Parsing Options:
· Based on the data structure of your API response, configure parsing options within the Data Source settings to properly map and format the incoming data as required.
- Applying Transformations:
· Navigate to the Transformation tab and apply transformations to refine your dataset. Use this tab to filter, aggregate, or reorganize data to meet the needs of your custom panel.
For more details on specific APIs and their parameters, refer to the IBM Storage Insights Swagger documentation, which provides an exhaustive list of available API endpoints and configurations.
7. Support for Issues and Improvement Requests
As an open-source project, the IBM Storage Insights Grafana dashboard follows a community-supported model, so IBM does not provide formal support for this dashboard. Designed as a do-it-yourself solution, it allows users to customize and troubleshoot independently. However, if you encounter issues or have ideas for improvements, you can connect with the project’s contributors and submit your requests via the project’s GitHub page.
8. References
- For more details on IBM Storage Insights Integration with Grafana, refer to the below link
- https://www.ibm.com/docs/en/storage-insights?topic=ecosystem-grafana
- For a complete list of Rest APIs available in IBM Storage Insights, refer to the below link
- https://insights.ibm.com/restapi/docs/
- For more details on Grafana Dashboard Json file and Configuration steps, refer to the below link
- https://github.com/ibmstorage/ibm-storageinsights-ecosystem-grafana
#Highlights#Highlights-home#Featured-area-2