Storage Fusion

Introduction

What is Cloud Native?

Cloud Native Architecture is a popular trend and a transformative approach to designing and running applications. This approach helps run resilient, responsive, and scalable applications anywhere in public, private, or hybrid clouds. Cloud Native is scalable, flexible, and resilient even during turbulent times.

IBM Storage Fusion

IBM Storage Fusion is a container-native hybrid cloud data platform that offers simplified deployment and data management for Kubernetes applications on Red Hat® OpenShift® Container Platform.

Read more about this topic here - IBM Storage Fusion - Product Overview.

TLDR version – IBM Storage Fusion is a cloud-native data services solution that gives enterprises the flexibility to run modern and stateful Kubernetes applications.

Organizations deal with only a single copy and let the product worry about storage and backup to provide simplified data accessibility and availability across hybrid clouds.

The blog covers the steps for installing Grafana and configuring Prometheus data source along with setting up IBM Storage Fusion’s Dashboards for better Monitoring and Observability of their cluster.

What is Monitoring and Observability?

Both Observability and Monitoring play a crucial role for any product that runs on an enterprise scale. By definition, Observability helps to gain insight into the health and statuses of the different applications and resources, whereas Monitoring is a collection and analysis of metrics from these applications. With applications evolving, the complexities within the system deepen. For the system to run impeccably, focus on Observability and Monitoring is an absolute imperative that cannot be ignored. A wide array of tools are available for this purpose, with Prometheus and Grafana being the most popular ones.

Popular Open Source Tools

Prometheus is an open-source tool designed to provide monitoring and alerting functionality for cloud-native environments. This tool enables you to scrape and store these data and metrics from the applications in a time series database, that is, with a timestamp. For visualization of this data, Grafana is the go-to choice because of its powerful dashboard visualizing capabilities and seamless integration with Prometheus.

Monitoring with IBM Storage Fusion

For monitoring various components within Fusion, an exporter service collects metrics from two different components - Compute and Storage. The service pulls the critical metrics of these components like Energy consumption, Temperature, Read/Write Ops, Disk capacity, Read/Write throughput, and Bytes Sent/Received making them available for Prometheus to consume.

Prometheus comes default with Red Hat OpenShift, but not Grafana (from Fusion's 2.6 release).

Steps to install Grafana in your OpenShift Container Platform (OCP)

1. Log in to the OCP using the administrator credentials.

2. From the left panel, go to Operators > OperatorHub.

3. Search for the Grafana Operator and then click Install.

4. Select the Update Channel as v4. Select or Create a new namespace for the installation location.
   
   

5. After the installation is completed, go to View Operator and then click the Create instance from the Grafana tab.

6. Configure by using the YAML view, and update the spec based on the following sample:

apiVersion: integreatly.org/v1alpha1
kind: Grafana
metadata:
  name: grafana
  namespace: grafana-ns
spec:
  config:
    auth:
      disable_signout_menu: true
    auth.anonymous:
      enabled: true
    log:
      level: warn
      mode: console
    security:
      admin_password: secret
      admin_user: root
  ingress:
    enabled: true

7. To validate, do the following steps:

1. From the left panel, go to Networking > Routes.

2. Click the entry for grafana-route. It redirects you to the login page of Grafana. 

3. Log in by using the credentials mentioned in the YAML.

The Grafana application is now up and running.

Configure Grafana with Prometheus

Now that you have Grafana installed either in cluster or externally, the quickest way is to use the OAuth token and the route directly but to avoid the hassle of updating the token every time, follow the steps from our documentation to set a service account secret. After following the steps in the link, you should have a Secret and Prometheus-API route. This will be needed to configure the Data Source for Grafana.

Configuring the Data Source:

1. Log in to the in-cluster/external Grafana user interface. 

2. Go to the Data Sources section.
   

3. Click Add DataSource and select Prometheus.

1. In the URL mention the Prometheus route - https://prometheus-k8s-openshift-monitoring.apps.<domain>.com (without the API suffix)

2. In the Auth section, enable the “Skip TLS verify” option.

3. Add a custom HTTP header with Header as “Authorization” and Value as “Bearer <secret/ auth token>”.

4. Select HTTP Method as GET.
   

4. Click Save & Test.

   

Importing the Dashboards of IBM Storage Fusion

After our Prometheus data source is configured, IBM Storage Fusion dashboards provide the most insightful observability for the three components - Compute, Storage, and Network. Download the JSON of these dashboards from here. 

From the homepage, select the + icon and then click Import.

You cannot simultaneously select multiple JSONs, so repeat the steps for all 3 JSONs.

Observability with Fusion

You should now have three dashboards each portraying a component in a visually clear and coherent structure. In the interest of conciseness, we are focusing on a singular dashboard serving as an illustrative example of how Storage Fusion employs both tools to deliver comprehensive Monitoring and Observability for the cluster.

The Compute Dashboard provides an overview of Energy consumption for the Node and its CPU along with various Thermal statistics.

The lines are displayed based on the number of nodes in the rack. You have the option to select and delve into the usage details of a specific node for a detailed examination of its usage through the graphs.

Dashboards with similar functionality for both Network and Storage components are available.

Conclusion

The article demonstrates how IBM Storage Fusion leverages the two leading open-source tools within its solution to provide complete visibility of the system and its various components.


--
Special Thanks to Vaibhav Gadge and Shyamala Rajagopalan for their assistance in reviewing and providing feedback on the article.