Modernised application With Prometheus and Grafana on RedHat OpenShift running on IBM POWER Systems
This article explains on how to scrape Prometheus metrics from a go application instrumented for Prometheus data, Deploying the application on the RedHat OpenShift along with Prometheus and Grafana running on IBM POWER Systems
Step1: Running a Prometheus instrumented application on RedHat OpenShift
This article makes use of a go application instrumented for Prometheus explained in the Prometheus documentation https://prometheus.io/docs/guides/go-application/
The Docker file to deploy the above sample application can be found at
https://github.com/mithunhr87/metrics_first/
An image using above docker file is built and pushed to quay repository
#podman build . -t quay.io/mithunibm87/metrics:latest
#podman push quay.io/mithunibm87/metrics:latest
Step 2: Deploying the Prometheus instance
Prometheus is deployed using the publicly available docker image available for IBM Linux on POWER Systems (ppc64le) at – docker.io
#docker pull prom/Prometheus
Step3: Create a Prometheus.yml config map containing the scrape information
The default Prometheus will not know where to get/scrape the information from, for our instrumented application , Hence first a config map with these details needs to be created.
Contents of prometheus.yml
global:
scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
alerting:
alertmanagers:
- static_configs:
- targets:
# - alertmanager:9093
# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
scrape_configs:
- job_name: 'mv-metrics'
scrape_interval: 5s
metrics_path: /metrics
static_configs:
- targets: ['metrics.metrics-demo.svc.cluster.local:9080']
Same file could be found at - https://github.com/mithunhr87/metrics_first/blob/main/prometheus.yml
Step4: In the default Prometheus instance, create volumes and volume mounts. Then, load the prometheus.yml file and force it to read the updated prometheus.yml file
Under volumes in Prometheus deployment file add below data
volumes:
- name: prometheus-config
configMap:
name: prometheus-config
items:
- key: prometheus.yml
path: prometheus.yml
mode: 420
defaultMode: 420
Under volumeMount in the Prometheus deployment file add below data
volumeMounts:
- name: prometheus-config
mountPath: /etc/prometheus/prometheus.yml
subPath: prometheus.yml
pass below arguments to image to read new Prometheus file
args:
- '--config.file=/etc/prometheus/prometheus.yml'
- '--storage.tsdb.path=/prometheus'
- '--web.console.libraries=/usr/share/prometheus/console_libraries'
- '--web.console.templates=/usr/share/prometheus/consoles'
Complete Prometheus modified deployment file could be found at - https://github.com/mithunhr87/metrics_first/blob/main/prometheus_deployment.yaml
Step5 : Check for targets on Prometheus
Now the Prometheus will have the target displayed for instrumented go application
Check for myapp_processed_ops_total parameter to check the total operations processed
Step6: Deploying Grafana
Grafana Image for IBM Linux on POWER Systems (ppc64le) can be found at. https://catalog.redhat.com/software/containers/rhel9/grafana/6144937d1ea8a4ec32b378f0?architecture=ppc64le&image=669951f71dd5ee73859b28c8>i-tabs=registry-tokens
registry.redhat.io/rhel9/Grafana
Deploy the above grafana image on RedHat OpenShift
Select data sources from the menu to add a new data source
Provide URL of the Prometheus instance deployed
Once you save and test, on success, A message “data source is working” will be displayed
Create a new dashboard, Add the new panel, with data souce Prometheus. You can query all the data , Here we can see 'myapp_processed_ops_total' is displayed for past 1 hour
In this way a complete monitoring stack along with custom Prometheus configuration and Grafana could be deployed on RedHat OpenShift running on IBM Power Systems