Gaurav Dangaich
This is a simple guide to set up JMeter Grafana Dashboard using InfluxDB which describes the step-by-step setup process.
With JMeter or any performance testing tool, one can see the detailed error logs only after the test run completes. If you have a development team monitoring the test or any stakeholder asks the reason on why a certain request/API failed, you will not have an answer unless you had time to analyse the test results. It’s more true for a headless test run such as non-GUI executions where you cannot access the ‘View Results Tree’ or ‘Summary Table’ during the test.
Wouldn’t it be great to have this information handy at any point of time during the test, to dynamically view test results such as slower transactions and detailed error information amongst API calls/requests in real-time?
An easy way to perceive this is to have a non-relational database, like InfluxDB, record the real-time test logs into a table and then use a graphing tool, like Grafana, to display the live data. As long as the database connects to the Graphing tool, one can view the historic as well as real-time data in terms of charts and graphs.
Overview:
JMeter is the most popular and easy to use open source load testing tool. It is designed to load test functional behaviour and measure performance. You can use JMeter to analyse and measure the performance of web application or a variety of services.
InfluxDB is a time-series database built for high-performance handling of time-stamped data. JMeter provides a backend listener InfluxdbBackendListenerClient to write the data to InfluxDB.
Grafana is an open-source metric analytics and visualization suite. It is most commonly used for visualizing time series data for infrastructure and application analytics.
Data Flow:
JMeter write the real-time test results into Influx database using backend listener and Grafana will read the data from InfluxDB and display it on its dashboard.
Data flow from JMeter to Grafana
Below are the steps to integrate JMeter with InfluxDB and Grafana:
- Download and setup InfluxDB
- Setup InfluxDB backend listener in JMeter
- Download and Setup Grafana
- Configure Grafana dashboard
- Run JMeter Test and monitor results
Download and setup InfluxDB
Installing InfluxDB through brew is very simple. You just need to run these two commands if brew is installed on your machine.
Start InfluxDB
Using brew command: brew services start influxdb
To verify that InfluxDB is up and running, all you need to do is to open a terminal window and run this command:
If the installation was completed successfully and the database is up and running, you will see an InfluxDB command line interface. This can be used for interacting with the database.
By default, InfluxDB uses the following network ports:
- TCP port 8086 is available for client-server communication using the InfluxDB API.
- TCP port 8088 is available for the RPC service to perform backup and restore operations.
Setup & Configure InfluxDB
We will need to create a database to store our JMeter Results.
Go to below URL from the browser (Below is just a one-time activity to setup database and other necessary configuration)
http://localhost:8086/onboarding/0
Click on ‘Get Started’:
Note: In the above image:
- Username: provide any suitable username to login to influxdb
- Password: provide any suitable password to login
- Initial Organization Name: this field will serve as a workspace name for different users to work on the same.
- Initial Bucket Name: Bucket is same as database in general, provide a suitable db name in this field
Once the above fields are filled, click on ‘Continue’, it will take to below page:
Setup Influxdb backend listener in JMeter
Here, we will see how to configure the backend listener in JMeter to write metrics to InfluxDB.
- Add a new backend listener to the thread group. Right Click on a Thread Group -> Add -> Listener -> Backend Listener
- Change its name to InfluxDB Backend Listener
- In the Backend Listener Implementation dropdown select org.apache.jmeter.visualizers.backend.influxdb.InfluxdbBackendListenerClient
- Configure backend listener as shown below in image
- InfluxdbMetricsSender — this is the class for sending metrics to InfluxDB. We will keep it as default.
- InfluxdbUrl — this is the URL of InfluxDB is in the following format: http://%5Binfluxdb_host%5D:%5Binfluxdb_port%5D/write?db=%5Bdatabase_name%5D. As we are running it on a local machine with a default port (8086), then, in this case, the URL will be:
- application — we can store metrics from different applications under one database.
- measurement — the name of the measurement/table that will be stored in InfluxDB. Use default ‘jmeter’.
- summaryOnly — if you want to keep summary results in the database and do not want to collect all the detailed metrics during test execution set to ‘true’ or else set this to false.
- samplersRegexp — use the regexp expression to filter specific metrics which you want to store in a database. Use the default “.*” value if you want to store all your metrics.
- percentiles — metrics percentiles that should be sent to the database.
- testTitle — give your test a name.
- eventTags — a list of tags which will be stored in the ‘events’ measurement of InfluxDB.
- influxdbToken — token generated at the time of db creation/registration must be passed here
- TAGS_{variable} — any specific variable from jmeter user defined variables to be passed into db must be declared under this.
In case the influx db access is user restricted then username and password need to be added in URL:
http://localhost:8086/api/v2/write?org=${org_name}&bucket=${bucket_name}&u=username&p=password
Here we are done with our setup of Backend Listener in JMeter. Now let’s move to our next setup of Grafana
Download and Setup Grafana
Installing Grafana through brew is verysimple. You just need to run the below command If brew is installed on your machine.
Start Grafana
Grafana can be started with the below brew command
After that, Grafana should be available on http://localhost:3000.
Use ‘admin’ both as a default username and password to log in.
Configure Grafana dashboard
In previous step we are able to start Grafana in localhost. Once logged in you will be presented with the Home Dashboard. From here you can create data sources, manage users, create report dashboards etc.
Add Data Source
The first thing we need to do is create a data source to pull data from our InfluxDB.
- Click Add Data Source and choose InfluxDB as the data source type.
- Specify the url: http://localhost:8086 and the name of the influxdb database we created earlier: jmeter.
- Configure minInterval time as 5s. You can also configure other settings for authentication.
- Click Save & Test hopefully you should see the Data source is working.
Create Dashboard
Instead of creating a fresh Dashboard in Grafana, we will use existing dashboard template (this is just a sample template, we can use any template as per the requirement and availability in the Grafrana_Dashboard_Template): SampleDashboard
- In Grafana , click on the ‘New’ icon in top right corner.
- Select Import, browse the downloaded JSON file or enter the dashboard ID we copied (19950) and click Load.
- On the import dashboard screen please configure with following values:
- Name — specify the dashboard name
- Folder — Grafana allows you to organize dashboards into folders.
- Unique identifier — provide an unique value to this field
- InfluxDB — select the name of the datasource we selected for it, in this case it is ‘influxdb’
- send_interval — this should match the JMeter property backend_influxdb.send_interval which default value is 5 seconds.
The sample Dashboard will look like as follows:
We are done with our Grafana setup. Now it’s time to run the actual JMeter test and monitor results in Grafana.
This completes our integration setup for Jmeter with InfluxDB and Grafana to view real time metrics results.
To learn more about InfluxDB internal storage:
I hope you will find this blog helpful.
Thanks for reading, any feedback is more than welcome!!
Happy Testing!!