​

Introduction

​A simple, yet powerful reporting infrastructure is one of the most sought after features from an enterprise grade performance testing tool. Performance analysts rely on reports for visualizing statistical counters that quantify application performance to detect problems.

In addition to a rich and intuitive set of default reporting templates included with the product, Performance Tester provides users with powerful features to customize existing reports or create one from scratch. Performance Testers and analysts have been leveraging these capabilities for their custom requirements and the customization capability that the tool traditionally offered has been a popular topic in discussion forums. It has also been elaborately described in tutorials such as the one available here.

Performance Tester 9.0.1 introduced a new Web Analytics reporting framework that leverages modern web technologies to offer a better user experience. Report customization has consequently undergone significant changes. This article describes the structure of the new Web Analytics statistics report and its customization.

To describe customization, we will use some real-life performance testing scenarios where customization could be valuable. Using the example scenarios, we will address three aspects of customization viz. enhancement of an existing report, usage of filters for easy viewing, and creation of a new report from scratch. Finally, we will see how to copy a new custom report into a different workspace.

Structure of a Web Analytics report
Performance Tester Web Analytics report consists of multiple pages with each page typically clubbing together visualizations of counters meant for inspection and analysis of a specific aspect of application performance. For example, counters pertaining to byte rates are visualized in the “Server Throughput” page of the default “Performance Report”.

Each page contains charts arranged in the form of a grid with one or more “rows” and each “row” contains one or more columns. Grids contain multiple “views” which in turn contain charts. All charts except “Rich Text” are linked to one or more statistical counters defined by Performance Tester. The “Rich Text” chart type is meant for displaying annotations or other static snippets of information that could be useful for users.

​The underlying structure of a report is not visible by default, that is when the report is in “read-only” mode. The structure is exposed only when the report is taken to the “edit” mode by a user. The following screen shows the report in the edit mode.
​

Note: The screenshots referred in this article are from Performance Tester 9.2. There may be minor differences in the appearance of reports in other versions of Performance Tester.

Enhancement of an existing report
As an example, where enhancement of an existing report could potentially be useful, let’s assume that we are working on an environment where there are limited load generation agent resources available and they are being utilized almost to the edge. In such a scenario, the engineers see a possibility of reports containing incorrect performance numbers because of resource starvation in load generation agents. A report where performance numbers are reported alongside the health metrics captured from the load generating agents could be a valuable visual aid for correlation and to detect anomalies.

Considering this requirement, let’s enhance the “Response vs. Time Summary” page of the default “Performance Report” to insert a view that displays the agent host CPU and memory counters.

​The “Response vs. Time Summary” page of the “Performance Report”, in its original form, is as follows.
​

1.    Note that the page contains two rows with two columns each and beneath them, we can locate enough vacant space left in the viewing area. Let’s insert the proposed new view as a third row into the vacant area.

To begin, let’s click the Menu button on the top-left corner of the Web Analytics report and select Edit.


2.    The report will now transition to the edit mode. We’ll click the button, “+ Click to insert a row” beneath the existing views to create a new empty row container and select 1 column for our chart. We’ll continue by clicking the button, “Click to insert a view” inside the row container. We should select the visualization type next.


3.    For this example, let’s select Line Chart because that’s an apt visualization for time-series data.
4.    To continue working with the view, let’s open the “View Settings” Page by clicking Settings.


5.    Let’s set a title for the view using the Options tab in the Settings page. For example, “Load Generation Agent Health”.


6.    Next, let’s add the required counters to the view. As mentioned earlier, for this example, we will use Percent Agent Host CPU and Agent Host Memory counters.
7.    To add a counter, let’s move to the Counters tab of the View Settings page and click the drop-down list adjacent to an empty text container.


8.    This lists all the counters available for pick up in this report that is being edited. Let’s pick “Percent Agent Host CPU” under the “Agent Health” > “CPU and Memory” category first. After selection, we’ll click the “+” button on top of the first counter that we added to create another empty horizontal text container with a drop-down list for selecting the next counter. Here, let’s select “Agent Host Memory” counter to complete counters configuration for our newly added view.


9.    After configuring the view, let’s close the “View Settings” page by clicking Apply on the top-right of the page. After returning to the page, we should save our changes using Menu  > Save and then go back to the viewing mode of the report by clicking on Menu > Edit once again.

The “Response vs. Time Summary” page in the modified “Performance Report” now contains our newly added view along with the previous ones. Having Page Performance counters along with Agent Health counters on the same page could ease analysis considerably and our engineers can quickly rule out resource starvation as a cause of performance deterioration if any.


​

Using filters to refine views

You can configure filters for some counter types to refine views further.

In this section, we will discuss the steps to use a filter using a sample scenario. One of the pages in our test run contain more than 20 Page Elements. Since this makes the viewing slightly inconvenient, let’s view only the slowest 5 Page Elements.

​The following screen shows the filter before it was modified.
​

To modify or define a filter in a view, we must enter in to the edit mode in the report. Once the report transitions to the edit mode, we can locate the required view and click Settings.

​In the “View Settings” page, we can move to the “Filters” tab and select the “Page Element Response Time Average” as the counter for filtering and select a value for the filter. In this example, set the value to the highest 5 so that the graph displays the slowest 5 page elements.

After performing the changes, we can apply the changes and complete the activity by clicking Menu > Edit. After saving the changes, the report transitions back to the viewing mode.

Creating a new report

Some users prefer creating a new report from scratch instead of modifying existing ones for customization. This section outlines the required steps by using a special project requirement as an example. In this example, analysts are mostly interested in transactions per second (TPS) and have observed in the past that factors of schedule design, such as delays, think times, and pacing of loops, have been a problem in the past. The analysts are looking for a report where Transaction Throughput counters are alongside Loop Iteration Counts for easier analysis. Examination of this new report helps you determine whether the pace of loop iterations impacts the transactions per second.

1.    To begin, we have to open an existing Web Analytics report in either the workbench or a browser and then switch to the edit mode by clicking Menu on the top-left corner of the report and select New in the context menu. Let’s provide a name and a short description for the new report for identification later. Let’s use the name “TPSwithLoops Report”.


​2.    Once you click “Create”, a blank report is displayed with a single “New Page” along with a button, “Click to insert a row”.

3.    Firstly, let us edit the page name to a meaningful one. To modify the name, click the report area above the “+ Click to insert a row” button.


4. To start adding charts, let’s start with a single row and add three columns, one each meant for Transactions throughput, Loop Iteration Attempts, and User Load.


5.    Let’s add Transaction Throughput counters into the first grid by clicking “+ Click to insert a view” and select the chart type. Let’s select Line Chart.


6.    An empty Line Chart is inserted into the report. Now, let’s add the relevant counters into the view by using the “View Settings” page by clicking Settings.


7.    In the View Settings page, move to the “Counters” tab for defining counters for the view. Firstly, depending on the counters that we like to visualize, let’s select the counter type.
8.    We are looking for Transactions Throughput in this graph, therefore, we must change the counter type to “Per Transaction”.


9.    Next, let’s click the drop-down list on the first empty horizontal bar to expose a list of counters for selection.


10.    Let’s select Transaction – Attempts. The defaults for units (automatic) and filters (no filters) are appropriate, so let’s leave them as they are.
11.    The “View Settings” page now displays the updated chart with a line graph.


12.    Optionally, we can set a title for the view by using the Options tab adjacent to Counters.


​13.    Let’s save our changes for the first view by clicking “Apply”. You can repeat the above steps for the second and third views by choosing line graphs both times and use the counters Global > Active Users and Per Loop > Completed, respectively.
14.    After completing the views, let us save the new report using Menu > Save and stop editing by clicking on Menu > Edit again.  The following screenshot displays a new report.

Once the report is saved, the “TPSwithLoops Report” template that we created would also be available as one of the report types for future use in the same workspace.
​

Sharing a report template

​If we want to use a custom report in a different workspace than where it was originally created or modified, we will have to export the template and import it into the new workspace.
To export the template, go to File > Export > Test > Report Definition as the export type. In the Export wizard, select the report and destination file.
​

To import the exported file into a new workspace, you must first switch Performance Tester to the new workspace and then go to File > Import > Test > Report Definition. Then, you must select the .report file that was exported previously to complete the import.
​

After the import is complete, the custom report should now be available for selection among the other default report types.

Conclusion
​Power users typically extract more value from tools, utilizing customization to extend it based on preferences, custom objectives, and business priorities. In performance testing, reporting is one such area with significant possibilities of augmented value addition via customization with Performance Tester. This article provides a brief background on customization of reports along with a few examples that should get you started on a journey of discovery. Enjoy testing!