Skip to content
This repository has been archived by the owner on Feb 22, 2024. It is now read-only.

Boxplot report

Jiří Holuša edited this page Aug 7, 2015 · 1 revision

Boxplot report

One of the reports that PerfRepo provides is to create boxplot charts. This Wiki page contains all necessary information about this kind of report, so you can start using it right away!

Data structure

The first thing you need to know is how to store the data in PerfRepo, so the boxplot report can work with them. Every report deals with the data differently, so this is essential.

PerfRepo has something called multi-value test execution. As you know from "Introduction to PerfRepo", every test execution can have values associated with specified metric. Typically a single value per metric is enough. However, there is also option to store many values for single metric - that's what we call "multi-value".

Boxplot chart is by definition used in this way - you have multiple runs where you receive many results. You want to apply the statistical magic and you want to compute these nice boxplots. So the only thing you need to do - create a test execution (either via PerfRepo client, GUI or purely XML REST) with all these values in one metric. This will eventually represent one "boxplot item". Please see also information below for possible customization with test execution parameters.

Did you notice that you don't have to compute minimum, maximum or anything ahead? That's right, PerfRepo will do it for you!

Report creation

Creating a boxplot report starts by clicking Reports and choosing Boxplot. After that creating wizard is initiated and consists of several step. We'll go through them right now step by step.

Step 1. Creating charts

One report can contain multiple charts, so we create them now by clicking the "+" icon. Every chart has to specify few things:

  • chart name - obviously
  • test - you have to select test which contains all test executions you want. In other words: one chart -> test executions assigned to single test
  • X-axis labels - one might want to have different type of values on the X-axis, here we specify. Very typical usecase is to have date there, but sometimes you want to have something completely different like kernel build number. You can choose option "parameter" and in the newly displayed text box specify name of the test execution parameter which provides the value.
  • X-axis sort - there is also three possible ways how to sort the test executions, all in ascending order. Option date is quite obvious. Next option is parameter and by selecting this, text box will appear. You can specify a test execution parameter and on the values lexicographical ordering will be applied. Therefore you can create any custom ordering by specifying right values in the parameter. Last option version is for special usecase, when you want to order by version. You also have to provide name of the test execution parameter which contains the version. The version ordering conforms to Linux sort -V command.

Step 2. Creating series

Creating series is much easier, but let's give an example of what we could want: Let's say we have a chart where we want to compare to multiple versions of product in two criterias: memory consumption and throughput and we want to have it nicely in one chart. Exactly those to criterias will be two different series. The optimal data organization might be: every test execution has two metrics with multiple values, one metric for throughput and the second for memory consuption.

You can also specify filtering tags that every test execution must have. Please note, that the test execution must have all tags present, but might have some other.

The resulting chart will contain two differently colored series as you see below (image TBD).

Step 3. Creating baselines

What we call baseline is a single-value metric which containt exactly one number. You need to specify the exact test execution ID and metric. The resulting chart will draw one line per baseline on the level of the baseline value.

Step 4. Preview

This is a preview page. The resulting report will look exactly like this, so now it's the time to check everything and maybe come back and correct something.

Step 5. Finishing step

Specify the report name under which it will be store. You can also adjust the permissions to report by which you control the availability of other users/groups. For details about permission, see the Permissions Wiki page.

Tips & tricks

The underlaying JavaScript library is very powerfull and you can do many things with it by right-click. However, the very cool feature is somehow hidden. When dragging throug the chart with left-mouse click, you can zoom the chart horizontaly, when holding Shift, you can zoom verticaly, which I found extremely useful.

Enjoy boxplot report!