-
Notifications
You must be signed in to change notification settings - Fork 188
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Introduce templated benchmarks. (#301)
This PR introduces templated benchmarks for BenchBase. In essence, users can define their own query templates that are parametrized with one or more values at runtime. If multiple parameter combinations are possible, we will continue to cycle through all available combinations.
A templated benchmark has the following structure:
```xml
<templates>
<template name="$QueryTemplateName">
<query>$SQLQuery</query>
<types>
<type>$ParameterType1</type>
<type>$ParameterType2</type>
</types>
<values>
<value>$ParameterValueA1</value>
<value>$ParameterValueA2</value>
...
</values>
<values>
<value>$ParameterValueB1</value>
<value>$ParameterValueB2</value>
...
</values>
</template> ...
<templates>
```
where `$ParameterType` is the integer `java.sql.Types` value (i.e., `INTEGER`, `VARCHAR`, etc.) and each value tag within `values` contains the values for one instantiation of the parameters set in `$SQLQuery`. The SQL query string is read as a `PreparedStatement`, i.e., parameters are defined in the string via a `?` placeholder. An example for a templated benchmark can be found in `data/templated/example.xml`. The file path for the XML template has to be defined in the workload configuration using the `query_templates_file` tag. An example configuration can be found in `config/sqlserver/sample_template_config.xml`. The example can be executed if a loaded TPC-C instance is used as JDBC endpoint.
Templated benchmarks are instantiated using `templated` as benchmark class when running BenchBase via the command line.- Loading branch information
1 parent
109064e
commit 611f3d4
Showing
33 changed files
with
1,775 additions
and
43 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| <?xml version="1.0"?> | ||
| <parameters> | ||
|
|
||
| <!-- Connection details --> | ||
| <type>MARIADB</type> | ||
| <driver>org.mariadb.jdbc.Driver</driver> | ||
| <url>jdbc:mariadb://localhost:3306/benchbase?useServerPrepStmts</url> | ||
| <username>admin</username> | ||
| <password>password</password> | ||
| <isolation>TRANSACTION_SERIALIZABLE</isolation> | ||
| <batchsize>128</batchsize> | ||
|
|
||
| <!-- Note: these example template reuses TPC-C queries and data as a demonstration, but others are possible. | ||
| To test this sample as is, you must first run benchbase with parameters `load=true` and | ||
| `config=sample_tpcc_config.xml` to load the TPC-C data to the target database. | ||
| Other templated benchmarks need to ensure that the data is loaded manually prior to running the benchmark. | ||
| --> | ||
|
|
||
| <query_templates_file>data/templated/example.xml</query_templates_file> | ||
|
|
||
| <!-- The workload --> | ||
| <terminals>1</terminals> | ||
| <works> | ||
| <work> | ||
| <time>10</time> | ||
| <rate>100</rate> | ||
| <weights>30,20,10,30,10</weights> | ||
| </work> | ||
| </works> | ||
|
|
||
| <!-- Select all templates that are executed. --> | ||
| <transactiontypes> | ||
| <transactiontype> | ||
| <name>GetOrder</name> | ||
| </transactiontype> | ||
| <transactiontype> | ||
| <name>GetCust</name> | ||
| </transactiontype> | ||
| <transactiontype> | ||
| <name>GetCustNull</name> | ||
| </transactiontype> | ||
| <transactiontype> | ||
| <name>GetWarehouse</name> | ||
| </transactiontype> | ||
| <transactiontype> | ||
| <name>GetItemByPrice</name> | ||
| </transactiontype> | ||
| </transactiontypes> | ||
| </parameters> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| <?xml version="1.0"?> | ||
| <parameters> | ||
|
|
||
| <!-- Connection details --> | ||
| <type>MYSQL</type> | ||
| <driver>com.mysql.cj.jdbc.Driver</driver> | ||
| <url>jdbc:mysql://localhost:3306/benchbase?rewriteBatchedStatements=true&allowPublicKeyRetrieval=True&sslMode=DISABLED</url> | ||
| <username>admin</username> | ||
| <password>password</password> | ||
| <isolation>TRANSACTION_SERIALIZABLE</isolation> | ||
| <batchsize>128</batchsize> | ||
|
|
||
| <!-- Note: these example template reuses TPC-C queries and data as a demonstration, but others are possible. | ||
| To test this sample as is, you must first run benchbase with parameters `load=true` and | ||
| `config=sample_tpcc_config.xml` to load the TPC-C data to the target database. | ||
| Other templated benchmarks need to ensure that the data is loaded manually prior to running the benchmark. | ||
| --> | ||
|
|
||
| <query_templates_file>data/templated/example.xml</query_templates_file> | ||
|
|
||
| <!-- The workload --> | ||
| <terminals>1</terminals> | ||
| <works> | ||
| <work> | ||
| <time>10</time> | ||
| <rate>100</rate> | ||
| <weights>30,20,10,30,10</weights> | ||
| </work> | ||
| </works> | ||
|
|
||
| <!-- Select all templates that are executed. --> | ||
| <transactiontypes> | ||
| <transactiontype> | ||
| <name>GetOrder</name> | ||
| </transactiontype> | ||
| <transactiontype> | ||
| <name>GetCust</name> | ||
| </transactiontype> | ||
| <transactiontype> | ||
| <name>GetCustNull</name> | ||
| </transactiontype> | ||
| <transactiontype> | ||
| <name>GetWarehouse</name> | ||
| </transactiontype> | ||
| <transactiontype> | ||
| <name>GetItemByPrice</name> | ||
| </transactiontype> | ||
| </transactiontypes> | ||
| </parameters> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| <?xml version="1.0"?> | ||
| <parameters> | ||
|
|
||
| <!-- Connection details --> | ||
| <type>POSTGRES</type> | ||
| <driver>org.postgresql.Driver</driver> | ||
| <url>jdbc:postgresql://localhost:5432/benchbase?sslmode=disable&ApplicationName=templated&reWriteBatchedInserts=true</url> | ||
| <username>admin</username> | ||
| <password>password</password> | ||
| <isolation>TRANSACTION_SERIALIZABLE</isolation> | ||
| <batchsize>128</batchsize> | ||
|
|
||
| <!-- Note: these example template reuses TPC-C queries and data as a demonstration, but others are possible. | ||
| To test this sample as is, you must first run benchbase with parameters `load=true` and | ||
| `config=sample_tpcc_config.xml` to load the TPC-C data to the target database. | ||
| Other templated benchmarks need to ensure that the data is loaded manually prior to running the benchmark. | ||
| --> | ||
|
|
||
| <query_templates_file>data/templated/example.xml</query_templates_file> | ||
|
|
||
| <!-- The workload --> | ||
| <terminals>1</terminals> | ||
| <works> | ||
| <work> | ||
| <time>10</time> | ||
| <rate>100</rate> | ||
| <weights>30,20,10,30,10</weights> | ||
| </work> | ||
| </works> | ||
|
|
||
| <!-- Select all templates that are executed. --> | ||
| <transactiontypes> | ||
| <transactiontype> | ||
| <name>GetOrder</name> | ||
| </transactiontype> | ||
| <transactiontype> | ||
| <name>GetCust</name> | ||
| </transactiontype> | ||
| <transactiontype> | ||
| <name>GetCustNull</name> | ||
| </transactiontype> | ||
| <transactiontype> | ||
| <name>GetWarehouse</name> | ||
| </transactiontype> | ||
| <transactiontype> | ||
| <name>GetItemByPrice</name> | ||
| </transactiontype> | ||
| </transactiontypes> | ||
| </parameters> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| <?xml version="1.0"?> | ||
| <parameters> | ||
|
|
||
| <!-- Connection details --> | ||
| <type>SQLITE</type> | ||
| <driver>org.sqlite.JDBC</driver> | ||
| <url>jdbc:sqlite:tpcc.db</url> | ||
| <isolation>TRANSACTION_SERIALIZABLE</isolation> | ||
| <batchsize>128</batchsize> | ||
|
|
||
| <!-- Note: these example template reuses TPC-C queries and data as a demonstration, but others are possible. | ||
| To test this sample as is, you must first run benchbase with parameters `load=true` and | ||
| `config=sample_tpcc_config.xml` to load the TPC-C data to the target database. | ||
| Other templated benchmarks need to ensure that the data is loaded manually prior to running the benchmark. | ||
| --> | ||
|
|
||
| <query_templates_file>data/templated/example.xml</query_templates_file> | ||
|
|
||
| <!-- The workload --> | ||
| <terminals>1</terminals> | ||
| <works> | ||
| <work> | ||
| <time>10</time> | ||
| <rate>100</rate> | ||
| <weights>30,20,10,30,10</weights> | ||
| </work> | ||
| </works> | ||
|
|
||
| <!-- Select all templates that are executed. --> | ||
| <transactiontypes> | ||
| <transactiontype> | ||
| <name>GetOrder</name> | ||
| </transactiontype> | ||
| <transactiontype> | ||
| <name>GetCust</name> | ||
| </transactiontype> | ||
| <transactiontype> | ||
| <name>GetCustNull</name> | ||
| </transactiontype> | ||
| <transactiontype> | ||
| <name>GetWarehouse</name> | ||
| </transactiontype> | ||
| <transactiontype> | ||
| <name>GetItemByPrice</name> | ||
| </transactiontype> | ||
| </transactiontypes> | ||
| </parameters> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.