Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: Support for Distributions in Templated Workloads #385

Merged
merged 36 commits into from
Mar 25, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
760ede7
Added Random distributions
ETHenzlere Oct 26, 2023
9acfc19
Added Binomial Distribution
ETHenzlere Oct 26, 2023
36174c0
Min/Max for all Distributions
ETHenzlere Oct 30, 2023
b28f7ee
Added Timestamps, Time,Date
ETHenzlere Nov 7, 2023
d42d950
HashMap Solution with type casting
ETHenzlere Nov 10, 2023
572ca57
Multi random support
ETHenzlere Nov 14, 2023
6f287d9
Architecture change | Uniform distribution implemented
ETHenzlere Nov 17, 2023
33bef1e
Switched Dist and Type, changed subarray calculation
ETHenzlere Nov 20, 2023
bab1086
Merge branch 'main' into feature/templated-distributions
ETHenzlere Nov 29, 2023
573967c
Renamings and Documentation
ETHenzlere Nov 29, 2023
c12e60a
Enum, More Documentation
ETHenzlere Nov 30, 2023
3d3c9b2
Merge branch 'main' into feature/templated-distributions
bpkroth Dec 1, 2023
51f0f1d
Renamed autogenerated methods, readme format
ETHenzlere Dec 4, 2023
c97d35e
Resolved merge conflicts
ETHenzlere Dec 7, 2023
38747ae
Testing increased, size reduction of massive switch statement
ETHenzlere Dec 7, 2023
dafb90a
Test functionality after merge
ETHenzlere Jan 5, 2024
961d5c4
Lightweight constructor added
ETHenzlere Jan 17, 2024
5454f52
Insert statment modification
ETHenzlere Jan 23, 2024
3f59530
Merged with main
ETHenzlere Jan 24, 2024
06d3f39
Min-Max check
ETHenzlere Jan 24, 2024
949c93d
Changed distributions due to FK contraints
ETHenzlere Jan 24, 2024
19fad84
Typo
ETHenzlere Jan 24, 2024
c9dc450
Typo 2, build checked
ETHenzlere Jan 24, 2024
36318bd
Merge branch 'main' into feature/templated-distributions
bpkroth Jan 24, 2024
ce028cb
Merge branch 'main' into feature/templated-distributions
bpkroth Jan 24, 2024
ee8f4c1
Enums, removal of aliases, nits
ETHenzlere Jan 25, 2024
0235de7
Adapted description
ETHenzlere Jan 25, 2024
180b592
Merge branch 'main' into feature/templated-distributions
ETHenzlere Mar 13, 2024
9bb153f
Merge branch 'main' into feature/templated-distributions
bpkroth Mar 14, 2024
7154f6e
Merge branch 'main' into feature/templated-distributions
ETHenzlere Mar 20, 2024
ad94565
Templated Values with type information
ETHenzlere Mar 20, 2024
556ff15
README cleanup + Assert pattern
ETHenzlere Mar 21, 2024
c039d92
README update for access pattern
ETHenzlere Mar 25, 2024
1b25aa1
Merge branch 'main' into feature/templated-distributions
ETHenzlere Mar 25, 2024
5de5361
README adjustment showing a possible solution for value weighting
ETHenzlere Mar 25, 2024
1054ac9
Merge branch 'main' into feature/templated-distributions
bpkroth Mar 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions data/templated/example.xml
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@
</types>
<values>
<value dist="uniform" min="1" max="3"/>
<value dist="zipfian" min="1"/>
<value dist="zipfian" min="1" max="2"/>
</values>
<values>
<value dist="scrambled" seed="999"/>
<value dist="scrambled" min="0" max="4" seed="999"/>
<value dist="normal" min="1" max="3" seed="1"/>
</values>
</template>
Expand Down Expand Up @@ -54,7 +54,7 @@
</types>
<values>
<value dist="normal" min="10" max="12.5"/>
<value dist="uniform" min="15.1"/>
<value dist="uniform" min="15.1" max="100.22"/>
</values>
<values>
<value>10.50</value>
Expand Down Expand Up @@ -106,9 +106,9 @@
<value>1</value>
<value>1</value>
<value>1</value>
<value dist="uniform" min="1674477220000" max="1706013220000"/>
<value dist="uniform" min="1999-12-12 01:01:55" max="1706013220000"/>
ETHenzlere marked this conversation as resolved.
Show resolved Hide resolved
<value dist="normal" min="0.5" max="12.5"/>
<value dist="uniform" max="23"/>
<value dist="uniform" min="0" max="23"/>
</values>
</template>
</templates>
14 changes: 7 additions & 7 deletions src/main/java/com/oltpbenchmark/benchmarks/templated/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ Each time the query is run, a different value will substitute the `?`.
<value dist="uniform" min="0" max="1000" seed="1"/>
</values>
<values>
<value dist="zipf" max="1000" seed="1"/>
<value dist="zipf" min="0" max="1000" seed="1"/>
</values>
</template>
<!-- ... -->
Expand All @@ -93,17 +93,17 @@ The distributions are dependent on the type of the value. Currently, the followi
The following properties can be set on the value:

- `dist`: The distribution of the values
- `min`: The minimum value the generator can produce. Defaults to 0
- `max`: The maximum value the generator can produce. Defaults to 1
- `seed`: A seed for the generator to ensure consistency
- `min`: The minimum value the generator can produce.
- `max`: The maximum value the generator can produce.
- `seed`: A seed for the generator to ensure consistency. Default is 0

*NOTE:* For `Timestamp`, `Date`, and `Time` types, the min and max values must be converted to UNIX time (milliseconds).
*NOTE:* For `Timestamp`, `Date`, and `Time` types, the min and max values must be converted to UNIX time (milliseconds) OR given as strings in the following format: (yyyy-MM-dd HH:mm:ss).
The value is interpreted as the milliseconds since January 1, 1970, 00:00:00 GMT/UTC.
ETHenzlere marked this conversation as resolved.
Show resolved Hide resolved
```xml
<value
dist="uniform"
min="1674477220000" // Mon Jan 23 2023 12:33:40
max="1706013220000" // Tue Jan 23 2024 12:33:40
min="2023-01-23 12:33:40" // Mon Jan 23 2023 12:33:40
max="1706013220000" // Tue Jan 23 2024 12:33:40
seed="0"
/>
```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -184,11 +184,14 @@ private CustomClassLoader loadQueryTemplates(String file) {

for (TemplateType template : templates.getTemplateList()) {
ImmutableParsedQueryTemplate.Builder b = ImmutableParsedQueryTemplate.builder();
List<String> templateTypes = template.getTypes().getTypeList();

b.name(template.getName());
b.query(template.getQuery());
b.paramsTypes(template.getTypes().getTypeList());
b.paramsTypes(templateTypes);

for (ValuesType paramValue : template.getValuesList()) {
int typeIndex = 0;
for (ValueType value : paramValue.getValueList()) {
/* Lightweight constructor used if no distribution is present */
if (value.getDist() == null
Expand All @@ -198,8 +201,13 @@ private CustomClassLoader loadQueryTemplates(String file) {
} else {
b.addParamsValues(
new TemplatedValue(
value.getDist(), value.getMin(), value.getMax(), value.getSeed()));
value.getDist(),
value.getMin(),
value.getMax(),
value.getSeed(),
templateTypes.get(typeIndex)));
}
typeIndex++;
}
}

Expand Down Expand Up @@ -274,6 +282,10 @@ private String buildTemplatedValueString(List<TemplatedValue> params) {
+ "\""
+ param.getSeed()
+ "\""
+ ","
+ "\""
+ param.getValueType()
+ "\""
+ "),";
} else {
result +=
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ public PreparedStatement getStatement(Connection conn, List<TemplatedValue> para
generatedInt = param.getNextLongScrambled().intValue();
break;
default:
throw param.createRuntimeException(paramType);
throw param.createRuntimeException();
}
stmt.setInt(i + 1, generatedInt);
break;
Expand All @@ -132,7 +132,7 @@ public PreparedStatement getStatement(Connection conn, List<TemplatedValue> para
generatedFloat = param.getNextFloatBinomial();
break;
default:
throw param.createRuntimeException(paramType);
throw param.createRuntimeException();
}
stmt.setFloat(i + 1, generatedFloat);
break;
Expand All @@ -152,7 +152,7 @@ public PreparedStatement getStatement(Connection conn, List<TemplatedValue> para
generatedLong = param.getNextLongScrambled();
break;
default:
throw param.createRuntimeException(paramType);
throw param.createRuntimeException();
}
stmt.setLong(i + 1, generatedLong);
break;
Expand All @@ -162,7 +162,7 @@ public PreparedStatement getStatement(Connection conn, List<TemplatedValue> para
stmt.setString(i + 1, param.getNextString());
break;
default:
throw param.createRuntimeException(paramType);
throw param.createRuntimeException();
}
break;
case TIMESTAMP:
Expand All @@ -183,7 +183,7 @@ public PreparedStatement getStatement(Connection conn, List<TemplatedValue> para
generatedTimestamp = param.getNextLongScrambled();
break;
default:
throw param.createRuntimeException(paramType);
throw param.createRuntimeException();
}
if (paramType == JDBCSupportedType.TIMESTAMP) {
stmt.setTimestamp(i + 1, new Timestamp(generatedTimestamp));
Expand Down
Loading
Loading