Skip to content

Commit

Permalink
fix: Restore Standalone Benchmark Artifact (#359)
Browse files Browse the repository at this point in the history
  • Loading branch information
stanbrub authored Oct 9, 2024
1 parent 686f77c commit 27b8167
Show file tree
Hide file tree
Showing 7 changed files with 46 additions and 11 deletions.
2 changes: 1 addition & 1 deletion .github/scripts/run-benchmarks-remote.sh
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ cd ${DEEPHAVEN_DIR};
title "-- Running Benchmarks --"
cd ${RUN_DIR}
cat ${RUN_TYPE}-scale-benchmark.properties | sed 's|${baseRowCount}|'"${ROW_COUNT}|g" | sed 's|${baseDistrib}|'"${DISTRIB}|g" > scale-benchmark.properties
JAVA_OPTS="-Dbenchmark.profile=scale-benchmark.properties -jar deephaven-benchmark-*.jar -cp standard-tests.jar"
JAVA_OPTS="-Dbenchmark.profile=scale-benchmark.properties -jar deephaven-benchmark-*-standalone.jar -cp standard-tests.jar"

if [ "${TAG_NAME}" = "Any" ]; then
java ${JAVA_OPTS} -p ${TEST_PACKAGE} -n "${TEST_PATTERN}"
Expand Down
2 changes: 1 addition & 1 deletion .github/scripts/run-publish-local.sh
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ sudo docker compose up -d
sleep 10

cd ${RUN_DIR}
java -Dbenchmark.profile=${BENCH_PROPS_NAME} -jar deephaven-benchmark-*.jar publish
java -Dbenchmark.profile=${BENCH_PROPS_NAME} -jar deephaven-benchmark-*-standalone.jar publish

cd ${DEEPHAVEN_DIR};
sudo docker compose down
Expand Down
8 changes: 4 additions & 4 deletions docs/CommandLine.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,24 @@
# Benchmark - Command Line Usage

Users can run Benchmark in an IDE with a standard JUnit test plugin or from the command line. The Benchmark artifact (deephaven-benchmark-1.0-SNAPSHOT.jar)
Users can run Benchmark in an IDE with a standard JUnit test plugin or from the command line. The Benchmark standalone jar (deephaven-benchmark-1.0-SNAPSHOT-standalone.jar)
contains all dependencies needed to run the framework. The standalone console launcher for JUnit is used to run the tests, so all of its command line options are
available from Benchmark's main jar.

## Examples

Have a look at the available arguments:
```
java -jar deephaven-benchmark-1.0-SNAPSHOT.jar --help
java -jar deephaven-benchmark-1.0-SNAPSHOT-standalone.jar --help
```

Run tests in a your own jar
```
java -jar deephaven-benchmark-1.0-SNAPSHOT.jar -cp your-tests.jar -p io.deephaven.your.tests
java -jar deephaven-benchmark-1.0-SNAPSHOT-standalone.jar -cp your-tests.jar -p io.deephaven.your.tests
```

Run tests in your own jar using your own property file
```
java -D"benchmark.profile"="your-benchmark.properties" -jar deephaven-benchmark-1.0-SNAPSHOT.jar -cp your-tests.jar -p your.tests
java -D"benchmark.profile"="your-benchmark.properties" -jar deephaven-benchmark-1.0-SNAPSHOT-standalone.jar -cp your-tests.jar -p your.tests
```

## Results
Expand Down
3 changes: 2 additions & 1 deletion docs/GettingStarted.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,8 @@ After checking out the Benchmark project, running *mvn verify* from the director
the code, package the uber main and test jars, and run some integration tests against your running Deephave Server.

The jar artifacts produce by the build are
- deephaven-benchmark-1.0-SNAPSHOT.jar: The main uber jar
- deephaven-benchmark-1.0-SNAPSHOT.jar: The main artifact
- deephaven-benchmark-1.0-SNAPSHOT-standalone.jar: The standalone super jar
- deephaven-benchmark-1.0-SNAPSHOT-tests.jar: The jar containing project tests

Run the uber jar according to the [Command Line Guide](CommandLine.md)
Expand Down
34 changes: 34 additions & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,40 @@
<target>21</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.5.1</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<shadedArtifactAttached>true</shadedArtifactAttached>
<shadedClassifierName>standalone</shadedClassifierName>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<manifestEntries>
<Main-Class>io.deephaven.benchmark.run.BenchmarkMain</Main-Class>
</manifestEntries>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
Expand Down
4 changes: 2 additions & 2 deletions src/main/java/io/deephaven/benchmark/run/BenchmarkMain.java
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@
* Main class for running the benchmark framework from the deephaven-benchmark artifact. This wraps the JUnit standalone
* console launcher.
* <p/>
* ex. java -Dbenchmark.profile=my-benchmark.properties -jar deephaven-benchmark-1.0-SNAPSHOT.jar -cp my-tests.jar -p
* my.tests
* ex. java -Dbenchmark.profile=my-benchmark.properties -jar deephaven-benchmark-1.0-SNAPSHOT-standalone.jar -cp
* my-tests.jar -p my.tests
* <p/>
* In addition to running benchmark tests using the console launcher, this class creates a
* <code>benchmark-summary-results.csv</code> that is a merge of any per-run results files that match
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ rm -rf /home/stan/Data/benchmark/results
rm -f /home/stan/Deephaven/deephaven-edge/data/*.def
rm -f /home/stan/Deephaven/deephaven-edge/data/*.parquet

JAVA_OPTS="-Dbenchmark.profile=scale-benchmark.properties -jar deephaven-benchmark-1.0-SNAPSHOT.jar -cp deephaven-benchmark-1.0-SNAPSHOT-tests.jar"
JAVA_OPTS="-Dbenchmark.profile=scale-benchmark.properties -jar deephaven-benchmark-1.0-SNAPSHOT-standalone.jar -cp deephaven-benchmark-1.0-SNAPSHOT-tests.jar"

# Run all benchmarks except tagged then run tagged benchmarks with iterations (Similar to what release and nightly do)
java ${JAVA_OPTS} -p io.deephaven.benchmark.tests.standard -n "^.*[.].*Test.*$" -T Iterate
Expand All @@ -28,7 +28,7 @@ java ${JAVA_OPTS} -p io.deephaven.benchmark.tests.standard -t Iterate
done

# Publish accumulated results for scores to slack
java -Dbenchmark.profile=smoke-test-scale-benchmark.properties -jar deephaven-benchmark-1.0-SNAPSHOT.jar publish
java -Dbenchmark.profile=smoke-test-scale-benchmark.properties -jar deephaven-benchmark-1.0-SNAPSHOT-standalone.jar publish

# Run all benchmarks regardless of tag (Similar to what adhoc and compare do)
rm -f /home/stan/Deephaven/deephaven-edge/data/*.def
Expand Down

0 comments on commit 27b8167

Please sign in to comment.