-
Notifications
You must be signed in to change notification settings - Fork 11
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Update README and explain how to compile benchmarks * Update CI workflow for C code * Remove workflow for .NET Code * Move HashTableBenchmark to separate folder
- Loading branch information
Showing
5 changed files
with
75 additions
and
176 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,218 +1,134 @@ | ||
| # Anybench benchmarks set | ||
|
|
||
| **`Benchmarks:`** | ||
| ## Generic benchmarks | ||
|
|
||
| ### Content and description | ||
|
|
||
| * Dhrystone v 2.1 | ||
| * Whetstone | ||
| * Whetstone MP | ||
| * Memspeed | ||
| * Linpack | ||
| * Lloops | ||
| * Busspeed | ||
| * MP MFlops | ||
| * Coremark | ||
| * Scimark 2 | ||
|
|
||
| Already contains precompiled binaries in `bin/` directory. | ||
| So you can just | ||
|
|
||
| ```sh | ||
| cd src/ | ||
| ./run.sh | ||
| ``` | ||
| * Whetstone MP - multi-thread version of classic Whetstone benchmark | ||
| * Memspeed - simple single-threaded memory speed test | ||
| * Linpack - classical computational benchmark, this is classic C version on small arrays (100x100). | ||
| * [Lloops](https://www.netlib.org/benchmark/livermorec) - Livermore loops, classic benchmark for supercomputers for late 80-s. | ||
| * [Busspeed](http://www.roylongbottom.org.uk/#anchorBusSpd2K) | ||
| * [MP MFlops](http://www.roylongbottom.org.uk/#anchorMPGFLOPS) - syntethic benchmark that tries to achieve peak FP performance on CPU. Significantly benefits from SIMD and it's highly recommended that you **modify march/mtune flag for your CPU and recompile it** | ||
| * [Coremark](https://github.com/eembc/coremark) - industry standard benchmark that stresses CPU Pipeline | ||
| * [Scimark 2](https://math.nist.gov/scimark2/index.html) - C version of classic Java scientific/computational benchmark | ||
| * [SuperPI](https://en.wikipedia.org/wiki/Super_PI) - classic software that computes Pi up to N digits. | ||
| * [STREAM](https://www.cs.virginia.edu/stream/) - multicore memory bandwidth benchmark | ||
| * [GSynth](https://github.com/EntityFX/anybench/pull/17) - contributed benchmark that evaluates CPU on audio synthesis tasks, code was taken from real-world software. | ||
|
|
||
| ## Configure | ||
| ### Running | ||
|
|
||
| You can generate build and run scripts: | ||
| There are some already precompiled binaries in `bin/` directory. | ||
| So you can just | ||
|
|
||
| ```sh | ||
| cd src/ | ||
| ./configure.sh | ||
| ./run.sh | ||
| ``` | ||
|
|
||
| ### Configure Coremark | ||
| However, it's highly recommended compiling your own binaries with platform-specific flags set and run them. | ||
|
|
||
| ```sh | ||
| cd src/ | ||
| ./configure-coremark.sh | ||
| ``` | ||
| ### Build | ||
|
|
||
| ### Configure Scimark 2 | ||
| This repo uses git submodules, if cloning: | ||
|
|
||
| ```sh | ||
| cd src/ | ||
| ./configure-coremark.sh | ||
| ```shell | ||
| git clone --recurse-submodules https://github.com/EntityFX/anybench.git | ||
| ``` | ||
|
|
||
| ## Compile | ||
|
|
||
| For **arm** architecture | ||
|
|
||
| ```sh | ||
| cd src/ | ||
| ./make-arm.sh | ||
| Alternatively: | ||
| ```shell | ||
| git clone https://github.com/EntityFX/anybench.git | ||
| cd anybench | ||
| git submodule update --init --recursive | ||
| ``` | ||
|
|
||
| For **amd64** architecture | ||
| If you want to modify compile flags, please edit corresponding `make-<arch>.sh` script and add them to corresponding parameter: | ||
| * `targetToFlags` - script will iterate over each flag specified there and compile a single version. | ||
| * `targetToFPU` - for the same key specified in `targetToFlags` all the flags will be used | ||
|
|
||
| ```sh | ||
| cd src/ | ||
| ./make-amd64.sh | ||
| ``` | ||
| There are separate scripts that set some optimization specific flags for particular CPU architecture. Some examples are: | ||
|
|
||
| For **e2k** architecture | ||
| #### For **arm** architecture (non-MacOS) | ||
|
|
||
| ```sh | ||
| ```shell | ||
| cd src/ | ||
| ./make-e2k.sh | ||
| ``` | ||
|
|
||
| ### Compile Coremark | ||
|
|
||
| For **arm** architecture | ||
|
|
||
| ```sh | ||
| cd src/ | ||
| ./make-coremark-arm.sh | ||
| ``` | ||
|
|
||
| For **amd64** architecture | ||
|
|
||
| ```sh | ||
| cd src/ | ||
| ./make-coremark-amd64.sh | ||
| ``` | ||
| For **e2k** architecture | ||
|
|
||
| ```sh | ||
| cd src/ | ||
| ./make-coremark-e2k.sh | ||
| ./make-arm.sh | ||
| ``` | ||
|
|
||
| ### Compile Scimark 2 | ||
|
|
||
| For **arm** architecture | ||
| #### For **arm** architecture (MacOS) | ||
|
|
||
| ```sh | ||
| ```shell | ||
| cd src/ | ||
| ./make-scimark2-arm.sh | ||
| ./make-mac-arm.sh | ||
| ``` | ||
|
|
||
| For **amd64** architecture | ||
|
|
||
| ```sh | ||
| #### For **amd64** architecture (using default CC) | ||
| **NOTE:** By default it will try to compile binary optimized for AMD Zen2, so if your CPU is different please modify script. That might still be ok for any other modern AMD CPU (it must support AVX2) and Intel CPUs that doesn't support AVX-512. | ||
| ```shell | ||
| cd src/ | ||
| ./make-scimark2-amd64.sh | ||
| ``` | ||
|
|
||
| For **e2k** architecture | ||
|
|
||
| ```sh | ||
| cd src/ | ||
| ./make-scimark2-e2k.sh | ||
| ./make-amd64.sh | ||
| ``` | ||
|
|
||
| ## Run | ||
|
|
||
| For **arm** architecture | ||
|
|
||
| ```sh | ||
| #### For **amd64** architecture (forcing clang) | ||
| **NOTE:** By default it will try to compile binary optimized for AMD Zen2, so if your CPU is different please modify script. That might still be ok for any other modern AMD CPU (it must support AVX2) and Intel CPUs that doesn't support AVX-512. | ||
| ```shell | ||
| cd src/ | ||
| ./run-arm.sh | ||
| ./make-amd64_clang.sh | ||
| ``` | ||
|
|
||
| For **amd64** architecture | ||
|
|
||
| ```sh | ||
| cd src/ | ||
| ./run-amd64.sh | ||
| ``` | ||
|
|
||
| For **e2k** architecture | ||
|
|
||
| ```sh | ||
| ```shell | ||
| cd src/ | ||
| ./run-e2k.sh | ||
| ./make-e2k.sh | ||
| ``` | ||
|
|
||
| ### Run Coremark | ||
| ### Results | ||
|
|
||
| For **arm** architecture | ||
| You can see results in `results/<your-arch>` direcotry. | ||
|
|
||
| ```sh | ||
| cd src/ | ||
| ./run-coremark-arm.sh | ||
| ``` | ||
| Please rename them to some identifiable thing before submitting. Recommended format: `<cpu-vendor>-<cpu-architecture>-<cpu-model>[-<optional-identifiers>]` | ||
|
|
||
| For **amd64** architecture | ||
| Some examples: | ||
|
|
||
| ```sh | ||
| cd src/ | ||
| ./run-coremark-amd64.sh | ||
| ``` | ||
| * `apple-arm-m1-max-perf-auto` - contains results from Apple M1 Max CPU on OSX with performance mode set to `auto`. | ||
| * `intel-x86-core_i7_2600-clang` - contains results from Intel Core i7 2600, compiled with clang. | ||
| * `nvidia-arm-tegra_xavier_t194-jetson_agx` - contains results obtained on NVidia ARM CPU Tegra T194, board Xavier AGX. | ||
|
|
||
| For **e2k** architecture | ||
|
|
||
| ```sh | ||
| cd src/ | ||
| ./run-coremark-e2k.sh | ||
| ``` | ||
|
|
||
| ### Run Scimark2 | ||
| ## Advanced benchmarks | ||
|
|
||
| For **arm** architecture | ||
| Those benchmarks requires certain software or libraries installed and might be hard to run on devices like smartphones | ||
|
|
||
| ```sh | ||
| cd src/ | ||
| ./run-scimark2-arm.sh | ||
| ``` | ||
| * [HashTableBenchmark](https://github.com/unum-cloud/HashTableBenchmark) - benchmark that tries to simulate typical in-memory database patterns. Basically it benchmarks performance of several hash table implementations, like `std::unordered_map`, `tsl::sparse_map` and others. | ||
|
|
||
| For **amd64** architecture | ||
| Currently there is no pre-build binaries for advanced benchmarks, as well as currently there is no single script to build them. | ||
|
|
||
| ```sh | ||
| cd src/ | ||
| ./run-scimark2-amd64.sh | ||
| ``` | ||
| ### Building | ||
|
|
||
| For **e2k** architecture | ||
| #### HashTableBenchmark | ||
| This benchmark requires CMake 3.x installed and it also require internet connection to fetch it's dependencies (it uses particular commits of gtest and gbenchmark libraries). | ||
|
|
||
| ```sh | ||
| cd src/ | ||
| ./run-scimark2-e2k.sh | ||
| ```shell | ||
| cd src/benchmarks/advanced/HashTableBenchmark | ||
| mkdir -p build && cmake -DCMAKE_BUILD_TYPE=Release -B ./build && make -j4 -C ./build | ||
| ``` | ||
|
|
||
| ## Results | ||
|
|
||
| You can see results in `results/<your-os>/<your-arch>` direcotry. | ||
| Additional results in `src\` direcotry. | ||
| ### Running | ||
|
|
||
| #### HashTableBenchmark | ||
|
|
||
| ## Other benchmarks | ||
| You must be in repository root | ||
|
|
||
| ### Whetstone Java | ||
|
|
||
| ```sh | ||
| cd src/java | ||
| java whetstc | ||
| ```shell | ||
| mkdir -p results/$(uname -m) | ||
| src/benchmarks/advanced/HashTableBenchmark/build/build/bin/mixed_ops 2>&1 | tee results/$(uname -m)/HashTableBenchmark-mixed_ops.log | ||
| src/benchmarks/advanced/HashTableBenchmark/build/build/bin/ram_speed 2>&1 | tee results/$(uname -m)/HashTableBenchmark-ram_speed.log | ||
| ``` | ||
|
|
||
| ### Whetstone JavaScript (nodejs or web browser) | ||
|
|
||
| ```sh | ||
| cd src/js | ||
| node whetstone.js | ||
| ``` | ||
|
|
||
| ### Whetstone C# (mono, net framework, dotnet core) | ||
|
|
||
| ```sh | ||
| cd bin/dotnet/mono/ | ||
| mono WhetstoneNetFramework.exe | ||
| ``` | ||
|
|
||
| ### Coremark for E2k (Elbrus) | ||
|
|
||
| ```sh | ||
| cd bin/linux/e2k/ | ||
| ./coremark_elbrus-<your_cpu> | ||
| ``` |