Building benchmarks for wasm32-wasi

[titzer]

Hello,

I would like to build the wasm32 benchmarks from source but I could not figure out how.

Starting from here: https://github.com/jedisct1/webassembly-benchmarks, I am able to run the .wasm binaries of the various benchmarks. However, some of they take a really, really, long time, particularly on the slower engines. I would like to adjust the amount of work done in some of the benchmark line items in order to make them run in a reasonable time (~1 second each).

I looked at the build scripts in build-dists/wasm32-wasi.sh but it seems to want a Makefile to have been generated, yet there is not a configure or configure.ac in this directory. I can build the library with the zig build but I don't know how to build the benchmarks.

[jedisct1]

Hi!

In order to build the benchmark, make sure that you are using the stable branch, then run dist-build/wasm32-wasi.sh --bench from the top directory.

The wasm32-wasi.sh script defines an -DITERATIONS=100. This is how many times individual tests are run. The value can be lowered, at the expense of accuracy.

[titzer]

Hi, thanks, I didn't realize this script should be run from the top-level directory.

With ITERATIONS=100, several benchmarks still take several minutes (on wasm3) to complete, while others take only milliseconds. I'm going to look into making adjustments so that an iteration of each benchmark is roughly the same amount amount of time. Specifically, for my use case, having a short iteration time (< 100ms or even < 10ms) allows running against engines that have an interpreter only. It also allows gathering many process executions (e.g. 100 samples) and then doing proper statistical analysis on them.

Further question: There is a lot of use of rand() in the benchmarks. Do any of them take more or less time based on the random numbers, or all of the random numbers used as "data" rather than "control"?