Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rename Json Files #315

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 12 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ dependencies.
Sandmark uses opam, with a static local repository, to build external
libraries and applications. It then builds any sandmark OCaml
benchmarks and any data dependencies. Following this it runs the
benchmarks as defined in the `run_config.json`
benchmarks as defined in the `sequential.json`

These stages are implemented in:

Expand All @@ -57,15 +57,15 @@ These stages are implemented in:

- Runplan: the list of benchmarks which will run along with the
measurement wrapper (e.g. orun or perf) is specified in
`run_config.json`. This config file is used to generate dune files
`sequential.json`. This config file is used to generate dune files
which will run the benchmarks.

- Build: dune is used to build all the sandmark OCaml benchmarks that
are in the `benchmarks` directory.

- Execute: dune is used to execute all the benchmarks sepcified in
the runplan using the benchmark wrapper defined in
`run_config.json` and specified via the `RUN_BENCH_TARGET` variable
`sequential.json` and specified via the `RUN_BENCH_TARGET` variable
passed to the makefile.

## Configuration of the compiler build
Expand Down Expand Up @@ -141,7 +141,7 @@ sudo setcap cap_sys_nice=ep /usr/bin/chrt
### Configuring the benchmark runs

A config file can be specified with the environment variable `RUN_CONFIG_JSON`,
and the default value is `run_config.json`. This file lists the executable to
and the default value is `sequential.json`. This file lists the executable to
run and the wrapper which will be used to collect data (e.g. orun or perf). You
can edit this file to change benchmark parameters or wrappers.

Expand Down Expand Up @@ -177,13 +177,13 @@ current tags are:
The benchmarking machine `turing` is an Intel Xeon Gold 5120 CPU with 64GB of
RAM housed at IITM.

The `run_config.json` file may be filtered based on the tag. For example,
The `sequential.json` file may be filtered based on the tag. For example,

```bash
$ TAG='"macro_bench"' make run_config_filtered.json
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$ TAG='"macro_bench"' make run_config_filtered.json
$ TAG='"macro_bench"' make sequential_filtered.json

The filtered files should also be changed to their new names. Leave the filtered as such and edit the name. This needs to be updated in the run_all* scripts too.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Sudha, thank you for the support, i just did the new changes but couldn't find the run_config_filtered.json file itself.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! You're right, run_config_filtered.json is created only after we run TAG='"macro_bench"' make run_config_filtered.json, so no need to modify that file.

Are you able to repeat this process for the other three files, like for example run_all_parallel.sh has multicore_parallel_run_config_filtered.json. git grep command should help here too. :)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah i did git grep run_config.json, you could see the changes in all the files. i made sure i changed that in all files. could you please check?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry if it was unclear; what I meant is the same process of replacing *_filtered.json with the new file name needs to be done for multicore_parallel_run_config_filtered.json, i.e it will be called parallel_turing_filtered.json and multicore_parallel_navajo_run_config_filtered.json will be parallel_navajo_filtered.json.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Sudha, no worries i just made the changes
could you please review?
Also, there is this file multicore_parallel_run_config_filtered_filtered_2domains, does it need renaming?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, there is this file multicore_parallel_run_config_filtered_filtered_2domains, does it need renaming?

Good point. Yes, we can call it parallel_turing_filtered_filtered_2domains.json. You can see how it's generated here: https://github.com/ocaml-bench/sandmark/blob/main/Makefile#L347-L348. This should make the CI happy I think. You might also want to edit the drone.yml file with the latest names.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Sudha, thank you so much for your endless support, i just did the changes and modified the drone.yml file too. please review for merge or correction of errors, thank you.

```

filters the `run_config.json` file to only contain the benchmarks tagged as
filters the `sequential.json` file to only contain the benchmarks tagged as
`macro_bench`.

### Running benchmarks
Expand Down Expand Up @@ -261,17 +261,17 @@ You can add new benchmarks as follows:

- **Add commands to run your applications:**
Add an entry for your benchmark run to the appropriate config file;
`run_config.json` for sequential benchmarks and
`multicore_parallel_run_config.json` for parallel benchmarks.
`sequential.json` for sequential benchmarks and
`parallel_turing.json` for parallel benchmarks.

### Config files

The `*_config.json` files used to build benchmarks

- **run_config.json** : Runs sequential benchmarks with stock OCaml variants in CI and sandmark-nightly on the IITM machine(turing)
- **multicore_parallel_run_config.json** : Runs parallel benchmarks with multicore OCaml in CI and sandmark-nightly on the IITM machine(turing)
- **multicore_parallel_navajo_run_config.json** : Runs parallel benchmarks with multicore OCaml in sandmark-nightly on Navajo (AMD EPYC 7551 32-Core Processor) machine
- **micro_multicore.json** : To locally run multicore specific micro benchmarks
- **sequential.json** : Runs sequential benchmarks with stock OCaml variants in CI and sandmark-nightly on the IITM machine(turing)
- **parallel_turing.json** : Runs parallel benchmarks with multicore OCaml in CI and sandmark-nightly on the IITM machine(turing)
- **parallel_navajo.json** : Runs parallel benchmarks with multicore OCaml in sandmark-nightly on Navajo (AMD EPYC 7551 32-Core Processor) machine
- **micro.json** : To locally run multicore specific micro benchmarks

### Benchmarks status

Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.