Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: benchmark scripts #909

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 21 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,13 +102,15 @@ detray-build/bin/detray_detector_validation \
```
In case of failures, this command will give a detailed debug output in the form of a log file, as well as an SVG representation of the failed tracks. The grid file is optional, but will trigger the use of spacial grids as acceleration structures during the navigation run.

Note: The `search_window` option defines the size of lookup area of the grid acceleration struction and is therefore detector dependent! Use `--search_window 3 3` (or larger) for the *toy detector* and *wire chamber* example detectors and `--search_window 0 0` otherwise.

### Material Validation

This tool checks whether the navigator picks up the material correctly by comparing the material found during a ray scan with the material collected during navigation by a specialized actor:
```shell
detray-build/bin/detray_material_validation \
--geometry_file toy_detector/toy_detector_geometry.json \
--material_file toy_detector/toy_detector_homogeneous_material.json \
--geometry_file ./toy_detector/toy_detector_geometry.json \
--material_file ./toy_detector/toy_detector_homogeneous_material.json \
--phi_steps 100 --eta_steps 100 --eta_range -4 4
```
Note: The correct material file must be loaded in addition to the geometry file!
Expand All @@ -122,9 +124,24 @@ detray-build/bin/detray_propagation_benchmark_<backend>_<algebra> \
--geometry_file ./toy_detector/toy_detector_geometry.json \
--grid_file ./toy_detector/toy_detector_surface_grids.json \
--material_file ./toy_detector/toy_detector_homogeneous_material.json \
--sort_tracks --randomize_charge --eta_range -3 3 -pT_range 1 100
--sort_tracks --randomize_charge --eta_range -3 3 --pT_range 0.5 100 \
--search_window 3 3 --covariance_transport
```
For every algebra-plugin that was built, a corresponding benchmark executable will be present. The CPU-backend benchmark is built by default and the CUDA-backend benchmark will be available if detray was built with CUDA enabled (`-DDETRAY_BUILD_CUDA=ON`). This executable can additionally be configured with any arguments targeted at [google benchmark](https://github.com/google/benchmark/blob/main/docs/user_guide.md).

If the data is dumped into json files using the options `--benchmark_out_format=json` and `--benchmark_out=<detector_name>_benchmark_data_<backend>_<algebra>.json`, it can afterwards be plotted with e.g.:
```shell
python3 detray/tests/tools/python/propagation_benchmarks.py \
--geometry_file ./toy_detector/toy_detector_geometry.json \
--algebra_plugins array eigen \
--cuda \
--data_files ./toy_detector_benchmark_data_cpu_array.json \
./toy_detector_benchmark_data_cpu_eigen.json \
./toy_detector_benchmark_data_cuda_array.json \
./toy_detector_benchmark_data_cuda_eigen.json
```
For every algebra-plugin that was built, a corresponding benchmark executable will be present. The CPU-backend benchmark is built by default and the CUDA-backend benchmark will be available if detray was built with CUDA enabled (`-DDETRAY_BUILD_CUDA=ON`).

using the *std::array* and [*Eigen3*](https://eigen.tuxfamily.org) based linear algebra plugins with CUDA backend as an example.

### Continuous benchmark

Expand Down
2 changes: 1 addition & 1 deletion core/include/detray/propagator/propagation_config.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ struct config {
<< "----------------------------\n"
<< cfg.stepping << "\nGeometry Context\n"
<< "----------------------------\n"
<< cfg.context.get() << "\n";
<< " No. : " << cfg.context.get() << "\n";

return out;
}
Expand Down
49 changes: 44 additions & 5 deletions core/include/detray/utils/type_list.hpp
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
/** Detray library, part of the ACTS project (R&D line)
*
* (c) 2023 CERN for the benefit of the ACTS project
* (c) 2023-2025 CERN for the benefit of the ACTS project
*
* Mozilla Public License Version 2.0
*/
Expand All @@ -12,6 +12,7 @@
#include "detray/utils/type_traits.hpp"

// System include(s)
#include <ranges>
#include <string>
#include <string_view>
#include <type_traits>
Expand Down Expand Up @@ -142,16 +143,54 @@ std::string demangle_type_name() {
return std::string{function.substr(start, size)};
}

/// @returns the name of a type as string
/// @tparam T the type
template <typename T>
std::string get_name(bool full = false) {
std::string tp_str{""};
try {
tp_str = detray::types::demangle_type_name<T>();
} catch (...) {
return "unknown";
}

if (tp_str.empty()) {
return "unknown";
}

if (full) {
return tp_str;
}

// Remove the template argument list
dvector<std::string> tokens{};
for (const auto t : std::views::split(tp_str, '<')) {
tokens.emplace_back(t.begin(), t.end());
}

// Split a the first ocurrence of '<'
tp_str = tokens.front();
tokens.clear();

// Strip the namespaces and qualifiers
for (const auto t : std::views::split(tp_str, ':')) {
tokens.emplace_back(t.begin(), t.end());
}

// Split at the last occurrence of ':'
return tokens.back();
}

template <typename = void>
struct print {};

template <typename... Ts>
struct print<list<Ts...>> {

template <typename P = void, typename... Ps>
void print_typeid() {
void print_typeid(bool full) {

std::printf("%s", demangle_type_name<P>().c_str());
std::printf("%s", get_name<P>(full).c_str());

// Keep unrolling the pack
if constexpr (sizeof...(Ps) > 0) {
Expand All @@ -160,9 +199,9 @@ struct print<list<Ts...>> {
}
}

print() {
print(bool full = true) {
std::printf("type_list<");
print_typeid<Ts...>();
print_typeid<Ts...>(full);
std::printf(">\n");
}
};
Expand Down
6 changes: 6 additions & 0 deletions tests/tools/python/impl/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
from .plot_benchmark_results import (
read_benchmark_data,
prepare_benchmark_data,
plot_benchmark_case,
plot_benchmark_data,
)
from .plot_navigation_validation import (
read_scan_data,
read_navigation_data,
Expand Down
Loading
Loading