Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-threaded implementation of HyFD algorithm #409

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

cone-forest
Copy link
Contributor

The HyFD algorithm was originally single-threaded,
which goes against its intended implementation.
If ThreadNumber option is specified in the algorithm configuration step,
the algorithm will utilize the specified number of threads.

This implementation was benchmarked on several large datasets such as:

  • iowa1kk
  • EpicMeds
  • flight

The general trend is displayed in the following graph:
hyfd_bench_threads_iowa1kk
(data for iowa1kk dataset on i9-13905h)

Alternative implementations showing worse performance:

  • Desbordante's utils::ParallelForEach
  • std::future + std::async
  • std::execution::par
  • OpenMP parallel for macro
  • TBB parallel for function

The final implementation showed high time consumption by memory allocation/deallocation.
In attempt of optimizing them the following allocators were benchmarked:

  • tbb::scalable_allocator
  • tcmalloc
  • jemalloc
  • mimalloc
    All of which showed similar/worse performance,
    therefore implementing custom allocator is probably useless.

src/core/algorithms/fd/hyfd/hyfd.cpp Outdated Show resolved Hide resolved
src/core/algorithms/fd/hyfd/hyfd.h Outdated Show resolved Hide resolved
@cone-forest cone-forest force-pushed the feature/hyfd-multithreading branch from 5eaa93b to 7540633 Compare May 13, 2024 22:30
@cone-forest cone-forest requested a review from BUYT-1 May 13, 2024 22:41
The HyFD algorithm was originally single-threaded,
which goes against its intended implementation.
If ThreadNumber option is specified in the algorithm configuration step,
the algorithm will utilize the specified number of threads.
@chernishev chernishev force-pushed the feature/hyfd-multithreading branch from 0a76047 to 5e8ee61 Compare September 2, 2024 13:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants