Skip to content

Commit e6ab09d

Browse files
tnaglertnagler
authored
Update quickpool (#56)
* pull new quickpool * adapt library code * update benchmarks * save plots in benchmarks folder * log scale on x axis * use global pool * update quickpool * adapt thread pool * prettify benchmarks * add free push/async/wait * better benchmarks * actually benchmark quickpool * change plot size everywhere * affinity * better benchmarks * update benchmarks * backoff algorithm * less iter for imbalanced * update benchmarks * fix nested parallel for * inline free functions * don't move into parallel for * bump version * update docs * update parallel for doc/args * fix single threaded loops * backwards compatible batched loops * update quickpool * update docs * try to get codecov running * try on linux * yeah that won't work * keep distr as it was * update quickpool (resizing) * allow parallel loops to limit threads * reduce number of tests * finalize * safe exception tests * do-while in wait() * safer resize * update NEWS and README
1 parent 8725ca8 commit e6ab09d

27 files changed

+1533
-745
lines changed

‎.Rbuildignore

+1
Original file line numberDiff line numberDiff line change
@@ -15,3 +15,4 @@ revdep/
1515
.vscode/
1616
new-benchmarks.R
1717
bench*
18+
revdep/

‎.github/workflows/R-CMD-check.yaml

+1-7
Original file line numberDiff line numberDiff line change
@@ -80,7 +80,7 @@ jobs:
8080
_R_CHECK_CRAN_INCOMING_: false
8181
run: rcmdcheck::rcmdcheck(args = c("--no-manual", "--as-cran"), error_on = "warning", check_dir = "check")
8282
shell: Rscript {0}
83-
# run: cd .. && R CMD build RcppThread && R CMD check RcppThread_1.1.0.tar.gz
83+
# run: cd .. && R CMD build RcppThread && R CMD check RcppThread_1.1.0.tar.gz
8484
# shell: bash
8585

8686
- name: Show testthat output
@@ -94,9 +94,3 @@ jobs:
9494
with:
9595
name: ${{ runner.os }}-r${{ matrix.config.r }}-results
9696
path: check
97-
98-
# - name: Test coverage
99-
# if: matrix.config.os == 'macOS-latest' && matrix.config.r == 'release'
100-
# run: |
101-
# Rscript -e 'remotes::install_github("r-lib/covr@gh-actions")'
102-
# Rscript -e 'covr::codecov(token = "${{secrets.CODECOV_TOKEN}}")'

‎DESCRIPTION

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
Package: RcppThread
22
Title: R-Friendly Threading in C++
3-
Version: 1.1.0
3+
Version: 2.0.0
44
Authors@R: c(
55
person("Thomas", "Nagler",, "[email protected]", role = c("aut", "cre"),
66
comment = c(ORCID = "0000-0003-1855-0046"))
@@ -13,7 +13,7 @@ Encoding: UTF-8
1313
SystemRequirements: C++11
1414
URL: https://github.com/tnagler/RcppThread
1515
BugReports: https://github.com/tnagler/RcppThread/issues
16-
RoxygenNote: 7.1.1
16+
RoxygenNote: 7.1.2
1717
Suggests:
1818
testthat,
1919
R.rsp,

‎NEWS.md

+6-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# RcppThread 1.1.0
1+
# RcppThread 2.0.0
22

33
* Add R function `detectCores()` (#48).
44

@@ -10,6 +10,11 @@
1010
* Free-standing `parallelFor()` and `parallelForEach()` functions now dispatch
1111
to a global thread pool that persists for the entire session. This
1212
significantly speeds up programs that repeatedly call these functions. (#54)
13+
14+
* New free-standing `push()`, `pushReturn()`/`async()`, and `wait()`, mirroring
15+
functionality from `ThreadPool`. (#56)
16+
17+
* Option to resize a thread pool (#56).
1318

1419

1520
# RcppThread 1.0.0

‎README.md

+5-3
Original file line numberDiff line numberDiff line change
@@ -27,13 +27,15 @@ For a detailed description of its functionality and examples, see the associated
2727
[JSS paper](https://doi.org/10.18637/jss.v097.c01)
2828
or the [API documentation](https://tnagler.github.io/RcppThread/).
2929

30-
Since then, the following features have been added:
30+
Since then, the following **new features** have been added:
3131

32-
- Free-standing `parallelFor()` and `parallelForEach()` functions now dispatch
32+
- Free-standing functions like `parallelFor()` now dispatch
3333
to a global thread pool that persists for the entire session. This
3434
significantly speeds up programs that repeatedly call these functions.
3535

36-
- Faster runtimes due to a work stealing task queue with lock-free pops (from [quickpool](https://github.com/tnagler/quickpool)).
36+
- Faster runtimes due to lock-free work stealing queue and loops (from [quickpool](https://github.com/tnagler/quickpool)).
37+
38+
- Option to resize a thread pool.
3739

3840
- An R function `RcppThread::detectCores()` to determine the number of (logical)
3941
cores on your machine.

‎benchmarks/benchmarks.R

+85
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
library("tidyverse")
2+
library("Rcpp")
3+
library("RcppParallel")
4+
library("RcppThread")
5+
library("ggthemes")
6+
7+
8+
Rcpp::sourceCpp(here::here("benchmarks/benchmarks.cpp"))
9+
10+
11+
wait_for <- 5
12+
13+
plot_df <- function(df, title = NULL) {
14+
p <- df %>%
15+
pivot_longer(-n, "call") %>%
16+
ggplot(aes(n, value, color = call)) +
17+
geom_line(size = 0.6) +
18+
expand_limits(y = 0) +
19+
labs(color = "", linetype = "") +
20+
naglr::theme_naglr(plot_title_size = 12) +
21+
theme(legend.margin = margin(1, 1, 1, 1)) +
22+
theme(legend.position = "bottom") +
23+
scale_x_log10() +
24+
ylab("speedup") +
25+
labs(title = title)
26+
print(p)
27+
}
28+
29+
30+
ns <- 10^(2:6)
31+
res <- benchEmpty(rev(ns), wait_for)
32+
df <- cbind(data.frame(n = rev(ns)), as.data.frame(res[, -1]))
33+
plot_df(df, "empty jobs")
34+
ggsave(here::here("benchmarks/benchEmptyThread.pdf"), width = 7.5, height = 3)
35+
36+
37+
ns <- 10^(2:6)
38+
res <- benchSqrt(rev(ns), wait_for)
39+
df <- cbind(data.frame(n = rev(ns)), as.data.frame(res[, -1]))
40+
plot_df(df, "1000x sqrt")
41+
ggsave(here::here("benchmarks/benchSqrt.pdf"), width = 7.5, height = 3)
42+
43+
44+
# ns <- 10^(2:6)
45+
# res <- benchSqrtWrite(rev(ns), wait_for)
46+
# df <- cbind(data.frame(n = rev(ns)), as.data.frame(res[, -1]))
47+
# plot_df(df, "1000x sqrt modify inplace")
48+
# ggsave(here::here("benchmarks/benchSqrtWrite.pdf"), width = 7.5, height = 3)
49+
50+
ns <- 10^(2:4)
51+
res <- benchSqrtImbalanced(rev(ns), wait_for)
52+
df <- cbind(data.frame(n = rev(ns)), as.data.frame(res[, -1]))
53+
plot_df(df, "n times sqrt (imbalanced)")
54+
ggsave(here::here("benchmarks/benchSqrtImbalanced.pdf"), width = 7.5, height = 3)
55+
56+
# ns <- 10^(2:4)
57+
# res <- benchSqrtWriteImbalanced(rev(ns), wait_for)
58+
# df <- cbind(data.frame(n = rev(ns)), as.data.frame(res[, -1]))
59+
# plot_df(df, "n times sqrt modify inplace (imbalanced)")
60+
# ggsave(here::here("benchmarks/benchSqrtWriteImbalanced.pdf"), width = 7.5, height = 3)
61+
62+
ns <- 10^(2:5)
63+
res <- benchKDE(rev(ns), 100, wait_for)
64+
df <- cbind(data.frame(n = rev(ns)), as.data.frame(res[, -1]))
65+
plot_df(df, "kernel density d = 10")
66+
ggsave(here::here("benchmarks/benchKDE-10.pdf"), width = 7.5, height = 3)
67+
68+
69+
ns <- 10^(2:5)
70+
res <- benchKDE(rev(ns), 100, wait_for)
71+
df <- cbind(data.frame(n = rev(ns)), as.data.frame(res[, -1]))
72+
plot_df(df, "kernel density d = 100")
73+
ggsave(here::here("benchmarks/benchKDE-100.pdf"), width = 7.5, height = 3)
74+
75+
ns <- 10^(2:4)
76+
res <- benchKendall(rev(ns), 10, wait_for)
77+
df <- cbind(data.frame(n = rev(ns)), as.data.frame(res[, -1]))
78+
plot_df(df, "Kendall matrix (unbalanced) d = 10")
79+
ggsave(here::here("benchmarks/benchKendall-10.pdf"), width = 7.5, height = 3)
80+
81+
ns <- 10^(2:4)
82+
res <- benchKendall(rev(ns), 100, wait_for)
83+
df <- cbind(data.frame(n = rev(ns)), as.data.frame(res[, -1]))
84+
plot_df(df, "Kendall matrix (unbalanced) d = 100")
85+
ggsave(here::here("benchmarks/benchKendall-100.pdf"), width = 7.5, height = 3)

0 commit comments

Comments
 (0)