Skip to content

Commit e8b7cba

Browse files
lionel-hadleyjennybc
authored
Write withr 3.0.0 post (#678)
* Write withr 3.0.0 post * We're using LIFO not FIFO (thanks Hannah!) * his -> its * Apply suggestions from code review Co-authored-by: Hadley Wickham <[email protected]> * More suggestions from review * More LIFO/FIFO confusion 😬 * Apply suggestions from code review Co-authored-by: Jennifer (Jenny) Bryan <[email protected]> * More intuitive order of side effects * Generate md * Update date --------- Co-authored-by: Hadley Wickham <[email protected]> Co-authored-by: Jennifer (Jenny) Bryan <[email protected]>
1 parent 451035b commit e8b7cba

File tree

4 files changed

+489
-0
lines changed

4 files changed

+489
-0
lines changed

content/blog/withr-3-0-0/index.Rmd

Lines changed: 226 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,226 @@
1+
---
2+
output: hugodown::hugo_document
3+
4+
slug: withr-3-0-0
5+
title: withr 3.0.0
6+
date: 2024-01-18
7+
author: Lionel Henry
8+
description: >
9+
withr is the tidyverse solution for automatically cleaning
10+
up after yourselves (temporary files, options, etc). This milestone makes withr much faster.
11+
12+
photo:
13+
url: https://unsplash.com/photos/brown-and-black-brush-on-brown-wooden-table-V0cSTljC92k
14+
author: Neal E. Johnson
15+
16+
categories: [package]
17+
tags: [r-lib, withr]
18+
---
19+
20+
It's not without jubilant bearing that we announce the release of the 3.0.0 version of [withr](https://withr.r-lib.org/), the tidyverse solution for automatic cleanup of resources! In this release, the internals of withr were rewritten to improve the performance and increase the compatibility with base R's `on.exit()` mechanism.
21+
22+
You can install it from CRAN with:
23+
24+
```{r, eval = FALSE}
25+
install.packages("withr")
26+
```
27+
28+
In this blog post we'll go over the changes that made this rewrite possible, but first we'll review the cleanup strategies made possible by withr.
29+
30+
You can see a full list of changes in the [release notes](https://withr.r-lib.org/news/index.html#withr-300).
31+
32+
```{r setup, echo = FALSE}
33+
# To avoid distracting calls to `eval()` in `local()`
34+
stop <- rlang::abort
35+
```
36+
37+
38+
## Cleaning up resources with base R and with withr
39+
40+
Traditionally, resource cleanup in R is done with `base::on.exit()`. Cleaning up in the on-exit hook ensures that the cleanup happens both in the normal case, when the code has finished running without error, and in the error case, when something went wrong and execution is interrupted.
41+
42+
`on.exit()` is meant to be used inside functions but it also works within `local()`, which we'll use here for our examples:
43+
44+
```{r}
45+
local({
46+
on.exit(message("Cleaning time!"))
47+
print(1 + 2)
48+
})
49+
```
50+
51+
```{r, error = TRUE}
52+
local({
53+
on.exit(message("Cleaning time!"))
54+
stop("uh oh")
55+
print(1 + 2)
56+
})
57+
```
58+
59+
`on.exit()` is guaranteed to run no matter what and this property makes it invaluable for resource cleaning. No more accidental littering!
60+
61+
However the process of cleaning up this way can be a bit verbose and feel too manual. Here is how you'd create and clean up a temporary file for instance:
62+
63+
```{r}
64+
local({
65+
my_file <- tempfile()
66+
67+
file.create(my_file)
68+
on.exit(file.remove(my_file))
69+
70+
writeLines(c("a", "b"), con = my_file)
71+
})
72+
```
73+
74+
Wouldn't it be great if we could wrap this code up in a function? That's the goal of withr's `local_`-prefixed functions. They combine both the creation or modification of a resource and its (eventual) restoration to the original state into a single function:
75+
76+
```{r}
77+
local({
78+
my_file <- withr::local_tempfile()
79+
80+
writeLines(c("a", "b"), con = my_file)
81+
})
82+
```
83+
84+
In this case we have created a resource (a file), but the same principle applies to modifying resources such as global options:
85+
86+
```{r}
87+
local({
88+
# Let's temporarily print with a single decimal place
89+
withr::local_options(digits = 1)
90+
print(1/3)
91+
})
92+
93+
# The original option value has been restored
94+
getOption("digits")
95+
96+
print(1/3)
97+
```
98+
99+
And you can equivalently use the `with_`-prefixed variants (from which the package takes its name!), this way you don't need to wrap in `local()`:
100+
101+
```{r}
102+
withr::with_options(list(digits = 1), print(1/3))
103+
```
104+
105+
The `with_` functions are useful for creating very small scopes for given resources, inside or outside a function.
106+
107+
108+
## The withr 3.0.0 rewrite
109+
110+
Traditionally, withr implemented its own exit event system on top of `on.exit()`. We needed an extra layer because of a couple of missing features:
111+
112+
- When multiple resources are managed by a piece of code, the order in which these resources are restored or cleaned up sometimes matter. The most consistent order for cleanup is last-in first-out (LIFO). In other words the oldest resource, on which younger resources might depend, is cleaned up last. But historically R only supported first-in first-out (FIFO) order.
113+
114+
- The other missing piece was being able to inspect the contents of the exit hook. The `sys.on.exit()` R helper was created for this purpose but was affected by a bug that prevented it from working inside functions.
115+
116+
We contributed two changes to R 3.5.0 that filled these missing pieces, fixing the `sys.on.exit()` bug and adding an `after` argument to `on.exit()` to allow last-in first-out ordering.
117+
118+
Until now, we haven't been able to leverage these contributions because of our policy of [supporting the current and previous four versions of R](https://www.tidyverse.org/blog/2019/04/r-version-support). Now that enough time has passed, it was time for a rewrite! Our version of `base::on.exit()` is `withr::defer()`. Along with better default behaviour, `withr::defer()` allows the clean up of resources non-locally (ironically an essential feature for implementing `local_` functions). Given the changes in R 3.5.0, `withr::defer()` can now be implemented as a simple wrapper around `on.exit()`.
119+
120+
One benefit of the rewrite is that mixing withr tools and `on.exit()` in the same function now correctly interleaves cleanup:
121+
122+
```{r}
123+
local({
124+
on.exit(print(1))
125+
126+
withr::defer(print(2))
127+
128+
on.exit(print(3), add = TRUE, after = FALSE)
129+
130+
withr::defer(print(4))
131+
132+
print(5)
133+
})
134+
```
135+
136+
But the main benefit is increased performance. Here is how `defer()` compared to `on.exit()` in the previous version:
137+
138+
```{r, eval = FALSE}
139+
base <- function() on.exit(NULL)
140+
withr <- function() defer(NULL)
141+
142+
# withr 2.5.2
143+
bench::mark(base(), withr(), check = FALSE)[1:8]
144+
#> # A tibble: 2 × 8
145+
#> expression min median `itr/sec` mem_alloc `gc/sec` n_itr n_gc
146+
#> <bch:expr> <bch:tm> <bch:> <dbl> <bch:byt> <dbl> <int> <dbl>
147+
#> 1 base() 0 82ns 6954952. 0B 696. 9999 1
148+
#> 2 withr() 26.2µs 27.9µs 35172. 88.4KB 52.8 9985 15
149+
```
150+
151+
withr 3.0.0 has now caught up to `on.exit()` quite a bit:
152+
153+
```{r, eval = FALSE}
154+
# withr 3.0.0
155+
bench::mark(base(), withr(), check = FALSE)[1:8]
156+
#> # A tibble: 2 × 8
157+
#> expression min median `itr/sec` mem_alloc `gc/sec` n_itr n_gc
158+
#> <bch:expr> <bch:tm> <bch:> <dbl> <bch:byt> <dbl> <int> <dbl>
159+
#> 1 base() 0 82ns 7329829. 0B 0 10000 0
160+
#> 2 withr() 2.95µs 3.4µs 280858. 0B 225. 9992 8
161+
```
162+
163+
Of course `on.exit()` is still much faster, in part because `defer()` supports more features (more on that below), but mostly because `on.exit` is a primitive function whereas `defer()` is implemented as a normal R function. That said, we hope that we now have made `defer()` (and the `local_` and `with_` functions that use it) sufficiently fast to be used even in performance-critical micro-tools.
164+
165+
166+
## Improved withr features
167+
168+
Over the successive releases of withr we've improved the behaviour of cleanup expressions interactively, in scripts executed with `source()`, and in knitr. `on.exit()` is a bit inconsistent when it is used outside of a function:
169+
170+
- Interactively, it doesn't do anything.
171+
- In `source()` and in knitr, it runs immediately instead of a the end of the script
172+
173+
`withr::defer()` and the `withr::local_` helpers try to be more helpful for these cases.
174+
175+
Interactively, it saves the cleanup action in a special global hook and you get information about how to actually perform the cleanup:
176+
177+
```{r, eval = FALSE}
178+
file <- withr::local_tempfile()
179+
#> Setting global deferred event(s).
180+
#> i These will be run:
181+
#> * Automatically, when the R session ends.
182+
#> * On demand, if you call `withr::deferred_run()`.
183+
#> i Use `withr::deferred_clear()` to clear them without executing.
184+
185+
# Clean up now
186+
withr::deferred_run()
187+
#> Ran 1/1 deferred expressions
188+
```
189+
190+
In knitr or `source()`[^1], the cleanup is performed at the end of the document or of the script. If you need chunk-level cleanup, use `local()` as we've been doing in the examples of this blog post:
191+
192+
`````md
193+
194+
Cleaning up at the end of the document:
195+
196+
```r
197+
document_wide_file <- withr::local_tempfile()
198+
```
199+
200+
Cleaning up at the end of the chunk:
201+
202+
```r
203+
local({
204+
local_file <- withr::local_tempfile()
205+
})
206+
```
207+
`````
208+
209+
Starting from withr 3.0.0, you can also run `deferred_run()` inside of a chunk:
210+
211+
`````md
212+
```r
213+
withr::deferred_run()
214+
#> Ran 1/1 deferred expressions
215+
```
216+
`````
217+
218+
219+
[^1]: `source()` is only supported by default when running in the global environment, which is usually the case. For the special case of sourcing in a local environment, you need to set `options(withr.hook_source = TRUE)` first.
220+
221+
222+
## Acknowledgements
223+
224+
Thanks to the github contributors who helped us with this release!
225+
226+
[&#x0040;ashbythorpe](https://github.com/ashbythorpe), [&#x0040;bastistician](https://github.com/bastistician), [&#x0040;DavisVaughan](https://github.com/DavisVaughan), [&#x0040;fkohrt](https://github.com/fkohrt), [&#x0040;gaborcsardi](https://github.com/gaborcsardi), [&#x0040;gdurif](https://github.com/gdurif), [&#x0040;hadley](https://github.com/hadley), [&#x0040;HenrikBengtsson](https://github.com/HenrikBengtsson), [&#x0040;honghaoli42](https://github.com/honghaoli42), [&#x0040;IndrajeetPatil](https://github.com/IndrajeetPatil), [&#x0040;jameslairdsmith](https://github.com/jameslairdsmith), [&#x0040;jennybc](https://github.com/jennybc), [&#x0040;jonkeane](https://github.com/jonkeane), [&#x0040;krlmlr](https://github.com/krlmlr), [&#x0040;lionel-](https://github.com/lionel-), [&#x0040;maelle](https://github.com/maelle), [&#x0040;MichaelChirico](https://github.com/MichaelChirico), [&#x0040;MLopez-Ibanez](https://github.com/MLopez-Ibanez), [&#x0040;moodymudskipper](https://github.com/moodymudskipper), [&#x0040;multimeric](https://github.com/multimeric), [&#x0040;orichters](https://github.com/orichters), [&#x0040;pfuehrlich-pik](https://github.com/pfuehrlich-pik), [&#x0040;solmos](https://github.com/solmos), [&#x0040;tillea](https://github.com/tillea), and [&#x0040;vanhry](https://github.com/vanhry).

0 commit comments

Comments
 (0)