Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure reproducible if using model cache #470

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Conversation

gowerc
Copy link
Collaborator

@gowerc gowerc commented Jan 8, 2025

Closes #469

FYI @luwidmer

@gowerc gowerc requested a review from gravesti January 8, 2025 16:50
@luwidmer
Copy link

luwidmer commented Jan 9, 2025

@gowerc doesn't .Random.seed always exist? At least I get

Restarting R session...

> exists(".Random.seed")
[1] TRUE
> ls()
character(0)
> .Random.seed <<- .Random.seed
> ls()
character(0)
> exists(".Random.seed")
[1] TRUE

@gravesti
Copy link
Collaborator

gravesti commented Jan 9, 2025

I think it doesn't necessarily exist. eg if I start a new R session from the terminal

gravesti$ R

R version 4.4.1 (2024-06-14) -- "Race for Your Life"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: aarch64-apple-darwin20
... snip ...

> .Random.seed
Error: object '.Random.seed' not found
> rnorm(1)
[1] -0.6881252
> .Random.seed
  [1]       10403           2 -1831660992  -462780898 -1349857047   996995611
  [7]   676991530   594638156 -1127753969  1257593413  1702644732  1213026898
 [13]  -333959219  -261150329 -1072259986   635315272  1119354443    93778857
 [19]   864664312   193269286  -542546111 -1416680237  1130410114   418352820
 [25]   -98325961  -999378643 -1509490108  1850580842   640379861   544927087

# Note that .Random.seed is only set if the seed has been set or if a random number
# has been generated.
if (exists(".Random.seed")) {
current_seed_state <- .Random.seed
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gowerc
To be a bit more robust, eg in case the function has an error, you can do something like

Suggested change
current_seed_state <- .Random.seed
current_seed_state <- .Random.seed
on.exit(suspendInterrupts(
assign(".Random.seed", value = current_seed_state, envir = globalenv(), inherits = FALSE)
))

In another package we have

    # Make sure to leave '.Random.seed' as-is on exit
    old_seed <- globalenv()$.Random.seed
    on.exit(suspendInterrupts(set_random_seed(old_seed)))

with the function

# Restore the RNG back to a previous state using the global .Random.seed
set_random_seed <- function(old_seed) {
  if (is.null(old_seed)) {
    rm(".Random.seed", envir = globalenv(), inherits = FALSE)
  } else {
    assign(".Random.seed", value = old_seed, envir = globalenv(), inherits = FALSE)
  }

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yer good idea will update !

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@luwidmer / @gravesti - Sorry if a stupid question but what exactly is suspendInterrupts supposed to be doing here ? My guess based on the sparse docs would be that it block SIG* interrupts however SIGTERM still seems to kill the program regardless:

image

I then wondered if it meant R based signals but it has no affect on that either:

> tryCatch(
+   {
+     suspendInterrupts({
+       warning("hi")
+       print("hello")
+     })
+   },
+   warning = \(e) print("Caught warning")
+ )
[1] "Caught warning"

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it just doesn't work properly on MacOS / Linux:
HenrikBengtsson/Wishlist-for-R#82

Comment on lines -583 to +606
rstan::stan_model(
model <- rstan::stan_model(
file = model_file,
auto_write = TRUE,
model_name = "rbmi_mmrm"
)

if (exists("current_seed_state")) {
.Random.seed <- current_seed_state
}
return(model)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These changes wouldn't be needed if you use on.exit()

@luwidmer
Copy link

luwidmer commented Jan 9, 2025

Yup, turns out I didn't have a completely new R session even after restarting! Thank you for testing @gowerc & @gravesti ! IMO the on.exit() solution (when .Random.seed exists) would be a pretty robust one.

@gowerc
Copy link
Collaborator Author

gowerc commented Jan 9, 2025

Yup, turns out I didn't have a completely new R session even after restarting! Thank you for testing @gowerc !

In fairness I had no idea it wasn't created on new R sessions until all the CICD tests failed unexpectedly 😓

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

rbmi 1.3.1: first fit after compiling model not reproducible with subsequent fits
3 participants