One approach for full uncertainty propagation of general brms models

[lunafazio]

As you show in the tutorial, brms's predict functions aren't very easy to use for models with many levels of dependencies. It seems like you worked things out by hand for one particular example, so I thought you might be interested in an automatic way of producing predictive draws of general brms models. I implemented one approach for purposes of dataset generation in the SBC package, here.

Of course installing a whole unrelated package might be a bit too much just for this, but it may give you some ideas in case you want to make your own implementation (for instance there's the brms_response_sequence function which figures out the model's DAG from the formula).

[jebyrnes]

HAHAHA - oh no - I realized I linked to the wrong repo for issues! HAHAHAHAHAHA

Can you repost over here - https://github.com/jebyrnes/bayesian_sem/issues - and I'll fix the links

[jebyrnes]

Huh - does brms_response_sequence() go through and put the relationships in order? We have something for that in piecewiseSEM - getSortedPsem() https://github.com/jslefche/piecewiseSEM/blob/5f2246375f98b926c2d603ef655c49a59af6226b/R/helpers.R#L696 - which turns a model list (like a bf!) into a DAG and then sorts it. We might have gakked the code from ggm... https://github.com/jslefche/piecewiseSEM/blob/5f2246375f98b926c2d603ef655c49a59af6226b/R/getDAG.R#L53

But I think something like this could then automate fitting. I have a package called sinterval - which I should get back to one day - that uses this to propogate error through piecewiseSEM fits based off of some of what I did here. https://github.com/jebyrnes/sinterval/blob/main/R/piecewisesem_psem_predict_sims.R

But - I wonder if this would be easy enough to take what you've done and adapt it to the keeley example........ I'd 100% put it in the tutorial

[jebyrnes]

Hrm. Kinda works, but, difference in density........ load all libraries from tutorial and SBC and....

data(keeley)

rich_bf <- 
  bf(rich ~ age + firesev) +
  bf(firesev ~ age) +
  set_rescor(FALSE)

rich_brms <- brm(rich_bf, data = keeley,
                 file = "fit_models/rich_brms.rds")

# run code from 
# https://github.com/hyunjimoon/SBC/blob/c4f832cb54e44a088cc937f377aefa3177f7de42/R/generator-brms.R#L159
pred_sbc <- 
  brms_full_ppred(rich_brms, 
                  newdata = data.frame(age = 10, 
                                       rich = NA, 
                                       firesev = NA)) |>
  purrr::list_rbind()

new_age_dat_more <- data.frame(age = 10, firesev = NA)

rich_pred <- epred_draws(rich_brms, 
                               newdata = new_age_dat_more, 
                               ndraws = 100) |>
  filter(.category=="firesev") |>
  ungroup() |>
  select(age, .epred) |>
  rename(firesev = .epred) |>
  epred_draws(rich_brms, 
                newdata = _,
                ndraws = 100) |>
  ungroup() |>
  select(-firesev) |>
  pivot_wider(names_from = ".category", 
              values_from = ".epred")

combined <- bind_rows(
  pred_sbc |> mutate(type = "SBC"),
  rich_pred |> mutate(type = "by_hand")
)

ggplot(combined,
       aes(x = firesev, fill = type)) +
  geom_density(alpha = 0.5)


ggplot(combined,
       aes(x = rich, fill = type)) +
  geom_density(alpha = 0.5)