Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to avg slope reporting in modelsummary vignette #809

Open
grantmcdermott opened this issue Sep 5, 2024 · 2 comments
Open

Switch to avg slope reporting in modelsummary vignette #809

grantmcdermott opened this issue Sep 5, 2024 · 2 comments

Comments

@grantmcdermott
Copy link
Contributor

Minor issue, but I think the group contrast examples starting here are potentially confusing, since the tables are simply reporting the number of observations in each group rather than the slopes coefficients, which are almost certainly going to be of more interest for tabling output.

Maybe switch to avg_comparisons?

library(marginaleffects)
library(modelsummary)

dat <- mtcars
dat$gear <- as.factor(dat$gear)
mod <- glm(vs ~ gear + mpg, data = dat, family = binomial)

cmp <- avg_comparisons(mod)
modelsummary(cmp, shape = term + contrast ~ model) 
gear mean(4) - mean(3) 0.037
(0.137)
mean(5) - mean(3) -0.340
(0.099)
mpg mean(+1) 0.061
(0.013)
Num.Obs. 32
AIC 26.2
BIC 32.1
Log.Lik. -9.101
F 2.389
RMSE 0.31

Created on 2024-09-05 with reprex v2.1.1

@grantmcdermott grantmcdermott changed the title Unexpected output in modelsummary vignette Switch to avg slope reporting in modelsummary vignette Sep 5, 2024
@vincentarelbundock
Copy link
Owner

I would call these group contrasts "counterfactual comparisons." For example, mean(4)-mean(3) is computed by:

  1. Copying the entire 32 observations, and setting gear to 4
  2. Copying the entire 32 observations, and setting gear to 3
  3. Computing predictions in both sets
  4. Taking the average difference between those predictions

So each contrast actually uses all 32 observations, which makes me feel like the "Num.Obs." row makes sense.

If you want to compute differences between subgroup average predictions, I'd do something like:

avg_predictions(mod, by = "gear", hypothesis = ~ sequential)

@grantmcdermott
Copy link
Contributor Author

(Ugh, I just realised that I raised this in the wrong repo; should be in marginaleffects. Rushing too much this morning. Sorry!)

Cool, cool. Happy to close, but I will just say that reporting grouped avg_slopes/comparisons is one of the canonical use-cases of marginaleffects+modelsummaary IMO, since we are often interested in the modulating effect across groups. I think the docs do an excellent job demonstrating this elsewhere; it would be nice to see an example of it in the dedicated "Presentation tables" vignette. (I was motivated to raise this issue after helping a colleague that was stuck trying to do this.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants