Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upload Causal IV case study in education folder #219

Merged
merged 5 commits into from
Sep 26, 2023

Conversation

joonho112
Copy link
Contributor

Hello, I've uploaded a case study document titled "Instrumental Variables Analysis of Randomized Experiments with One-Sided Noncompliance." This study is a collaborative work with Profs. Avi Feller and Sophia Rabe-Hesketh from UC Berkeley. The document is a result of our work on the IES project, which you can review here: IES Project Link. I would greatly appreciate it if you could approve this merge request. Thank you.

@andrjohns
Copy link
Collaborator

Can you remove the two .DS_Store files and update the deprecated syntax in your Stan models?

@davkoh would you be able to review this one? You can see the rendered HTML at this link (it takes a second to load)

Copy link
Collaborator

@andrjohns andrjohns left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few suggestions for your Stan code

Comment on lines 49 to 51
target += log_sum_exp(
log(1 - pi_c) + bernoulli_lpmf(Y[n] | eta_n), // Never-taker
log(pi_c) + bernoulli_lpmf(Y[n] | eta_c0)); // Complier
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is more numerically stable and efficient as:

target += log_mix(pi_c, bernoulli_lpmf(Y[n] | eta_c0), bernoulli_lpmf(Y[n] | eta_n));

// Superpopulation complier average causal effect (CACE)
// in per-1000 units
real CACE;
CACE = (eta_c1 - eta_c0)*10^3;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't look like you use this CACE value in your model. If it's just a derived quantity you'll be using in post-processing/analysis then you should move this to the generated quantities block

@@ -0,0 +1,57 @@
data {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My comments from the other Stan model also apply to this one

Comment on lines 38 to 45
if(Z[n] == 1 && W[n] == 1){
target += log(pi_c) + bernoulli_lpmf(Y[n] | eta_c1) ;
}

// Never-taker (assigned to treatment)
else if(Z[n] == 1 && W[n] == 0){
target += log(1 - pi_c) + bernoulli_lpmf(Y[n] | eta_n);
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a bit more efficient to calculate log(pi_c) and log(1-pi_c) once and then reuse them:

model {
  real log_pi_c = log(pi_c);
  real log1m_pi_c = log1m(pi_c);
  ...
}

@joonho112
Copy link
Contributor Author

Hello @andrjohns, Thank you for your insights and guidance. I've updated the Stan code in accordance with your suggestions and removed the .DS_Store files. Please take a moment to review the modifications, and let me know if there are any additional adjustments needed prior to the publication of these files.

@andrjohns
Copy link
Collaborator

Hello @andrjohns, Thank you for your insights and guidance. I've updated the Stan code in accordance with your suggestions and removed the .DS_Store files. Please take a moment to review the modifications, and let me know if there are any additional adjustments needed prior to the publication of these files.

The CI (model compilation) is still failing because your Stan models are using the old array syntax which has been removed from Stan. See this section of the manual for guidance on how to update

@joonho112
Copy link
Contributor Author

I have updated all Stan models with the new array syntax.

Copy link
Collaborator

@andrjohns andrjohns left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks for the updates! Just a couple of very minor Stan changes remaining and then will leave to @davkoh for final sign-off

transformed parameters {
// Superpopulation complier average causal effect (CACE)
// in per-1000 units
real CACE = (eta_c1 - eta_c0) * 10^3;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move to generated quantities block

Comment on lines 21 to 22
real CACE = (eta_c1 - eta_c0) * 10^3;
real NACE = (eta_n1 - eta_n0) * 10^3;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move to generated quantities block

Copy link
Contributor

@bob-carpenter bob-carpenter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The .DS_Store files should not be checked in.

@joonho112
Copy link
Contributor Author

joonho112 commented Sep 23, 2023

I have made the necessary adjustments to the Stan code, transitioning the codes from the "transformed parameters" section to the "generated quantities" section. Additionally, I have removed all .DS_Store files from the repository.

Could you proceed to merge this Pull Request? I am looking forward to the final sign-off.

@bob-carpenter
Copy link
Contributor

Thanks. Looks good. That label from GitHub that says they're -6kb is confusing, because they appear to have been deleted when I look at your branch. I can merge.

@bob-carpenter bob-carpenter merged commit f73317a into stan-dev:master Sep 26, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants