Skip to content

Commit

Permalink
quick typo review
Browse files Browse the repository at this point in the history
  • Loading branch information
rdf310ca committed Feb 19, 2024
1 parent 12a11cf commit bf09a47
Show file tree
Hide file tree
Showing 8 changed files with 17 additions and 16 deletions.
4 changes: 2 additions & 2 deletions vignettes/approx_fi.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ summary(gl_fits, central.tendency = c("mean","median","mode"), prob = .90)

Another group of fit indices compares the hypothesized model with the *worst* possible model, so they are called incremental indices. Such indices compare your model's $\chi^2_H$ to the *null* model's $\chi^2_0$ in different ways. Indices include the Comparative Fit Index (CFI), Tucker-Lewis Index (TLI), and Normed Fit Index (NFI).

To estimate these indices we need to defined and estimate the respective *null* model. The standard *null* model used by default in frequentist SEM programs (like *lavaan*) includes only the indicators variances and intercepts, and no covariances between items.
To estimate these indices we need to define and estimate the respective *null* model. The standard *null* model used by default in frequentist SEM programs (like *lavaan*) includes only the indicators variances and intercepts, and no covariances between items.

You can specify your *null* model by including only the respective indicator variances in your model syntax, such as

Expand Down Expand Up @@ -98,7 +98,7 @@ gl_fits_all <- blavFitIndices(fit, baseline.model = fit_null)
summary(gl_fits_all, central.tendency = c("mean","median","mode"), prob = .90)
```

The ```summary()``` method now presents the central tendicy measure you asked for, standard deviation, and credible interval for the noncentrality and incremental fit indices.
The ```summary()``` method now presents the central tendency measure you asked for, standard deviation, and credible interval for the noncentrality and incremental fit indices.

### Access the indices posterior distributions

Expand Down
4 changes: 2 additions & 2 deletions vignettes/convergence_efficiency.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ fit <- bsem(model, data=PoliticalDemocracy,

### Convergence

The primary convergence diagnostic is that $\hat{R}$, which compares the between- and within-chain samples of model parameters and other univariate quantities of interest [@new_rhat]. If chains have not mixed well (ie, the between- and within-chain estimates don't agree), $\hat{R}$ is larger than 1. We recommend running at least three chains by default and only using the posterior samples if $\hat{R} < 1.05$ for all the parameters.
The primary convergence diagnostic is $\hat{R}$, which compares the between- and within-chain samples of model parameters and other univariate quantities of interest [@new_rhat]. If chains have not mixed well (ie, the between- and within-chain estimates don't agree), $\hat{R}$ is larger than 1. We recommend running at least three chains by default and only using the posterior samples if $\hat{R} < 1.05$ for all the parameters.

```blavaan``` presents the $\hat{R}$ reported by the underlying MCMC program, either Stan or JAGS (Stan by default). We can obtain the $\hat{R}$ from the ```summary()``` function, and we can also extract it with the ```blavInspect()``` function

Expand Down Expand Up @@ -95,7 +95,7 @@ and/or change the model priors with the ```dpriors()``` function. These address

We should also evaluate the efficiency of the posterior samples. Effective sample size (ESS) is a useful measure for sampling efficiency, and is well defined even if the chains do not have finite mean or variance [@new_rhat].

In short, the posterior samples produced by MCMC are autocorrelated. This means that, if you draw 500 posterior samples, you do not have 500 independent pieces of information about the posterior distribution, because the samples are autocorlated. The ESS metric is <a href="https://www.johndcook.com/blog/2017/06/27/effective-sample-size-for-mcmc/">like a currency conversion,</a> telling you how much your autocorrelated samples are worth if we were to convert them to indepndent samples. In ```blavaan``` we can print it from the ```summary``` function with the ```neff``` argument
In short, the posterior samples produced by MCMC are autocorrelated. This means that, if you draw 500 posterior samples, you do not have 500 independent pieces of information about the posterior distribution, because the samples are autocorlated. The ESS metric is <a href="https://www.johndcook.com/blog/2017/06/27/effective-sample-size-for-mcmc/">like a currency conversion,</a> telling you how much your autocorrelated samples are worth if we were to convert them to independent samples. In ```blavaan``` we can print it from the ```summary``` function with the ```neff``` argument

```{r, eval=F}
summary(fit, neff=T)
Expand Down
6 changes: 3 additions & 3 deletions vignettes/convergence_loop.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,11 @@ In many cases you would need to run the BSEM models multiple times until it has

### Convergence loop

You will start by writing the model syntax as always. Then instead of running the ```blavaan``` functions as usual, we will run then inside the ```while``` loop as follows.
You will start by writing the model syntax as always. Then instead of running the ```blavaan``` functions as usual, we will run them inside the ```while``` loop as follows.

Before the loop starts you need to define a starting ```BURN <- 0``` number of iterations, and a convergence value higher than the desired such as ```rhat <- 20```.

Then the loop will be set sto stop when the convergence criteria (```rhat```) is lower than a desired value, like $\hat{R} < 1.05$, we specify this with ```while(rhat > 1.05)```, meaning the the loop will continue as long as rhat is higher than 1.05.
Then the loop will be set stop when the convergence criteria (```rhat```) is lower than a desired value, like $\hat{R} < 1.05$, we specify this with ```while(rhat > 1.05)```, meaning the the loop will continue as long as rhat is higher than 1.05.

```{r, eval=T, include=FALSE, cache=TRUE}
HS.model <- ' visual =~ x1 + x2 + x3
Expand Down Expand Up @@ -66,7 +66,7 @@ while(rhat > 1.05) {
```


Thn inside the loop we will increase the number of ```BURN``` iterations by 1000 in this example. And after estimating the model, we will evaluate the convergence by getting the highest estimated $\hat{R}$, and printing it in the screen so you will see how far is the model from converging.
Then inside the loop we will increase the number of ```BURN``` iterations by 1000 in this example. And after estimating the model, we will evaluate the convergence by getting the highest estimated $\hat{R}$, and printing it in the screen so you will see how far is the model from converging.

```{r, eval=T}
print(paste0("Rhat=",rhat))
Expand Down
2 changes: 1 addition & 1 deletion vignettes/estimate.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ library(blavaan, quietly=TRUE)
library(lavaan, quietly=TRUE)
```

Models are specified using *lavaan* syntax, and prior distribution specification can already been covered on the [previous page](prior.html). The specified model can then be estimated via the `bsem()` command, with other models being estimated via `bcfa()`, `bgrowth()`, or `blavaan()`. Regardless of the command, there are many arguments that allow you to tailor the model estimation to your needs. We discuss here some of the most popular arguments, as well as some easy-to-miss arguments.
Models are specified using *lavaan* syntax, and prior distribution specification has already been covered on the [previous page](prior.html). The specified model can then be estimated via the `bsem()` command, with other models being estimated via `bcfa()`, `bgrowth()`, or `blavaan()`. Regardless of the command, there are many arguments that allow you to tailor the model estimation to your needs. We discuss here some of the most popular arguments, as well as some easy-to-miss arguments.

### Primary arguments
Primary arguments to the model estimation commands include `burnin`, `sample`, `n.chains`, and `target`. The `burnin` and `sample` arguments are used to specify the desired number of burn-in iterations and posterior samples for each of `n.chains` chains (and the `burnin` argument controls the *warm-up* iterations in Stan). The `target` argument, on the other hand, is used to specify the MCMC strategy used for estimation. The default, `target = "stan"`, tends to be fastest and most efficient. Other options are slightly more flexible, including `target = "stanclassic"` and `target = "jags"`. Both of these approaches sample latent variables as if they are model parameters, whereas `target = "stan"` marginalizes out the latent variables. For more detail of these approaches, see the [JSS paper](https://www.jstatsoft.org/article/view/v100i06).
Expand Down
4 changes: 2 additions & 2 deletions vignettes/model_comparison.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ As both WAIC and LOO approximate the models' performance across posterior draws,

The model differences estimate the differences across the Expected Log-Pointwise-Predictive-Density (elpd), and the standard error of the respective difference.

There are no clear cutoff rules on how to interpret and present these comparisons, and the researchers need to use their expert knowledge as part of the decision process. The best recommendation is the present the differences in elpd $\Delta elpd$, the standard error, and the ratio between them. If the ratio is at least 2 can be consider evidence of differences between the models, and a ratio of 4 would be considered stronger evidence.
There are no clear cutoff rules on how to interpret and present these comparisons, and the researchers need to use their expert knowledge as part of the decision process. The best recommendation is to present the differences in elpd $\Delta elpd$, the standard error, and the ratio between them. If the ratio is at least 2 can be consider evidence of differences between the models, and a ratio of 4 would be considered stronger evidence.

For the first example, we will compare the standard political democracy model, with a model where all factor regressions are fixed to 0.

Expand Down Expand Up @@ -317,7 +317,7 @@ abs(bc14$diff_loo[,"elpd_diff"] / bc14$diff_loo[,"se_diff"])

### Bayes factor

In the Bayesian literature you will the the use of the Bayes factor (BF) to compare models. There are a number of criticisms related to the use of the BF in BSEM, including (1) the BF is unstable for large models (like most SEMs), (2) it is highly sensitive to model priors, (3) it requires strong priors to have stable estimation of it, (4) it can require large number of posterior draws, (5) the estimation using the marginal likelihood ignores a lot of information from the posterior distributions. For more details on this discussion please see @tendeiro_review_2019 and @schad_workflow_2022. These criticisms lead us to recommend against use of the BF in everyday BSEM estimation. For researchers who commit to their prior distributions and who commit to exploring the noise in their computations, the BF can used to describe the relative odds of one model over another, which is more intuitive than some other model comparison metrics.
In the Bayesian literature you will make use of the Bayes factor (BF) to compare models. There are a number of criticisms related to the use of the BF in BSEM, including (1) the BF is unstable for large models (like most SEMs), (2) it is highly sensitive to model priors, (3) it requires strong priors to have stable estimation of it, (4) it can require large number of posterior draws, (5) the estimation using the marginal likelihood ignores a lot of information from the posterior distributions. For more details on this discussion please see @tendeiro_review_2019 and @schad_workflow_2022. These criticisms lead us to recommend against use of the BF in everyday BSEM estimation. For researchers who commit to their prior distributions and who commit to exploring the noise in their computations, the BF can used to describe the relative odds of one model over another, which is more intuitive than some other model comparison metrics.


### Summary
Expand Down
9 changes: 5 additions & 4 deletions vignettes/prior_pred_checks.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ The prior predictive distribution is just like the posterior predictive distribu

This is easy to carry out mechanically by simulating parameters $θ^{sim}∼p(\theta)$ according to the priors, then simulating data $y^{sim}∼p(y∣ \theta^{sim})$ according to the sampling distribution given the simulated parameters. The result is a simulation from the joint distribution, $(y^{sim},θ^{sim})∼p(y,\theta)$ and thus $y^{sim}∼p(y)$ is a simulation from the prior predictive distribution.

In *blavaan* we can get PPC when we use the argument ```prisamp=TRUE``` , this will tell blavaan to ignore the data and buil distributions only from the priors. Here we will start by adjusting the priors, instead of using the default priors.
In *blavaan* we can get PPC when we use the argument ```prisamp=TRUE``` , this will tell blavaan to ignore the data and build distributions only from the priors. Here we will start by adjusting the priors, instead of using the default priors.

#### Weakly informative priors

Expand Down Expand Up @@ -77,11 +77,12 @@ You can also pick which parameters to plot, like the factor correlations by chos
plot(fit_wi, pars=19:21, plot.type = "dens")
```

From the factor loadings distributions we see that the first loading on each factor is bounded by 0, this is due to a modeling identification constraint in blavaan, and with a maximum values aroun 6. And for all other distributions they range between -6 to 6 or -4 to 4, and for all priors the most likely value is around 0. This would be described as weakly informative priors as it allows them to range between begative and positive values but without allowing crazy high/low values
From the factor loading distributions we see that the first loading on each factor is bounded by 0, this is due to a modeling identification constraint in blavaan, and with a maximum value around 6. And for all other distributions they range between -6 to 6 or -4 to 4, and for all priors the most likely value is around 0. This would be described as weakly informative as it allows negative and positive values but without allowing crazy high/low values.

Note that what would be a *realistic* range would dependen on the parameter, model specification, and data. So, consider these priors in function of those characterictics.
What would be a *realistic* range would depend on the parameter, model specification, and data. So, consider these priors in function of those characteristics.

For the factor correlations we kept the default diffuse priors, so these allowed very high and low correlation, but the prior distributions are not flat across all possible correlation values.

For the factor correlations we kept the deafult diffuse priors, so these allowed from very high and low correlation, but the prior distributions are not flat across all possible correlation values.

#### Default priors

Expand Down
2 changes: 1 addition & 1 deletion vignettes/probability_direction.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ To calculate the probability of direction we will use a function from the packag
library(brms)
```

Ans we will need to extract the posterior draws a matrix,
And we will need to extract the posterior draws as a matrix,

```{r, eval=T}
mc_out <- as.matrix(blavInspect(fit, "mcmc", add.labels = FALSE))
Expand Down
2 changes: 1 addition & 1 deletion vignettes/resources.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Other resources are listed below:

- A Stan case study on *blavaan*, by Feng Ji and colleagues: <a href="https://mc-stan.org/users/documentation/case-studies/sem.html">html</a>

- Many recorded presentations are available on the internets:
- Many recorded presentations are available on the internet:
- The longest talk with the most details: <a href="https://youtu.be/xJy5zIkzgvg">Presentation to the Oslo UseR Group,</a> including real-time coding examples near the end. (Code from those examples is <a href="https://ecmerkle.github.io/pub/merkle_oslo_script.R">here.</a>)

- Details about how the package works: <a href="https://www.youtube.com/watch?v=Mm6oP_Ss6qg">A virtual presentation</a> to the <a href="https://www.psychoco.org/2022/">2022 International Workshop on Psychometric Computing.</a>
Expand Down

0 comments on commit bf09a47

Please sign in to comment.