From 98d541d46c9d5c6252eb5022c2a03ed5bae31cc2 Mon Sep 17 00:00:00 2001 From: Rae Woong Park Date: Sun, 24 Nov 2019 19:17:14 +0900 Subject: [PATCH] meaning of coverage in the fig 18.9 I am not sure about the meaning of coverage in Y axis of the figure 18.9. Some dots are pale and some are dark. What is the difference? What is the meaning of small dots packed around dashed line; small dots scattered all the coverage; small dots packed around 0%-25%? --- MethodValidity.Rmd | 2 ++ 1 file changed, 2 insertions(+) diff --git a/MethodValidity.Rmd b/MethodValidity.Rmd index 19ab6ed..4ca9756 100644 --- a/MethodValidity.Rmd +++ b/MethodValidity.Rmd @@ -365,9 +365,11 @@ We have run all the methods in the OHDSI Methods Library through this benchmark, [^methodEvalViewerUrl]: http://data.ohdsi.org/MethodEvalViewer/ +===> ```{r methodEval, fig.cap='Coverage of the 95\\% confidence interval for the methods in the Methods Library. Each dot represents the performance of a specific set of analysis choices. The dashed line indicates nominal performance (95\\% coverage). SCCS = Self-Controlled Case Series, GI = Gastrointestinal, IBD = inflammatory bowel disease.',echo=FALSE, out.width='100%', fig.align='center', fig.pos='h'} knitr::include_graphics("images/MethodValidity/methodEval.png") ``` +<=== This emphasizes the need for empirical evaluation and calibration: if no empirical evaluation is performed, which is true for almost all published observational studies, we must assume a prior informed by the results in Figure \@ref(fig:methodEval), and conclude that it is likely that the true effect size is not contained in the 95% confidence interval!