my_formula <- formula(Y ~ treatment1 + treatment2)
class(my_formula)2.0.1 Think About Your Analytical Goals
-Throughout this guide, we have tried to explicitly state the goals of each analysis. This helps informs how to approach the analysis of an experiment. It can be difficult, especially for new scientists-in-training (i.e. graduate students), to understand what it is they want to estimate. You may have been handed a data set you had no role in generating and told to “analyze this” with no additional context. Or perhaps you may have conducted a large study that has some overall goals that are lofty, yet vague.
+Throughout this guide, we have tried to explicitly state the goals of each analysis. This helps informs how to approach the analysis of an experiment. It can be difficult, especially for new scientists-in-training (i.e. graduate students), to understand what it is they want to estimate. You may have been handed a data set you had no role in generating and told to “analyze this” with no additional context. Or perhaps you may have conducted a large study that has some overall goals that are lofty, yet vague. And now you must translate the vague aims into clear statistical questions.
It can helpful to think about the exact results you are hoping to get. What does this look like exactly? Do you want to estimate the changes in plant diversity as the result of a herbicide spraying program? Do you want to find out if a fertilizer treatment changed protein content in a crop and by how much? Do you want to know about changes in human diet due to an intervention? What are quantifiable difference that you and/or experts in your domain would find meaningful?
-Consider what the results would look like for (1) the best case scenario when your wildest dreams come true, and (2) null results, when you find out that your treatment or invention had no effect. It’s very helpful to understand and recognize both situations.
-By “consider”, we mean: imagine the final plot or table, or summary sentence you want to present, either in a peer-reviewed manuscript, or some output for stakeholders. From this, you work backwards to determine the analytical approach needed to arrive at that final output. Or you may determine that your data are unsuitable to generate the desired output, in which case, it’s best to determine that as soon as possible.
-By “consider”, we also mean: imagine exactly what the spreadsheet of results would say - what columns are present and what data are in the cells. If you are planning an experiment, this can help ensure you plan it properly to actually test whatever it is you want to evaluate. If the experiment is done, this enables you to evaluate if you have the information present to test your hypothesis.
-By taking the time to reflect on what it is you exactly want to analyze, this can save time and prevent you from doing unneeded analyzes that don’t serve this final goal. There is rarely (never?) one way to analyze an experiment or a data set, so use your limited time wisely and focus on what matters.
+Consider what the results would look like for (1) the best case scenario where your wildest research dreams come true, and (2) null results, when you find out that your treatment or invention had no effect. It’s very helpful to understand and recognize exactly what both situations look like.
+By “consider”, we mean: imagine the final plot or table, or summary sentence you want to present, either in a peer-reviewed manuscript, or some output for stakeholders. From this, you can work backwards to determine the analytical approach needed to arrive at that desired final output. Or you may determine that your data are unsuitable to generate the desired output, in which case, it’s best to determine that as soon as possible.
+By “consider”, we also mean: imagine exactly what the spreadsheet of results would contain after a successful trial. What columns are present and what data are in those cells. If you are planning an experiment, this can help ensure you plan it properly to actually test whatever it is you want to evaluate. If the experiment is done, this enables you to evaluate if you have the information present to test your hypothesis.
+By taking the time to reflect on what it is you exactly want to analyze, this can save time and prevent you from doing unneeded analyzes that don’t serve this final goal. There is rarely (never?) one way to analyze an experiment or a data set, so use your limited time wisely and focus on what matters to you most.
2.0.2 Know That Data Cleaning is Time Consuming
@@ -346,7 +347,7 @@1 This will take way longer than the actual analysis! It is needed to ensure you can actually get correct results in an analysis, and hence data cleaning is worth the time it requires. +
This has and will continue to occupy the majority of researcher’s time when conducting an analysis. Truly, we are sorry for this. But, please know it is not you, it is the nature of data. Plan for and prepare yourself mentally to spend time cleaning and preparing your data for analysis.1 This will likely take way longer than the actual analysis! It is needed to ensure you can actually get correct results in an analysis, and hence data cleaning is worth the time it requires.
1 For an excellent set of basic instructions on data preparation, please see: Broman, K. W., & Woo, K. H. (2018). Data Organization in Spreadsheets. The American Statistician, 72(1), 2–10.
2.0.3 Interpret ANOVA and P-values with Caution
@@ -367,11 +368,15 @@
-Image from a paper published in 2024. Although this was a fully crossed factorial experiment, compact letter display was implemented across all treatment combinations, resulting in some nonsensical comparisons among some more informative contrasts.
+Image from a paper published in 2024. Although this was a fully crossed factorial experiment, compact letter display was implemented across all treatment combinations, resulting in some nonsensical comparisons among some more informative contrasts. What a waste.
Implementing compact letter display can kill statistical power (the probability of detecting true differences) because it requires that all pairwise comparison being made. Doing this, especially when there are many treatment levels, has its perils. The biggest problem is that this creates a multiple testing problem. The RCBD example in this guide has 42 treatments, resulting in a total of 861 comparisons (\(=42*(42-1)/2\)), that are then adjusted for multiple tests. With that many tests, a severe adjustment is likely and hence things that are different are not detected. With so many tests, it could be that there is an overall effect due to treatment, but they all share the same letter!
The second problem is one of interpretation. Just because two treatments or varieties share a letter does not mean they are equivalent. It only means that they were not found to be different. A funny distinction, but alas. There is an entire branch of statistics, ‘equivalence testing’ devoted to just this topic - how to test if two things are actually the same. This involves the user declaring a maximum allowable numeric difference for a variable in order to determine if two items are statistically different or equivalent - something that these pairwise comparisons are not doing.]
-Another problem is that doing all pairwise comparison may not align with experimental goals. In many circumstances, not every pairwise combination is of any interest or relevance to the study. Additionally, complex treatment structure may necessitate custom constrasts that highlight differences between the marginal estimate of multiple treatments versus another. For example, there may be 2 levels of ‘high’ nitrogen fertilizer treatment with two different sources (i.e. types of fertilizer). A researcher may want to contrast those two levels together against ‘low’ nitrogen treatment levels.
-Often, researchers have embedded additional structure in the treatments that is not fully reflected in the statistical model. For example, perhaps a study is looking at five different intercropping mixtures, two that incorporate a legume and 3 that do not. Conducting all pairwise comparisons with miss estimating the difference due to including a legume in an intercropping mix and not incorporating one. Soil fertility and other agronomic studies often have complex treatment structure. When it is not practical or financially feasible to have a full factorial experiment, embedding different treatment combinations in the main factor of analysis can accomplish this. This is a good study design approach, and we have statistical tools to analyze it.
+Another problem is that doing all pairwise comparison may not align with experimental goals. In many circumstances, not every pairwise combination is of any interest or relevance to the study. Additionally, complex treatment structure may necessitate custom contrasts that highlight differences between the marginal estimate of multiple treatments versus another. For example, there may be 2 levels of ‘high’ nitrogen fertilizer treatment with two different sources (i.e. types of fertilizer). A researcher may want to contrast those two levels together against ‘low’ nitrogen treatment levels.
+Often, researchers have embedded additional structure in the treatments that is not fully reflected in the statistical model. For example, perhaps a study is looking at five different intercropping mixtures, two that incorporate a legume and 3 that do not. Conducting all pairwise comparisons with miss estimating the difference due to including a legume in an intercropping mix and not incorporating one. Soil fertility and other agronomic studies often have complex treatment structure. When it is not practical or financially feasible to have a full factorial experiment, embedding different treatment combinations in the main factor of analysis can accomplish this. This is a good study design approach, but compact letter display is an efficient way to report results. In such cases, custom contrasts are a better choice for hypothesis testing.The emmeans chapter covers how to do this.
+
Image from a paper published in 2024. Although this was a fully crossed factorial experiment, compact letter display was implemented across all treatment combinations, resulting in some nonsensical comparisons among some more informative contrasts.
+Image from a paper published in 2024. Although this was a fully crossed factorial experiment, compact letter display was implemented across all treatment combinations, resulting in some nonsensical comparisons among some more informative contrasts. What a waste.
2.0.5 Final Thoughts
+Good statistical analysis requires a thoughtful, intentional approach. If you have gone to the trouble to conduct a well designed experiment or assemble a useful data set, take the time and effort to analyze it properly.
-This is reasonably good. Things do tend to fall apart at the tails.
+This is reasonably good. Things do tend to fall apart at the tails a little, so this is not concerning.
4.2.3.2 New Way
+4.2.3.2 New Way
Nowadays, we can take advantage of the performance package, which provides a comprehensive suite of diagnostic plots.
-
Please look for check_model() in help tab to find what other checks you can perform using this function. If you would like to check all assumptions you can use check = “all”.
Please look for check_model() in help tab to find what other checks you can perform using this function. If you would like to check all assumptions you can use the argument check = "all".
check_model(model_rcbd_lmer, check = c('normality', 'linearity'))
@@ -661,7 +662,7 @@ 4.2.4 Inference
4.2.4 Inference
-R syntax for esimating model marginal means is the same for lme4 and nlme.
+R syntax for estimating model marginal means is the same for lme4 and nlme.
Estimates for each treatment level can be obtained with the ‘emmeans’ package.
rcbd_emm <- emmeans(model_rcbd_lmer, ~ variety)
@@ -720,7 +721,7 @@ If you want to run ANOVA, it can be done quite easily. By default, the Kenward-Rogers method of degrees of freedom approximation is used.
-The Type I method is sometimes referred to as the “sequential” sum of squares, because it involves a process of adding terms to the model one at a time. Type I sum of squares is the default hypothesis testing method used by the anova() function.
+The Type I method is sometimes referred to as the “sequential” sum of squares, because it involves a process of adding terms to the model one at a time. Type I sum of squares is the default hypothesis testing method used by the anova() function. This only matters when a data set is unbalanced across treatments, either due to design or missing data points.
diff --git a/chapters/variance-components.html b/chapters/variance-components.html
index de607ce..274f226 100644
--- a/chapters/variance-components.html
+++ b/chapters/variance-components.html
@@ -310,7 +310,7 @@ 13
library(nlme); library(emmeans); library(performance)
library(lme4)
diff --git a/index.html b/index.html
index 28e628d..8afa198 100644
--- a/index.html
+++ b/index.html
@@ -292,13 +292,13 @@ Preface
“Path in the Wilderness” by Erich Taeubel, Jr.
Running mixed models in R is no easy task. There are dozens of packages supporting these aims, each with varying functionality, syntax, and conventions. The linear mixed model ecosystem in R consists of over 80 libraries that either construct and solve mixed model equations or helper packages the process the results from mixed model analysis. These libraries provide a patchwork of overlapping and unique functionality regarding the fundamental structure of mixed models: allowable distributions, nested and crossed random effects, heterogeneous error structures and other facets. No single library has all possible functionality enabled.
-This patchwork of packages makes it very challenging for statisticians to conduct mixed model analysis and to teach others how to run mixed models in R. The purpose of this guide to to provide some recipes for handling common analytical scenario’s that require mixed models. As a field guide, it is intended to be succinct, and help researchers meet their analytic goals.
+This patchwork of packages makes it very challenging for statisticians to conduct mixed model analysis and to teach others how to run mixed models in R. The purpose of this guide to to provide some recipes for handling common analytical scenario’s that require mixed models. As a field guide, it is intended to be succinct, and to help researchers meet their analytic goals.
In general, the content from this website may not be copied or reproduced without attribution. However, the example code and required data sets to run the code are MIT licensed. These can be accessed on GitHub.
What This Does Not Cover
-Generalized linear models. We do address cases of unequal variance, but if another distribution and/or a link function is required for the model, that is not addressed in this guide.
Basic principles of experimental design. We assume you know this, but if you do not, please check out the Grammar of Experimental Design.
Generalized linear models where the response variable does not follow a normal distribution. We do address cases of unequal variance, but if another distribution and/or a link function is required for the model, that is not addressed in this guide.
Basic principles of experimental design. We assume you know this, but if you do not, please check out the Grammar of Experimental Design for guidance on these topics.
Instructions in using R. We assume familiarity with R. If you need help in learning R, there are numerous guides, including our introductory R course.
diff --git a/search.json b/search.json
index baed886..20aa7aa 100644
--- a/search.json
+++ b/search.json
@@ -4,7 +4,7 @@
"href": "index.html",
"title": "Field Guide to the R Mixed Model Wilderness",
"section": "",
- "text": "Preface\n“Path in the Wilderness” by Erich Taeubel, Jr.\nRunning mixed models in R is no easy task. There are dozens of packages supporting these aims, each with varying functionality, syntax, and conventions. The linear mixed model ecosystem in R consists of over 80 libraries that either construct and solve mixed model equations or helper packages the process the results from mixed model analysis. These libraries provide a patchwork of overlapping and unique functionality regarding the fundamental structure of mixed models: allowable distributions, nested and crossed random effects, heterogeneous error structures and other facets. No single library has all possible functionality enabled.\nThis patchwork of packages makes it very challenging for statisticians to conduct mixed model analysis and to teach others how to run mixed models in R. The purpose of this guide to to provide some recipes for handling common analytical scenario’s that require mixed models. As a field guide, it is intended to be succinct, and help researchers meet their analytic goals.\nIn general, the content from this website may not be copied or reproduced without attribution. However, the example code and required data sets to run the code are MIT licensed. These can be accessed on GitHub.",
+ "text": "Preface\n“Path in the Wilderness” by Erich Taeubel, Jr.\nRunning mixed models in R is no easy task. There are dozens of packages supporting these aims, each with varying functionality, syntax, and conventions. The linear mixed model ecosystem in R consists of over 80 libraries that either construct and solve mixed model equations or helper packages the process the results from mixed model analysis. These libraries provide a patchwork of overlapping and unique functionality regarding the fundamental structure of mixed models: allowable distributions, nested and crossed random effects, heterogeneous error structures and other facets. No single library has all possible functionality enabled.\nThis patchwork of packages makes it very challenging for statisticians to conduct mixed model analysis and to teach others how to run mixed models in R. The purpose of this guide to to provide some recipes for handling common analytical scenario’s that require mixed models. As a field guide, it is intended to be succinct, and to help researchers meet their analytic goals.\nIn general, the content from this website may not be copied or reproduced without attribution. However, the example code and required data sets to run the code are MIT licensed. These can be accessed on GitHub.",
"crumbs": [
"Preface"
]
@@ -14,7 +14,7 @@
"href": "index.html#what-this-does-not-cover",
"title": "Field Guide to the R Mixed Model Wilderness",
"section": "What This Does Not Cover",
- "text": "What This Does Not Cover\n\nGeneralized linear models. We do address cases of unequal variance, but if another distribution and/or a link function is required for the model, that is not addressed in this guide.\nBasic principles of experimental design. We assume you know this, but if you do not, please check out the Grammar of Experimental Design.\nInstructions in using R. We assume familiarity with R. If you need help in learning R, there are numerous guides, including our introductory R course.",
+ "text": "What This Does Not Cover\n\nGeneralized linear models where the response variable does not follow a normal distribution. We do address cases of unequal variance, but if another distribution and/or a link function is required for the model, that is not addressed in this guide.\nBasic principles of experimental design. We assume you know this, but if you do not, please check out the Grammar of Experimental Design for guidance on these topics.\nInstructions in using R. We assume familiarity with R. If you need help in learning R, there are numerous guides, including our introductory R course.",
"crumbs": [
"Preface"
]
@@ -34,7 +34,7 @@
"href": "chapters/intro.html",
"title": "1 Introduction",
"section": "",
- "text": "1.1 Terms\nThis guide is focused on frequent implementations of mixed models in R, covering different scenarios common in the agricultural sciences.\nThis is not intended to be a guide to the theory of mixed models, it is focused on implementations of models only.\nPlease read this section and refer back to if when you forget what these terms mean.",
+ "text": "1.1 Terms\nThis guide is focused on frequentist implementations of mixed models in R, covering different scenarios common in the agricultural and life sciences.\nThis is not intended to be a guide to the theory of mixed models, it is focused on implementations of models only.\nPlease read this section and refer back to if when you forget what these terms mean.",
"crumbs": [
"1 Introduction"
]
@@ -64,7 +64,7 @@
"href": "chapters/analysis-tips.html",
"title": "2 Tips on Analysis",
"section": "",
- "text": "Below are some things our office frequently says to researchers.\n\n2.0.1 Think About Your Analytical Goals\nThroughout this guide, we have tried to explicitly state the goals of each analysis. This helps informs how to approach the analysis of an experiment. It can be difficult, especially for new scientists-in-training (i.e. graduate students), to understand what it is they want to estimate. You may have been handed a data set you had no role in generating and told to “analyze this” with no additional context. Or perhaps you may have conducted a large study that has some overall goals that are lofty, yet vague.\nIt can helpful to think about the exact results you are hoping to get. What does this look like exactly? Do you want to estimate the changes in plant diversity as the result of a herbicide spraying program? Do you want to find out if a fertilizer treatment changed protein content in a crop and by how much? Do you want to know about changes in human diet due to an intervention? What are quantifiable difference that you and/or experts in your domain would find meaningful?\nConsider what the results would look like for (1) the best case scenario when your wildest dreams come true, and (2) null results, when you find out that your treatment or invention had no effect. It’s very helpful to understand and recognize both situations.\nBy “consider”, we mean: imagine the final plot or table, or summary sentence you want to present, either in a peer-reviewed manuscript, or some output for stakeholders. From this, you work backwards to determine the analytical approach needed to arrive at that final output. Or you may determine that your data are unsuitable to generate the desired output, in which case, it’s best to determine that as soon as possible.\nBy “consider”, we also mean: imagine exactly what the spreadsheet of results would say - what columns are present and what data are in the cells. If you are planning an experiment, this can help ensure you plan it properly to actually test whatever it is you want to evaluate. If the experiment is done, this enables you to evaluate if you have the information present to test your hypothesis.\nBy taking the time to reflect on what it is you exactly want to analyze, this can save time and prevent you from doing unneeded analyzes that don’t serve this final goal. There is rarely (never?) one way to analyze an experiment or a data set, so use your limited time wisely and focus on what matters.\n\n\n2.0.2 Know That Data Cleaning is Time Consuming\n\n\n\n\n\n\n\n\n\nFigure 2.1: How you will spend your time\n\n\n\n\nThis has and will continue to occupy the majority of researcher’s time when conducting an analysis. Truly, we are sorry for this. But, please know it is not you, it is the nature of data. Please plan for and prepare yourself mentally to spend time cleaning and preparing your data for analysis.1 This will take way longer than the actual analysis! It is needed to ensure you can actually get correct results in an analysis, and hence data cleaning is worth the time it requires.\n1 For an excellent set of basic instructions on data preparation, please see: Broman, K. W., & Woo, K. H. (2018). Data Organization in Spreadsheets. The American Statistician, 72(1), 2–10.\n\n2.0.3 Interpret ANOVA and P-values with Caution\n\nInformally, a p-value is the probability under a specified statistical model that a statistical summary of the data (e.g., the sample mean difference between two compared groups) would be equal to or more extreme than its observed value.\n---American Statistical Association\n\nThe great majority of researched are deeply interested in p-values. This is not a bad thing per se, but sometimes the focus is so strong it comes at the expense of other valuable pieces of information, like treatment estimates! Russ Leanth, author of the emmeans package refers to this particular practice as “star gazing”.\nIt is important to evaluate why you want to do ANOVA, what extra information it will bring and what you plan to do with those results. Sometimes, researchers want to conduct an ANOVA even though the original goals of analysis were reached without it. Running an ANOVA may increase or decrease confidence in your other results. That is not at all what ANOVA is intended to do, nor is this what p-values can tell us. ANOVA compares across group variation to within group variation. It cannot tell us if anything is the ‘same’ (there’s a separate branch of analysis, ‘equivalence testing’, for that), and it cannot tell us specifically what is different, unless you are fortunate enough to only have 2 levels in your treatment structure. P-values provide no guarantee that something is truly different or not; it only quantifies the probability you could have observed these results by chance.\nThe American Statistics Association recommends that “Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.”2 That article also explains what p-values are telling us and how to avoid committing analytical errors and/or misinterpreting p-values. If you have time to read the full article, it will benefit your research!\n2 Wasserstein, R. L., & Lazar, N. A. (2016). The ASA Statement on p-Values: Context, Process, and Purpose. The American Statistician, 70(2), 129–133.The main problematic behavior I see is researchers using p-values as the sole criteria on whether to present results: “We wanted to test if x, y and z had an effect. We ran some model and found that that only x had a significant effect, and those results indicate…” (while results with a p-value > 0.05 are ignored).\nA better option would be to discuss the the results of the analysis and how they addressed the research questions: how did the dependent variable change (or not change) as a result of the treatments/interventions/independent variables? What are the parameters or treatment predictions and what do they tell us with regard to the research goals? And to bolster those estimates, what are the confidence intervals on those estimates? What are the p-values for the statistical tests? P-values can support the results and conclusions, but the main results desired by a researcher are usually the estimates themselves - so lead with that!\nTo learn more about common pitfalls in interpreting p-values, check out our blog post on the subject and/or this paper3 on the subject.\n3 Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, Altman DG. (2016) Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol. 31(4):337-50.\n\n2.0.4 Comments on Hypothesis Testing and Usage of Treatment Letters\nOften, I see researchers use compact letter display (e.g. “A”, “B”, “C”, ….) for indicating differences among treatments. This makes for concise presentation of results in tables and figures, but it can both kill statistical power and misses nuance in the results.\n\n\n\nImage from a paper published in 2024. Although this was a fully crossed factorial experiment, compact letter display was implemented across all treatment combinations, resulting in some nonsensical comparisons among some more informative contrasts.\nImplementing compact letter display can kill statistical power (the probability of detecting true differences) because it requires that all pairwise comparison being made. Doing this, especially when there are many treatment levels, has its perils. The biggest problem is that this creates a multiple testing problem. The RCBD example in this guide has 42 treatments, resulting in a total of 861 comparisons (\\(=42*(42-1)/2\\)), that are then adjusted for multiple tests. With that many tests, a severe adjustment is likely and hence things that are different are not detected. With so many tests, it could be that there is an overall effect due to treatment, but they all share the same letter!\nThe second problem is one of interpretation. Just because two treatments or varieties share a letter does not mean they are equivalent. It only means that they were not found to be different. A funny distinction, but alas. There is an entire branch of statistics, ‘equivalence testing’ devoted to just this topic - how to test if two things are actually the same. This involves the user declaring a maximum allowable numeric difference for a variable in order to determine if two items are statistically different or equivalent - something that these pairwise comparisons are not doing.]\nAnother problem is that doing all pairwise comparison may not align with experimental goals. In many circumstances, not every pairwise combination is of any interest or relevance to the study. Additionally, complex treatment structure may necessitate custom constrasts that highlight differences between the marginal estimate of multiple treatments versus another. For example, there may be 2 levels of ‘high’ nitrogen fertilizer treatment with two different sources (i.e. types of fertilizer). A researcher may want to contrast those two levels together against ‘low’ nitrogen treatment levels.\nOften, researchers have embedded additional structure in the treatments that is not fully reflected in the statistical model. For example, perhaps a study is looking at five different intercropping mixtures, two that incorporate a legume and 3 that do not. Conducting all pairwise comparisons with miss estimating the difference due to including a legume in an intercropping mix and not incorporating one. Soil fertility and other agronomic studies often have complex treatment structure. When it is not practical or financially feasible to have a full factorial experiment, embedding different treatment combinations in the main factor of analysis can accomplish this. This is a good study design approach, and we have statistical tools to analyze it.\n\n\n\n\nBroman, Karl W., and Kara H. Woo. 2018. “Data Organization in Spreadsheets.” The American Statistician 72 (1): 2–10. https://doi.org/10.1080/00031305.2017.1375989.\n\n\nGreenland, Sander, Stephen J. Senn, Kenneth J. Rothman, John B. Carlin, Charles Poole, Steven N. Goodman, and Douglas G. Altman. 2016. “Statistical Tests, P Values, Confidence Intervals, and Power: A Guide to Misinterpretations.” European Journal of Epidemiology 31 (4): 337–50. https://doi.org/10.1007/s10654-016-0149-3.\n\n\nWasserstein, Ronald L., and Nicole A. Lazar. 2016. “The ASA Statement on p-Values: Context, Process, and Purpose.” The American Statistician 70 (2): 129–33. https://doi.org/10.1080/00031305.2016.1154108.",
+ "text": "Below are some things our office frequently says to researchers.\n\n2.0.1 Think About Your Analytical Goals\nThroughout this guide, we have tried to explicitly state the goals of each analysis. This helps informs how to approach the analysis of an experiment. It can be difficult, especially for new scientists-in-training (i.e. graduate students), to understand what it is they want to estimate. You may have been handed a data set you had no role in generating and told to “analyze this” with no additional context. Or perhaps you may have conducted a large study that has some overall goals that are lofty, yet vague. And now you must translate the vague aims into clear statistical questions.\nIt can helpful to think about the exact results you are hoping to get. What does this look like exactly? Do you want to estimate the changes in plant diversity as the result of a herbicide spraying program? Do you want to find out if a fertilizer treatment changed protein content in a crop and by how much? Do you want to know about changes in human diet due to an intervention? What are quantifiable difference that you and/or experts in your domain would find meaningful?\nConsider what the results would look like for (1) the best case scenario where your wildest research dreams come true, and (2) null results, when you find out that your treatment or invention had no effect. It’s very helpful to understand and recognize exactly what both situations look like.\nBy “consider”, we mean: imagine the final plot or table, or summary sentence you want to present, either in a peer-reviewed manuscript, or some output for stakeholders. From this, you can work backwards to determine the analytical approach needed to arrive at that desired final output. Or you may determine that your data are unsuitable to generate the desired output, in which case, it’s best to determine that as soon as possible.\nBy “consider”, we also mean: imagine exactly what the spreadsheet of results would contain after a successful trial. What columns are present and what data are in those cells. If you are planning an experiment, this can help ensure you plan it properly to actually test whatever it is you want to evaluate. If the experiment is done, this enables you to evaluate if you have the information present to test your hypothesis.\nBy taking the time to reflect on what it is you exactly want to analyze, this can save time and prevent you from doing unneeded analyzes that don’t serve this final goal. There is rarely (never?) one way to analyze an experiment or a data set, so use your limited time wisely and focus on what matters to you most.\n\n\n2.0.2 Know That Data Cleaning is Time Consuming\n\n\n\n\n\n\n\n\n\nFigure 2.1: How you will spend your time\n\n\n\n\nThis has and will continue to occupy the majority of researcher’s time when conducting an analysis. Truly, we are sorry for this. But, please know it is not you, it is the nature of data. Plan for and prepare yourself mentally to spend time cleaning and preparing your data for analysis.1 This will likely take way longer than the actual analysis! It is needed to ensure you can actually get correct results in an analysis, and hence data cleaning is worth the time it requires.\n1 For an excellent set of basic instructions on data preparation, please see: Broman, K. W., & Woo, K. H. (2018). Data Organization in Spreadsheets. The American Statistician, 72(1), 2–10.\n\n2.0.3 Interpret ANOVA and P-values with Caution\n\nInformally, a p-value is the probability under a specified statistical model that a statistical summary of the data (e.g., the sample mean difference between two compared groups) would be equal to or more extreme than its observed value.\n---American Statistical Association\n\nThe great majority of researched are deeply interested in p-values. This is not a bad thing per se, but sometimes the focus is so strong it comes at the expense of other valuable pieces of information, like treatment estimates! Russ Leanth, author of the emmeans package refers to this particular practice as “star gazing”.\nIt is important to evaluate why you want to do ANOVA, what extra information it will bring and what you plan to do with those results. Sometimes, researchers want to conduct an ANOVA even though the original goals of analysis were reached without it. Running an ANOVA may increase or decrease confidence in your other results. That is not at all what ANOVA is intended to do, nor is this what p-values can tell us. ANOVA compares across group variation to within group variation. It cannot tell us if anything is the ‘same’ (there’s a separate branch of analysis, ‘equivalence testing’, for that), and it cannot tell us specifically what is different, unless you are fortunate enough to only have 2 levels in your treatment structure. P-values provide no guarantee that something is truly different or not; it only quantifies the probability you could have observed these results by chance.\nThe American Statistics Association recommends that “Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.”2 That article also explains what p-values are telling us and how to avoid committing analytical errors and/or misinterpreting p-values. If you have time to read the full article, it will benefit your research!\n2 Wasserstein, R. L., & Lazar, N. A. (2016). The ASA Statement on p-Values: Context, Process, and Purpose. The American Statistician, 70(2), 129–133.The main problematic behavior I see is researchers using p-values as the sole criteria on whether to present results: “We wanted to test if x, y and z had an effect. We ran some model and found that that only x had a significant effect, and those results indicate…” (while results with a p-value > 0.05 are ignored).\nA better option would be to discuss the the results of the analysis and how they addressed the research questions: how did the dependent variable change (or not change) as a result of the treatments/interventions/independent variables? What are the parameters or treatment predictions and what do they tell us with regard to the research goals? And to bolster those estimates, what are the confidence intervals on those estimates? What are the p-values for the statistical tests? P-values can support the results and conclusions, but the main results desired by a researcher are usually the estimates themselves - so lead with that!\nTo learn more about common pitfalls in interpreting p-values, check out our blog post on the subject and/or this paper3 on the subject.\n3 Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, Altman DG. (2016) Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol. 31(4):337-50.\n\n2.0.4 Comments on Hypothesis Testing and Usage of Treatment Letters\nOften, I see researchers use compact letter display (e.g. “A”, “B”, “C”, ….) for indicating differences among treatments. This makes for concise presentation of results in tables and figures, but it can both kill statistical power and misses nuance in the results.\n\n\n\nImage from a paper published in 2024. Although this was a fully crossed factorial experiment, compact letter display was implemented across all treatment combinations, resulting in some nonsensical comparisons among some more informative contrasts. What a waste.\nImplementing compact letter display can kill statistical power (the probability of detecting true differences) because it requires that all pairwise comparison being made. Doing this, especially when there are many treatment levels, has its perils. The biggest problem is that this creates a multiple testing problem. The RCBD example in this guide has 42 treatments, resulting in a total of 861 comparisons (\\(=42*(42-1)/2\\)), that are then adjusted for multiple tests. With that many tests, a severe adjustment is likely and hence things that are different are not detected. With so many tests, it could be that there is an overall effect due to treatment, but they all share the same letter!\nThe second problem is one of interpretation. Just because two treatments or varieties share a letter does not mean they are equivalent. It only means that they were not found to be different. A funny distinction, but alas. There is an entire branch of statistics, ‘equivalence testing’ devoted to just this topic - how to test if two things are actually the same. This involves the user declaring a maximum allowable numeric difference for a variable in order to determine if two items are statistically different or equivalent - something that these pairwise comparisons are not doing.]\nAnother problem is that doing all pairwise comparison may not align with experimental goals. In many circumstances, not every pairwise combination is of any interest or relevance to the study. Additionally, complex treatment structure may necessitate custom contrasts that highlight differences between the marginal estimate of multiple treatments versus another. For example, there may be 2 levels of ‘high’ nitrogen fertilizer treatment with two different sources (i.e. types of fertilizer). A researcher may want to contrast those two levels together against ‘low’ nitrogen treatment levels.\nOften, researchers have embedded additional structure in the treatments that is not fully reflected in the statistical model. For example, perhaps a study is looking at five different intercropping mixtures, two that incorporate a legume and 3 that do not. Conducting all pairwise comparisons with miss estimating the difference due to including a legume in an intercropping mix and not incorporating one. Soil fertility and other agronomic studies often have complex treatment structure. When it is not practical or financially feasible to have a full factorial experiment, embedding different treatment combinations in the main factor of analysis can accomplish this. This is a good study design approach, but compact letter display is an efficient way to report results. In such cases, custom contrasts are a better choice for hypothesis testing.The emmeans chapter covers how to do this.\n\n\n2.0.5 Final Thoughts\nGood statistical analysis requires a thoughtful, intentional approach. If you have gone to the trouble to conduct a well designed experiment or assemble a useful data set, take the time and effort to analyze it properly.\n\n\n\n\nBroman, Karl W., and Kara H. Woo. 2018. “Data Organization in Spreadsheets.” The American Statistician 72 (1): 2–10. https://doi.org/10.1080/00031305.2017.1375989.\n\n\nGreenland, Sander, Stephen J. Senn, Kenneth J. Rothman, John B. Carlin, Charles Poole, Steven N. Goodman, and Douglas G. Altman. 2016. “Statistical Tests, P Values, Confidence Intervals, and Power: A Guide to Misinterpretations.” European Journal of Epidemiology 31 (4): 337–50. https://doi.org/10.1007/s10654-016-0149-3.\n\n\nWasserstein, Ronald L., and Nicole A. Lazar. 2016. “The ASA Statement on p-Values: Context, Process, and Purpose.” The American Statistician 70 (2): 129–33. https://doi.org/10.1080/00031305.2016.1154108.",
"crumbs": [
"2 Tao of Analysis"
]
@@ -74,7 +74,7 @@
"href": "chapters/background.html",
"title": "3 Mixed model theory and background",
"section": "",
- "text": "3.1 Model\nMixed-effects models are called “mixed” because they simultaneously model fixed and random effects. Fixed effects (e.g. treatments) represent population-level (i.e., average) effects that should persist across experiments. Fixed effects are similar to the parameters found in “traditional” regression techniques like ordinary least squares. Random effects are discrete units sampled from some population (e.g. plots, participants), and thus they are inherently categorical.\nRecall simple linear regression with intercept (\\(\\beta_0\\)) and slope (\\(\\beta_1\\)) effect for subject i. The slope and intercept are chosen in a way so that the residual sum of squares is minimized.\n\\[ Y = \\beta_0 + \\beta_1 X + \\epsilon \\]\nIf we consider this model in a mixed model framework, \\(\\beta_0\\) and \\(\\beta_0\\) are considered fixed effects (also known as the population-averaged values) and \\(b_i\\) is a random effect for subject i. The random effect can be thought of as each subject’s deviation from the fixed intercept parameter. The key assumption about \\(b_i\\) is that it is independent, identically and normally distributed with a mean of zero and associated variance. Random effects are especially useful when we have (1) lots of levels (e.g., many species or blocks), (2) relatively little data on each level (although we need multiple samples from most of the levels), and (3) uneven sampling across levels.\nFor example, if we let the intercept be a random effect, it takes the form:\n\\[ Y = \\beta_0 + b_i + \\beta_1 X + \\epsilon \\]\nIn this model, predictions would vary depending on each subject’s random intercept term, but slopes would be the same.\nIn second case, we can have a fixed intercept and a random slope. The model will be:\n\\[ Y = \\beta_0 + (\\beta_1 + b_i)(X) + \\epsilon\\]\nIn this model, the bi is a random effect for subject i applied to the slope. Predictions would vary with random slope term, but the intercept will be the same:\nThird case would be the mixed model with random slope and intercept:\n\\[ Y = (\\beta_0 + a_i) + (\\beta_1 + b_i)(X) + \\epsilon\\]\nIn this model, \\(a_i\\) and \\(b_i\\) are random effects for subject i applied to the intercept and slope, respectively. Predictions would vary depending on each subject’s slope and intercept terms:",
+ "text": "3.1 Model\nMixed-effects models are called “mixed” because they simultaneously model fixed and random effects. Fixed effects (e.g. treatments) represent population-level (average) effects that should persist across experiments. Fixed effects are similar to the parameters found in “traditional” regression techniques like ordinary least squares. Random effects are discrete units sampled from some population (e.g. plots, participants), and thus they are inherently categorical.\nRecall simple linear regression with intercept (\\(\\beta_0\\)) and slope (\\(\\beta_1\\)) effect for subject \\(i\\). The slope and intercept are chosen in a way so that the residual sum of squares is minimized.\n\\[ Y = \\beta_0 + \\beta_1 X + \\epsilon \\]\nIf we consider this model in a mixed model framework, \\(\\beta_0\\) and \\(\\beta_0\\) are considered fixed effects (also known as the population-averaged values) and \\(b_i\\) is a random effect for subject i. The random effect can be thought of as each subject’s deviation from the fixed intercept parameter. The key assumption about \\(b_i\\) is that it is independent, identically and normally distributed with a mean of zero and associated variance. Random effects are especially useful when we have (1) lots of levels (e.g., many species or blocks), (2) relatively little data on each level (although we need multiple samples from most of the levels), and (3) uneven sampling across levels.\nFor example, if we let the intercept be a random effect, it takes the form:\n\\[ Y = \\beta_0 + b_i + \\beta_1 X + \\epsilon \\]\nIn this model, predictions would vary depending on each subject’s random intercept term, but slopes would be the same.\nIn second case, we can have a fixed intercept and a random slope. The model will be:\n\\[ Y = \\beta_0 + (\\beta_1 + b_i)(X) + \\epsilon\\]\nIn this model, the \\(\\beta_i\\) is a random effect for subject \\(i\\). Predictions would vary with random slope term, but the intercept will be the same:\nThird case would be the mixed model with random slope and intercept:\n\\[ Y = (\\beta_0 + a_i) + (\\beta_1 + b_i)(X) + \\epsilon\\]\nIn this model, \\(a_i\\) and \\(b_i\\) are random effects for subject \\(i\\) applied to the intercept and slope, respectively. Predictions would vary depending on each subject’s slope and intercept terms:",
"crumbs": [
"3 Mixed Model Background"
]
@@ -94,7 +94,7 @@
"href": "chapters/background.html#formula-notation",
"title": "3 Mixed model theory and background",
"section": "3.2 R Formula Syntax for Random and Fixed Effects",
- "text": "3.2 R Formula Syntax for Random and Fixed Effects\nFormula notation is often used in the R syntax for linear models. It looks like this: \\(Y ~ X\\), where Y is the dependent variable (the response) and X is/are the independent variable(s) (e.g. the experimental treatments).\n\nmy_formula <- formula(Y ~ treatment1 + treatment2)\nclass(my_formula)\n\n[1] \"formula\"\n\n\nThe package ‘lme4’ has some additional conventions regarding the formula. Random effects are put in parentheses and a 1| is used to denote random intercepts (rather than random slopes). The table below provides several examples of random effects in mixed models. The names of grouping factors are denoted g, g1, and g2, and covariate as x.\n\n\n\n\n\n\n\n\nFormula\nAlternative\nMeaning\n\n\n\n\n(1 | g)\n1 + (1 | g)\nRandom intercept with a fixed mean\n\n\n(1 | g1/g2)\n(1 | g1) + (1 | g1:g2)\nIntercept varying among g1 and g2 within g1.\n\n\n(1 | g1) + (1 | g2)\n1 + (1 | g1) + (1| g2)\nIntercept varying among g1 and g2.\n\n\nx + (x | g)\n1 + x + (1 + x | g)\nCorrelated random intercept and slope\n\n\nx + (x || g)\n1 + x + (1 | g) + (0 + x | g)\nUncorrelated random intercept and slope.",
+ "text": "3.2 R Formula Syntax for Random and Fixed Effects\nFormula notation is often used in the R syntax for linear models. It looks like this: \\(Y ~ X\\), where \\(Y\\) is the dependent variable (the response) and \\(X\\) is/are the independent variable(s) that is, the experimental treatments or interventions.\n\nmy_formula <- formula(Y ~ treatment1 + treatment2)\nclass(my_formula)\n\n[1] \"formula\"\n\n\nThe package ‘lme4’ has some additional conventions regarding the formula. Random effects are put in parentheses and a 1| is used to denote random intercepts (rather than random slopes). The table below provides several examples of random effects in mixed models. The names of grouping factors are denoted g, g1, and g2, and covariates as x.\n\n\n\n\n\n\n\n\nFormula\nAlternative\nMeaning\n\n\n\n\n(1|g)\n1 + (1|g)\nRandom intercept with a fixed mean\n\n\n(1|g1/g2)\n(1| 1) + (1|g1:g2)\nIntercept varying among g1 and g2 within g1\n\n\n(1|g1) + (1|g2)\n1 + (1|g1) + (1|g2)\nIntercept varying among g1 and g2\n\n\nx + (x|g)\n1 + x + (1 + x|g)\nCorrelated random intercept and slope\n\n\nx + (x||g)\n1 + x + (1|g) + (0 + x|g)\nUncorrelated random intercept and slope\n\n\n\nThe first example, (1|g) suffices for most models and is the only structure used in this guide.",
"crumbs": [
"3 Mixed Model Background"
]
@@ -104,7 +104,7 @@
"href": "chapters/rcbd.html",
"title": "4 Randomized Complete Block Design",
"section": "",
- "text": "4.1 Background\nThis is a simple model that can serve as a good entrance point to mixed models.\nIt is very common design where experimental treatments are applied at random to experimental units within each block. The blocks are intended to control for a nuisance source of variation, such as over time, spatial variance, changes in equipment or operators, or myriad other causes.\nThe statistical model:\n\\[y_{ij} = \\mu + \\alpha_i + \\beta_j + \\epsilon_{ij}\\] Where:\n\\(\\mu\\) = overall experimental mean \\(\\alpha\\) = treatment effects (fixed) \\(\\beta\\) = block effects (random) \\(\\epsilon\\) = error terms\n\\[ \\epsilon \\sim N(0, \\sigma)\\]\n\\[ \\beta \\sim N(0, \\sigma_b)\\]\nBoth the overall error and the block effects are assumed to be normally distributed with a mean of zero and standard deviations of \\(\\sigma\\) and \\(sigma_B\\), respectively.",
+ "text": "4.1 Background\nThis is a simple model that can serve as a good entrance point to mixed models.\nRandomized complete block design (RCBD) is very common design where experimental treatments are applied at random to experimental units within each block. The block can represent a spatial or temporal unit or even different technicians taking data. The blocks are intended to control for a nuisance source of variation, such as over time, spatial variance, changes in equipment or operators, or myriad other causes. They are a random effect where the actual blocks used in the study are a random sample of a distribution of other blocks.\nThe statistical model:\n\\[y_{ij} = \\mu + \\alpha_i + \\beta_j + \\epsilon_{ij}\\] Where:\n\\(\\mu\\) = overall experimental mean \\(\\alpha\\) = treatment effects (fixed) \\(\\beta\\) = block effects (random) \\(\\epsilon\\) = error terms\n\\[ \\epsilon \\sim N(0, \\sigma)\\]\n\\[ \\beta \\sim N(0, \\sigma_b)\\]\nBoth the overall error and the block effects are assumed to be normally distributed with a mean of zero and standard deviations of \\(\\sigma\\) and \\(sigma_B\\), respectively.",
"crumbs": [
"Experiment designs",
"4 Randomized Complete Block Design"
@@ -126,7 +126,7 @@
"href": "chapters/rcbd.html#example-analysis",
"title": "4 Randomized Complete Block Design",
"section": "4.2 Example Analysis",
- "text": "4.2 Example Analysis\nFirst, load the libraries for analysis and estimation:\n\nlme4nlme\n\n\n\nlibrary(lme4); library(lmerTest); library(emmeans)\nlibrary(dplyr); library(performance)\n\n\n\n\nlibrary(nlme); library(performance); library(emmeans)\nlibrary(dplyr)\n\n\n\n\nNext, let’s load some data. It is located here if you want to download it yourself (recommended).\nThis data set is for a single wheat variety trial conducted in Aberdeen, Idaho in 2015. The trial includes 4 blocks and 42 different treatments (wheat varieties in this case). This experiment consists of a series of plots (the experimental unit) laid out in a rectangular grid in a farm field. The goal of this analysis is the estimate the yield and test weight of each variety and the determine the rankings of each variety with regard to yield.\n\nvar_trial <- read.csv(here::here(\"data\", \"aberdeen2015.csv\"))\n\n\nTable of variables in the data set\n\n\n\n\n\n\nblock\nblocking unit\n\n\nrange\ncolumn position for each plot\n\n\nrow\nrow position for each plot\n\n\nvariety\ncrop variety (the treatment) being evaluated\n\n\nstand_pct\npercentage of the plot with actual plants growing in them\n\n\ndays_to_heading_julian\nJulian days (starting January 1st) until plot “headed” (first spike emerged)\n\n\nlodging\npercentage of plants in the plot that fell down and hence could not be harvested\n\n\nyield_bu_a\nyield (bushels per acre)\n\n\n\nThere are several variables present that are not useful for this analysis. The only thing we are concerned about is block, variety, yield_bu_a, and test_weight.\n\n4.2.1 Data integrity checks\nThe first thing is to make sure the data is what we expect. There are two steps:\n\nmake sure data are the expected data type\ncheck the extent of missing data\ninspect the independent variables and make sure the expected levels are present in the data\ninspect the dependent variable to ensure its distribution is following expectations\n\n\nstr(var_trial)\n\n'data.frame': 168 obs. of 10 variables:\n $ block : int 4 4 4 4 4 4 4 4 4 4 ...\n $ range : int 1 1 1 1 1 1 1 1 1 1 ...\n $ row : int 1 2 3 4 5 6 7 8 9 10 ...\n $ variety : chr \"DAS004\" \"Kaseberg\" \"Bruneau\" \"OR2090473\" ...\n $ stand_pct : int 100 98 96 100 98 100 100 100 99 100 ...\n $ days_to_heading_julian: int 149 146 149 146 146 151 145 145 146 146 ...\n $ height : int 39 35 33 31 33 44 30 36 36 29 ...\n $ lodging : int 0 0 0 0 0 0 0 0 0 0 ...\n $ yield_bu_a : num 128 130 119 115 141 ...\n $ test_weight : num 56.4 55 55.3 54.1 54.1 56.4 54.7 57.5 56.1 53.8 ...\n\n\nThese look okay except for block, which is currently coded as integer (numeric). We don’t want run a regression of block, where block 1 has twice the effect of block 2, and so on. So, converting it to a character will fix that. It can also be converted to a factor, but I find character easier to work with, and ultimately, equivalent to factor conversion\n\nvar_trial$block <- as.character(var_trial$block)\n\nNext, check the independent variables. Running a cross tabulations is often sufficient to ascertain this.\n\ntable(var_trial$variety, var_trial$block)\n\n \n 1 2 3 4\n 06-03303B 1 1 1 1\n Bobtail 1 1 1 1\n Brundage 1 1 1 1\n Bruneau 1 1 1 1\n DAS003 1 1 1 1\n DAS004 1 1 1 1\n Eltan 1 1 1 1\n IDN-01-10704A 1 1 1 1\n IDN-02-29001A 1 1 1 1\n IDO1004 1 1 1 1\n IDO1005 1 1 1 1\n Jasper 1 1 1 1\n Kaseberg 1 1 1 1\n LCS Artdeco 1 1 1 1\n LCS Biancor 1 1 1 1\n LCS Drive 1 1 1 1\n LOR-833 1 1 1 1\n LOR-913 1 1 1 1\n LOR-978 1 1 1 1\n Madsen 1 1 1 1\n Madsen / Eltan (50/50) 1 1 1 1\n Mary 1 1 1 1\n Norwest Duet 1 1 1 1\n Norwest Tandem 1 1 1 1\n OR2080637 1 1 1 1\n OR2080641 1 1 1 1\n OR2090473 1 1 1 1\n OR2100940 1 1 1 1\n Rosalyn 1 1 1 1\n Stephens 1 1 1 1\n SY Ovation 1 1 1 1\n SY 107 1 1 1 1\n SY Assure 1 1 1 1\n UI Castle CLP 1 1 1 1\n UI Magic CLP 1 1 1 1\n UI Palouse 1 1 1 1\n UI Sparrow 1 1 1 1\n UI-WSU Huffman 1 1 1 1\n WB 456 1 1 1 1\n WB 528 1 1 1 1\n WB1376 CLP 1 1 1 1\n WB1529 1 1 1 1\n\n\nThere are 42 varieties and there appears to be no misspellings among them that might confuse R into thinking varieties are different when they are actually the same. R is sensitive to case and white space, which can make it easy to create near duplicate treatments, such as “eltan” and “Eltan” and “Eltan”. There is no evidence of that in this data set. Additionally, it is perfectly balanced, with exactly one observation per treatment per rep. Please note that this does not tell us anything about the extent of missing data.\n\n\n\n\n\n\nMissing Data\n\n\n\nHere is a quick check to count the number of missing data in each column. This is not neededfor the data sets in this tutorial that have already been comprehensively examined, but it is helpful to check that the level of missingness displayed in an R session is what you expect.\n\napply(var_trial, 2, function(x) sum(is.na(x)))\n\n block range row \n 0 0 0 \n variety stand_pct days_to_heading_julian \n 0 0 0 \n height lodging yield_bu_a \n 0 0 0 \n test_weight \n 0 \n\n\nAlas, no missing data!\n\n\nIf there were independent variables with a continuous distribution (a covariate), I would plot those data.\nLast, check the dependent variable. A histogram is often quite sufficient to accomplish this. This is designed to be a quick check, so no need to spend time making the plot look good.\n\n\n\n\n\n\n\n\n\nFigure 4.1: Histogram of the dependent variable.\n\n\n\n\n\nhist(var_trial$yield_bu_a, main = \"\", xlab = \"yield\")\n\nThe range is roughly falling into the range we expect. I know this from talking with the person who generated the data, not through my own intuition. I do not see any large spikes of points at a single value (indicating something odd), nor do I see any extreme values (low or high) that might indicate some larger problems.\nData are not expected to be normally distributed at this point, so don’t bother running any Shapiro-Wilk tests. This histogram is a check to ensure the the data are entered correctly and they appear valid. It requires a mixture of domain knowledge and statistical training to know this, but over time, if you look at these plots with regularity, you will gain a feel for what your data should look like at this stage.\nThese are not complicated checks. They are designed to be done quickly and should be done for every analysis if you not previously already inspected the data as thus. We do this before every analysis and often discover surprising things! Best to discover these things early, since they are likely to impact the final analysis.\nThis data set is ready for analysis!\n\n\n4.2.2 Model Building\n\n\nRecall the model:\n\\[y_{ij} = \\mu + \\alpha_i + \\beta_j + \\epsilon_{ij}\\]\nFor this model, \\(\\alpha_i\\) is the variety effect (fixed) and \\(\\beta_j\\) is the block effect (random).\nHere is the R syntax for the RCBD statistical model:\n\nlme4nlme\n\n\n\nmodel_rcbd_lmer <- lmer(yield_bu_a ~ variety + (1|block),\n data = var_trial, \n na.action = na.exclude)\n\n\n\n\nmodel_rcbd_lme <- lme(yield_bu_a ~ variety,\n random = ~ 1|block,\n data = var_trial, \n na.action = na.exclude)\n\n\n\n\nThe parentheses are used to indicate that ‘block’ is a random effect, and this particular notation (1|block) indicates that a ‘random intercept’ model is being fit. This is the most common approach. It means there is one overall effect fit for each block. I use the argument na.action = na.exclude as instruction for how to handle missing data: conduct the analysis, adjusting as needed for the missing data, and when prediction or residuals are output, please pad them in the appropriate places for missing data so they can be easily merged into the main data set if need be.\n\n\n4.2.3 Check Model Assumptions\n\n\nR syntax for checking model assumptions is the same for lme4 and nlme.\nRemember those iid assumptions? Let’s make sure we actually met them.\n\n4.2.3.1 Old Way\nThere are special plotting function written for lme4 and nlme objects (ie.plot(lmer_object)) for checking the homoscedasticity (constant variance).\n\n\n\n\n\n\n\n\n\nFigure 4.2: Plot of residuals versus fitted values\n\n\n\n\n\nplot(model_rcbd_lmer, resid(., scaled=TRUE) ~ fitted(.), \n xlab = \"fitted values\", ylab = \"studentized residuals\")\n\nWe are looking for a random and uniform distribution of points. This looks good!\nChecking normality requiring first extracting the model residuals with resid() and then generating a qq-plot and line.\n\n\n\n\n\n\n\n\n\nFigure 4.3: QQ-plot of residuals\n\n\n\n\n\nqqnorm(resid(model_rcbd_lmer), main = NULL); qqline(resid(model_rcbd_lmer))\n\nThis is reasonably good. Things do tend to fall apart at the tails.\n\n\n4.2.3.2 New Way\nNowadays, we can take advantage of the performance package, which provides a comprehensive suite of diagnostic plots.\n\n\nPlease look for check_model() in help tab to find what other checks you can perform using this function. If you would like to check all assumptions you can use check = “all”.\n\ncheck_model(model_rcbd_lmer, check = c('normality', 'linearity'))\n\n\n\n\n\n\n\n\n\n\n\n4.2.4 Inference\n\n\nR syntax for esimating model marginal means is the same for lme4 and nlme.\nEstimates for each treatment level can be obtained with the ‘emmeans’ package.\n\nrcbd_emm <- emmeans(model_rcbd_lmer, ~ variety)\nas.data.frame(rcbd_emm) %>% arrange(desc(emmean))\n\n variety emmean SE df lower.CL upper.CL\n Rosalyn 155.2703 7.212203 77.85 140.91149 169.6292\n IDO1005 153.5919 7.212203 77.85 139.23310 167.9508\n OR2080641 152.6942 7.212203 77.85 138.33536 167.0530\n Bobtail 151.6403 7.212203 77.85 137.28149 165.9992\n UI Sparrow 151.6013 7.212203 77.85 137.24245 165.9601\n Kaseberg 150.9768 7.212203 77.85 136.61794 165.3356\n IDN-01-10704A 148.9861 7.212203 77.85 134.62729 163.3450\n 06-03303B 148.8300 7.212203 77.85 134.47116 163.1888\n WB1529 148.2445 7.212203 77.85 133.88568 162.6034\n DAS003 145.2000 7.212203 77.85 130.84116 159.5588\n IDN-02-29001A 144.5755 7.212203 77.85 130.21665 158.9343\n Bruneau 143.9900 7.212203 77.85 129.63116 158.3488\n SY 107 143.6387 7.212203 77.85 129.27987 157.9975\n WB 528 142.9752 7.212203 77.85 128.61633 157.3340\n OR2080637 141.7652 7.212203 77.85 127.40633 156.1240\n Jasper 141.2968 7.212203 77.85 126.93794 155.6556\n UI Magic CLP 139.5403 7.212203 77.85 125.18149 153.8992\n Madsen 139.2671 7.212203 77.85 124.90826 153.6259\n LCS Biancor 139.1110 7.212203 77.85 124.75213 153.4698\n SY Ovation 138.6426 7.212203 77.85 124.28375 153.0014\n OR2090473 137.8229 7.212203 77.85 123.46407 152.1817\n Madsen / Eltan (50/50) 136.9642 7.212203 77.85 122.60536 151.3230\n UI-WSU Huffman 135.4810 7.212203 77.85 121.12213 149.8398\n Mary 134.8564 7.212203 77.85 120.49762 149.2153\n Norwest Tandem 134.3490 7.212203 77.85 119.99020 148.7079\n Brundage 134.0758 7.212203 77.85 119.71697 148.4346\n IDO1004 132.5145 7.212203 77.85 118.15568 146.8733\n DAS004 132.2413 7.212203 77.85 117.88245 146.6001\n Norwest Duet 132.0852 7.212203 77.85 117.72633 146.4440\n Eltan 131.4606 7.212203 77.85 117.10181 145.8195\n LCS Artdeco 130.8361 7.212203 77.85 116.47729 145.1950\n UI Palouse 130.4848 7.212203 77.85 116.12600 144.8437\n LOR-978 130.4458 7.212203 77.85 116.08697 144.8046\n LCS Drive 128.7674 7.212203 77.85 114.40858 143.1262\n Stephens 127.1671 7.212203 77.85 112.80826 141.5259\n OR2100940 126.1523 7.212203 77.85 111.79342 140.5111\n UI Castle CLP 125.5277 7.212203 77.85 111.16891 139.8866\n WB1376 CLP 123.6932 7.212203 77.85 109.33439 138.0521\n LOR-833 122.7565 7.212203 77.85 108.39762 137.1153\n LOR-913 118.7752 7.212203 77.85 104.41633 133.1340\n WB 456 118.4629 7.212203 77.85 104.10407 132.8217\n SY Assure 111.0468 7.212203 77.85 96.68794 125.4056\n\nDegrees-of-freedom method: kenward-roger \nConfidence level used: 0.95 \n\n\nThis table indicates the estimated marginal means (“emmeans”, sometimes called “least squares means”), the standard error (“SE”) of those means, the degrees of freedom and the upper and lower bounds of the 95% confidence interval. As an additional step, the emmeans were sorted from largest to smallest.\nAt this point, the analysis goals have been met: we know the estimated means for each treatment and their rankings.\nIf you want to run ANOVA, it can be done quite easily. By default, the Kenward-Rogers method of degrees of freedom approximation is used.\n\n\nThe Type I method is sometimes referred to as the “sequential” sum of squares, because it involves a process of adding terms to the model one at a time. Type I sum of squares is the default hypothesis testing method used by the anova() function.\n\nlme4nlme\n\n\n\nanova(model_rcbd_lmer, type = \"1\")\n\nType I Analysis of Variance Table with Satterthwaite's method\n Sum Sq Mean Sq NumDF DenDF F value Pr(>F) \nvariety 18354 447.65 41 123 2.4528 8.017e-05 ***\n---\nSignif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n\n\n\n\n\nanova(model_rcbd_lme, type = \"sequential\")\n\n numDF denDF F-value p-value\n(Intercept) 1 123 2514.1283 <.0001\nvariety 41 123 2.4528 1e-04\n\n\n\n\n\n\n\n\n\n\n\nna.action = na.exclude\n\n\n\nYou may have noticed the final argument for na.action in the model statement:\nmodel_rcbd_lmer <- lmer(yield_bu_a ~ variety + (1|block),\n data = var_trial, \n na.action = na.exclude)\nThe argument na.action = na.exclude provides instructions for how to handle missing data. na.exclude removes the missing data points before proceeding with the analysis. When any obervation-levels model outputs is generated (e.g. predictions, residuals), they are padded in the appropriate place to account for missing data. This is handy because it makes it easier to add those results to the original data set if so desired.\nSince there are no missing data, this step was not strictly necessary, but it’s a good habit to be in.",
+ "text": "4.2 Example Analysis\nFirst, load the libraries for analysis and estimation:\n\nlme4nlme\n\n\n\nlibrary(lme4); library(lmerTest); library(emmeans)\nlibrary(dplyr); library(performance)\n\n\n\n\nlibrary(nlme); library(performance); library(emmeans)\nlibrary(dplyr)\n\n\n\n\nNext, let’s load some data. It is located here if you want to download it yourself (recommended).\nThis data set is for a single wheat variety trial conducted in Aberdeen, Idaho in 2015. The trial includes 4 blocks and 42 different treatments (wheat varieties in this case). This experiment consists of a series of plots (the experimental unit) laid out in a rectangular grid in a farm field. The goal of this analysis is the estimate the yield of each variety and the determine the rankings of each variety for the variable.\n\nvar_trial <- read.csv(here::here(\"data\", \"aberdeen2015.csv\"))\n\n\nTable of variables in the data set\n\n\n\n\n\n\nblock\nblocking unit\n\n\nrange\ncolumn position for each plot\n\n\nrow\nrow position for each plot\n\n\nvariety\ncrop variety (the treatment) being evaluated\n\n\nstand_pct\npercentage of the plot with actual plants growing in them\n\n\ndays_to_heading_julian\nJulian days (starting January 1st) until plot “headed” (first spike emerged)\n\n\nlodging\npercentage of plants in the plot that fell down and hence could not be harvested\n\n\nyield_bu_a\nyield (bushels per acre)\n\n\n\nThere are several variables present that are not useful for this analysis. The only thing we are concerned about is block, variety, yield_bu_a, and test_weight.\n\n4.2.1 Data integrity checks\nThe first thing is to make sure the data is what we expect. There are two steps:\n\nmake sure data are the expected data type\ncheck the extent of missing data\ninspect the independent variables and make sure the expected levels are present in the data\ninspect the dependent variable to ensure its distribution is following expectations\n\n\nstr(var_trial)\n\n'data.frame': 168 obs. of 10 variables:\n $ block : int 4 4 4 4 4 4 4 4 4 4 ...\n $ range : int 1 1 1 1 1 1 1 1 1 1 ...\n $ row : int 1 2 3 4 5 6 7 8 9 10 ...\n $ variety : chr \"DAS004\" \"Kaseberg\" \"Bruneau\" \"OR2090473\" ...\n $ stand_pct : int 100 98 96 100 98 100 100 100 99 100 ...\n $ days_to_heading_julian: int 149 146 149 146 146 151 145 145 146 146 ...\n $ height : int 39 35 33 31 33 44 30 36 36 29 ...\n $ lodging : int 0 0 0 0 0 0 0 0 0 0 ...\n $ yield_bu_a : num 128 130 119 115 141 ...\n $ test_weight : num 56.4 55 55.3 54.1 54.1 56.4 54.7 57.5 56.1 53.8 ...\n\n\nThese look okay except for block, which is currently coded as integer (numeric). We don’t want run a regression of block, where block 1 has twice the effect of block 2, and so on. So, converting it to a character will fix that. It can also be converted to a factor, but character variables are a bit easier to work with, and ultimately, equivalent to factor conversion\n\nvar_trial$block <- as.character(var_trial$block)\n\nNext, check the independent variables. Running a cross tabulations is often sufficient to ascertain this.\n\ntable(var_trial$variety, var_trial$block)\n\n \n 1 2 3 4\n 06-03303B 1 1 1 1\n Bobtail 1 1 1 1\n Brundage 1 1 1 1\n Bruneau 1 1 1 1\n DAS003 1 1 1 1\n DAS004 1 1 1 1\n Eltan 1 1 1 1\n IDN-01-10704A 1 1 1 1\n IDN-02-29001A 1 1 1 1\n IDO1004 1 1 1 1\n IDO1005 1 1 1 1\n Jasper 1 1 1 1\n Kaseberg 1 1 1 1\n LCS Artdeco 1 1 1 1\n LCS Biancor 1 1 1 1\n LCS Drive 1 1 1 1\n LOR-833 1 1 1 1\n LOR-913 1 1 1 1\n LOR-978 1 1 1 1\n Madsen 1 1 1 1\n Madsen / Eltan (50/50) 1 1 1 1\n Mary 1 1 1 1\n Norwest Duet 1 1 1 1\n Norwest Tandem 1 1 1 1\n OR2080637 1 1 1 1\n OR2080641 1 1 1 1\n OR2090473 1 1 1 1\n OR2100940 1 1 1 1\n Rosalyn 1 1 1 1\n Stephens 1 1 1 1\n SY Ovation 1 1 1 1\n SY 107 1 1 1 1\n SY Assure 1 1 1 1\n UI Castle CLP 1 1 1 1\n UI Magic CLP 1 1 1 1\n UI Palouse 1 1 1 1\n UI Sparrow 1 1 1 1\n UI-WSU Huffman 1 1 1 1\n WB 456 1 1 1 1\n WB 528 1 1 1 1\n WB1376 CLP 1 1 1 1\n WB1529 1 1 1 1\n\n\nThere are 42 varieties and there appears to be no mis-spellings among them that might confuse R into thinking varieties are different when they are actually the same. R is sensitive to case and white space, which can make it easy to create near duplicate treatments, such as “eltan” and “Eltan” and “Eltan”. There is no evidence of that in this data set. Additionally, it is perfectly balanced, with exactly one observation per treatment per rep. Please note that this does not tell us anything about the extent of missing data.\n\n\n\n\n\n\nMissing Data\n\n\n\nHere is a quick check to count the number of missing data in each column. This is not neededfor the data sets in this tutorial that have already been comprehensively examined, but it is helpful to check that the level of missingness displayed in an R session is what you expect.\n\napply(var_trial, 2, function(x) sum(is.na(x)))\n\n block range row \n 0 0 0 \n variety stand_pct days_to_heading_julian \n 0 0 0 \n height lodging yield_bu_a \n 0 0 0 \n test_weight \n 0 \n\n\nAlas, no missing data!\n\n\nIf there were independent variables with a continuous distribution (a covariate), plot those data.\nLast, check the dependent variable. A histogram is often quite sufficient to accomplish this. This is designed to be a quick check, so no need to spend time making the plot look good.\n\n\n\n\n\n\n\n\n\nFigure 4.1: Histogram of the dependent variable.\n\n\n\n\n\nhist(var_trial$yield_bu_a, main = \"\", xlab = \"yield\")\n\nThe range is roughly falling into the range we expect. We (the authors) know this from talking with the person who generated the data, not through our own intuition. There are mp large spikes of points at a single value (indicating something odd), nor are there any extreme values (low or high) that might indicate problems.\nData are not expected to be normally distributed at this point, so don’t bother running any Shapiro-Wilk tests. This histogram is a check to ensure the the data are entered correctly and they appear valid. It requires a mixture of domain knowledge and statistical training to know this, but over time, if you look at these plots with regularity, you will gain a feel for what your data should look like at this stage.\nThese are not complicated checks. They are designed to be done quickly and should be done for every analysis if you not previously already inspected the data as thus. We do this before every analysis and often discover surprising things! Best to discover these things early, since they are likely to impact the final analysis.\nThis data set is ready for analysis!\n\n\n4.2.2 Model Building\n\n\nRecall the model:\n\\[y_{ij} = \\mu + \\alpha_i + \\beta_j + \\epsilon_{ij}\\]\nFor this model, \\(\\alpha_i\\) is the variety effect (fixed) and \\(\\beta_j\\) is the block effect (random).\nHere is the R syntax for the RCBD statistical model:\n\nlme4nlme\n\n\n\nmodel_rcbd_lmer <- lmer(yield_bu_a ~ variety + (1|block),\n data = var_trial, \n na.action = na.exclude)\n\n\n\n\nmodel_rcbd_lme <- lme(yield_bu_a ~ variety,\n random = ~ 1|block,\n data = var_trial, \n na.action = na.exclude)\n\n\n\n\nThe parentheses are used to indicate that ‘block’ is a random effect, and this particular notation (1|block) indicates that a ‘random intercept’ model is being fit. This is the most common approach. It means there is one overall effect fit for each block.\nWe use the argument na.action = na.exclude as instruction for how to handle missing data: conduct the analysis, adjusting as needed for the missing data, and when prediction or residuals are output, please pad them in the appropriate places for missing data so they can be easily merged into the main data set if need be.\n\n\n4.2.3 Check Model Assumptions\n\n\nR syntax for checking model assumptions is the same for lme4 and nlme.\nRemember those iid assumptions? Let’s make sure we actually met them.\n\n4.2.3.1 Old Way\nThere are special plotting function written for lme4 and nlme objects (ie.plot(lmer_object)) for checking the homoscedasticity (constant variance).\n\n\n\n\n\n\n\n\n\nFigure 4.2: Plot of residuals versus fitted values\n\n\n\n\n\nplot(model_rcbd_lmer, resid(., scaled=TRUE) ~ fitted(.), \n xlab = \"fitted values\", ylab = \"studentized residuals\")\n\nWe are looking for a random and uniform distribution of points. This looks good!\nChecking normality requiring first extracting the model residuals with resid() and then generating a qq-plot and line.\n\n\n\n\n\n\n\n\n\nFigure 4.3: QQ-plot of residuals\n\n\n\n\n\nqqnorm(resid(model_rcbd_lmer), main = NULL); qqline(resid(model_rcbd_lmer))\n\nThis is reasonably good. Things do tend to fall apart at the tails a little, so this is not concerning.\n\n\n4.2.3.2 New Way\nNowadays, we can take advantage of the performance package, which provides a comprehensive suite of diagnostic plots.\n\n\nPlease look for check_model() in help tab to find what other checks you can perform using this function. If you would like to check all assumptions you can use the argument check = \"all\".\n\ncheck_model(model_rcbd_lmer, check = c('normality', 'linearity'))\n\n\n\n\n\n\n\n\n\n\n\n4.2.4 Inference\n\n\nR syntax for estimating model marginal means is the same for lme4 and nlme.\nEstimates for each treatment level can be obtained with the ‘emmeans’ package.\n\nrcbd_emm <- emmeans(model_rcbd_lmer, ~ variety)\nas.data.frame(rcbd_emm) %>% arrange(desc(emmean))\n\n variety emmean SE df lower.CL upper.CL\n Rosalyn 155.2703 7.212203 77.85 140.91149 169.6292\n IDO1005 153.5919 7.212203 77.85 139.23310 167.9508\n OR2080641 152.6942 7.212203 77.85 138.33536 167.0530\n Bobtail 151.6403 7.212203 77.85 137.28149 165.9992\n UI Sparrow 151.6013 7.212203 77.85 137.24245 165.9601\n Kaseberg 150.9768 7.212203 77.85 136.61794 165.3356\n IDN-01-10704A 148.9861 7.212203 77.85 134.62729 163.3450\n 06-03303B 148.8300 7.212203 77.85 134.47116 163.1888\n WB1529 148.2445 7.212203 77.85 133.88568 162.6034\n DAS003 145.2000 7.212203 77.85 130.84116 159.5588\n IDN-02-29001A 144.5755 7.212203 77.85 130.21665 158.9343\n Bruneau 143.9900 7.212203 77.85 129.63116 158.3488\n SY 107 143.6387 7.212203 77.85 129.27987 157.9975\n WB 528 142.9752 7.212203 77.85 128.61633 157.3340\n OR2080637 141.7652 7.212203 77.85 127.40633 156.1240\n Jasper 141.2968 7.212203 77.85 126.93794 155.6556\n UI Magic CLP 139.5403 7.212203 77.85 125.18149 153.8992\n Madsen 139.2671 7.212203 77.85 124.90826 153.6259\n LCS Biancor 139.1110 7.212203 77.85 124.75213 153.4698\n SY Ovation 138.6426 7.212203 77.85 124.28375 153.0014\n OR2090473 137.8229 7.212203 77.85 123.46407 152.1817\n Madsen / Eltan (50/50) 136.9642 7.212203 77.85 122.60536 151.3230\n UI-WSU Huffman 135.4810 7.212203 77.85 121.12213 149.8398\n Mary 134.8564 7.212203 77.85 120.49762 149.2153\n Norwest Tandem 134.3490 7.212203 77.85 119.99020 148.7079\n Brundage 134.0758 7.212203 77.85 119.71697 148.4346\n IDO1004 132.5145 7.212203 77.85 118.15568 146.8733\n DAS004 132.2413 7.212203 77.85 117.88245 146.6001\n Norwest Duet 132.0852 7.212203 77.85 117.72633 146.4440\n Eltan 131.4606 7.212203 77.85 117.10181 145.8195\n LCS Artdeco 130.8361 7.212203 77.85 116.47729 145.1950\n UI Palouse 130.4848 7.212203 77.85 116.12600 144.8437\n LOR-978 130.4458 7.212203 77.85 116.08697 144.8046\n LCS Drive 128.7674 7.212203 77.85 114.40858 143.1262\n Stephens 127.1671 7.212203 77.85 112.80826 141.5259\n OR2100940 126.1523 7.212203 77.85 111.79342 140.5111\n UI Castle CLP 125.5277 7.212203 77.85 111.16891 139.8866\n WB1376 CLP 123.6932 7.212203 77.85 109.33439 138.0521\n LOR-833 122.7565 7.212203 77.85 108.39762 137.1153\n LOR-913 118.7752 7.212203 77.85 104.41633 133.1340\n WB 456 118.4629 7.212203 77.85 104.10407 132.8217\n SY Assure 111.0468 7.212203 77.85 96.68794 125.4056\n\nDegrees-of-freedom method: kenward-roger \nConfidence level used: 0.95 \n\n\nThis table indicates the estimated marginal means (“emmeans”, sometimes called “least squares means”), the standard error (“SE”) of those means, the degrees of freedom and the upper and lower bounds of the 95% confidence interval. As an additional step, the emmeans were sorted from largest to smallest.\nAt this point, the analysis goals have been met: we know the estimated means for each treatment and their rankings.\nIf you want to run ANOVA, it can be done quite easily. By default, the Kenward-Rogers method of degrees of freedom approximation is used.\n\n\nThe Type I method is sometimes referred to as the “sequential” sum of squares, because it involves a process of adding terms to the model one at a time. Type I sum of squares is the default hypothesis testing method used by the anova() function. This only matters when a data set is unbalanced across treatments, either due to design or missing data points.\n\nlme4nlme\n\n\n\nanova(model_rcbd_lmer, type = \"1\")\n\nType I Analysis of Variance Table with Satterthwaite's method\n Sum Sq Mean Sq NumDF DenDF F value Pr(>F) \nvariety 18354 447.65 41 123 2.4528 8.017e-05 ***\n---\nSignif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n\n\n\n\n\nanova(model_rcbd_lme, type = \"sequential\")\n\n numDF denDF F-value p-value\n(Intercept) 1 123 2514.1283 <.0001\nvariety 41 123 2.4528 1e-04\n\n\n\n\n\n\n\n\n\n\n\nna.action = na.exclude\n\n\n\nYou may have noticed the final argument for na.action in the model statement:\nmodel_rcbd_lmer <- lmer(yield_bu_a ~ variety + (1|block),\n data = var_trial, \n na.action = na.exclude)\nThe argument na.action = na.exclude provides instructions for how to handle missing data. na.exclude removes the missing data points before proceeding with the analysis. When any obervation-levels model outputs is generated (e.g. predictions, residuals), they are padded in the appropriate place to account for missing data. This is handy because it makes it easier to add those results to the original data set if so desired.\nSince there are no missing data, this step was not strictly necessary, but it’s a good habit to be in.",
"crumbs": [
"Experiment designs",
"4 Randomized Complete Block Design"
@@ -454,7 +454,7 @@
"href": "chapters/variance-components.html",
"title": "13 Variance & Variance Components",
"section": "",
- "text": "13.1 Unequal Variance\nMixed models provide the advantage of being able to estimate the variance of random variables. The decision of how to assign",
+ "text": "13.1 Unequal Variance\nMixed models provide the advantage of being able to estimate the variance of random variables. Instead of looking at a variable as a collection of specific levels to estimate, random effects view variables as being a random drawn from a normal distribution with a standard deviation. The decision of how to designate a variable as random or fixed depends on",
"crumbs": [
"13 Variance and Variance Components"
]
-
R syntax for esimating model marginal means is the same for lme4 and nlme.
+R syntax for estimating model marginal means is the same for lme4 and nlme.
rcbd_emm <- emmeans(model_rcbd_lmer, ~ variety)
@@ -720,7 +721,7 @@ If you want to run ANOVA, it can be done quite easily. By default, the Kenward-Rogers method of degrees of freedom approximation is used.
-The Type I method is sometimes referred to as the “sequential” sum of squares, because it involves a process of adding terms to the model one at a time. Type I sum of squares is the default hypothesis testing method used by the anova() function.
+The Type I method is sometimes referred to as the “sequential” sum of squares, because it involves a process of adding terms to the model one at a time. Type I sum of squares is the default hypothesis testing method used by the anova() function. This only matters when a data set is unbalanced across treatments, either due to design or missing data points.
diff --git a/chapters/variance-components.html b/chapters/variance-components.html
index de607ce..274f226 100644
--- a/chapters/variance-components.html
+++ b/chapters/variance-components.html
@@ -310,7 +310,7 @@ 13
library(nlme); library(emmeans); library(performance)
library(lme4)
diff --git a/index.html b/index.html
index 28e628d..8afa198 100644
--- a/index.html
+++ b/index.html
@@ -292,13 +292,13 @@ Preface
“Path in the Wilderness” by Erich Taeubel, Jr.
Running mixed models in R is no easy task. There are dozens of packages supporting these aims, each with varying functionality, syntax, and conventions. The linear mixed model ecosystem in R consists of over 80 libraries that either construct and solve mixed model equations or helper packages the process the results from mixed model analysis. These libraries provide a patchwork of overlapping and unique functionality regarding the fundamental structure of mixed models: allowable distributions, nested and crossed random effects, heterogeneous error structures and other facets. No single library has all possible functionality enabled.
-This patchwork of packages makes it very challenging for statisticians to conduct mixed model analysis and to teach others how to run mixed models in R. The purpose of this guide to to provide some recipes for handling common analytical scenario’s that require mixed models. As a field guide, it is intended to be succinct, and help researchers meet their analytic goals.
+This patchwork of packages makes it very challenging for statisticians to conduct mixed model analysis and to teach others how to run mixed models in R. The purpose of this guide to to provide some recipes for handling common analytical scenario’s that require mixed models. As a field guide, it is intended to be succinct, and to help researchers meet their analytic goals.
In general, the content from this website may not be copied or reproduced without attribution. However, the example code and required data sets to run the code are MIT licensed. These can be accessed on GitHub.
What This Does Not Cover
-Generalized linear models. We do address cases of unequal variance, but if another distribution and/or a link function is required for the model, that is not addressed in this guide.
Basic principles of experimental design. We assume you know this, but if you do not, please check out the Grammar of Experimental Design.
Generalized linear models where the response variable does not follow a normal distribution. We do address cases of unequal variance, but if another distribution and/or a link function is required for the model, that is not addressed in this guide.
Basic principles of experimental design. We assume you know this, but if you do not, please check out the Grammar of Experimental Design for guidance on these topics.
Instructions in using R. We assume familiarity with R. If you need help in learning R, there are numerous guides, including our introductory R course.
diff --git a/search.json b/search.json
index baed886..20aa7aa 100644
--- a/search.json
+++ b/search.json
@@ -4,7 +4,7 @@
"href": "index.html",
"title": "Field Guide to the R Mixed Model Wilderness",
"section": "",
- "text": "Preface\n“Path in the Wilderness” by Erich Taeubel, Jr.\nRunning mixed models in R is no easy task. There are dozens of packages supporting these aims, each with varying functionality, syntax, and conventions. The linear mixed model ecosystem in R consists of over 80 libraries that either construct and solve mixed model equations or helper packages the process the results from mixed model analysis. These libraries provide a patchwork of overlapping and unique functionality regarding the fundamental structure of mixed models: allowable distributions, nested and crossed random effects, heterogeneous error structures and other facets. No single library has all possible functionality enabled.\nThis patchwork of packages makes it very challenging for statisticians to conduct mixed model analysis and to teach others how to run mixed models in R. The purpose of this guide to to provide some recipes for handling common analytical scenario’s that require mixed models. As a field guide, it is intended to be succinct, and help researchers meet their analytic goals.\nIn general, the content from this website may not be copied or reproduced without attribution. However, the example code and required data sets to run the code are MIT licensed. These can be accessed on GitHub.",
+ "text": "Preface\n“Path in the Wilderness” by Erich Taeubel, Jr.\nRunning mixed models in R is no easy task. There are dozens of packages supporting these aims, each with varying functionality, syntax, and conventions. The linear mixed model ecosystem in R consists of over 80 libraries that either construct and solve mixed model equations or helper packages the process the results from mixed model analysis. These libraries provide a patchwork of overlapping and unique functionality regarding the fundamental structure of mixed models: allowable distributions, nested and crossed random effects, heterogeneous error structures and other facets. No single library has all possible functionality enabled.\nThis patchwork of packages makes it very challenging for statisticians to conduct mixed model analysis and to teach others how to run mixed models in R. The purpose of this guide to to provide some recipes for handling common analytical scenario’s that require mixed models. As a field guide, it is intended to be succinct, and to help researchers meet their analytic goals.\nIn general, the content from this website may not be copied or reproduced without attribution. However, the example code and required data sets to run the code are MIT licensed. These can be accessed on GitHub.",
"crumbs": [
"Preface"
]
@@ -14,7 +14,7 @@
"href": "index.html#what-this-does-not-cover",
"title": "Field Guide to the R Mixed Model Wilderness",
"section": "What This Does Not Cover",
- "text": "What This Does Not Cover\n\nGeneralized linear models. We do address cases of unequal variance, but if another distribution and/or a link function is required for the model, that is not addressed in this guide.\nBasic principles of experimental design. We assume you know this, but if you do not, please check out the Grammar of Experimental Design.\nInstructions in using R. We assume familiarity with R. If you need help in learning R, there are numerous guides, including our introductory R course.",
+ "text": "What This Does Not Cover\n\nGeneralized linear models where the response variable does not follow a normal distribution. We do address cases of unequal variance, but if another distribution and/or a link function is required for the model, that is not addressed in this guide.\nBasic principles of experimental design. We assume you know this, but if you do not, please check out the Grammar of Experimental Design for guidance on these topics.\nInstructions in using R. We assume familiarity with R. If you need help in learning R, there are numerous guides, including our introductory R course.",
"crumbs": [
"Preface"
]
@@ -34,7 +34,7 @@
"href": "chapters/intro.html",
"title": "1 Introduction",
"section": "",
- "text": "1.1 Terms\nThis guide is focused on frequent implementations of mixed models in R, covering different scenarios common in the agricultural sciences.\nThis is not intended to be a guide to the theory of mixed models, it is focused on implementations of models only.\nPlease read this section and refer back to if when you forget what these terms mean.",
+ "text": "1.1 Terms\nThis guide is focused on frequentist implementations of mixed models in R, covering different scenarios common in the agricultural and life sciences.\nThis is not intended to be a guide to the theory of mixed models, it is focused on implementations of models only.\nPlease read this section and refer back to if when you forget what these terms mean.",
"crumbs": [
"1 Introduction"
]
@@ -64,7 +64,7 @@
"href": "chapters/analysis-tips.html",
"title": "2 Tips on Analysis",
"section": "",
- "text": "Below are some things our office frequently says to researchers.\n\n2.0.1 Think About Your Analytical Goals\nThroughout this guide, we have tried to explicitly state the goals of each analysis. This helps informs how to approach the analysis of an experiment. It can be difficult, especially for new scientists-in-training (i.e. graduate students), to understand what it is they want to estimate. You may have been handed a data set you had no role in generating and told to “analyze this” with no additional context. Or perhaps you may have conducted a large study that has some overall goals that are lofty, yet vague.\nIt can helpful to think about the exact results you are hoping to get. What does this look like exactly? Do you want to estimate the changes in plant diversity as the result of a herbicide spraying program? Do you want to find out if a fertilizer treatment changed protein content in a crop and by how much? Do you want to know about changes in human diet due to an intervention? What are quantifiable difference that you and/or experts in your domain would find meaningful?\nConsider what the results would look like for (1) the best case scenario when your wildest dreams come true, and (2) null results, when you find out that your treatment or invention had no effect. It’s very helpful to understand and recognize both situations.\nBy “consider”, we mean: imagine the final plot or table, or summary sentence you want to present, either in a peer-reviewed manuscript, or some output for stakeholders. From this, you work backwards to determine the analytical approach needed to arrive at that final output. Or you may determine that your data are unsuitable to generate the desired output, in which case, it’s best to determine that as soon as possible.\nBy “consider”, we also mean: imagine exactly what the spreadsheet of results would say - what columns are present and what data are in the cells. If you are planning an experiment, this can help ensure you plan it properly to actually test whatever it is you want to evaluate. If the experiment is done, this enables you to evaluate if you have the information present to test your hypothesis.\nBy taking the time to reflect on what it is you exactly want to analyze, this can save time and prevent you from doing unneeded analyzes that don’t serve this final goal. There is rarely (never?) one way to analyze an experiment or a data set, so use your limited time wisely and focus on what matters.\n\n\n2.0.2 Know That Data Cleaning is Time Consuming\n\n\n\n\n\n\n\n\n\nFigure 2.1: How you will spend your time\n\n\n\n\nThis has and will continue to occupy the majority of researcher’s time when conducting an analysis. Truly, we are sorry for this. But, please know it is not you, it is the nature of data. Please plan for and prepare yourself mentally to spend time cleaning and preparing your data for analysis.1 This will take way longer than the actual analysis! It is needed to ensure you can actually get correct results in an analysis, and hence data cleaning is worth the time it requires.\n1 For an excellent set of basic instructions on data preparation, please see: Broman, K. W., & Woo, K. H. (2018). Data Organization in Spreadsheets. The American Statistician, 72(1), 2–10.\n\n2.0.3 Interpret ANOVA and P-values with Caution\n\nInformally, a p-value is the probability under a specified statistical model that a statistical summary of the data (e.g., the sample mean difference between two compared groups) would be equal to or more extreme than its observed value.\n---American Statistical Association\n\nThe great majority of researched are deeply interested in p-values. This is not a bad thing per se, but sometimes the focus is so strong it comes at the expense of other valuable pieces of information, like treatment estimates! Russ Leanth, author of the emmeans package refers to this particular practice as “star gazing”.\nIt is important to evaluate why you want to do ANOVA, what extra information it will bring and what you plan to do with those results. Sometimes, researchers want to conduct an ANOVA even though the original goals of analysis were reached without it. Running an ANOVA may increase or decrease confidence in your other results. That is not at all what ANOVA is intended to do, nor is this what p-values can tell us. ANOVA compares across group variation to within group variation. It cannot tell us if anything is the ‘same’ (there’s a separate branch of analysis, ‘equivalence testing’, for that), and it cannot tell us specifically what is different, unless you are fortunate enough to only have 2 levels in your treatment structure. P-values provide no guarantee that something is truly different or not; it only quantifies the probability you could have observed these results by chance.\nThe American Statistics Association recommends that “Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.”2 That article also explains what p-values are telling us and how to avoid committing analytical errors and/or misinterpreting p-values. If you have time to read the full article, it will benefit your research!\n2 Wasserstein, R. L., & Lazar, N. A. (2016). The ASA Statement on p-Values: Context, Process, and Purpose. The American Statistician, 70(2), 129–133.The main problematic behavior I see is researchers using p-values as the sole criteria on whether to present results: “We wanted to test if x, y and z had an effect. We ran some model and found that that only x had a significant effect, and those results indicate…” (while results with a p-value > 0.05 are ignored).\nA better option would be to discuss the the results of the analysis and how they addressed the research questions: how did the dependent variable change (or not change) as a result of the treatments/interventions/independent variables? What are the parameters or treatment predictions and what do they tell us with regard to the research goals? And to bolster those estimates, what are the confidence intervals on those estimates? What are the p-values for the statistical tests? P-values can support the results and conclusions, but the main results desired by a researcher are usually the estimates themselves - so lead with that!\nTo learn more about common pitfalls in interpreting p-values, check out our blog post on the subject and/or this paper3 on the subject.\n3 Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, Altman DG. (2016) Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol. 31(4):337-50.\n\n2.0.4 Comments on Hypothesis Testing and Usage of Treatment Letters\nOften, I see researchers use compact letter display (e.g. “A”, “B”, “C”, ….) for indicating differences among treatments. This makes for concise presentation of results in tables and figures, but it can both kill statistical power and misses nuance in the results.\n\n\n\nImage from a paper published in 2024. Although this was a fully crossed factorial experiment, compact letter display was implemented across all treatment combinations, resulting in some nonsensical comparisons among some more informative contrasts.\nImplementing compact letter display can kill statistical power (the probability of detecting true differences) because it requires that all pairwise comparison being made. Doing this, especially when there are many treatment levels, has its perils. The biggest problem is that this creates a multiple testing problem. The RCBD example in this guide has 42 treatments, resulting in a total of 861 comparisons (\\(=42*(42-1)/2\\)), that are then adjusted for multiple tests. With that many tests, a severe adjustment is likely and hence things that are different are not detected. With so many tests, it could be that there is an overall effect due to treatment, but they all share the same letter!\nThe second problem is one of interpretation. Just because two treatments or varieties share a letter does not mean they are equivalent. It only means that they were not found to be different. A funny distinction, but alas. There is an entire branch of statistics, ‘equivalence testing’ devoted to just this topic - how to test if two things are actually the same. This involves the user declaring a maximum allowable numeric difference for a variable in order to determine if two items are statistically different or equivalent - something that these pairwise comparisons are not doing.]\nAnother problem is that doing all pairwise comparison may not align with experimental goals. In many circumstances, not every pairwise combination is of any interest or relevance to the study. Additionally, complex treatment structure may necessitate custom constrasts that highlight differences between the marginal estimate of multiple treatments versus another. For example, there may be 2 levels of ‘high’ nitrogen fertilizer treatment with two different sources (i.e. types of fertilizer). A researcher may want to contrast those two levels together against ‘low’ nitrogen treatment levels.\nOften, researchers have embedded additional structure in the treatments that is not fully reflected in the statistical model. For example, perhaps a study is looking at five different intercropping mixtures, two that incorporate a legume and 3 that do not. Conducting all pairwise comparisons with miss estimating the difference due to including a legume in an intercropping mix and not incorporating one. Soil fertility and other agronomic studies often have complex treatment structure. When it is not practical or financially feasible to have a full factorial experiment, embedding different treatment combinations in the main factor of analysis can accomplish this. This is a good study design approach, and we have statistical tools to analyze it.\n\n\n\n\nBroman, Karl W., and Kara H. Woo. 2018. “Data Organization in Spreadsheets.” The American Statistician 72 (1): 2–10. https://doi.org/10.1080/00031305.2017.1375989.\n\n\nGreenland, Sander, Stephen J. Senn, Kenneth J. Rothman, John B. Carlin, Charles Poole, Steven N. Goodman, and Douglas G. Altman. 2016. “Statistical Tests, P Values, Confidence Intervals, and Power: A Guide to Misinterpretations.” European Journal of Epidemiology 31 (4): 337–50. https://doi.org/10.1007/s10654-016-0149-3.\n\n\nWasserstein, Ronald L., and Nicole A. Lazar. 2016. “The ASA Statement on p-Values: Context, Process, and Purpose.” The American Statistician 70 (2): 129–33. https://doi.org/10.1080/00031305.2016.1154108.",
+ "text": "Below are some things our office frequently says to researchers.\n\n2.0.1 Think About Your Analytical Goals\nThroughout this guide, we have tried to explicitly state the goals of each analysis. This helps informs how to approach the analysis of an experiment. It can be difficult, especially for new scientists-in-training (i.e. graduate students), to understand what it is they want to estimate. You may have been handed a data set you had no role in generating and told to “analyze this” with no additional context. Or perhaps you may have conducted a large study that has some overall goals that are lofty, yet vague. And now you must translate the vague aims into clear statistical questions.\nIt can helpful to think about the exact results you are hoping to get. What does this look like exactly? Do you want to estimate the changes in plant diversity as the result of a herbicide spraying program? Do you want to find out if a fertilizer treatment changed protein content in a crop and by how much? Do you want to know about changes in human diet due to an intervention? What are quantifiable difference that you and/or experts in your domain would find meaningful?\nConsider what the results would look like for (1) the best case scenario where your wildest research dreams come true, and (2) null results, when you find out that your treatment or invention had no effect. It’s very helpful to understand and recognize exactly what both situations look like.\nBy “consider”, we mean: imagine the final plot or table, or summary sentence you want to present, either in a peer-reviewed manuscript, or some output for stakeholders. From this, you can work backwards to determine the analytical approach needed to arrive at that desired final output. Or you may determine that your data are unsuitable to generate the desired output, in which case, it’s best to determine that as soon as possible.\nBy “consider”, we also mean: imagine exactly what the spreadsheet of results would contain after a successful trial. What columns are present and what data are in those cells. If you are planning an experiment, this can help ensure you plan it properly to actually test whatever it is you want to evaluate. If the experiment is done, this enables you to evaluate if you have the information present to test your hypothesis.\nBy taking the time to reflect on what it is you exactly want to analyze, this can save time and prevent you from doing unneeded analyzes that don’t serve this final goal. There is rarely (never?) one way to analyze an experiment or a data set, so use your limited time wisely and focus on what matters to you most.\n\n\n2.0.2 Know That Data Cleaning is Time Consuming\n\n\n\n\n\n\n\n\n\nFigure 2.1: How you will spend your time\n\n\n\n\nThis has and will continue to occupy the majority of researcher’s time when conducting an analysis. Truly, we are sorry for this. But, please know it is not you, it is the nature of data. Plan for and prepare yourself mentally to spend time cleaning and preparing your data for analysis.1 This will likely take way longer than the actual analysis! It is needed to ensure you can actually get correct results in an analysis, and hence data cleaning is worth the time it requires.\n1 For an excellent set of basic instructions on data preparation, please see: Broman, K. W., & Woo, K. H. (2018). Data Organization in Spreadsheets. The American Statistician, 72(1), 2–10.\n\n2.0.3 Interpret ANOVA and P-values with Caution\n\nInformally, a p-value is the probability under a specified statistical model that a statistical summary of the data (e.g., the sample mean difference between two compared groups) would be equal to or more extreme than its observed value.\n---American Statistical Association\n\nThe great majority of researched are deeply interested in p-values. This is not a bad thing per se, but sometimes the focus is so strong it comes at the expense of other valuable pieces of information, like treatment estimates! Russ Leanth, author of the emmeans package refers to this particular practice as “star gazing”.\nIt is important to evaluate why you want to do ANOVA, what extra information it will bring and what you plan to do with those results. Sometimes, researchers want to conduct an ANOVA even though the original goals of analysis were reached without it. Running an ANOVA may increase or decrease confidence in your other results. That is not at all what ANOVA is intended to do, nor is this what p-values can tell us. ANOVA compares across group variation to within group variation. It cannot tell us if anything is the ‘same’ (there’s a separate branch of analysis, ‘equivalence testing’, for that), and it cannot tell us specifically what is different, unless you are fortunate enough to only have 2 levels in your treatment structure. P-values provide no guarantee that something is truly different or not; it only quantifies the probability you could have observed these results by chance.\nThe American Statistics Association recommends that “Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.”2 That article also explains what p-values are telling us and how to avoid committing analytical errors and/or misinterpreting p-values. If you have time to read the full article, it will benefit your research!\n2 Wasserstein, R. L., & Lazar, N. A. (2016). The ASA Statement on p-Values: Context, Process, and Purpose. The American Statistician, 70(2), 129–133.The main problematic behavior I see is researchers using p-values as the sole criteria on whether to present results: “We wanted to test if x, y and z had an effect. We ran some model and found that that only x had a significant effect, and those results indicate…” (while results with a p-value > 0.05 are ignored).\nA better option would be to discuss the the results of the analysis and how they addressed the research questions: how did the dependent variable change (or not change) as a result of the treatments/interventions/independent variables? What are the parameters or treatment predictions and what do they tell us with regard to the research goals? And to bolster those estimates, what are the confidence intervals on those estimates? What are the p-values for the statistical tests? P-values can support the results and conclusions, but the main results desired by a researcher are usually the estimates themselves - so lead with that!\nTo learn more about common pitfalls in interpreting p-values, check out our blog post on the subject and/or this paper3 on the subject.\n3 Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, Altman DG. (2016) Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol. 31(4):337-50.\n\n2.0.4 Comments on Hypothesis Testing and Usage of Treatment Letters\nOften, I see researchers use compact letter display (e.g. “A”, “B”, “C”, ….) for indicating differences among treatments. This makes for concise presentation of results in tables and figures, but it can both kill statistical power and misses nuance in the results.\n\n\n\nImage from a paper published in 2024. Although this was a fully crossed factorial experiment, compact letter display was implemented across all treatment combinations, resulting in some nonsensical comparisons among some more informative contrasts. What a waste.\nImplementing compact letter display can kill statistical power (the probability of detecting true differences) because it requires that all pairwise comparison being made. Doing this, especially when there are many treatment levels, has its perils. The biggest problem is that this creates a multiple testing problem. The RCBD example in this guide has 42 treatments, resulting in a total of 861 comparisons (\\(=42*(42-1)/2\\)), that are then adjusted for multiple tests. With that many tests, a severe adjustment is likely and hence things that are different are not detected. With so many tests, it could be that there is an overall effect due to treatment, but they all share the same letter!\nThe second problem is one of interpretation. Just because two treatments or varieties share a letter does not mean they are equivalent. It only means that they were not found to be different. A funny distinction, but alas. There is an entire branch of statistics, ‘equivalence testing’ devoted to just this topic - how to test if two things are actually the same. This involves the user declaring a maximum allowable numeric difference for a variable in order to determine if two items are statistically different or equivalent - something that these pairwise comparisons are not doing.]\nAnother problem is that doing all pairwise comparison may not align with experimental goals. In many circumstances, not every pairwise combination is of any interest or relevance to the study. Additionally, complex treatment structure may necessitate custom contrasts that highlight differences between the marginal estimate of multiple treatments versus another. For example, there may be 2 levels of ‘high’ nitrogen fertilizer treatment with two different sources (i.e. types of fertilizer). A researcher may want to contrast those two levels together against ‘low’ nitrogen treatment levels.\nOften, researchers have embedded additional structure in the treatments that is not fully reflected in the statistical model. For example, perhaps a study is looking at five different intercropping mixtures, two that incorporate a legume and 3 that do not. Conducting all pairwise comparisons with miss estimating the difference due to including a legume in an intercropping mix and not incorporating one. Soil fertility and other agronomic studies often have complex treatment structure. When it is not practical or financially feasible to have a full factorial experiment, embedding different treatment combinations in the main factor of analysis can accomplish this. This is a good study design approach, but compact letter display is an efficient way to report results. In such cases, custom contrasts are a better choice for hypothesis testing.The emmeans chapter covers how to do this.\n\n\n2.0.5 Final Thoughts\nGood statistical analysis requires a thoughtful, intentional approach. If you have gone to the trouble to conduct a well designed experiment or assemble a useful data set, take the time and effort to analyze it properly.\n\n\n\n\nBroman, Karl W., and Kara H. Woo. 2018. “Data Organization in Spreadsheets.” The American Statistician 72 (1): 2–10. https://doi.org/10.1080/00031305.2017.1375989.\n\n\nGreenland, Sander, Stephen J. Senn, Kenneth J. Rothman, John B. Carlin, Charles Poole, Steven N. Goodman, and Douglas G. Altman. 2016. “Statistical Tests, P Values, Confidence Intervals, and Power: A Guide to Misinterpretations.” European Journal of Epidemiology 31 (4): 337–50. https://doi.org/10.1007/s10654-016-0149-3.\n\n\nWasserstein, Ronald L., and Nicole A. Lazar. 2016. “The ASA Statement on p-Values: Context, Process, and Purpose.” The American Statistician 70 (2): 129–33. https://doi.org/10.1080/00031305.2016.1154108.",
"crumbs": [
"2 Tao of Analysis"
]
@@ -74,7 +74,7 @@
"href": "chapters/background.html",
"title": "3 Mixed model theory and background",
"section": "",
- "text": "3.1 Model\nMixed-effects models are called “mixed” because they simultaneously model fixed and random effects. Fixed effects (e.g. treatments) represent population-level (i.e., average) effects that should persist across experiments. Fixed effects are similar to the parameters found in “traditional” regression techniques like ordinary least squares. Random effects are discrete units sampled from some population (e.g. plots, participants), and thus they are inherently categorical.\nRecall simple linear regression with intercept (\\(\\beta_0\\)) and slope (\\(\\beta_1\\)) effect for subject i. The slope and intercept are chosen in a way so that the residual sum of squares is minimized.\n\\[ Y = \\beta_0 + \\beta_1 X + \\epsilon \\]\nIf we consider this model in a mixed model framework, \\(\\beta_0\\) and \\(\\beta_0\\) are considered fixed effects (also known as the population-averaged values) and \\(b_i\\) is a random effect for subject i. The random effect can be thought of as each subject’s deviation from the fixed intercept parameter. The key assumption about \\(b_i\\) is that it is independent, identically and normally distributed with a mean of zero and associated variance. Random effects are especially useful when we have (1) lots of levels (e.g., many species or blocks), (2) relatively little data on each level (although we need multiple samples from most of the levels), and (3) uneven sampling across levels.\nFor example, if we let the intercept be a random effect, it takes the form:\n\\[ Y = \\beta_0 + b_i + \\beta_1 X + \\epsilon \\]\nIn this model, predictions would vary depending on each subject’s random intercept term, but slopes would be the same.\nIn second case, we can have a fixed intercept and a random slope. The model will be:\n\\[ Y = \\beta_0 + (\\beta_1 + b_i)(X) + \\epsilon\\]\nIn this model, the bi is a random effect for subject i applied to the slope. Predictions would vary with random slope term, but the intercept will be the same:\nThird case would be the mixed model with random slope and intercept:\n\\[ Y = (\\beta_0 + a_i) + (\\beta_1 + b_i)(X) + \\epsilon\\]\nIn this model, \\(a_i\\) and \\(b_i\\) are random effects for subject i applied to the intercept and slope, respectively. Predictions would vary depending on each subject’s slope and intercept terms:",
+ "text": "3.1 Model\nMixed-effects models are called “mixed” because they simultaneously model fixed and random effects. Fixed effects (e.g. treatments) represent population-level (average) effects that should persist across experiments. Fixed effects are similar to the parameters found in “traditional” regression techniques like ordinary least squares. Random effects are discrete units sampled from some population (e.g. plots, participants), and thus they are inherently categorical.\nRecall simple linear regression with intercept (\\(\\beta_0\\)) and slope (\\(\\beta_1\\)) effect for subject \\(i\\). The slope and intercept are chosen in a way so that the residual sum of squares is minimized.\n\\[ Y = \\beta_0 + \\beta_1 X + \\epsilon \\]\nIf we consider this model in a mixed model framework, \\(\\beta_0\\) and \\(\\beta_0\\) are considered fixed effects (also known as the population-averaged values) and \\(b_i\\) is a random effect for subject i. The random effect can be thought of as each subject’s deviation from the fixed intercept parameter. The key assumption about \\(b_i\\) is that it is independent, identically and normally distributed with a mean of zero and associated variance. Random effects are especially useful when we have (1) lots of levels (e.g., many species or blocks), (2) relatively little data on each level (although we need multiple samples from most of the levels), and (3) uneven sampling across levels.\nFor example, if we let the intercept be a random effect, it takes the form:\n\\[ Y = \\beta_0 + b_i + \\beta_1 X + \\epsilon \\]\nIn this model, predictions would vary depending on each subject’s random intercept term, but slopes would be the same.\nIn second case, we can have a fixed intercept and a random slope. The model will be:\n\\[ Y = \\beta_0 + (\\beta_1 + b_i)(X) + \\epsilon\\]\nIn this model, the \\(\\beta_i\\) is a random effect for subject \\(i\\). Predictions would vary with random slope term, but the intercept will be the same:\nThird case would be the mixed model with random slope and intercept:\n\\[ Y = (\\beta_0 + a_i) + (\\beta_1 + b_i)(X) + \\epsilon\\]\nIn this model, \\(a_i\\) and \\(b_i\\) are random effects for subject \\(i\\) applied to the intercept and slope, respectively. Predictions would vary depending on each subject’s slope and intercept terms:",
"crumbs": [
"3 Mixed Model Background"
]
@@ -94,7 +94,7 @@
"href": "chapters/background.html#formula-notation",
"title": "3 Mixed model theory and background",
"section": "3.2 R Formula Syntax for Random and Fixed Effects",
- "text": "3.2 R Formula Syntax for Random and Fixed Effects\nFormula notation is often used in the R syntax for linear models. It looks like this: \\(Y ~ X\\), where Y is the dependent variable (the response) and X is/are the independent variable(s) (e.g. the experimental treatments).\n\nmy_formula <- formula(Y ~ treatment1 + treatment2)\nclass(my_formula)\n\n[1] \"formula\"\n\n\nThe package ‘lme4’ has some additional conventions regarding the formula. Random effects are put in parentheses and a 1| is used to denote random intercepts (rather than random slopes). The table below provides several examples of random effects in mixed models. The names of grouping factors are denoted g, g1, and g2, and covariate as x.\n\n\n\n\n\n\n\n\nFormula\nAlternative\nMeaning\n\n\n\n\n(1 | g)\n1 + (1 | g)\nRandom intercept with a fixed mean\n\n\n(1 | g1/g2)\n(1 | g1) + (1 | g1:g2)\nIntercept varying among g1 and g2 within g1.\n\n\n(1 | g1) + (1 | g2)\n1 + (1 | g1) + (1| g2)\nIntercept varying among g1 and g2.\n\n\nx + (x | g)\n1 + x + (1 + x | g)\nCorrelated random intercept and slope\n\n\nx + (x || g)\n1 + x + (1 | g) + (0 + x | g)\nUncorrelated random intercept and slope.",
+ "text": "3.2 R Formula Syntax for Random and Fixed Effects\nFormula notation is often used in the R syntax for linear models. It looks like this: \\(Y ~ X\\), where \\(Y\\) is the dependent variable (the response) and \\(X\\) is/are the independent variable(s) that is, the experimental treatments or interventions.\n\nmy_formula <- formula(Y ~ treatment1 + treatment2)\nclass(my_formula)\n\n[1] \"formula\"\n\n\nThe package ‘lme4’ has some additional conventions regarding the formula. Random effects are put in parentheses and a 1| is used to denote random intercepts (rather than random slopes). The table below provides several examples of random effects in mixed models. The names of grouping factors are denoted g, g1, and g2, and covariates as x.\n\n\n\n\n\n\n\n\nFormula\nAlternative\nMeaning\n\n\n\n\n(1|g)\n1 + (1|g)\nRandom intercept with a fixed mean\n\n\n(1|g1/g2)\n(1| 1) + (1|g1:g2)\nIntercept varying among g1 and g2 within g1\n\n\n(1|g1) + (1|g2)\n1 + (1|g1) + (1|g2)\nIntercept varying among g1 and g2\n\n\nx + (x|g)\n1 + x + (1 + x|g)\nCorrelated random intercept and slope\n\n\nx + (x||g)\n1 + x + (1|g) + (0 + x|g)\nUncorrelated random intercept and slope\n\n\n\nThe first example, (1|g) suffices for most models and is the only structure used in this guide.",
"crumbs": [
"3 Mixed Model Background"
]
@@ -104,7 +104,7 @@
"href": "chapters/rcbd.html",
"title": "4 Randomized Complete Block Design",
"section": "",
- "text": "4.1 Background\nThis is a simple model that can serve as a good entrance point to mixed models.\nIt is very common design where experimental treatments are applied at random to experimental units within each block. The blocks are intended to control for a nuisance source of variation, such as over time, spatial variance, changes in equipment or operators, or myriad other causes.\nThe statistical model:\n\\[y_{ij} = \\mu + \\alpha_i + \\beta_j + \\epsilon_{ij}\\] Where:\n\\(\\mu\\) = overall experimental mean \\(\\alpha\\) = treatment effects (fixed) \\(\\beta\\) = block effects (random) \\(\\epsilon\\) = error terms\n\\[ \\epsilon \\sim N(0, \\sigma)\\]\n\\[ \\beta \\sim N(0, \\sigma_b)\\]\nBoth the overall error and the block effects are assumed to be normally distributed with a mean of zero and standard deviations of \\(\\sigma\\) and \\(sigma_B\\), respectively.",
+ "text": "4.1 Background\nThis is a simple model that can serve as a good entrance point to mixed models.\nRandomized complete block design (RCBD) is very common design where experimental treatments are applied at random to experimental units within each block. The block can represent a spatial or temporal unit or even different technicians taking data. The blocks are intended to control for a nuisance source of variation, such as over time, spatial variance, changes in equipment or operators, or myriad other causes. They are a random effect where the actual blocks used in the study are a random sample of a distribution of other blocks.\nThe statistical model:\n\\[y_{ij} = \\mu + \\alpha_i + \\beta_j + \\epsilon_{ij}\\] Where:\n\\(\\mu\\) = overall experimental mean \\(\\alpha\\) = treatment effects (fixed) \\(\\beta\\) = block effects (random) \\(\\epsilon\\) = error terms\n\\[ \\epsilon \\sim N(0, \\sigma)\\]\n\\[ \\beta \\sim N(0, \\sigma_b)\\]\nBoth the overall error and the block effects are assumed to be normally distributed with a mean of zero and standard deviations of \\(\\sigma\\) and \\(sigma_B\\), respectively.",
"crumbs": [
"Experiment designs",
"4 Randomized Complete Block Design"
@@ -126,7 +126,7 @@
"href": "chapters/rcbd.html#example-analysis",
"title": "4 Randomized Complete Block Design",
"section": "4.2 Example Analysis",
- "text": "4.2 Example Analysis\nFirst, load the libraries for analysis and estimation:\n\nlme4nlme\n\n\n\nlibrary(lme4); library(lmerTest); library(emmeans)\nlibrary(dplyr); library(performance)\n\n\n\n\nlibrary(nlme); library(performance); library(emmeans)\nlibrary(dplyr)\n\n\n\n\nNext, let’s load some data. It is located here if you want to download it yourself (recommended).\nThis data set is for a single wheat variety trial conducted in Aberdeen, Idaho in 2015. The trial includes 4 blocks and 42 different treatments (wheat varieties in this case). This experiment consists of a series of plots (the experimental unit) laid out in a rectangular grid in a farm field. The goal of this analysis is the estimate the yield and test weight of each variety and the determine the rankings of each variety with regard to yield.\n\nvar_trial <- read.csv(here::here(\"data\", \"aberdeen2015.csv\"))\n\n\nTable of variables in the data set\n\n\n\n\n\n\nblock\nblocking unit\n\n\nrange\ncolumn position for each plot\n\n\nrow\nrow position for each plot\n\n\nvariety\ncrop variety (the treatment) being evaluated\n\n\nstand_pct\npercentage of the plot with actual plants growing in them\n\n\ndays_to_heading_julian\nJulian days (starting January 1st) until plot “headed” (first spike emerged)\n\n\nlodging\npercentage of plants in the plot that fell down and hence could not be harvested\n\n\nyield_bu_a\nyield (bushels per acre)\n\n\n\nThere are several variables present that are not useful for this analysis. The only thing we are concerned about is block, variety, yield_bu_a, and test_weight.\n\n4.2.1 Data integrity checks\nThe first thing is to make sure the data is what we expect. There are two steps:\n\nmake sure data are the expected data type\ncheck the extent of missing data\ninspect the independent variables and make sure the expected levels are present in the data\ninspect the dependent variable to ensure its distribution is following expectations\n\n\nstr(var_trial)\n\n'data.frame': 168 obs. of 10 variables:\n $ block : int 4 4 4 4 4 4 4 4 4 4 ...\n $ range : int 1 1 1 1 1 1 1 1 1 1 ...\n $ row : int 1 2 3 4 5 6 7 8 9 10 ...\n $ variety : chr \"DAS004\" \"Kaseberg\" \"Bruneau\" \"OR2090473\" ...\n $ stand_pct : int 100 98 96 100 98 100 100 100 99 100 ...\n $ days_to_heading_julian: int 149 146 149 146 146 151 145 145 146 146 ...\n $ height : int 39 35 33 31 33 44 30 36 36 29 ...\n $ lodging : int 0 0 0 0 0 0 0 0 0 0 ...\n $ yield_bu_a : num 128 130 119 115 141 ...\n $ test_weight : num 56.4 55 55.3 54.1 54.1 56.4 54.7 57.5 56.1 53.8 ...\n\n\nThese look okay except for block, which is currently coded as integer (numeric). We don’t want run a regression of block, where block 1 has twice the effect of block 2, and so on. So, converting it to a character will fix that. It can also be converted to a factor, but I find character easier to work with, and ultimately, equivalent to factor conversion\n\nvar_trial$block <- as.character(var_trial$block)\n\nNext, check the independent variables. Running a cross tabulations is often sufficient to ascertain this.\n\ntable(var_trial$variety, var_trial$block)\n\n \n 1 2 3 4\n 06-03303B 1 1 1 1\n Bobtail 1 1 1 1\n Brundage 1 1 1 1\n Bruneau 1 1 1 1\n DAS003 1 1 1 1\n DAS004 1 1 1 1\n Eltan 1 1 1 1\n IDN-01-10704A 1 1 1 1\n IDN-02-29001A 1 1 1 1\n IDO1004 1 1 1 1\n IDO1005 1 1 1 1\n Jasper 1 1 1 1\n Kaseberg 1 1 1 1\n LCS Artdeco 1 1 1 1\n LCS Biancor 1 1 1 1\n LCS Drive 1 1 1 1\n LOR-833 1 1 1 1\n LOR-913 1 1 1 1\n LOR-978 1 1 1 1\n Madsen 1 1 1 1\n Madsen / Eltan (50/50) 1 1 1 1\n Mary 1 1 1 1\n Norwest Duet 1 1 1 1\n Norwest Tandem 1 1 1 1\n OR2080637 1 1 1 1\n OR2080641 1 1 1 1\n OR2090473 1 1 1 1\n OR2100940 1 1 1 1\n Rosalyn 1 1 1 1\n Stephens 1 1 1 1\n SY Ovation 1 1 1 1\n SY 107 1 1 1 1\n SY Assure 1 1 1 1\n UI Castle CLP 1 1 1 1\n UI Magic CLP 1 1 1 1\n UI Palouse 1 1 1 1\n UI Sparrow 1 1 1 1\n UI-WSU Huffman 1 1 1 1\n WB 456 1 1 1 1\n WB 528 1 1 1 1\n WB1376 CLP 1 1 1 1\n WB1529 1 1 1 1\n\n\nThere are 42 varieties and there appears to be no misspellings among them that might confuse R into thinking varieties are different when they are actually the same. R is sensitive to case and white space, which can make it easy to create near duplicate treatments, such as “eltan” and “Eltan” and “Eltan”. There is no evidence of that in this data set. Additionally, it is perfectly balanced, with exactly one observation per treatment per rep. Please note that this does not tell us anything about the extent of missing data.\n\n\n\n\n\n\nMissing Data\n\n\n\nHere is a quick check to count the number of missing data in each column. This is not neededfor the data sets in this tutorial that have already been comprehensively examined, but it is helpful to check that the level of missingness displayed in an R session is what you expect.\n\napply(var_trial, 2, function(x) sum(is.na(x)))\n\n block range row \n 0 0 0 \n variety stand_pct days_to_heading_julian \n 0 0 0 \n height lodging yield_bu_a \n 0 0 0 \n test_weight \n 0 \n\n\nAlas, no missing data!\n\n\nIf there were independent variables with a continuous distribution (a covariate), I would plot those data.\nLast, check the dependent variable. A histogram is often quite sufficient to accomplish this. This is designed to be a quick check, so no need to spend time making the plot look good.\n\n\n\n\n\n\n\n\n\nFigure 4.1: Histogram of the dependent variable.\n\n\n\n\n\nhist(var_trial$yield_bu_a, main = \"\", xlab = \"yield\")\n\nThe range is roughly falling into the range we expect. I know this from talking with the person who generated the data, not through my own intuition. I do not see any large spikes of points at a single value (indicating something odd), nor do I see any extreme values (low or high) that might indicate some larger problems.\nData are not expected to be normally distributed at this point, so don’t bother running any Shapiro-Wilk tests. This histogram is a check to ensure the the data are entered correctly and they appear valid. It requires a mixture of domain knowledge and statistical training to know this, but over time, if you look at these plots with regularity, you will gain a feel for what your data should look like at this stage.\nThese are not complicated checks. They are designed to be done quickly and should be done for every analysis if you not previously already inspected the data as thus. We do this before every analysis and often discover surprising things! Best to discover these things early, since they are likely to impact the final analysis.\nThis data set is ready for analysis!\n\n\n4.2.2 Model Building\n\n\nRecall the model:\n\\[y_{ij} = \\mu + \\alpha_i + \\beta_j + \\epsilon_{ij}\\]\nFor this model, \\(\\alpha_i\\) is the variety effect (fixed) and \\(\\beta_j\\) is the block effect (random).\nHere is the R syntax for the RCBD statistical model:\n\nlme4nlme\n\n\n\nmodel_rcbd_lmer <- lmer(yield_bu_a ~ variety + (1|block),\n data = var_trial, \n na.action = na.exclude)\n\n\n\n\nmodel_rcbd_lme <- lme(yield_bu_a ~ variety,\n random = ~ 1|block,\n data = var_trial, \n na.action = na.exclude)\n\n\n\n\nThe parentheses are used to indicate that ‘block’ is a random effect, and this particular notation (1|block) indicates that a ‘random intercept’ model is being fit. This is the most common approach. It means there is one overall effect fit for each block. I use the argument na.action = na.exclude as instruction for how to handle missing data: conduct the analysis, adjusting as needed for the missing data, and when prediction or residuals are output, please pad them in the appropriate places for missing data so they can be easily merged into the main data set if need be.\n\n\n4.2.3 Check Model Assumptions\n\n\nR syntax for checking model assumptions is the same for lme4 and nlme.\nRemember those iid assumptions? Let’s make sure we actually met them.\n\n4.2.3.1 Old Way\nThere are special plotting function written for lme4 and nlme objects (ie.plot(lmer_object)) for checking the homoscedasticity (constant variance).\n\n\n\n\n\n\n\n\n\nFigure 4.2: Plot of residuals versus fitted values\n\n\n\n\n\nplot(model_rcbd_lmer, resid(., scaled=TRUE) ~ fitted(.), \n xlab = \"fitted values\", ylab = \"studentized residuals\")\n\nWe are looking for a random and uniform distribution of points. This looks good!\nChecking normality requiring first extracting the model residuals with resid() and then generating a qq-plot and line.\n\n\n\n\n\n\n\n\n\nFigure 4.3: QQ-plot of residuals\n\n\n\n\n\nqqnorm(resid(model_rcbd_lmer), main = NULL); qqline(resid(model_rcbd_lmer))\n\nThis is reasonably good. Things do tend to fall apart at the tails.\n\n\n4.2.3.2 New Way\nNowadays, we can take advantage of the performance package, which provides a comprehensive suite of diagnostic plots.\n\n\nPlease look for check_model() in help tab to find what other checks you can perform using this function. If you would like to check all assumptions you can use check = “all”.\n\ncheck_model(model_rcbd_lmer, check = c('normality', 'linearity'))\n\n\n\n\n\n\n\n\n\n\n\n4.2.4 Inference\n\n\nR syntax for esimating model marginal means is the same for lme4 and nlme.\nEstimates for each treatment level can be obtained with the ‘emmeans’ package.\n\nrcbd_emm <- emmeans(model_rcbd_lmer, ~ variety)\nas.data.frame(rcbd_emm) %>% arrange(desc(emmean))\n\n variety emmean SE df lower.CL upper.CL\n Rosalyn 155.2703 7.212203 77.85 140.91149 169.6292\n IDO1005 153.5919 7.212203 77.85 139.23310 167.9508\n OR2080641 152.6942 7.212203 77.85 138.33536 167.0530\n Bobtail 151.6403 7.212203 77.85 137.28149 165.9992\n UI Sparrow 151.6013 7.212203 77.85 137.24245 165.9601\n Kaseberg 150.9768 7.212203 77.85 136.61794 165.3356\n IDN-01-10704A 148.9861 7.212203 77.85 134.62729 163.3450\n 06-03303B 148.8300 7.212203 77.85 134.47116 163.1888\n WB1529 148.2445 7.212203 77.85 133.88568 162.6034\n DAS003 145.2000 7.212203 77.85 130.84116 159.5588\n IDN-02-29001A 144.5755 7.212203 77.85 130.21665 158.9343\n Bruneau 143.9900 7.212203 77.85 129.63116 158.3488\n SY 107 143.6387 7.212203 77.85 129.27987 157.9975\n WB 528 142.9752 7.212203 77.85 128.61633 157.3340\n OR2080637 141.7652 7.212203 77.85 127.40633 156.1240\n Jasper 141.2968 7.212203 77.85 126.93794 155.6556\n UI Magic CLP 139.5403 7.212203 77.85 125.18149 153.8992\n Madsen 139.2671 7.212203 77.85 124.90826 153.6259\n LCS Biancor 139.1110 7.212203 77.85 124.75213 153.4698\n SY Ovation 138.6426 7.212203 77.85 124.28375 153.0014\n OR2090473 137.8229 7.212203 77.85 123.46407 152.1817\n Madsen / Eltan (50/50) 136.9642 7.212203 77.85 122.60536 151.3230\n UI-WSU Huffman 135.4810 7.212203 77.85 121.12213 149.8398\n Mary 134.8564 7.212203 77.85 120.49762 149.2153\n Norwest Tandem 134.3490 7.212203 77.85 119.99020 148.7079\n Brundage 134.0758 7.212203 77.85 119.71697 148.4346\n IDO1004 132.5145 7.212203 77.85 118.15568 146.8733\n DAS004 132.2413 7.212203 77.85 117.88245 146.6001\n Norwest Duet 132.0852 7.212203 77.85 117.72633 146.4440\n Eltan 131.4606 7.212203 77.85 117.10181 145.8195\n LCS Artdeco 130.8361 7.212203 77.85 116.47729 145.1950\n UI Palouse 130.4848 7.212203 77.85 116.12600 144.8437\n LOR-978 130.4458 7.212203 77.85 116.08697 144.8046\n LCS Drive 128.7674 7.212203 77.85 114.40858 143.1262\n Stephens 127.1671 7.212203 77.85 112.80826 141.5259\n OR2100940 126.1523 7.212203 77.85 111.79342 140.5111\n UI Castle CLP 125.5277 7.212203 77.85 111.16891 139.8866\n WB1376 CLP 123.6932 7.212203 77.85 109.33439 138.0521\n LOR-833 122.7565 7.212203 77.85 108.39762 137.1153\n LOR-913 118.7752 7.212203 77.85 104.41633 133.1340\n WB 456 118.4629 7.212203 77.85 104.10407 132.8217\n SY Assure 111.0468 7.212203 77.85 96.68794 125.4056\n\nDegrees-of-freedom method: kenward-roger \nConfidence level used: 0.95 \n\n\nThis table indicates the estimated marginal means (“emmeans”, sometimes called “least squares means”), the standard error (“SE”) of those means, the degrees of freedom and the upper and lower bounds of the 95% confidence interval. As an additional step, the emmeans were sorted from largest to smallest.\nAt this point, the analysis goals have been met: we know the estimated means for each treatment and their rankings.\nIf you want to run ANOVA, it can be done quite easily. By default, the Kenward-Rogers method of degrees of freedom approximation is used.\n\n\nThe Type I method is sometimes referred to as the “sequential” sum of squares, because it involves a process of adding terms to the model one at a time. Type I sum of squares is the default hypothesis testing method used by the anova() function.\n\nlme4nlme\n\n\n\nanova(model_rcbd_lmer, type = \"1\")\n\nType I Analysis of Variance Table with Satterthwaite's method\n Sum Sq Mean Sq NumDF DenDF F value Pr(>F) \nvariety 18354 447.65 41 123 2.4528 8.017e-05 ***\n---\nSignif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n\n\n\n\n\nanova(model_rcbd_lme, type = \"sequential\")\n\n numDF denDF F-value p-value\n(Intercept) 1 123 2514.1283 <.0001\nvariety 41 123 2.4528 1e-04\n\n\n\n\n\n\n\n\n\n\n\nna.action = na.exclude\n\n\n\nYou may have noticed the final argument for na.action in the model statement:\nmodel_rcbd_lmer <- lmer(yield_bu_a ~ variety + (1|block),\n data = var_trial, \n na.action = na.exclude)\nThe argument na.action = na.exclude provides instructions for how to handle missing data. na.exclude removes the missing data points before proceeding with the analysis. When any obervation-levels model outputs is generated (e.g. predictions, residuals), they are padded in the appropriate place to account for missing data. This is handy because it makes it easier to add those results to the original data set if so desired.\nSince there are no missing data, this step was not strictly necessary, but it’s a good habit to be in.",
+ "text": "4.2 Example Analysis\nFirst, load the libraries for analysis and estimation:\n\nlme4nlme\n\n\n\nlibrary(lme4); library(lmerTest); library(emmeans)\nlibrary(dplyr); library(performance)\n\n\n\n\nlibrary(nlme); library(performance); library(emmeans)\nlibrary(dplyr)\n\n\n\n\nNext, let’s load some data. It is located here if you want to download it yourself (recommended).\nThis data set is for a single wheat variety trial conducted in Aberdeen, Idaho in 2015. The trial includes 4 blocks and 42 different treatments (wheat varieties in this case). This experiment consists of a series of plots (the experimental unit) laid out in a rectangular grid in a farm field. The goal of this analysis is the estimate the yield of each variety and the determine the rankings of each variety for the variable.\n\nvar_trial <- read.csv(here::here(\"data\", \"aberdeen2015.csv\"))\n\n\nTable of variables in the data set\n\n\n\n\n\n\nblock\nblocking unit\n\n\nrange\ncolumn position for each plot\n\n\nrow\nrow position for each plot\n\n\nvariety\ncrop variety (the treatment) being evaluated\n\n\nstand_pct\npercentage of the plot with actual plants growing in them\n\n\ndays_to_heading_julian\nJulian days (starting January 1st) until plot “headed” (first spike emerged)\n\n\nlodging\npercentage of plants in the plot that fell down and hence could not be harvested\n\n\nyield_bu_a\nyield (bushels per acre)\n\n\n\nThere are several variables present that are not useful for this analysis. The only thing we are concerned about is block, variety, yield_bu_a, and test_weight.\n\n4.2.1 Data integrity checks\nThe first thing is to make sure the data is what we expect. There are two steps:\n\nmake sure data are the expected data type\ncheck the extent of missing data\ninspect the independent variables and make sure the expected levels are present in the data\ninspect the dependent variable to ensure its distribution is following expectations\n\n\nstr(var_trial)\n\n'data.frame': 168 obs. of 10 variables:\n $ block : int 4 4 4 4 4 4 4 4 4 4 ...\n $ range : int 1 1 1 1 1 1 1 1 1 1 ...\n $ row : int 1 2 3 4 5 6 7 8 9 10 ...\n $ variety : chr \"DAS004\" \"Kaseberg\" \"Bruneau\" \"OR2090473\" ...\n $ stand_pct : int 100 98 96 100 98 100 100 100 99 100 ...\n $ days_to_heading_julian: int 149 146 149 146 146 151 145 145 146 146 ...\n $ height : int 39 35 33 31 33 44 30 36 36 29 ...\n $ lodging : int 0 0 0 0 0 0 0 0 0 0 ...\n $ yield_bu_a : num 128 130 119 115 141 ...\n $ test_weight : num 56.4 55 55.3 54.1 54.1 56.4 54.7 57.5 56.1 53.8 ...\n\n\nThese look okay except for block, which is currently coded as integer (numeric). We don’t want run a regression of block, where block 1 has twice the effect of block 2, and so on. So, converting it to a character will fix that. It can also be converted to a factor, but character variables are a bit easier to work with, and ultimately, equivalent to factor conversion\n\nvar_trial$block <- as.character(var_trial$block)\n\nNext, check the independent variables. Running a cross tabulations is often sufficient to ascertain this.\n\ntable(var_trial$variety, var_trial$block)\n\n \n 1 2 3 4\n 06-03303B 1 1 1 1\n Bobtail 1 1 1 1\n Brundage 1 1 1 1\n Bruneau 1 1 1 1\n DAS003 1 1 1 1\n DAS004 1 1 1 1\n Eltan 1 1 1 1\n IDN-01-10704A 1 1 1 1\n IDN-02-29001A 1 1 1 1\n IDO1004 1 1 1 1\n IDO1005 1 1 1 1\n Jasper 1 1 1 1\n Kaseberg 1 1 1 1\n LCS Artdeco 1 1 1 1\n LCS Biancor 1 1 1 1\n LCS Drive 1 1 1 1\n LOR-833 1 1 1 1\n LOR-913 1 1 1 1\n LOR-978 1 1 1 1\n Madsen 1 1 1 1\n Madsen / Eltan (50/50) 1 1 1 1\n Mary 1 1 1 1\n Norwest Duet 1 1 1 1\n Norwest Tandem 1 1 1 1\n OR2080637 1 1 1 1\n OR2080641 1 1 1 1\n OR2090473 1 1 1 1\n OR2100940 1 1 1 1\n Rosalyn 1 1 1 1\n Stephens 1 1 1 1\n SY Ovation 1 1 1 1\n SY 107 1 1 1 1\n SY Assure 1 1 1 1\n UI Castle CLP 1 1 1 1\n UI Magic CLP 1 1 1 1\n UI Palouse 1 1 1 1\n UI Sparrow 1 1 1 1\n UI-WSU Huffman 1 1 1 1\n WB 456 1 1 1 1\n WB 528 1 1 1 1\n WB1376 CLP 1 1 1 1\n WB1529 1 1 1 1\n\n\nThere are 42 varieties and there appears to be no mis-spellings among them that might confuse R into thinking varieties are different when they are actually the same. R is sensitive to case and white space, which can make it easy to create near duplicate treatments, such as “eltan” and “Eltan” and “Eltan”. There is no evidence of that in this data set. Additionally, it is perfectly balanced, with exactly one observation per treatment per rep. Please note that this does not tell us anything about the extent of missing data.\n\n\n\n\n\n\nMissing Data\n\n\n\nHere is a quick check to count the number of missing data in each column. This is not neededfor the data sets in this tutorial that have already been comprehensively examined, but it is helpful to check that the level of missingness displayed in an R session is what you expect.\n\napply(var_trial, 2, function(x) sum(is.na(x)))\n\n block range row \n 0 0 0 \n variety stand_pct days_to_heading_julian \n 0 0 0 \n height lodging yield_bu_a \n 0 0 0 \n test_weight \n 0 \n\n\nAlas, no missing data!\n\n\nIf there were independent variables with a continuous distribution (a covariate), plot those data.\nLast, check the dependent variable. A histogram is often quite sufficient to accomplish this. This is designed to be a quick check, so no need to spend time making the plot look good.\n\n\n\n\n\n\n\n\n\nFigure 4.1: Histogram of the dependent variable.\n\n\n\n\n\nhist(var_trial$yield_bu_a, main = \"\", xlab = \"yield\")\n\nThe range is roughly falling into the range we expect. We (the authors) know this from talking with the person who generated the data, not through our own intuition. There are mp large spikes of points at a single value (indicating something odd), nor are there any extreme values (low or high) that might indicate problems.\nData are not expected to be normally distributed at this point, so don’t bother running any Shapiro-Wilk tests. This histogram is a check to ensure the the data are entered correctly and they appear valid. It requires a mixture of domain knowledge and statistical training to know this, but over time, if you look at these plots with regularity, you will gain a feel for what your data should look like at this stage.\nThese are not complicated checks. They are designed to be done quickly and should be done for every analysis if you not previously already inspected the data as thus. We do this before every analysis and often discover surprising things! Best to discover these things early, since they are likely to impact the final analysis.\nThis data set is ready for analysis!\n\n\n4.2.2 Model Building\n\n\nRecall the model:\n\\[y_{ij} = \\mu + \\alpha_i + \\beta_j + \\epsilon_{ij}\\]\nFor this model, \\(\\alpha_i\\) is the variety effect (fixed) and \\(\\beta_j\\) is the block effect (random).\nHere is the R syntax for the RCBD statistical model:\n\nlme4nlme\n\n\n\nmodel_rcbd_lmer <- lmer(yield_bu_a ~ variety + (1|block),\n data = var_trial, \n na.action = na.exclude)\n\n\n\n\nmodel_rcbd_lme <- lme(yield_bu_a ~ variety,\n random = ~ 1|block,\n data = var_trial, \n na.action = na.exclude)\n\n\n\n\nThe parentheses are used to indicate that ‘block’ is a random effect, and this particular notation (1|block) indicates that a ‘random intercept’ model is being fit. This is the most common approach. It means there is one overall effect fit for each block.\nWe use the argument na.action = na.exclude as instruction for how to handle missing data: conduct the analysis, adjusting as needed for the missing data, and when prediction or residuals are output, please pad them in the appropriate places for missing data so they can be easily merged into the main data set if need be.\n\n\n4.2.3 Check Model Assumptions\n\n\nR syntax for checking model assumptions is the same for lme4 and nlme.\nRemember those iid assumptions? Let’s make sure we actually met them.\n\n4.2.3.1 Old Way\nThere are special plotting function written for lme4 and nlme objects (ie.plot(lmer_object)) for checking the homoscedasticity (constant variance).\n\n\n\n\n\n\n\n\n\nFigure 4.2: Plot of residuals versus fitted values\n\n\n\n\n\nplot(model_rcbd_lmer, resid(., scaled=TRUE) ~ fitted(.), \n xlab = \"fitted values\", ylab = \"studentized residuals\")\n\nWe are looking for a random and uniform distribution of points. This looks good!\nChecking normality requiring first extracting the model residuals with resid() and then generating a qq-plot and line.\n\n\n\n\n\n\n\n\n\nFigure 4.3: QQ-plot of residuals\n\n\n\n\n\nqqnorm(resid(model_rcbd_lmer), main = NULL); qqline(resid(model_rcbd_lmer))\n\nThis is reasonably good. Things do tend to fall apart at the tails a little, so this is not concerning.\n\n\n4.2.3.2 New Way\nNowadays, we can take advantage of the performance package, which provides a comprehensive suite of diagnostic plots.\n\n\nPlease look for check_model() in help tab to find what other checks you can perform using this function. If you would like to check all assumptions you can use the argument check = \"all\".\n\ncheck_model(model_rcbd_lmer, check = c('normality', 'linearity'))\n\n\n\n\n\n\n\n\n\n\n\n4.2.4 Inference\n\n\nR syntax for estimating model marginal means is the same for lme4 and nlme.\nEstimates for each treatment level can be obtained with the ‘emmeans’ package.\n\nrcbd_emm <- emmeans(model_rcbd_lmer, ~ variety)\nas.data.frame(rcbd_emm) %>% arrange(desc(emmean))\n\n variety emmean SE df lower.CL upper.CL\n Rosalyn 155.2703 7.212203 77.85 140.91149 169.6292\n IDO1005 153.5919 7.212203 77.85 139.23310 167.9508\n OR2080641 152.6942 7.212203 77.85 138.33536 167.0530\n Bobtail 151.6403 7.212203 77.85 137.28149 165.9992\n UI Sparrow 151.6013 7.212203 77.85 137.24245 165.9601\n Kaseberg 150.9768 7.212203 77.85 136.61794 165.3356\n IDN-01-10704A 148.9861 7.212203 77.85 134.62729 163.3450\n 06-03303B 148.8300 7.212203 77.85 134.47116 163.1888\n WB1529 148.2445 7.212203 77.85 133.88568 162.6034\n DAS003 145.2000 7.212203 77.85 130.84116 159.5588\n IDN-02-29001A 144.5755 7.212203 77.85 130.21665 158.9343\n Bruneau 143.9900 7.212203 77.85 129.63116 158.3488\n SY 107 143.6387 7.212203 77.85 129.27987 157.9975\n WB 528 142.9752 7.212203 77.85 128.61633 157.3340\n OR2080637 141.7652 7.212203 77.85 127.40633 156.1240\n Jasper 141.2968 7.212203 77.85 126.93794 155.6556\n UI Magic CLP 139.5403 7.212203 77.85 125.18149 153.8992\n Madsen 139.2671 7.212203 77.85 124.90826 153.6259\n LCS Biancor 139.1110 7.212203 77.85 124.75213 153.4698\n SY Ovation 138.6426 7.212203 77.85 124.28375 153.0014\n OR2090473 137.8229 7.212203 77.85 123.46407 152.1817\n Madsen / Eltan (50/50) 136.9642 7.212203 77.85 122.60536 151.3230\n UI-WSU Huffman 135.4810 7.212203 77.85 121.12213 149.8398\n Mary 134.8564 7.212203 77.85 120.49762 149.2153\n Norwest Tandem 134.3490 7.212203 77.85 119.99020 148.7079\n Brundage 134.0758 7.212203 77.85 119.71697 148.4346\n IDO1004 132.5145 7.212203 77.85 118.15568 146.8733\n DAS004 132.2413 7.212203 77.85 117.88245 146.6001\n Norwest Duet 132.0852 7.212203 77.85 117.72633 146.4440\n Eltan 131.4606 7.212203 77.85 117.10181 145.8195\n LCS Artdeco 130.8361 7.212203 77.85 116.47729 145.1950\n UI Palouse 130.4848 7.212203 77.85 116.12600 144.8437\n LOR-978 130.4458 7.212203 77.85 116.08697 144.8046\n LCS Drive 128.7674 7.212203 77.85 114.40858 143.1262\n Stephens 127.1671 7.212203 77.85 112.80826 141.5259\n OR2100940 126.1523 7.212203 77.85 111.79342 140.5111\n UI Castle CLP 125.5277 7.212203 77.85 111.16891 139.8866\n WB1376 CLP 123.6932 7.212203 77.85 109.33439 138.0521\n LOR-833 122.7565 7.212203 77.85 108.39762 137.1153\n LOR-913 118.7752 7.212203 77.85 104.41633 133.1340\n WB 456 118.4629 7.212203 77.85 104.10407 132.8217\n SY Assure 111.0468 7.212203 77.85 96.68794 125.4056\n\nDegrees-of-freedom method: kenward-roger \nConfidence level used: 0.95 \n\n\nThis table indicates the estimated marginal means (“emmeans”, sometimes called “least squares means”), the standard error (“SE”) of those means, the degrees of freedom and the upper and lower bounds of the 95% confidence interval. As an additional step, the emmeans were sorted from largest to smallest.\nAt this point, the analysis goals have been met: we know the estimated means for each treatment and their rankings.\nIf you want to run ANOVA, it can be done quite easily. By default, the Kenward-Rogers method of degrees of freedom approximation is used.\n\n\nThe Type I method is sometimes referred to as the “sequential” sum of squares, because it involves a process of adding terms to the model one at a time. Type I sum of squares is the default hypothesis testing method used by the anova() function. This only matters when a data set is unbalanced across treatments, either due to design or missing data points.\n\nlme4nlme\n\n\n\nanova(model_rcbd_lmer, type = \"1\")\n\nType I Analysis of Variance Table with Satterthwaite's method\n Sum Sq Mean Sq NumDF DenDF F value Pr(>F) \nvariety 18354 447.65 41 123 2.4528 8.017e-05 ***\n---\nSignif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n\n\n\n\n\nanova(model_rcbd_lme, type = \"sequential\")\n\n numDF denDF F-value p-value\n(Intercept) 1 123 2514.1283 <.0001\nvariety 41 123 2.4528 1e-04\n\n\n\n\n\n\n\n\n\n\n\nna.action = na.exclude\n\n\n\nYou may have noticed the final argument for na.action in the model statement:\nmodel_rcbd_lmer <- lmer(yield_bu_a ~ variety + (1|block),\n data = var_trial, \n na.action = na.exclude)\nThe argument na.action = na.exclude provides instructions for how to handle missing data. na.exclude removes the missing data points before proceeding with the analysis. When any obervation-levels model outputs is generated (e.g. predictions, residuals), they are padded in the appropriate place to account for missing data. This is handy because it makes it easier to add those results to the original data set if so desired.\nSince there are no missing data, this step was not strictly necessary, but it’s a good habit to be in.",
"crumbs": [
"Experiment designs",
"4 Randomized Complete Block Design"
@@ -454,7 +454,7 @@
"href": "chapters/variance-components.html",
"title": "13 Variance & Variance Components",
"section": "",
- "text": "13.1 Unequal Variance\nMixed models provide the advantage of being able to estimate the variance of random variables. The decision of how to assign",
+ "text": "13.1 Unequal Variance\nMixed models provide the advantage of being able to estimate the variance of random variables. Instead of looking at a variable as a collection of specific levels to estimate, random effects view variables as being a random drawn from a normal distribution with a standard deviation. The decision of how to designate a variable as random or fixed depends on",
"crumbs": [
"13 Variance and Variance Components"
]