```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```
## Introduction
Chapter 11 talks about the problem of multiple tests. These examples will be fairly short but will highlight a few ways to adjust for multiple tests.
## Adjusting for Multiple Tests
1. Let's start by loading the `tidyverse` package and the `rlm` package.
```{r}
library(tidyverse)
library(rlm)
```
2. Import `gss` into R.
```{r, eval = FALSE}
data(gss)
```
3. Let's do a multiple with degree, race, and education predicting income (`income06`). Our first adjustment is using Bonferroni's adjustment. To do so, we will grab the p-values from the `summary()` and feed that to `p.adjust()`.
```{r}
fit <- gss %>%
lm(income06 ~ degree + race + educ, data = .) %>%
summary()
p_bonf <- fit %>%
coef() %>%
data.frame %>%
.[[4]] %>%
p.adjust("bonferroni")
fit %>%
coef() %>%
data.frame %>%
.[,1:3] %>%
cbind(p_bonf)
```
4. Are there significant mean differences between the non-reference levels and the reference level regarding income for either degree or race now that we have adjusted for multiple tests? What about the effect of education?
5. Let's try a different approach; namely, let's try `"fdr"` (the false discovery rate). This approach controls the number of false discoveries (as the name inplies). This approach is less conservative than Bonferroni's.
```{r}
p_fdr <- fit %>%
coef() %>%
data.frame %>%
.[[4]] %>%
p.adjust("fdr")
fit %>%
coef() %>%
data.frame %>%
.[,1:3] %>%
cbind(p_fdr)
```
6. Do any conclusions change based on using the false discovery rate rather than Bonferroni's? Which change? Why?
## Conclusion
Adjusting for multiple comparisons is often an important consideration. In R, there are several approaches that can be quickly applied. Although important to adjust, possibly more important is the consideration of which tests to run and the likelihood of the effect prior to making any tests.