Chapter 11 talks about the problem of multiple tests. These examples will be fairly short but will highlight a few ways to adjust for multiple tests.
tidyverse
package and the educ7610
package.library(tidyverse)
library(educ7610)
gss
into R.data(gss)
income06
). Our first adjustment is using Bonferroni’s adjustment. To do so, we will grab the p-values from the summary()
and feed that to p.adjust()
.fit <- gss %>%
lm(income06 ~ degree + race + educ, data = .)
mt_adjust(fit, "bonferroni")
## Estimate Std..Error t.value Pr...t..
## (Intercept) 7885.8124 4776.5311 1.650950 7.914133e-01
## degreeAssociates 29800.3711 4442.5264 6.707978 2.118946e-10
## degreeBachelors 45455.5562 4227.3359 10.752766 2.912338e-25
## degreeGraduate 57663.9272 5123.0925 11.255687 1.573189e-27
## degreeHS 20296.5103 3067.3139 6.617031 3.875839e-10
## raceOther 13466.5777 4026.0555 3.344856 6.723918e-03
## raceWhite 16327.4677 2740.7819 5.957230 2.471381e-08
## educ 706.0966 366.1085 1.928654 4.314717e-01
"fdr"
(the false discovery rate). This approach controls the number of false discoveries (as the name inplies). This approach is less conservative than Bonferroni’s.mt_adjust(fit, "fdr")
## Estimate Std..Error t.value Pr...t..
## (Intercept) 7885.8124 4776.5311 1.650950 9.892666e-02
## degreeAssociates 29800.3711 4442.5264 6.707978 7.063154e-11
## degreeBachelors 45455.5562 4227.3359 10.752766 1.456169e-25
## degreeGraduate 57663.9272 5123.0925 11.255687 1.573189e-27
## degreeHS 20296.5103 3067.3139 6.617031 9.689598e-11
## raceOther 13466.5777 4026.0555 3.344856 1.120653e-03
## raceWhite 16327.4677 2740.7819 5.957230 4.942763e-09
## educ 706.0966 366.1085 1.928654 6.163882e-02
Adjusting for multiple comparisons is often an important consideration. In R, there are several approaches that can be quickly applied. Although important to adjust, possibly more important is the consideration of which tests to run and the likelihood of the effect prior to making any tests.