3
3Test for Equality of Covariance (MCAR)
3.0.1 MissMech::TestMCARNormality()
The main purpose of this package is to test whether the missing data mechanism, for an incompletely observed data set, is one of missing completely at random (MCAR).
As a by product, however, this package has the capabilities of : * imputing incomplete data * performing a test to determine whether data have a multivariate normal distribution * performing a test of equality of covariances for groups * obtaining normal-theory maximum likelihood estimates for mean and covariance when data are incomplete.
The test of MCAR follows the methodology proposed by Jamshidian and Jalal (2010). It is based on testing equality of covariances between groups having identical missing data patterns.
The data are imputed, using two options of normality and distribution free, and the test of equality of covariances between groups with identical missing data patterns is performed also with options of assuming normality (Hawkins test) or non-parametrically.
Users can optionally use their own method of data imputation as well.
Multiple imputation is an additional feature of the program that can be used as a diagnostic tool to help identify cases or variables that contribute to rejection of MCAR, when the MCAR test is rejected (See Jamshidian and Jalal, 2010 for details).
As explained in Jamshidian, Jalal, and Jansen (2014), this package can also be used for imputing missing data, test of multivariate normality, and test of equality of covariances between several groups when data are completely observed.
Jamshidian, M. and Bentler, P. M. (1999). “ML estimation of mean and covariance structures with missing data using complete data routines.” Journal of Educational and Behavioral Statistics, 24, 21-41.
Jamshidian, M. and Jalal, S. (2010). “Tests of homoscedasticity, normality, and missing at random for incomplete multivariate data,” Psychometrika, 75, 649-674.
Jamshidian, M. Jalal, S., and Jansen, C. (2014). “ MissMech: An R Package for Testing Homoscedasticity, Multivariate Normality, and Missing Completely at Random (MCAR),” Journal of Statistical Software, 56(6), 1-31.
colon_s %>%
dplyr::select(age, sex.factor, nodes, extent.factor, perfor.factor,
mort_5yr, smoking_mcar, smoking_mar) %>%
dplyr::mutate_all(as.numeric) %>% # categorical variables must be changed to numeric
MissMech::TestMCARNormality()## Call:
## MissMech::TestMCARNormality(data = .)
##
## Number of Patterns: 6
##
## Total number of cases used in the analysis: 918
##
## Pattern(s) used:
## age sex.factor nodes extent.factor perfor.factor mort_5yr
## group.1 1 1 1 1 1 1
## group.2 1 1 1 1 1 1
## group.3 1 1 1 1 1 1
## group.4 1 1 NA 1 1 1
## group.5 1 1 1 1 1 NA
## group.6 1 1 1 1 1 1
## smoking_mcar smoking_mar Number of cases
## group.1 1 1 604
## group.2 1 NA 199
## group.3 NA 1 74
## group.4 1 1 11
## group.5 1 1 10
## group.6 NA NA 20
##
##
## Test of normality and Homoscedasticity:
## -------------------------------------------
##
## Hawkins Test:
##
## P-value for the Hawkins test of normality and homoscedasticity: 1.937163e-34
##
## Either the test of multivariate normality or homoscedasticity (or both) is rejected.
## Provided that normality can be assumed, the hypothesis of MCAR is
## rejected at 0.05 significance level.
##
## Non-Parametric Test:
##
## P-value for the non-parametric test of homoscedasticity: 1.789557e-08
##
## Hypothesis of MCAR is rejected at 0.05 significance level.
## The multivariate normality test is inconclusive.Conclusion: This provides evidence that missing data on the variables included in this test are NOT missing completely at random MCAR by Hawkins test, \(p < .001\). Note, the non-parametric test is also able to reach this conclusion, \(p < .001\).
3.0.2 Little’s Test, revised ‘LittleMCAR’
“R-function for Little’s test for data missing completely at random” by Eric Stemmler (2020/08/14)
Little’s test basically compares the estimated means of each variable between the different missing patterns. These leads to either the decision to use simple random imputation or to model the missingness mechanism and use that model for imputation. The former is appropriate if one can be certain that the means are not different for different missingness patterns, i.e. a non-significant result of the Little’s test. For a more detailed explanation about the different types of missingness (see Gelman and Hill, 2006).
Little himself “… expect[s] the test to be sensitive to departures from the normality assumption, and even under normality the asymptotic null distribution seems unlikely to be reliable unless the sample size is large.” (Little 1988).
Gelman, Andrew, and Jennifer Hill. 2006. Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge university press.
Little, Roderick JA. 1988. “A Test of Missing Completely at Random for Multivariate Data with Missing Values.” Journal of the American Statistical Association 83 (404). Taylor & Francis: 1198–1202.
Statistical inference based on incomplete data typically involves certain assumptions for the missing data mechanism. The validity of these assumptions requires formal evaluation before any further analysis.
For example, likelihood based inference is valid only if the missing data mechanism is ignorable (Rubin 1976), which usually relies on the missing at random assumption (MAR). MAR assumes that the missingness of the data may depend on the observed data, but is independent of the unobserved data. Therefore testing MAR is in general impossible since it requires unavailable information about the missing data.
Instead, the missing completely at random assumption (MCAR) assumes that the missingness of the data is independent of both the observed and unobserved data, which is stronger than MAR and possible to test using only the observed data.
When missing data mechanism depends on the unobserved data, data are missing not at random (MNAR). Although the likelihood inference only requires the MAR assumption, testing of MCAR is still of interest in real applications, since many simple missing data methods such as complete case analysis are valid only under MCAR (Chapter 3 of Little and Rubin 1987, also see the blood test example in Section 4). Also the maximum likelihood estimation for the multivariate normal model may be more sensitive to the distributional assumption when the data are not MCAR (Little 1988).
GitHub repository: https://github.com/rcst/little-test
devtools::source_url("https://raw.githubusercontent.com/rcst/little-test/master/mcar.R")## SHA-1 hash of file is 294bccdf1f2f0f3c5589b0cbcbb428aab2dcd6d8littletest_mcar <- colon_s %>%
dplyr::select(age, sex.factor, nodes, extent.factor, perfor.factor, obstruct.factor,
mort_5yr, smoking_mcar, smoking_mar,) %>%
mcar()## Iterations of EM:
## 1...2...3...4...5...6...7...
## this could take a whilelittletest_mcar$missing.patterns## [1] 14littletest_mcar$amount.missing## age sex.factor nodes extent.factor perfor.factor
## Number Missing 0 0 18.00000000 0 0
## Percent Missing 0 0 0.01937567 0 0
## obstruct.factor mort_5yr smoking_mcar smoking_mar
## Number Missing 21.00000000 14.00000000 98.0000000 227.0000000
## Percent Missing 0.02260495 0.01506997 0.1054898 0.2443488littletest_mcar$chi.square## [1] 171.9229littletest_mcar$df## [1] 95littletest_mcar$p.value## [1] 2.309862e-06Conclusion: This provides evidence that missing data on the variables included in this test are NOT missing completely at random MCAR), \(\chi^2(95) = 172.92, p < .001\). Note, this test does not tell you WHICH variables are asociated.
3.1 Test Associations (MAR)
3.1.1 finalfit::missing_compare()
colon_s %>%
finalfit::missing_compare(explanatory = c("age", "sex.factor", "nodes", "extent.factor", "perfor.factor",
"smoking_mcar", "smoking_mar"),
dependent = "mort_5yr")## Missing data analysis: Mortality 5 year Not missing
## Age (years) Mean (SD) 59.8 (11.9)
## Sex Female 437 (98.2)
## Male 478 (98.8)
## nodes Mean (SD) 3.7 (3.6)
## Extent of spread Submucosa 20 (95.2)
## Muscle 103 (97.2)
## Serosa 750 (98.8)
## Adjacent structures 42 (97.7)
## Perforation No 888 (98.4)
## Yes 27 (100.0)
## smoking_mcar Non-smoker 636 (98.6)
## Smoker 183 (98.4)
## smoking_mar Non-smoker 565 (98.6)
## Smoker 125 (96.9)
## Missing p
## 53.9 (12.7) 0.066
## 8 (1.8) 0.669
## 6 (1.2)
## 2.9 (2.8) 0.441
## 1 (4.8) 0.325
## 3 (2.8)
## 9 (1.2)
## 1 (2.3)
## 14 (1.6) 1.000
## 0 (0.0)
## 9 (1.4) 1.000
## 3 (1.6)
## 8 (1.4) 0.330
## 4 (3.1)Conclusion: There is no evidence that missingness in the five year mortality is associated with any of the variables: age, sex, extent of spread, perforation, and smoking (either varaible), \(p's > .066\).
colon_s %>%
finalfit::missing_compare(explanatory = c("age", "sex.factor", "nodes", "extent.factor", "perfor.factor",
"mort_5yr", "smoking_mar"),
dependent = "smoking_mcar")## Missing data analysis: smoking_mcar Not missing
## Age (years) Mean (SD) 59.7 (12.0)
## Sex Female 400 (89.9)
## Male 431 (89.0)
## nodes Mean (SD) 3.7 (3.6)
## Extent of spread Submucosa 19 (90.5)
## Muscle 92 (86.8)
## Serosa 682 (89.9)
## Adjacent structures 38 (88.4)
## Perforation No 806 (89.4)
## Yes 25 (92.6)
## Mortality 5 year Alive 456 (89.2)
## Died 363 (89.9)
## smoking_mar Non-smoker 510 (89.0)
## Smoker 115 (89.1)
## Missing p
## 60.1 (11.8) 0.768
## 45 (10.1) 0.758
## 53 (11.0)
## 3.6 (3.7) 0.920
## 2 (9.5) 0.801
## 14 (13.2)
## 77 (10.1)
## 5 (11.6)
## 96 (10.6) 0.825
## 2 (7.4)
## 55 (10.8) 0.847
## 41 (10.1)
## 63 (11.0) 1.000
## 14 (10.9)Conclusion: There is no evidence that missingness in the smoking variable (MCAR version) is associated with any of the variables: age, sex, extent of spread, and perforation, \(p's > .758\).
colon_s %>%
finalfit::missing_compare(explanatory = c("age", "sex.factor", "nodes", "extent.factor", "perfor.factor",
"mort_5yr", "smoking_mcar"),
dependent = "smoking_mar")## Missing data analysis: smoking_mar Not missing Missing
## Age (years) Mean (SD) 59.8 (11.8) 59.7 (12.4)
## Sex Female 276 (62.0) 169 (38.0)
## Male 426 (88.0) 58 (12.0)
## nodes Mean (SD) 3.5 (3.2) 4.2 (4.6)
## Extent of spread Submucosa 16 (76.2) 5 (23.8)
## Muscle 76 (71.7) 30 (28.3)
## Serosa 577 (76.0) 182 (24.0)
## Adjacent structures 33 (76.7) 10 (23.3)
## Perforation No 684 (75.8) 218 (24.2)
## Yes 18 (66.7) 9 (33.3)
## Mortality 5 year Alive 390 (76.3) 121 (23.7)
## Died 300 (74.3) 104 (25.7)
## smoking_mcar Non-smoker 490 (76.0) 155 (24.0)
## Smoker 135 (72.6) 51 (27.4)
## p
## 0.897
## <0.001
##
## 0.007
## 0.806
##
##
##
## 0.387
##
## 0.521
##
## 0.397
## Conclusion: There is evidence that missingness in the smoking variable (MAR version) is associated with sex, \(p < .001\). We can see there there is a much higher percent of missing values for females (38%) compared to males (12%).