Exercise 2

This exercise requires using multipleGroup() to estimate multiple-group models, and includes one example of how to perform a search for DIF. The data is available in the Exercise_02.Rdata file.

Load the dataset into R using the load() function, and inspect the defined objects. The object named data represents a 20-item dichtomously scored test. The group object consists of two groups: a set of individuals that had been trained in general test taking skills (with_training), and a set of individuals without any training (without_training).

  1. Explore the dataset, and determine the dimensionality with mirt() (could be different across groups, but assume that it isn’t for now).
library(mirt)
## Loading required package: stats4
## Loading required package: lattice
load('Exercise_02.Rdata')
xtabs(~group)
## group
## without_training    with_training 
##             1000             1000
head(data)
##      Item_1 Item_2 Item_3 Item_4 Item_5 Item_6 Item_7 Item_8 Item_9
## [1,]      0      0      0      1      0      0      1      1      0
## [2,]      1      1      0      0      0      0      1      1      1
## [3,]      1      1      1      0      1      1      1      1      1
## [4,]      0      0      0      0      0      0      0      1      0
## [5,]      1      0      1      0      1      1      1      1      1
## [6,]      1      0      0      0      1      0      1      1      1
##      Item_10 Item_11 Item_12 Item_13 Item_14 Item_15 Item_16 Item_17
## [1,]       1       0       0       0       0       0       1       0
## [2,]       0       1       1       1       0       1       1       1
## [3,]       0       1       1       1       1       0       1       1
## [4,]       0       0       0       0       0       0       0       0
## [5,]       1       1       0       1       0       0       0       1
## [6,]       0       1       1       0       1       1       1       1
##      Item_18 Item_19 Item_20
## [1,]       1       0       0
## [2,]       0       0       1
## [3,]       1       0       1
## [4,]       0       0       0
## [5,]       1       1       1
## [6,]       1       1       1
mod1 <- mirt(data, 1)
## 
Iteration: 1, Log-Lik: -23087.201, Max-Change: 0.71338
Iteration: 2, Log-Lik: -22600.090, Max-Change: 0.34008
Iteration: 3, Log-Lik: -22500.754, Max-Change: 0.29897
Iteration: 4, Log-Lik: -22459.606, Max-Change: 0.16363
Iteration: 5, Log-Lik: -22441.177, Max-Change: 0.11083
Iteration: 6, Log-Lik: -22432.080, Max-Change: 0.07520
Iteration: 7, Log-Lik: -22427.936, Max-Change: 0.05935
Iteration: 8, Log-Lik: -22425.843, Max-Change: 0.03423
Iteration: 9, Log-Lik: -22424.859, Max-Change: 0.03275
Iteration: 10, Log-Lik: -22424.215, Max-Change: 0.01310
Iteration: 11, Log-Lik: -22424.006, Max-Change: 0.00985
Iteration: 12, Log-Lik: -22423.885, Max-Change: 0.00711
Iteration: 13, Log-Lik: -22423.722, Max-Change: 0.00253
Iteration: 14, Log-Lik: -22423.702, Max-Change: 0.00192
Iteration: 15, Log-Lik: -22423.687, Max-Change: 0.00187
Iteration: 16, Log-Lik: -22423.634, Max-Change: 0.00137
Iteration: 17, Log-Lik: -22423.632, Max-Change: 0.00086
Iteration: 18, Log-Lik: -22423.631, Max-Change: 0.00083
Iteration: 19, Log-Lik: -22423.624, Max-Change: 0.00033
Iteration: 20, Log-Lik: -22423.624, Max-Change: 0.00017
Iteration: 21, Log-Lik: -22423.624, Max-Change: 0.00017
Iteration: 22, Log-Lik: -22423.623, Max-Change: 0.00020
Iteration: 23, Log-Lik: -22423.623, Max-Change: 0.00017
Iteration: 24, Log-Lik: -22423.623, Max-Change: 0.00015
Iteration: 25, Log-Lik: -22423.622, Max-Change: 0.00008
mod2 <- mirt(data, 2, verbose=FALSE)
## EM cycles terminated after 500 iterations.
anova(mod1, mod2) #unidimensional appears to be fine
## 
## Model 1: mirt(data = data, model = 1)
## Model 2: mirt(data = data, model = 2, verbose = FALSE)
##        AIC     AICc    SABIC      BIC    logLik     X2  df      p
## 1 44927.24 44928.92 45024.20 45151.28 -22423.62    NaN NaN    NaN
## 2 44934.98 44938.63 45077.98 45265.43 -22408.49 30.267  19 0.0485
  1. Use itemfit() to determine if there are any aberrant items in the model according to the S-X2 statistic. Use a false-discovery rate adjustment, such as the Benjamini and Hochberg adjustment, to control for multiple comparisons when inspecting the \(p\)-values (found in the p.adjust() function).
fit <- itemfit(mod1)
fit
##       item    S_X2 df.S_X2 p.S_X2
## 1   Item_1 21.7993      17 0.1926
## 2   Item_2 38.5962      17 0.0020
## 3   Item_3 29.9362      17 0.0268
## 4   Item_4 10.4637      16 0.8413
## 5   Item_5 13.5965      16 0.6287
## 6   Item_6 16.7100      17 0.4742
## 7   Item_7 21.2008      16 0.1709
## 8   Item_8 12.9224      17 0.7414
## 9   Item_9 18.8849      17 0.3352
## 10 Item_10 21.4008      16 0.1636
## 11 Item_11 11.7612      16 0.7602
## 12 Item_12 15.7208      16 0.4726
## 13 Item_13 15.4830      17 0.5607
## 14 Item_14 17.8869      17 0.3960
## 15 Item_15  6.3627      16 0.9837
## 16 Item_16 23.8454      16 0.0929
## 17 Item_17 14.9842      17 0.5966
## 18 Item_18 19.4301      17 0.3044
## 19 Item_19 14.5314      17 0.6292
## 20 Item_20  9.4825      17 0.9239
#adjust the S-X2 values for false-discovery
p.adjust(fit$p.S_X2, method = 'fdr')
##  [1] 0.6420000 0.0400000 0.2680000 0.9347778 0.8389333 0.8389333 0.6420000
##  [8] 0.8943529 0.8380000 0.6420000 0.8943529 0.8389333 0.8389333 0.8389333
## [15] 0.9837000 0.6193333 0.8389333 0.8380000 0.8389333 0.9725263
  1. Fit a completely independent multiple group model to the data with multipleGroup(), as well as the completely equal model (i.e., ignoring group membership/contraining all parameters to be equal across groups), and view the estimated coefficients. Plot the test-score and test-information functions (type = 'score' and type = 'info') for the completely independent model, and plot a few items tracelines with itemplot().
#completely independent
mod.ind <- multipleGroup(data, 1, group=group)
## 
Iteration: 1, Log-Lik: -23087.201, Max-Change: 0.93446
Iteration: 2, Log-Lik: -22437.665, Max-Change: 0.40803
Iteration: 3, Log-Lik: -22300.535, Max-Change: 0.24414
Iteration: 4, Log-Lik: -22238.576, Max-Change: 0.15137
Iteration: 5, Log-Lik: -22212.513, Max-Change: 0.13477
Iteration: 6, Log-Lik: -22196.281, Max-Change: 0.08028
Iteration: 7, Log-Lik: -22185.792, Max-Change: 0.06319
Iteration: 8, Log-Lik: -22179.020, Max-Change: 0.05588
Iteration: 9, Log-Lik: -22174.503, Max-Change: 0.06322
Iteration: 10, Log-Lik: -22166.680, Max-Change: 0.04069
Iteration: 11, Log-Lik: -22164.831, Max-Change: 0.02950
Iteration: 12, Log-Lik: -22163.929, Max-Change: 0.02025
Iteration: 13, Log-Lik: -22162.722, Max-Change: 0.01888
Iteration: 14, Log-Lik: -22162.267, Max-Change: 0.01677
Iteration: 15, Log-Lik: -22161.903, Max-Change: 0.01456
Iteration: 16, Log-Lik: -22160.674, Max-Change: 0.00801
Iteration: 17, Log-Lik: -22160.605, Max-Change: 0.00482
Iteration: 18, Log-Lik: -22160.565, Max-Change: 0.00544
Iteration: 19, Log-Lik: -22160.485, Max-Change: 0.00306
Iteration: 20, Log-Lik: -22160.470, Max-Change: 0.00314
Iteration: 21, Log-Lik: -22160.457, Max-Change: 0.00277
Iteration: 22, Log-Lik: -22160.406, Max-Change: 0.00697
Iteration: 23, Log-Lik: -22160.399, Max-Change: 0.00196
Iteration: 24, Log-Lik: -22160.395, Max-Change: 0.00166
Iteration: 25, Log-Lik: -22160.393, Max-Change: 0.00123
Iteration: 26, Log-Lik: -22160.391, Max-Change: 0.00088
Iteration: 27, Log-Lik: -22160.390, Max-Change: 0.00085
Iteration: 28, Log-Lik: -22160.387, Max-Change: 0.00243
Iteration: 29, Log-Lik: -22160.386, Max-Change: 0.00165
Iteration: 30, Log-Lik: -22160.386, Max-Change: 0.00144
Iteration: 31, Log-Lik: -22160.386, Max-Change: 0.00038
Iteration: 32, Log-Lik: -22160.386, Max-Change: 0.00034
Iteration: 33, Log-Lik: -22160.385, Max-Change: 0.00030
Iteration: 34, Log-Lik: -22160.385, Max-Change: 0.00032
Iteration: 35, Log-Lik: -22160.385, Max-Change: 0.00031
Iteration: 36, Log-Lik: -22160.385, Max-Change: 0.00018
Iteration: 37, Log-Lik: -22160.385, Max-Change: 0.00012
Iteration: 38, Log-Lik: -22160.385, Max-Change: 0.00011
Iteration: 39, Log-Lik: -22160.385, Max-Change: 0.00010
coef(mod.ind, simplify=TRUE)
## $without_training
## $without_training$items
##            a1      d g u
## Item_1  0.763  0.070 0 1
## Item_2  1.292 -0.483 0 1
## Item_3  0.833 -0.002 0 1
## Item_4  1.342 -0.430 0 1
## Item_5  1.659 -0.313 0 1
## Item_6  0.840 -0.247 0 1
## Item_7  1.616  0.576 0 1
## Item_8  1.160  0.398 0 1
## Item_9  0.995 -0.126 0 1
## Item_10 1.367 -0.606 0 1
## Item_11 1.884  0.512 0 1
## Item_12 1.882 -0.722 0 1
## Item_13 0.836 -0.541 0 1
## Item_14 1.130 -0.318 0 1
## Item_15 1.188 -0.485 0 1
## Item_16 1.320 -0.400 0 1
## Item_17 1.055  0.577 0 1
## Item_18 0.703 -0.065 0 1
## Item_19 1.122 -0.332 0 1
## Item_20 0.665  0.311 0 1
## 
## $without_training$means
## F1 
##  0 
## 
## $without_training$cov
##    F1
## F1  1
## 
## 
## $with_training
## $with_training$items
##            a1     d g u
## Item_1  1.157 0.855 0 1
## Item_2  0.745 0.221 0 1
## Item_3  2.162 1.509 0 1
## Item_4  2.146 1.143 0 1
## Item_5  2.386 1.472 0 1
## Item_6  1.072 0.666 0 1
## Item_7  2.407 2.553 0 1
## Item_8  1.597 1.608 0 1
## Item_9  1.539 0.982 0 1
## Item_10 2.437 0.990 0 1
## Item_11 2.535 2.394 0 1
## Item_12 2.921 1.461 0 1
## Item_13 1.280 0.290 0 1
## Item_14 1.557 0.683 0 1
## Item_15 1.928 0.972 0 1
## Item_16 1.844 0.975 0 1
## Item_17 1.487 1.805 0 1
## Item_18 0.915 0.536 0 1
## Item_19 1.752 1.759 0 1
## Item_20 1.266 1.267 0 1
## 
## $with_training$means
## F1 
##  0 
## 
## $with_training$cov
##    F1
## F1  1
#completely equal
mod.equal <- multipleGroup(data, 1, group=group, invariance = c('slopes', 'intercepts'))
## 
Iteration: 1, Log-Lik: -23087.201, Max-Change: 0.71338
Iteration: 2, Log-Lik: -22600.090, Max-Change: 0.34008
Iteration: 3, Log-Lik: -22500.754, Max-Change: 0.29897
Iteration: 4, Log-Lik: -22459.606, Max-Change: 0.16363
Iteration: 5, Log-Lik: -22441.177, Max-Change: 0.11083
Iteration: 6, Log-Lik: -22432.080, Max-Change: 0.07520
Iteration: 7, Log-Lik: -22427.936, Max-Change: 0.05935
Iteration: 8, Log-Lik: -22425.843, Max-Change: 0.03423
Iteration: 9, Log-Lik: -22424.859, Max-Change: 0.03275
Iteration: 10, Log-Lik: -22424.215, Max-Change: 0.01310
Iteration: 11, Log-Lik: -22424.006, Max-Change: 0.00985
Iteration: 12, Log-Lik: -22423.885, Max-Change: 0.00711
Iteration: 13, Log-Lik: -22423.722, Max-Change: 0.00253
Iteration: 14, Log-Lik: -22423.702, Max-Change: 0.00192
Iteration: 15, Log-Lik: -22423.687, Max-Change: 0.00187
Iteration: 16, Log-Lik: -22423.634, Max-Change: 0.00137
Iteration: 17, Log-Lik: -22423.632, Max-Change: 0.00086
Iteration: 18, Log-Lik: -22423.631, Max-Change: 0.00083
Iteration: 19, Log-Lik: -22423.624, Max-Change: 0.00033
Iteration: 20, Log-Lik: -22423.624, Max-Change: 0.00017
Iteration: 21, Log-Lik: -22423.624, Max-Change: 0.00017
Iteration: 22, Log-Lik: -22423.623, Max-Change: 0.00020
Iteration: 23, Log-Lik: -22423.623, Max-Change: 0.00017
Iteration: 24, Log-Lik: -22423.623, Max-Change: 0.00015
Iteration: 25, Log-Lik: -22423.622, Max-Change: 0.00008
coef(mod.equal, simplify=TRUE) 
## $without_training
## $without_training$items
##            a1      d g u
## Item_1  1.027  0.448 0 1
## Item_2  0.994 -0.095 0 1
## Item_3  1.501  0.612 0 1
## Item_4  1.860  0.291 0 1
## Item_5  2.184  0.528 0 1
## Item_6  1.062  0.203 0 1
## Item_7  2.153  1.480 0 1
## Item_8  1.485  0.980 0 1
## Item_9  1.356  0.396 0 1
## Item_10 1.998  0.103 0 1
## Item_11 2.354  1.407 0 1
## Item_12 2.595  0.273 0 1
## Item_13 1.142 -0.142 0 1
## Item_14 1.419  0.167 0 1
## Item_15 1.693  0.187 0 1
## Item_16 1.715  0.259 0 1
## Item_17 1.386  1.165 0 1
## Item_18 0.860  0.232 0 1
## Item_19 1.659  0.617 0 1
## Item_20 1.036  0.752 0 1
## 
## $without_training$means
## F1 
##  0 
## 
## $without_training$cov
##    F1
## F1  1
## 
## 
## $with_training
## $with_training$items
##            a1      d g u
## Item_1  1.027  0.448 0 1
## Item_2  0.994 -0.095 0 1
## Item_3  1.501  0.612 0 1
## Item_4  1.860  0.291 0 1
## Item_5  2.184  0.528 0 1
## Item_6  1.062  0.203 0 1
## Item_7  2.153  1.480 0 1
## Item_8  1.485  0.980 0 1
## Item_9  1.356  0.396 0 1
## Item_10 1.998  0.103 0 1
## Item_11 2.354  1.407 0 1
## Item_12 2.595  0.273 0 1
## Item_13 1.142 -0.142 0 1
## Item_14 1.419  0.167 0 1
## Item_15 1.693  0.187 0 1
## Item_16 1.715  0.259 0 1
## Item_17 1.386  1.165 0 1
## Item_18 0.860  0.232 0 1
## Item_19 1.659  0.617 0 1
## Item_20 1.036  0.752 0 1
## 
## $with_training$means
## F1 
##  0 
## 
## $with_training$cov
##    F1
## F1  1
plot(mod.ind)

plot(mod.ind, type = 'info')

itemplot(mod.ind, 1)

itemplot(mod.ind, 2)

itemplot(mod.ind, 3)

  1. The items appear to be behaving differently for each group in the completely independent model. This is an indication that there are either population difference between the groups, the items are showing DIF, or both.
mod.freegroup <- multipleGroup(data, 1, group=group, 
                      invariance = c('intercepts','slopes', 'free_means', 'free_varcov'))
## 
Iteration: 1, Log-Lik: -23087.201, Max-Change: 0.70187
Iteration: 2, Log-Lik: -22483.951, Max-Change: 0.35565
Iteration: 3, Log-Lik: -22393.077, Max-Change: 0.21786
Iteration: 4, Log-Lik: -22350.901, Max-Change: 0.12034
Iteration: 5, Log-Lik: -22328.549, Max-Change: 0.08712
Iteration: 6, Log-Lik: -22316.395, Max-Change: 0.06607
Iteration: 7, Log-Lik: -22307.995, Max-Change: 0.04264
Iteration: 8, Log-Lik: -22302.204, Max-Change: 0.03669
Iteration: 9, Log-Lik: -22297.842, Max-Change: 0.03851
Iteration: 10, Log-Lik: -22291.973, Max-Change: 0.02951
Iteration: 11, Log-Lik: -22289.196, Max-Change: 0.03846
Iteration: 12, Log-Lik: -22286.826, Max-Change: 0.03029
Iteration: 13, Log-Lik: -22282.645, Max-Change: 0.02520
Iteration: 14, Log-Lik: -22280.793, Max-Change: 0.02810
Iteration: 15, Log-Lik: -22279.373, Max-Change: 0.02142
Iteration: 16, Log-Lik: -22277.138, Max-Change: 0.02206
Iteration: 17, Log-Lik: -22276.114, Max-Change: 0.02068
Iteration: 18, Log-Lik: -22275.200, Max-Change: 0.01632
Iteration: 19, Log-Lik: -22273.469, Max-Change: 0.01688
Iteration: 20, Log-Lik: -22272.840, Max-Change: 0.01718
Iteration: 21, Log-Lik: -22272.281, Max-Change: 0.01363
Iteration: 22, Log-Lik: -22271.272, Max-Change: 0.01290
Iteration: 23, Log-Lik: -22270.878, Max-Change: 0.00964
Iteration: 24, Log-Lik: -22270.574, Max-Change: 0.00952
Iteration: 25, Log-Lik: -22269.459, Max-Change: 0.02240
Iteration: 26, Log-Lik: -22269.112, Max-Change: 0.00654
Iteration: 27, Log-Lik: -22268.976, Max-Change: 0.00525
Iteration: 28, Log-Lik: -22268.767, Max-Change: 0.00542
Iteration: 29, Log-Lik: -22268.675, Max-Change: 0.00439
Iteration: 30, Log-Lik: -22268.602, Max-Change: 0.00437
Iteration: 31, Log-Lik: -22268.326, Max-Change: 0.01052
Iteration: 32, Log-Lik: -22268.240, Max-Change: 0.00307
Iteration: 33, Log-Lik: -22268.207, Max-Change: 0.00265
Iteration: 34, Log-Lik: -22268.117, Max-Change: 0.00471
Iteration: 35, Log-Lik: -22268.088, Max-Change: 0.00213
Iteration: 36, Log-Lik: -22268.071, Max-Change: 0.00211
Iteration: 37, Log-Lik: -22268.007, Max-Change: 0.00513
Iteration: 38, Log-Lik: -22267.986, Max-Change: 0.00124
Iteration: 39, Log-Lik: -22267.979, Max-Change: 0.00100
Iteration: 40, Log-Lik: -22267.968, Max-Change: 0.00126
Iteration: 41, Log-Lik: -22267.962, Max-Change: 0.00088
Iteration: 42, Log-Lik: -22267.958, Max-Change: 0.00087
Iteration: 43, Log-Lik: -22267.943, Max-Change: 0.00219
Iteration: 44, Log-Lik: -22267.938, Max-Change: 0.00064
Iteration: 45, Log-Lik: -22267.936, Max-Change: 0.00064
Iteration: 46, Log-Lik: -22267.928, Max-Change: 0.00155
Iteration: 47, Log-Lik: -22267.925, Max-Change: 0.00039
Iteration: 48, Log-Lik: -22267.924, Max-Change: 0.00036
Iteration: 49, Log-Lik: -22267.924, Max-Change: 0.00008
#MUCH better fit with differing group parameters
anova(mod.freegroup, mod.equal)
## 
## Model 1: multipleGroup(data = data, model = 1, group = group, invariance = c("slopes", 
##     "intercepts"))
## Model 2: multipleGroup(data = data, model = 1, group = group, invariance = c("intercepts", 
##     "slopes", "free_means", "free_varcov"))
##        AIC     AICc    SABIC      BIC    logLik      X2  df   p
## 1 44927.24 44928.92 45024.20 45151.28 -22423.62     NaN NaN NaN
## 2 44617.85 44619.61 44717.23 44847.49 -22267.92 311.396   1   0
#group with training appears to have a higher group mean, but with more variance
coef(mod.freegroup)[[2]]$GroupPars
##     MEAN_1 COV_11
## par  0.879      1
plot(mod.freegroup) # completely overlapping.....this shouldn't surprise you!

  1. Check if the newly acquired model from the previous question still fits worse than the completely independent model (it should, but not by much). A difference between the independent and group freed model likely indicates that, over and above general group differences, there may be items that are not functioning the same within each group (i.e., contain DIF). For this particular model, some items may have different slopes, intercepts, or both (other IRT models may have additional parameter differences that could be inspected).
mirtCluster()
DIF(mod.freegroup, c('a1', 'd'), scheme = 'drop')
## $Item_1
##        AIC     AICc    SABIC      BIC    logLik    X2  df      p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92   NaN NaN    NaN
## 2 44619.08 44621.01 44723.31 44859.92 -22266.54 2.768   2 0.2506
## 
## $Item_2
##        AIC     AICc    SABIC      BIC    logLik     X2  df   p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92    NaN NaN NaN
## 2 44586.44 44588.37 44690.66 44827.28 -22250.22 35.411   2   0
## 
## $Item_3
##        AIC     AICc    SABIC      BIC    logLik     X2  df   p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92    NaN NaN NaN
## 2 44586.30 44588.23 44690.52 44827.14 -22250.15 35.551   2   0
## 
## $Item_4
##        AIC     AICc    SABIC      BIC    logLik    X2  df      p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92   NaN NaN    NaN
## 2 44617.16 44619.10 44721.39 44858.00 -22265.58 4.688   2 0.0959
## 
## $Item_5
##        AIC     AICc    SABIC      BIC    logLik    X2  df      p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92   NaN NaN    NaN
## 2 44620.28 44622.22 44724.51 44861.12 -22267.14 1.565   2 0.4573
## 
## $Item_6
##        AIC     AICc    SABIC      BIC    logLik    X2  df      p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92   NaN NaN    NaN
## 2 44621.49 44623.43 44725.72 44862.33 -22267.75 0.354   2 0.8378
## 
## $Item_7
##        AIC     AICc    SABIC      BIC    logLik    X2  df      p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92   NaN NaN    NaN
## 2 44619.78 44621.72 44724.01 44860.62 -22266.89 2.065   2 0.3561
## 
## $Item_8
##        AIC     AICc    SABIC      BIC    logLik    X2  df      p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92   NaN NaN    NaN
## 2 44621.04 44622.97 44725.26 44861.87 -22267.52 0.814   2 0.6656
## 
## $Item_9
##        AIC     AICc    SABIC      BIC    logLik    X2  df      p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92   NaN NaN    NaN
## 2 44618.53 44620.47 44722.76 44859.37 -22266.27 3.315   2 0.1906
## 
## $Item_10
##        AIC     AICc    SABIC      BIC    logLik     X2  df      p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92    NaN NaN    NaN
## 2 44611.46 44613.39 44715.68 44852.30 -22262.73 10.391   2 0.0055
## 
## $Item_11
##        AIC     AICc    SABIC      BIC    logLik    X2  df      p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92   NaN NaN    NaN
## 2 44620.76 44622.70 44724.99 44861.60 -22267.38 1.087   2 0.5807
## 
## $Item_12
##        AIC     AICc    SABIC      BIC    logLik    X2  df      p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92   NaN NaN    NaN
## 2 44618.78 44620.71 44723.00 44859.62 -22266.39 3.069   2 0.2156
## 
## $Item_13
##        AIC     AICc    SABIC      BIC    logLik    X2  df     p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92   NaN NaN   NaN
## 2 44618.43 44620.36 44722.66 44859.27 -22266.22 3.419   2 0.181
## 
## $Item_14
##        AIC     AICc    SABIC      BIC    logLik    X2  df      p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92   NaN NaN    NaN
## 2 44617.83 44619.76 44722.05 44858.67 -22265.91 4.022   2 0.1339
## 
## $Item_15
##        AIC     AICc    SABIC      BIC    logLik    X2  df      p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92   NaN NaN    NaN
## 2 44616.68 44618.62 44720.91 44857.52 -22265.34 5.166   2 0.0755
## 
## $Item_16
##        AIC     AICc    SABIC      BIC    logLik    X2  df      p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92   NaN NaN    NaN
## 2 44620.86 44622.80 44725.09 44861.70 -22267.43 0.985   2 0.6111
## 
## $Item_17
##        AIC     AICc    SABIC      BIC    logLik    X2  df      p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92   NaN NaN    NaN
## 2 44620.98 44622.91 44725.20 44861.82 -22267.49 0.871   2 0.6469
## 
## $Item_18
##        AIC     AICc    SABIC      BIC    logLik    X2  df      p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92   NaN NaN    NaN
## 2 44620.45 44622.39 44724.68 44861.29 -22267.23 1.396   2 0.4976
## 
## $Item_19
##        AIC     AICc    SABIC      BIC    logLik     X2  df   p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92    NaN NaN NaN
## 2 44574.84 44576.77 44679.06 44815.67 -22244.42 47.013   2   0
## 
## $Item_20
##        AIC     AICc    SABIC      BIC    logLik    X2  df      p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92   NaN NaN    NaN
## 2 44613.52 44615.45 44717.74 44854.36 -22263.76 8.331   2 0.0155
DIFs <- DIF(mod.freegroup, c('a1', 'd'), scheme = 'drop', items2test = c(2,3,19), return_models = TRUE)
itemplot(DIFs$Item_2, 2)

itemplot(DIFs$Item_3, 3)

itemplot(DIFs$Item_19, 19)

DIF(mod.freegroup, 'a1', scheme = 'drop', items2test = c(2,3,19))
## $Item_2
##        AIC     AICc    SABIC      BIC    logLik     X2  df   p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92    NaN NaN NaN
## 2 44585.16 44587.01 44686.96 44820.40 -22250.58 34.689   1   0
## 
## $Item_3
##        AIC     AICc    SABIC      BIC    logLik     X2  df   p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92    NaN NaN NaN
## 2 44586.91 44588.75 44688.71 44822.15 -22251.45 32.941   1   0
## 
## $Item_19
##        AIC     AICc    SABIC      BIC    logLik     X2  df     p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92    NaN NaN   NaN
## 2 44607.05 44608.90 44708.85 44842.29 -22261.53 12.798   1 3e-04
DIF(mod.freegroup, 'd', scheme = 'drop', items2test = c(2,3,19))
## $Item_2
##        AIC     AICc    SABIC      BIC    logLik    X2  df      p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92   NaN NaN    NaN
## 2 44617.26 44619.10 44719.06 44852.49 -22266.63 2.592   1 0.1074
## 
## $Item_3
##        AIC     AICc    SABIC      BIC    logLik    X2  df      p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92   NaN NaN    NaN
## 2 44619.81 44621.65 44721.61 44855.05 -22267.90 0.042   1 0.8376
## 
## $Item_19
##        AIC     AICc    SABIC      BIC    logLik     X2  df   p
## 1 44617.85 44619.61 44717.23 44847.49 -22267.92    NaN NaN NaN
## 2 44575.47 44577.32 44677.28 44810.71 -22245.74 44.375   1   0
  1. Build a final model (using the mirt.model() syntax) that accounts for the DIF effects you previously found, and compare this to the completely independent model you computed in questions 2). How does this model compare, and what would you conclude about the current version of the model? As well, perform a global fit test with M2() on both models to determine if the more contrained representation affects the overall fit of the data.
model <- mirt.model('Theta = 1-20
                    CONSTRAINB = (1,4-18,20, a1), (1-18,20, d)')
final.mod <- multipleGroup(data, model, group=group, SE=TRUE,
                           invariance = c('free_means', 'free_var'))
## 
Iteration: 1, Log-Lik: -23087.201, Max-Change: 0.73089
Iteration: 2, Log-Lik: -22332.074, Max-Change: 0.37417
Iteration: 3, Log-Lik: -22271.940, Max-Change: 0.12651
Iteration: 4, Log-Lik: -22241.820, Max-Change: 0.07129
Iteration: 5, Log-Lik: -22227.442, Max-Change: 0.07300
Iteration: 6, Log-Lik: -22217.401, Max-Change: 0.05129
Iteration: 7, Log-Lik: -22209.553, Max-Change: 0.06249
Iteration: 8, Log-Lik: -22203.217, Max-Change: 0.05767
Iteration: 9, Log-Lik: -22198.001, Max-Change: 0.04732
Iteration: 10, Log-Lik: -22193.380, Max-Change: 0.04319
Iteration: 11, Log-Lik: -22190.032, Max-Change: 0.03121
Iteration: 12, Log-Lik: -22187.329, Max-Change: 0.02893
Iteration: 13, Log-Lik: -22178.433, Max-Change: 0.07644
Iteration: 14, Log-Lik: -22176.822, Max-Change: 0.02948
Iteration: 15, Log-Lik: -22175.614, Max-Change: 0.01317
Iteration: 16, Log-Lik: -22175.239, Max-Change: 0.01249
Iteration: 17, Log-Lik: -22174.792, Max-Change: 0.01115
Iteration: 18, Log-Lik: -22174.408, Max-Change: 0.01013
Iteration: 19, Log-Lik: -22173.209, Max-Change: 0.03396
Iteration: 20, Log-Lik: -22172.880, Max-Change: 0.01226
Iteration: 21, Log-Lik: -22172.669, Max-Change: 0.00855
Iteration: 22, Log-Lik: -22172.598, Max-Change: 0.00604
Iteration: 23, Log-Lik: -22172.517, Max-Change: 0.00554
Iteration: 24, Log-Lik: -22172.447, Max-Change: 0.00511
Iteration: 25, Log-Lik: -22172.249, Max-Change: 0.01458
Iteration: 26, Log-Lik: -22172.149, Max-Change: 0.00714
Iteration: 27, Log-Lik: -22172.119, Max-Change: 0.00488
Iteration: 28, Log-Lik: -22172.074, Max-Change: 0.00645
Iteration: 29, Log-Lik: -22172.050, Max-Change: 0.00379
Iteration: 30, Log-Lik: -22172.036, Max-Change: 0.00277
Iteration: 31, Log-Lik: -22172.009, Max-Change: 0.00526
Iteration: 32, Log-Lik: -22171.993, Max-Change: 0.00294
Iteration: 33, Log-Lik: -22171.985, Max-Change: 0.00203
Iteration: 34, Log-Lik: -22171.973, Max-Change: 0.00318
Iteration: 35, Log-Lik: -22171.966, Max-Change: 0.00203
Iteration: 36, Log-Lik: -22171.961, Max-Change: 0.00110
Iteration: 37, Log-Lik: -22171.957, Max-Change: 0.00188
Iteration: 38, Log-Lik: -22171.953, Max-Change: 0.00094
Iteration: 39, Log-Lik: -22171.950, Max-Change: 0.00083
Iteration: 40, Log-Lik: -22171.945, Max-Change: 0.00236
Iteration: 41, Log-Lik: -22171.941, Max-Change: 0.00110
Iteration: 42, Log-Lik: -22171.939, Max-Change: 0.00142
Iteration: 43, Log-Lik: -22171.936, Max-Change: 0.00039
Iteration: 44, Log-Lik: -22171.936, Max-Change: 0.00030
Iteration: 45, Log-Lik: -22171.936, Max-Change: 0.00009
## 
## Calculating information matrix...
anova(final.mod, mod.ind)
## 
## Model 1: multipleGroup(data = data, model = model, group = group, invariance = c("free_means", 
##     "free_var"), SE = TRUE)
## Model 2: multipleGroup(data = data, model = 1, group = group)
##        AIC     AICc    SABIC      BIC    logLik     X2  df      p
## 1 44435.87 44438.09 44547.37 44693.51 -22171.94    NaN NaN    NaN
## 2 44480.77 44487.52 44674.68 44928.84 -22160.39 23.101  34 0.9213
M2(mod.ind)
##             M2  df         p       RMSEA RMSEA_5   RMSEA_95
## stats 369.1507 340 0.1328907 0.006549056       0 0.01083244
##       SRMSR.without_training SRMSR.with_training       TLI       CFI
## stats             0.02593815          0.02386541 0.9987662 0.9988961
M2(final.mod)
##             M2  df         p       RMSEA RMSEA_5   RMSEA_95
## stats 393.2452 374 0.2369247 0.005073631       0 0.00972443
##       SRMSR.without_training SRMSR.with_training       TLI       CFI
## stats             0.02815207          0.02637266 0.9992595 0.9992712
  1. (EXTRA) Perform a test for differential test functioning using DTF() using the final model defined in the previous question (you will need to compute the information matrix here; see help(mirt) and help(DTF) for examples). Inspect the raw DTF scores, obtain the inferential hypothesis test regarding the \(sDTF\) statistic, and create an expected total score plot with the imputed confidence intervals.
DTF(final.mod)
##    sDTF.score sDTF(%).score    uDTF.score uDTF(%).score 
##   -0.02527068   -0.12635338    0.08355475    0.41777377
DTF(final.mod, draws = 1000)
## $observed
##    sDTF.score sDTF(%).score    uDTF.score uDTF(%).score 
##   -0.02527068   -0.12635338    0.08355475    0.41777377 
## 
## $CIs
##          sDTF.score sDTF(%).score uDTF.score uDTF(%).score
## CI_97.5 -0.00466036    -0.0233018 0.13047591     0.6523796
## CI_2.5  -0.04995020    -0.2497510 0.04781641     0.2390820
## 
## $tests
## P(sDTF.score = 0) 
##        0.02848051
DTF(final.mod, draws = 1000, plot="func")