DTF {mirt}R Documentation

Differential test functioning statistics

Description

Function performs various omnibus differential test functioning procedures on an object estimated with multipleGroup(). If the latent means/covariances are suspected to differ then the input object should contain a set of 'anchor' items to ensure that only differential test features are being detected rather than group differences. Returns signed (average area above and below) and unsigned (total area) statistics, with descriptives such as the percent average bias between group total scores for each statistic. If a grid of Theta values is passed, these can be evaluated as well to determine specific DTF location effects. For best results, the baseline model should contain a set of 'anchor' items and have freely estimated hyper-parameters in the focal groups. See DIF for details.

Usage

DTF(
  mod,
  draws = NULL,
  CI = 0.95,
  npts = 1000,
  theta_lim = c(-6, 6),
  Theta_nodes = NULL,
  plot = "none",
  auto.key = list(space = "right", points = FALSE, lines = TRUE),
  ...
)

Arguments

mod

a multipleGroup object which estimated only 2 groups

draws

a number indicating how many draws to take to form a suitable multiple imputation estimate of the expected test scores (usually 100 or more). Returns a list containing the imputation distribution and null hypothesis test for the sDTF statistic

CI

range of confidence interval when using draws input

npts

number of points to use in the integration. Default is 1000

theta_lim

lower and upper limits of the latent trait (theta) to be evaluated, and is used in conjunction with npts

Theta_nodes

an optional matrix of Theta values to be evaluated in the draws for the sDTF statistic. However, these values are not averaged across, and instead give the bootstrap confidence intervals at the respective Theta nodes. Useful when following up a large uDTF/sDTF statistic to determine where the difference between the test curves are large (while still accounting for sampling variability). Returns a matrix with observed variability

plot

a character vector indicating which plot to draw. Possible values are 'none', 'func' for the test score functions, and 'sDTF' for the evaluated sDTF values across the integration grid. Each plot is drawn with imputed confidence envelopes

auto.key

logical; automatically generate key in lattice plot?

...

additional arguments to be passed to lattice and boot

Author(s)

Phil Chalmers rphilip.chalmers@gmail.com

References

Chalmers, R., P. (2012). mirt: A Multidimensional Item Response Theory Package for the R Environment. Journal of Statistical Software, 48(6), 1-29. doi:10.18637/jss.v048.i06

Chalmers, R. P., Counsell, A., and Flora, D. B. (2016). It might not make a big DIF: Improved Differential Test Functioning statistics that account for sampling variability. Educational and Psychological Measurement, 76, 114-140. doi:10.1177/0013164415584576

See Also

multipleGroup, DIF

Examples

## No test: 
set.seed(1234)
n <- 30
N <- 500

# only first 5 items as anchors
model <- 'F = 1-30
          CONSTRAINB = (1-5, a1), (1-5, d)'

a <- matrix(1, n)
d <- matrix(rnorm(n), n)
group <- c(rep('Group_1', N), rep('Group_2', N))

## -------------
# groups completely equal
dat1 <- simdata(a, d, N, itemtype = '2PL')
dat2 <- simdata(a, d, N, itemtype = '2PL')
dat <- rbind(dat1, dat2)
mod <- multipleGroup(dat, model, group=group, SE=TRUE,
                     invariance=c('free_means', 'free_var'))
plot(mod)

plot of chunk unnamed-chunk-1

DTF(mod)

##    sDTF.score sDTF(%).score    uDTF.score uDTF(%).score 
##     0.1744653     0.5815508     0.1771799     0.5905996

if(interactive()) mirtCluster()
DTF(mod, draws = 1000) #95% C.I. for sDTF containing 0. uDTF is very small

## $observed
##    sDTF.score sDTF(%).score    uDTF.score uDTF(%).score 
##     0.1744653     0.5815508     0.1771799     0.5905996 
## 
## $CIs
##         sDTF.score sDTF(%).score uDTF.score uDTF(%).score
## CI_97.5  0.5039569     1.6798563 0.74876717     2.4958906
## CI_2.5  -0.1403432    -0.4678107 0.08578107     0.2859369
## 
## $tests
## P(sDTF.score = 0) 
##         0.2872505

DTF(mod, draws = 1000, plot='sDTF') #sDTF 95% C.I.'s across Theta always include 0

plot of chunk unnamed-chunk-1

## -------------
## random slopes and intercepts for 15 items, and latent mean difference
##    (no systematic DTF should exist, but DIF will be present)
set.seed(1234)
dat1 <- simdata(a, d, N, itemtype = '2PL', mu=.50, sigma=matrix(1.5))
dat2 <- simdata(a + c(numeric(15), runif(n-15, -.2, .2)),
                d + c(numeric(15), runif(n-15, -.5, .5)), N, itemtype = '2PL')
dat <- rbind(dat1, dat2)
mod1 <- multipleGroup(dat, 1, group=group)
plot(mod1) #does not account for group differences! Need anchors

plot of chunk unnamed-chunk-1

mod2 <- multipleGroup(dat, model, group=group, SE=TRUE,
                      invariance=c('free_means', 'free_var'))
plot(mod2)

plot of chunk unnamed-chunk-1

# significant DIF in multiple items....
# DIF(mod2, which.par=c('a1', 'd'), items2test=16:30)
DTF(mod2)

##    sDTF.score sDTF(%).score    uDTF.score uDTF(%).score 
##     0.1945029     0.6483428     0.1951426     0.6504754

DTF(mod2, draws=1000) #non-sig DTF due to item cancellation

## $observed
##    sDTF.score sDTF(%).score    uDTF.score uDTF(%).score 
##     0.1945029     0.6483428     0.1951426     0.6504754 
## 
## $CIs
##          sDTF.score sDTF(%).score uDTF.score uDTF(%).score
## CI_97.5  0.48981746     1.6327249 0.63675470     2.1225157
## CI_2.5  -0.08145747    -0.2715249 0.08279536     0.2759845
## 
## $tests
## P(sDTF.score = 0) 
##         0.1678257

## -------------
## systematic differing slopes and intercepts (clear DTF)
dat1 <- simdata(a, d, N, itemtype = '2PL', mu=.50, sigma=matrix(1.5))
dat2 <- simdata(a + c(numeric(15), rnorm(n-15, 1, .25)), d + c(numeric(15), rnorm(n-15, 1, .5)),
                N, itemtype = '2PL')
dat <- rbind(dat1, dat2)
mod3 <- multipleGroup(dat, model, group=group, SE=TRUE,
                      invariance=c('free_means', 'free_var'))
plot(mod3) #visable DTF happening

plot of chunk unnamed-chunk-1

# DIF(mod3, c('a1', 'd'), items2test=16:30)
DTF(mod3) #unsigned bias. Signed bias indicates group 2 scores generally higher on average

##    sDTF.score sDTF(%).score    uDTF.score uDTF(%).score 
##    -0.8275464    -2.7584881     0.8539061     2.8463537

DTF(mod3, draws=1000)

## $observed
##    sDTF.score sDTF(%).score    uDTF.score uDTF(%).score 
##    -0.8275464    -2.7584881     0.8539061     2.8463537 
## 
## $CIs
##         sDTF.score sDTF(%).score uDTF.score uDTF(%).score
## CI_97.5 -0.4971357     -1.657119  1.2031511      4.010504
## CI_2.5  -1.1803225     -3.934408  0.6384586      2.128195
## 
## $tests
## P(sDTF.score = 0) 
##      2.311111e-06

DTF(mod3, draws=1000, plot='func')

plot of chunk unnamed-chunk-1

DTF(mod3, draws=1000, plot='sDTF') #multiple DTF areas along Theta

plot of chunk unnamed-chunk-1

# evaluate specific values for sDTF
Theta_nodes <- matrix(seq(-6,6,length.out = 100))
sDTF <- DTF(mod3, Theta_nodes=Theta_nodes)
head(sDTF)

##             Theta       sDTF
## score.1 -6.000000 0.01507921
## score.2 -5.878788 0.01674080
## score.3 -5.757576 0.01852616
## score.4 -5.636364 0.02042988
## score.5 -5.515152 0.02244193
## score.6 -5.393939 0.02454674

sDTF <- DTF(mod3, Theta_nodes=Theta_nodes, draws=100)
head(sDTF)

##             Theta       sDTF   CI_97.5     CI_2.5
## score.1 -6.000000 0.01507921 0.1075343 -0.1669580
## score.2 -5.878788 0.01674080 0.1177759 -0.1820318
## score.3 -5.757576 0.01852616 0.1288388 -0.1979721
## score.4 -5.636364 0.02042988 0.1407533 -0.2147294
## score.5 -5.515152 0.02244193 0.1535422 -0.2322314
## score.6 -5.393939 0.02454674 0.1672177 -0.2503809

## End(No test)
[Package mirt version 1.40 Index]