Compute the empirical detection/rejection rate for Type I errors and Power

Source: R/summary_functions.R

Computes the detection/rejection rate for determining empirical Type I error and power rates using information from p-values.

EDR(p, alpha = 0.05, unname = FALSE)

Arguments

p

a numeric vector or matrix/data.frame of p-values from the desired statistical estimator. If a matrix, each statistic must be organized by column, where the number of rows is equal to the number of replications

alpha

the detection threshold (typical values are .10, .05, and .01). Default is .05

unname

logical; apply unname to the results to remove any variable names?

References

Chalmers, R. P., & Adkins, M. C. (2020). Writing Effective and Reliable Monte Carlo Simulations with the SimDesign Package. The Quantitative Methods for Psychology, 16(4), 248-280. doi:10.20982/tqmp.16.4.p248

Sigal, M. J., & Chalmers, R. P. (2016). Play it again: Teaching statistics with Monte Carlo simulation. Journal of Statistics Education, 24(3), 136-156. doi:10.1080/10691898.2016.1246953

See also

ECR, Bradley1978

Author

Phil Chalmers rphilip.chalmers@gmail.com

Examples


rates <- numeric(100)
for(i in 1:100){
   dat <- rnorm(100)
   rates[i] <- t.test(dat)$p.value
}

EDR(rates)
#> [1] 0.03
EDR(rates, alpha = .01)
#> [1] 0

# multiple rates at once
rates <- cbind(runif(1000), runif(1000))
EDR(rates)
#> [1] 0.034 0.044