Using Auxiliary Item Information in the Item Parameter Estimation of a Graded Response Model for a Small to Medium Sample Size: Empirical versus Hierarchical Bayes Estimation

Abstract

Marginal maximum likelihood estimation (MMLE) is largely considered the ``gold standard’’ for item response theory item parameter estimation. MMLE requires medium to large sample sizes to ensure convergence and to obtain accurate and precise item parameter estimates. However, sufficiently large sample sizes are not always possible for smaller research institutions or when studying rare populations. In this paper, empirical Bayes and hierarchical Bayes are presented as alternatives to MMLE in small sample sizes, using auxiliary item information to estimate the item parameters of a graded response model with higher accuracy. Empirical Bayes and hierarchical Bayes methods are compared with MMLE to determine under what conditions these Bayes methods can outperform MMLE, and to determine if hierarchical Bayes can act as an acceptable alternative to MMLE in conditions where MMLE is unable to converge. In addition, empirical Bayes and hierarchical Bayes methods are compared to show how hierarchical Bayes can result in estimates of posterior variance with greater accuracy than empirical Bayes by acknowledging the uncertainty of item parameter estimates. The proposed methods were illustrated using an empirical data set and evaluated via a simulation study. Simulation results showed that empirical Bayes and hierarchical Bayes methods can be acceptable alternatives to MMLE under various testing conditions, and we provide a guideline to indicate which methods would be recommend in different research situations. R functions are provided to implement these proposed methods.