| dc.description.abstract | Intervention fidelity (Nelson, Cordray, Hulleman, Darrow, & Sommer, in press) is the extent to which an intervention has been implemented as planned in the treatment group, and differentiated from the control group, in the context of a randomized controlled experiment (RCT). Education researchers are seeking more and better tools for measuring intervention fidelity, but approaches have varied widely among researchers, and there are few direct comparisons of different analytical methods.
IRT approaches may be especially capable of overcoming difficulties associated with analyzing intervention fidelity data, including skewed distributions, multidimensionality, and poorly-defined constructs. A recent development in explanatory multidimensional IRT (MIRT) is a model that detects group differences and individual differences simultaneously for multidimensional tests (Cho, Athay, & Preacher, in press). Model results then can be compared directly with factorial analysis of variance (ANOVA, Kirk, 1968) results.
The primary goal of this study is to demonstrate parallel analyses of empirical intervention fidelity data with both the traditional ANOVA using total scores and this particular MIRT model. The comparison shows the unique strengths of explanatory MIRT for intervention fidelity analyses, allowing researchers to assess its benefits over classical test theory (CTT) approaches, as well as the feasibility of MIRT analysis for their data. Secondarily, the results of this study show that choice of analytical method for intervention fidelity analysis can lead to somewhat different statistical conclusions. It is recommended that the sources of such deviations be investigated through simulation studies and other methods in the future. | |
