Assessment Across Online Language Education - Stephanie Link
9. A Systematic Approach to Vetting Reading Comprehension Items for Inclusion in Cloud-based Assessments
Assessment Across Online Language Education - Stephanie Link
Fabiana MacMillan [+]
Rosetta Stone
Description
Fabiana MacMillan reports the trial of a process for vetting reading comprehension items for inclusion in online placement and achievement assessments—the Systematic Approach to Item Vetting (SAIV). The chapter begins with an outline of the SAIV system used to perform a qualitative analysis of all items in the dataset prior to field testing. Next is a brief general description of the dataset and the tests for which the items were developed. This is followed by a description of the quantitative analysis of learner response data using the Rasch model (Rasch, 1980) to obtain difficulty parameter estimates and discrimination indices for each item. The penultimate section discusses results of the analyses and the extent to which qualitative data correlates with field test outcomes. Finally, her chapter concludes by showing that the SAIV process may help predict which items will be successful after field testing and has promise in also predicting item difficulty.