Abstract

This study investigates the effects of preselected model compositions and a multiple weighted trait scale on the consistency of raters' scores in a direct ESL writing assessment context. Twenty experienced ESL teachers at the institution where this study took place were divided into two equal groups. One group used model compositions and the weighted multiple trait scale to grade a set of sixteen authentic ESL exam papers. The second group graded the same set of compositions using the trait scale alone. Ratings were analyzed using traditional statistical techniques, such as a t -test analysis, as well as Minifac Facets software, a student version of the multi-faceted Rasch analysis program FACETS. Findings from the t-test showed more significant score variance among the teachers who used the trait scale alone to grade the compositions than those who used model compositions in addition to the trait scale. This difference between the two groups was supported in the Rasch analysis, which identified a greater spread in the over-all severity measures for the group of teachers who graded with the trait scale alone