Breadcrumb

 
 

Measuring rater consistency : an investigation into the effects of two testing instruments on raters' scores

Title:

Measuring rater consistency : an investigation into the effects of two testing instruments on raters' scores

Quintieri, Marcello (2005) Measuring rater consistency : an investigation into the effects of two testing instruments on raters' scores. Masters thesis, Concordia University.

[img]
Preview
PDF - Accepted Version
4Mb

Abstract

This study investigates the effects of preselected model compositions and a multiple weighted trait scale on the consistency of raters' scores in a direct ESL writing assessment context. Twenty experienced ESL teachers at the institution where this study took place were divided into two equal groups. One group used model compositions and the weighted multiple trait scale to grade a set of sixteen authentic ESL exam papers. The second group graded the same set of compositions using the trait scale alone. Ratings were analyzed using traditional statistical techniques, such as a t -test analysis, as well as Minifac Facets software, a student version of the multi-faceted Rasch analysis program FACETS. Findings from the t-test showed more significant score variance among the teachers who used the trait scale alone to grade the compositions than those who used model compositions in addition to the trait scale. This difference between the two groups was supported in the Rasch analysis, which identified a greater spread in the over-all severity measures for the group of teachers who graded with the trait scale alone

Divisions:Concordia University > Faculty of Arts and Science > Education
Item Type:Thesis (Masters)
Authors:Quintieri, Marcello
Pagination:x, 96 leaves : ill. ; 29 cm.
Institution:Concordia University
Degree Name:M.A.
Program:Education
Date:2005
Thesis Supervisor(s):Gatbonton, Elizabeth
ID Code:8735
Deposited By:Concordia University Libraries
Deposited On:18 Aug 2011 14:33
Last Modified:18 Aug 2011 15:14
Related URLs:
All items in Spectrum are protected by copyright, with all rights reserved. The use of items is governed by Spectrum's terms of access.

Repository Staff Only: item control page

Document Downloads

More statistics for this item...

Concordia University - Footer