Abstract

Comprehensibility, which refers to listeners’ perception of how easy or difficult it is for them to understand second language (L2) speakers, is an important global dimension of L2 speech. Comprehensibility has been included in high-stakes language assessment instruments and has been promoted as a learning goal in language teaching. However, listener-assessed comprehensibility can be affected by various variables, such as linguistic characteristics of L2 speech and individual differences among raters in their cognitive, experiential, and affective profiles. The overarching goal of this dissertation is to examine several of these sources of variance in L2 speech assessment, with a particular focus on comprehensibility.
Study 1 examined the role of discourse organization in L2 comprehensibility ratings. Twelve English for Academic Purposes teachers listened to 60 L2 speech samples elicited through a TOEFL-type integrated speaking task, evaluating each sample for comprehensibility and coherence (perceived interconnectedness of ideas). The speech samples were analyzed for the occurrence of various discourse features at micro and macro levels. Linear mixed-effects modelling revealed that coherence and comprehensibility were strongly related, and that comprehensibility was uniquely predicted by micro measures of discourse organization.
Study 2 investigated how individual differences in teachers’ experience and beliefs profiles are associated with their judgements of L2 speech. Participants included 50 in-service and 50 pre-service teachers who rated 40 audio-recorded speaking performances by Japanese secondary school students, evaluating these students’ L2 English comprehensibility, fluency, and accentedness. The teachers also completed online questionnaires targeting their beliefs about L2 pronunciation instruction and recorded their professional and personal experiences related to language teaching and learning. Results revealed that in-service teachers held significantly stronger beliefs about L2 pronunciation instruction compared to pre-service teachers. Notably, some of the teachers’ beliefs and experiences were related to their L2 speech assessments.
Finally, Study 3 explored how speakers themselves evaluate their own L2 speech, examining the extent to which individual differences in speakers’ metacognitive skills were relevant to their self-assessments. The study also examined whether a brief peer-assessment activity was effective in helping L2 speakers align their self-assessments with the evaluations provided by external listeners. For this study, 40 L2 English-speaking international students performed an academic oral summary task and then self-assessed their speech for comprehensibility. Their speech was also evaluated for comprehensibility by 30 external listeners from mixed language backgrounds. Results showed that peer-assessment, but not L2 speakers’ metacognition, was associated with greater alignment between L2 English speakers’ self-ratings and external listeners’ evaluations.