Login | Register

Teacher assessment of grammatical ability in second language academic writing: A case study


Teacher assessment of grammatical ability in second language academic writing: A case study

Neumann, Heike ORCID: https://orcid.org/0000-0001-8977-7219 (2014) Teacher assessment of grammatical ability in second language academic writing: A case study. Journal of Second Language Writing, 24 . pp. 83-107. ISSN 10603743

Text (accepted manuscript) (application/vnd.openxmlformats-officedocument.wordprocessingml.document)
Teacher_assessment_of_grammar_SPECTRUM.docx - Accepted Version
Available under License Creative Commons Attribution Non-commercial No Derivatives.

Official URL: http://dx.doi.org/10.1016/j.jslw.2014.04.002


In the language assessment literature, grammatical ability is widely accepted as a key component of second language (L2) ability in general and L2 writing ability in particular. Indicators of grammatical ability have been investigated in L2 writing research, but the indicators L2 writing teachers attend to when determining grammatical ability levels of their students have not been studied. Furthermore, there is no research on what students know about their teachers’ assessment criteria and how that knowledge might affect their writing and learning process. This mixed methods triangulation study examines these questions in university L2 academic writing classes through a quantitative text-based analysis of academic essay exams, student questionnaires, and teacher and student interviews. The combined results of all data sources indicate that the teachers in this study focus primarily on accuracy when assessing grammatical ability. This leads to risk avoidance behaviour by students and may have a negative impact on their learning as students adapt their writing to meet above all their teachers’ expectations for grammatical accuracy.

Divisions:Concordia University > Faculty of Arts and Science > Education
Item Type:Article
Authors:Neumann, Heike
Journal or Publication:Journal of Second Language Writing
Date:17 May 2014
Digital Object Identifier (DOI):10.1016/j.jslw.2014.04.002
Keywords:Second Language Writing, Academic Writing, Grammatical Ability, Assessment Criteria, Rating Scales
ID Code:985473
Deposited On:06 Jun 2019 17:52
Last Modified:06 Jun 2019 17:52


Alderson, J. C., & Wall, D. (1993). Does washback exist? Applied Linguistics, 14(2), 115-129. doi: 10.1093/applin/14.2.115
Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.
Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press.
Baker, B. A. (2010). In the service of the stakeholder: A critical, mixed methods program of research in high stakes language assessment. (Unpublished doctoral dissertation). McGill University, Montreal, Quebec, Canada, Montreal. Retrieved from http://digitool.Library.McGill.CA:8881/R/?func=dbin-jump-full&object_id=96783
Bardovi-Harlig, K., & Bofman, T. (1989). Attainment of syntactic and morphological accuracy by advanced language learners. Studies in Second Language Acquisition, 11(1), 17-34. doi: 10.1017/S0272263100007816
Barkaoui, K. (2010a). Explaining ESL essay holistic scores: A multilevel modeling approach. Language Testing, 27(4), 515-535. doi: 10.1177/0265532210368717
Barkaoui, K. (2010b). Variability in ESL essay rating processes: The role of the rating scale and rater experience. Language Assessment Quarterly, 7(1), 54 - 74. doi: 10.1080/15434300903464418
Beers, S., & Nagy, W. (2009). Syntactic complexity as a predictor of adolescent writing quality: Which measures? Which genre? Reading and Writing, 22(2), 185-200. doi: 10.1007/s11145-007-9107-5
Biber, D. (2006). University language: A corpus-based study of spoken and written registers. Amsterdam: John Benjamins.
Brown, H. D. (2000). Principles of language learning and teaching (4th ed.). White Plains, NY: Longman.
Brown, H. D. (2007). Teaching by principles: An interactive approach to language pedagogy (3rd ed.). White Plains, NY: Pearson.
Byrnes, H. (2002). The role of task and task-based assessment in a content-oriented collegiate foreign language curriculum. Language Testing, 19(4), 419-437. doi: 10.1191/0265532202lt238oa
Byrnes, H., Maxim, H. H., & Norris, J. M. (2010). Realizing advanced FL writing development in collegiate education: Curricular design, pedagogy, assessment. Modern Language Journal, 94(Monograph Issue). doi: 10.1111/j.1540-4781.2010.01147.x
Canale, M., & Swain, M. (1980). Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 1(1), 1-47. doi: 10.1093/applin/I.1.1
Charmaz, K. (2003). Qualitative interviewing and grounded theory analysis. In J. A. Holstein & J. F. Gubrium (Eds.), Inside interviewing: New lenses, new concerns (pp. 311-330). Thousand Oaks, CA: SAGE.
Charmaz, K. (2006). Constructing grounded theory: A practical guide through qualitative analysis. Thousand Oaks, CA: SAGE.
Corbin, J. M., & Strauss, A. L. (2008). Basics of qualitative research: Techniques and procedures for developing grounded theory (3rd ed.). Thousand Oaks, CA: SAGE.
Council of Europe. (2001). Common European framework of reference for languages: Learning, teaching, assessment. Retrieved from http://www.coe.int/t/dg4/linguistic/Source/Framework_EN.pdf
Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: SAGE.
Creswell, J. W., & Miller, D. L. (2000). Determining validity in qualitative inquiry. Theory into Practice, 39(3), 124-130. doi: 10.1207/s15430421tip3903_2
Creswell, J. W., & Plano Clark, V. L. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: SAGE.
Crusan, D. (2010). Assessment in the second language writing classroom. Ann Arbor, MI: The University of Michigan Press.
Cumming, A. (2004). Broadening, deepening, and consolidating. Language Assessment Quarterly, 1(1), 5-18. doi: 10.1207/s15434311laq0101_2
Cumming, A. (2013). Assessing integrated writing tasks for academic purposes: Promises and Perils. Language Assessment Quarterly, 10(1), 1-8. doi: 10.1080/15434303.2011.622016
Cumming, A., Kantor, R., Baba, K., Erdosy, U., Eouanzoui, K., & James, M. (2005). Differences in written discourse in independent and integrated prototype tasks for next generation TOEFL. Assessing Writing, 10(1), 5-43. doi: 10.1016/j.asw.2005.02.001
Educational Testing Service. (2004). Scoring Guides (Rubrics) for Writing Responses. Retrieved April 8, 2007, from www.ets.org under "iBT TOEFL Scoring Guides"
Ellis, R., & Yuan, F. (2004). The effects of planning on fluency, complexity, and accuracy in second language narrative writing. Studies in Second Language Acquisition, 26(1), 59-84. doi: 10.1017/S0272263104026130
Ferris, D. R. (2002). Treatment of error in second language student writing. Ann Arbor: University of Michigan Press.
Ferris, D. R. (2003). Response to student writing: Implications for second language students. Mahwah, N.J.: Lawrence Erlbaum Associates.
Ferris, D. R. (2007). Preparing teachers to respond to student writing. Journal of Second Language Writing, 16(3), 165-193. doi: 10.1016/j.jslw.2007.07.003
Ferris, D. R., & Hedgcock, J. (2005). Teaching ESL composition: Purpose, process, and practice. Mahwah, N.J.; London: Lawrence Erlbaum.
Fritz, E., & Ruegg, R. (2013). Rater sensitivity to lexical accuracy, sophistication and range when assessing writing. Assessing Writing, 18(2), 173-181. doi: http://dx.doi.org/10.1016/j.asw.2013.02.001
Gass, S. M., & Selinker, L. (2008). Second language acquisition: An introductory course. New York: Routledge/Taylor & Francis.
Grabe, W., & Kaplan, R. B. (1996). Theory & practice of writing: An applied linguistic perspective. New York: Addison Wesley Longman.
Halliday, M. A. K., & Martin, J. R. (1993). Writing science: Literacy and discursive power. Pittsburgh: University of Pittsburgh Press.
Halliday, M. A. K., & Matthiessen, C. M. I. M. (2004). An introduction to functional grammar (3rd ed.). London: Arnold.
Hamp-Lyons, L. (1991). What is a "writing test"? In L. Hamp-Lyons (Ed.), Assessing second language writing in academic contexts (pp. 5-15). Norwood, NJ: Ablex.
Hamp-Lyons, L. (2011). Writing assessment: Shifting issues, new tools, enduring questions. Assessing Writing, 16(1), 3-5. doi: http://dx.doi.org/10.1016/j.asw.2010.12.001
Hayes, J. R. (1996). A new framework for understanding cognition and affect in writing. In C. M. Levy & S. Randsdell (Eds.), The science of writing: Theories, methods, individual differences, and applications (pp. 1-27). Mahwah, NJ: Lawrence Erlbaum.
Hayes, J. R., & Flower, L. S. (1980). Identifying the organization of writing processes. In L. W. Gregg & E. R. Steinberg (Eds.), Cognitive processes in writing (pp. 3-30). Hillsdale, NJ: Erlbaum.
He, L., & Shi, L. (2012). Topical knowledge and ESL writing. Language Testing, 29(3), 443-464. doi: 10.1177/0265532212436659
Homburg, T. J. (1984). Holistic evaluation of ESL compositions: Can it be validated objectively? TESOL Quarterly, 18(1), 87-107.
Housen, A., & Kuiken, F. (2009a). Complexity, accuracy, and fluency in second language acquisition. Applied Linguistics, 30(4), 461-473. doi: 10.1093/applin/amp048
Housen, A., & Kuiken, F. (2009b). Complexity, accuracy, and fluency in second language acquisition [Special issue]. Applied Linguistics, 30(4), 461 - 626
Huang, J., & Foote, C. J. (2010). Grading between the lines: What really impacts professors’ holistic evaluation of ESL graduate student writing? Language Assessment Quarterly, 7(3), 219-233. doi: 10.1080/15434300903540894
Hunt, K. W. (1965). Grammatical structures written at three grade levels. Urbana, IL: The National Council of Teachers of English.
Hyland, K. (2006). English for academic purposes: An advanced resource book. London: Routledge.
International English Language Testing System. (n.d.-a). IELTS Task 1 writing band descriptors. Retrieved June 10, 2009, from http://www.ielts.org/researchers/score_processing_and_reporting.aspx
International English Language Testing System. (n.d.-b). IELTS Task 2 writing band descriptors. Retrieved June 10, 2009, from http://www.ielts.org/researchers/score_processing_and_reporting.aspx
Iwashita, N., Brown, A., McNamara, T., & O'Hagan, S. (2008). Assessed levels of second language speaking proficiency: How distinct? Applied Linguistics, 29(1), 24-49. doi: 10.1093/applin/amm017
Kuiken, F., & Vedder, I. (2008). Cognitive task complexity and written output in Italian and French as a foreign language. Journal of Second Language Writing, 17(1), 48-60. doi: 10.1016/j.jslw.2007.08.003
Leki, I. (2001). Hearing voices: L2 students' experiences in L2 writing courses. In T. Silva & P. K. Matsuda (Eds.), On second language writing (pp. 17-28). Mahwah, NJ: Lawrence Erlbaum.
Li, Y. (2000). Linguistic characteristics of ESL writing in task-based e-mail activities. System, 28(2), 229-245. doi: 10.1016/S0346-251X(00)00009-9
Lightbown, P. M., & Spada, N. (2013). How languages are learned (4th ed.). Oxford: Oxford University Press.
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills, CA: SAGE.
Lumley, T. (2005). Assessing second language writing: The rater's perspective. Frankfurt: Peter Lang.
McNamara, T. F. (1996). Measuring second language performance. New York: Longman.
McNamara, T. F. (2001). Rethinking alternative assessment. Language Testing, 18(4), 329-332. doi: 10.1177/026553220101800401
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13-103). New York: American Council on Education/Macmillan.
Ministère d'Éducation, Loisir et Sport. (n.d.). Programme de formation de l'école québécoise: Enseignment secondaire, deuxième cycle (English as a Second Language). [Quebec education program: Secondary school education, cycle two] Retrieved from http://www1.mels.gouv.qc.ca/sections/programmeFormation/secondaire2/
Neumann, H. (2011). What's in a grade? A mixed methods investigation of teacher assessment of grammatical ability in L2 academic writing. (Unpublished doctoral dissertation). McGill University, Montreal. Retrieved from http://digitool.Library.McGill.CA:8881/R/?func=dbin-jump-full&object_id=103454
Norris, J. M., Brown, J. D., Hudson, T., & Yoshioka, J. (1998). Designing second language performance assessments. Honolulu, HI: University of Hawai'i, Second Language Teaching and Curriculum Center.
Norris, J. M., & Ortega, L. (2009). Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics, 30(4), 555-578. doi: 10.1093/applin/amp044
Ortega, L. (2003). Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college-level L2 writing. Applied Linguistics, 24(4), 492-518. doi: 10.1093/applin/24.4.492
Oxford, R. L. (1990). Language learning strategies: What every teacher should know. New York: Newbury House.
Polio, C. G. (1997). Measures of linguistic accuracy in second language writing research. Language Learning, 47(1), 101-143. doi: 10.1111/0023-8333.31997003
Polio, C. G., & Williams, J. (2009). Teaching and Testing Writing. In M. H. Long & C. J. Doughty (Eds.), The Handbook of Language Teaching (pp. 486-517): Wiley-Blackwell.
Purpura, J. E. (2004). Assessing grammar. Cambridge: Cambridge University Press.
Quirk, R., Greenbaum, S., Leech, G., & Svartvik, J. (1985). A comprehensive grammar of the English language. London & New York: Longman.
Rea-Dickins, P. (1997). So, why do we need relationships with stakeholders in language testing? A view from the UK. Language Testing, 14(3), 304-314. doi: 10.1177/026553229701400307
Rea-Dickins, P. (2009). Classroom-based assessment. In E. Shohamy & N. H. Hornberger (Eds.), Encyclopedia of language and education (2nd ed., Vol. 7: Language testing and assessment, pp. 257-571). New York: Springer.
Scardamalia, M., & Bereiter, C. (1987). Knowledge telling and knowledge transforming in written composition. In S. Rosenberg (Ed.), Advances in applied psycholinguistics, Volume 2: Reading, writing, and language learning (pp. 142-175). Cambridge: Cambridge University Press.
Skehan, P. (1989). Individual differences in second-language learning. London: Arnold.
Skehan, P. (1996). A Framework for the implementation of task-based instruction. Applied Linguistics, 17(1), 38-62. doi: 10.1093/applin/17.1.38
Skehan, P. (1998). A cognitive approach to language learning. Oxford: Oxford University Press.
Skehan, P. (2009). Modelling second language performance: Integrating complexity, accuracy, fluency, and lexis. Applied Linguistics, 30(4), 510-532. doi: 10.1093/applin/amp047
Storch, N. (2002). Relationships formed in dyadic interaction and opportunity for learning. International Journal of Educational Research, 37(3–4), 305-322. doi: 10.1016/s0883-0355(03)00007-7
Storch, N. (2005). Collaborative writing: Product, process, and students’ reflections. Journal of Second Language Writing, 14(3), 153-173. doi: 10.1016/j.jslw.2005.05.002
Storch, N. (2009). The impact of studying in a second language (L2) medium university on the development of L2 writing. Journal of Second Language Writing, 18(2), 103-118. doi: 10.1016/j.jslw.2009.02.003
Storch, N., & Wigglesworth, G. (2007). Writing tasks: The effects of collaboration. In M. d. P. García Mayo (Ed.), Investigating tasks in formal language learning (pp. 157-177). Clevedon, England: Multilingual Matters.
Sweedler-Brown, C. O. (1993). ESL essay evaluation: The influence of sentence-level and rhetorical features. Journal of Second Language Writing, 2(1), 3-17. doi: http://dx.doi.org/10.1016/1060-3743(93)90003-L
Tapia, E. (1993). Cognitive demands as a factor in interlanguage syntax: A study in topics and texts. (Unpublished doctoral dissertation). Indiana University, Bloomington. Retrieved from http://proquest.umi.com/pqdweb?did=746315871&sid=3&Fmt=2&clientId=10843&RQT=309&VName=PQD
Teddlie, C., & Tashakkori, A. (2009). Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Thousand Oaks, CA: SAGE.
Turner, C. E., & Upshur, J. A. (2002). Rating scales derived from student samples: Effects of the scale maker and the student sample on scale content and student scores. TESOL Quarterly, 36(1), 49-70. doi: 10.2307/3588360
Upshur, J. A., & Turner, C. E. (1999). Systematic effects in the rating of second-language speaking ability: Test method and learner discourse. Language Testing, 16(1), 82-111. doi: 10.1177/026553229901600105
Wall, D., & Alderson, J. C. (1993). Examining washback: The Sri Lankan impact study. Language Testing, 10(1), 41-69. doi: 10.1177/026553229301000103
Weigle, S. C. (2002). Assessing writing. Cambridge: Cambridge University Press.
Weigle, S. C. (2012). Assessing writing. In C. Coombe, P. Davidson, B. O'Sullivan & S. Stoynoff (Eds.), The Cambridge guide to second language assessment (pp. 218-224). Cambridge: Cambridge University Press.
White, E. M. (1984). Holisticism. College Composition and Communication, 35(4), 400-409. doi: 10.2307/357792
White, E. M. (1995). An apologia for the timed impromptu essay test. College Composition and Communication, 46(1), 30-45. doi: 10.2307/358868
Wigglesworth, G., & Storch, N. (2009). Pair versus individual writing: Effects on fluency, complexity and accuracy. Language Testing, 26(3), 445-466. doi: 10.1177/0265532209104670
Wigglesworth, G., & Storch, N. (2012). What role for collaboration in writing and writing feedback. Journal of Second Language Writing, 21(4), 364-374. doi: http://dx.doi.org/10.1016/j.jslw.2012.09.005
Wolfe-Quintero, K., Inagaki, S., & Kim, H.-Y. (1998). Second language development in writing: Measures of fluency, accuracy & complexity. Honolulu, HI: University of Hawai'i, Second Language Teaching and Curriculum Center.
Yin, R. K. (2009). Case study research: Design and methods (4th ed.). Thousand Oaks, CA: SAGE.
Zentralstelle für das Auslandsschulwesen. (2012). Deutsches Sprachdiplom der Kultusministerkonferenz: Handreichungen für die schrifltliche Kommunikation (Niveaustufe B2/C1). [German language diploma of the education ministers' conference: Documents for the written communication (Level B2/C1)] Retrieved from http://www.bva.bund.de/DE/Organisation/Abteilungen/Abteilung_ZfA/Auslandsschularbeit/DSD/Handbuch/download_DSD_II_SK_Hand_2012_T2.pdf?__blob=publicationFile&v=2
All items in Spectrum are protected by copyright, with all rights reserved. The use of items is governed by Spectrum's terms of access.

Repository Staff Only: item control page

Downloads per month over past year

Research related to the current document (at the CORE website)
- Research related to the current document (at the CORE website)
Back to top Back to top