Show simple item record

dc.contributor.authorKhabbazbashi, Nahalen
dc.contributor.authorGalaczi, Evelina D.en
dc.date.accessioned2020-01-10T09:57:43Z
dc.date.available2020-01-10T09:57:43Z
dc.date.issued2020-01-24
dc.identifier.citationKhabbazbashi N, Galaczi E (2020) 'A comparison of holistic, analytic, and part marking models in speaking assessment', Language Testing, 37 (3), pp.333-360.en
dc.identifier.issn0265-5322
dc.identifier.doi10.1177/0265532219898635
dc.identifier.urihttp://hdl.handle.net/10547/623759
dc.description.abstractThis mixed methods study examined holistic, analytic, and part marking models (MMs) in terms of their measurement properties and impact on candidate CEFR classifications in a semi-direct online speaking test. Speaking performances of 240 candidates were first marked holistically and by part (phase 1). On the basis of phase 1 findings – which suggested stronger measurement properties for the part MM – phase 2 focused on a comparison of part and analytic MMs. Speaking performances of 400 candidates were rated analytically and by part during that phase. Raters provided open comments on their marking experiences. Results suggested a significant impact of MM; approximately 30% and 50% of candidates in phases 1 and 2 respectively were awarded different (adjacent) CEFR levels depending on the choice of MM used to assign scores. There was a trend of higher CEFR levels with the holistic MM and lower CEFR levels with the part MM. While strong correlations were found between all pairings of MMs, further analyses revealed important differences. The part MM was shown to display superior measurement qualities particularly in allowing raters to make finer distinctions between different speaking ability levels. These findings have implications for the scoring validity of speaking tests.
dc.language.isoenen
dc.publisherSAGEen
dc.relation.urlhttps://journals.sagepub.com/doi/abs/10.1177/0265532219898635
dc.rightsGreen - can archive pre-print and post-print or publisher's version/PDF
dc.rights.urihttp://creativecommons.org/licenses/by-nc-nd/4.0/*
dc.subjectlanguage assessmenten
dc.subjectEnglish language testingen
dc.subjectEnglish language assessmenten
dc.subjectspeakingen
dc.subjectQ110 Applied Linguisticsen
dc.titleA comparison of holistic, analytic, and part marking models in speaking assessmenten
dc.typeArticleen
dc.identifier.journalLanguage Testingen
dc.date.updated2020-01-10T09:55:29Z
html.description.abstractThis mixed methods study examined holistic, analytic, and part marking models (MMs) in terms of their measurement properties and impact on candidate CEFR classifications in a semi-direct online speaking test. Speaking performances of 240 candidates were first marked holistically and by part (phase 1). On the basis of phase 1 findings – which suggested stronger measurement properties for the part MM – phase 2 focused on a comparison of part and analytic MMs. Speaking performances of 400 candidates were rated analytically and by part during that phase. Raters provided open comments on their marking experiences. Results suggested a significant impact of MM; approximately 30% and 50% of candidates in phases 1 and 2 respectively were awarded different (adjacent) CEFR levels depending on the choice of MM used to assign scores. There was a trend of higher CEFR levels with the holistic MM and lower CEFR levels with the part MM. While strong correlations were found between all pairings of MMs, further analyses revealed important differences. The part MM was shown to display superior measurement qualities particularly in allowing raters to make finer distinctions between different speaking ability levels. These findings have implications for the scoring validity of speaking tests.


Files in this item

Thumbnail
Name:
LT-19-0034.R3_Proof_hi.pdf
Size:
525.2Kb
Format:
PDF

This item appears in the following Collection(s)

Show simple item record

Green - can archive pre-print and post-print or publisher's version/PDF
Except where otherwise noted, this item's license is described as Green - can archive pre-print and post-print or publisher's version/PDF