Comparing rating modes: analysing live, audio, and video ratings of IELTS Speaking Test performances
Name:
IELTS+Double+rating_LAQ_copy+f ...
Size:
1.281Mb
Format:
PDF
Description:
authors' accepted version
Issue Date
2020-08-26Subjects
language testingEnglish language testing
speaking
academic speaking
oral test
Subject Categories::X162 Teaching English as a Foreign Language (TEFL)
Metadata
Show full item recordAbstract
This mixed methods study compared IELTS examiners’ scores when assessing spoken performances under live and two ‘non-live’ testing conditions using audio and video recordings. Six IELTS examiners assessed 36 test-takers’ performances under the live, audio, and video rating conditions. Scores in the three rating modes were calibrated using the many-facet Rasch model (MFRM). For all three modes, examiners provided written justifications for their ratings, and verbal reports were also collected to gain insights into examiner perceptions towards performance under the audio and video conditions. Results showed that, for all rating criteria, audio ratings were significantly lower than live and video ratings. Examiners noticed more negative performance features under the two non-live rating conditions, compared to the live condition. However, richer information about test-taker performance in the video mode appeared to cause raters to rely less on such negative evidence than audio raters when awarding scores. Verbal report data showed that having visual information in the video-rating mode helped examiners to understand what the test-takers were saying, to comprehend better what test-takers were communicating using non-verbal means, and to understand with greater confidence the source of test-takers’ hesitation, pauses, and awkwardness.Citation
Nakatsuhara F, Inoue C, Taylor L (2020) 'Comparing rating modes: analysing live, audio, and video ratings of IELTS Speaking Test performances', Language Assessment Quarterly, 18 (2), pp.83-106.Publisher
Taylor & FrancisJournal
Language Assessment QuarterlyAdditional Links
https://www.tandfonline.com/doi/full/10.1080/15434303.2020.1799222Type
ArticleLanguage
enISSN
1543-4303Sponsors
This research was funded by the IELTS Partners: British Council, IDP: IELTS Australia, and Cambridge Assessment English under the IELTS Joint Funded Research Programme (Round 19).ae974a485f413a2113503eed53cd6c53
10.1080/15434303.2020.1799222
Scopus Count
Collections
The following license files are associated with this item:
- Creative Commons
Except where otherwise noted, this item's license is described as Green - can archive pre-print and post-print or publisher's version/PDF