Name:
Challenges+25+Khabbazbashi+Xu+ ...
Size:
318.6Kb
Format:
PDF
Description:
author's accepted version
Issue Date
2021-02-10Subjects
speakinglanguage assessment
learning technology
Subject Categories::X162 Teaching English as a Foreign Language (TEFL)
Metadata
Show full item recordOther Titles
Issues in Language Testing Around the World: Insights for Language Test Users.Abstract
The rapid advances in speech processing and machine learning technologies have attracted language testers’ strong interest in developing automated speaking assessment in which candidate responses are scored by computer algorithms rather than trained human examiners. Despite its increasing popularity, automatic evaluation of spoken language is still shrouded in mystery and technical jargon, often resembling an opaque "black box" that transforms candidate speech to scores in a matter of minutes. Our chapter explicitly problematizes this lack of transparency around test score interpretation and use and asks the following questions: What do automatically derived scores actually mean? What are the speaking constructs underlying them? What are some common problems encountered in automated assessment of speaking? And how can test users evaluate the suitability of automated speaking assessment for their proposed test uses? In addressing these questions, the purpose of our chapter is to explore the benefits, problems, and caveats associated with automated speaking assessment touching on key theoretical discussions on construct representation and score interpretation as well as practical issues such as the infrastructure necessary for capturing high quality audio and the difficulties associated with acquiring training data. We hope to promote assessment literacy by providing the necessary guidance for users to critically engage with automated speaking assessment, pose the right questions to test developers, and ultimately make informed decisions regarding the fitness for purpose of automated assessment solutions for their specific learning and assessment contexts.Citation
Khabbazbashi N, Xu J, Galaczi E (2021) 'Opening the black box: exploring automated speaking evaluation', in Lanteigne B, Coombe C, Brown JD (ed(s).). Issues in Language Testing Around the World: Insights for Language Test Users., Springer pp.-.Publisher
SpringerAdditional Links
https://www.springer.com/gp/book/9789813342316Type
Book chapterLanguage
enISBN
9789813342316ae974a485f413a2113503eed53cd6c53
10.1007/978-981-33-4232-3
Scopus Count
Collections
The following license files are associated with this item:
- Creative Commons
Except where otherwise noted, this item's license is described as Attribution-NonCommercial-NoDerivatives 4.0 International