A comparative study of the variables used to measure syntactic complexity and accuracy in task-based research

5.00
Hdl Handle:
http://hdl.handle.net/10547/621953
Title:
A comparative study of the variables used to measure syntactic complexity and accuracy in task-based research
Authors:
Inoue, Chihiro
Abstract:
The constructs of complexity, accuracy and fluency (CAF) have been used extensively to investigate learner performance on second language tasks. However, a serious concern is that the variables used to measure these constructs are sometimes used conventionally without any empirical justification. It is crucial for researchers to understand how results might be different depending on which measurements are used, and accordingly, choose the most appropriate variables for their research aims. The first strand of this article examines the variables conventionally used to measure syntactic complexity in order to identify which may be the best indicators of different proficiency levels, following suggestions by Norris and Ortega. The second strand compares the three variables used to measure accuracy in order to identify which one is most valid. The data analysed were spoken performances by 64 Japanese EFL students on two picture-based narrative tasks, which were rated at Common European Framework of Reference for Languages (CEFR) A2 to B2 according to Rasch-adjusted ratings by seven human judges. The tasks performed were very similar, but had different degrees of what Loschky and Bley-Vroman term ‘task-essentialness’ for subordinate clauses. It was found that the variables used to measure syntactic complexity yielded results that were not consistent with suggestions by Norris and Ortega. The variable found to be the most valid for measuring accuracy was errors per 100 words. Analysis of transcripts revealed that results were strongly influenced by the differing degrees of task-essentialness for subordination between the two tasks, as well as the spread of errors across different units of analysis. This implies that the characteristics of test tasks need to be carefully scrutinised, followed by careful piloting, in order to ensure greater validity and reliability in task-based research.
Affiliation:
University of Bedfordshire
Citation:
Inoue C (2016) 'A comparative study of the variables used to measure syntactic complexity and accuracy in task-based research', Language learning journal, 44 (4), pp.487-505.
Publisher:
Taylor & Francis (Routledge): SSH Titles
Journal:
Language learning journal
Issue Date:
12-Apr-2016
URI:
http://hdl.handle.net/10547/621953
DOI:
10.1080/09571736.2015.1130079
Additional Links:
http://www.tandfonline.com/doi/full/10.1080/09571736.2015.1130079
Type:
Article
Language:
en
ISSN:
0957-1736
EISSN:
1753-2167
Appears in Collections:
English language learning and assessment

Full metadata record

DC FieldValue Language
dc.contributor.authorInoue, Chihiroen
dc.date.accessioned2017-01-19T11:41:46Z-
dc.date.available2017-01-19T11:41:46Z-
dc.date.issued2016-04-12-
dc.identifier.citationInoue C (2016) 'A comparative study of the variables used to measure syntactic complexity and accuracy in task-based research', Language learning journal, 44 (4), pp.487-505.en
dc.identifier.issn0957-1736-
dc.identifier.doi10.1080/09571736.2015.1130079-
dc.identifier.urihttp://hdl.handle.net/10547/621953-
dc.description.abstractThe constructs of complexity, accuracy and fluency (CAF) have been used extensively to investigate learner performance on second language tasks. However, a serious concern is that the variables used to measure these constructs are sometimes used conventionally without any empirical justification. It is crucial for researchers to understand how results might be different depending on which measurements are used, and accordingly, choose the most appropriate variables for their research aims. The first strand of this article examines the variables conventionally used to measure syntactic complexity in order to identify which may be the best indicators of different proficiency levels, following suggestions by Norris and Ortega. The second strand compares the three variables used to measure accuracy in order to identify which one is most valid. The data analysed were spoken performances by 64 Japanese EFL students on two picture-based narrative tasks, which were rated at Common European Framework of Reference for Languages (CEFR) A2 to B2 according to Rasch-adjusted ratings by seven human judges. The tasks performed were very similar, but had different degrees of what Loschky and Bley-Vroman term ‘task-essentialness’ for subordinate clauses. It was found that the variables used to measure syntactic complexity yielded results that were not consistent with suggestions by Norris and Ortega. The variable found to be the most valid for measuring accuracy was errors per 100 words. Analysis of transcripts revealed that results were strongly influenced by the differing degrees of task-essentialness for subordination between the two tasks, as well as the spread of errors across different units of analysis. This implies that the characteristics of test tasks need to be carefully scrutinised, followed by careful piloting, in order to ensure greater validity and reliability in task-based research.en
dc.language.isoenen
dc.publisherTaylor & Francis (Routledge): SSH Titlesen
dc.relation.urlhttp://www.tandfonline.com/doi/full/10.1080/09571736.2015.1130079en
dc.rightsGreen - can archive pre-print and post-print or publisher's version/PDF-
dc.rights.urihttp://creativecommons.org/licenses/by-nc-nd/4.0/*
dc.subjectspeakingen
dc.subjecttask-based researchen
dc.subjectsyntactic complexityen
dc.subjectaccuracyen
dc.subjectspeech communicationen
dc.titleA comparative study of the variables used to measure syntactic complexity and accuracy in task-based researchen
dc.typeArticleen
dc.identifier.eissn1753-2167-
dc.contributor.departmentUniversity of Bedfordshireen
dc.identifier.journalLanguage learning journalen
dc.date.updated2017-01-19T10:58:06Z-
This item is licensed under a Creative Commons License
Creative Commons
All Items in UOBREP are protected by copyright, with all rights reserved, unless otherwise indicated.