Browsing English language learning and assessment by Subjects
Now showing items 1-4 of 4
Eye-tracking L2 students taking online multiple-choice reading tests: benefits and challengesRecently, there has been a marked increase in language testing research involving eye-tracking. It appears to offer a useful methodology for examining cognitive validity in language tests, i.e., the extent to which the mental processes that a language test elicits from test takers resemble those that they would employ in the target language use domains. This article reports on a recent study which examined reading processes of test takers at different proficiency levels on a reading proficiency test. Using a mixed-methods approach, the study collected cognitive validity evidence through eye-tracking and stimulated recall interviews. The study investigated whether there are differences in reading behaviour among test takers at CEFR B1, B2 and C1 levels on an online reading task. The main findings are reported and the implications of the findings are discussed to reflect on some fundamental questions regarding the use of eye-tracking in language testing research.
Placing construct definition at the heart of assessment: research, design and a priori validationIn this chapter, we will first highlight Professor Cyril Weir’s major research into the nature of academic reading. Using one of his test development pro- jects as an example, we will describe how the construct of academic reading was operationalised in the local context of a British university by theoretical construct definition together with empirical analyses of students’ reading patterns on the test through eye-tracking. As we progress through the chapter we reflect on how Weir’s various research projects fed into the development of the test and a new method of analysing eye-tracking data in relation to different types of reading.
Researching the cognitive validity of GEPT high-intermediate and advanced reading : an eye tracking an stimulated recall studyIt is important for any language test to establish its cognitive validity in order to ensure that the test elicits from test takers those cognitive processes which correspond to the processes which they would normally employ in the target real-life context (Weir 2005). This study investigates the cognitive validity of the GEPT Reading Test at two levels. High-intermediate (CEFR B2) and Advanced (CEFR C1), using innovative eye-tracking technology and detailed stimulated recall interviews and surveys. Representative reading items were carefully selected from across all parts of the GEPT High- Intermediate Level Reading Test and the GEPT Advanced Level Reading Test. Taiwanese students (n=24) studying Masters level programmes at British universities were asked to complete the test items on a computer, while the Tobii X2 Eye Tracker was used to track their gaze behaviour during completion of the test items. Immediately after they had completed each individual part, they were asked to report the cognitive process they employed by using a Reading Process Checklist, and a further (n=8) then participated in a detailed stimulated recall interview while viewing video footage of their gaze patterns. Taking into account all these sources of data, it was found that the High-Intermediate section of the GEPT test successfully elicited and tested an appropriate range of lower and higher cognitive processes, as defined in Khalifa and Weir (2009). It was also concluded that the Advanced sections of the test elicited the same set of cognitive processes as the High- Intermediate test, with the addition in the final section of the most difficult of all in Khalifa and Weir's scheme. In summary, it is apparent that the two elements of the GEPT test which were researched in this project were successful in requiring of candidates the range of cognitive processing activity commensurate with High-Intermediate and Advanced reading levels respectively, which is an important element in establishing the cognitive validity of the GEPT test.
Using eye-tracking research to inform language test validity and designThis paper reports on a recent study which used eye-tracking methodology to examine the cognitive validity of two level-specific English Proficiency Reading Tests (CEFR B2 and C1). Using a mixed-methods approach, the study investigated test takers’ reading patterns on six item types using eye-tracking, a self-report checklist and stimulated recall interviews. Twenty L2 participants completed 30 items on a computer, with the Tobii X2 Eye Tracker recording their eye movements on screen. Immediately after they had completed each item type, they reported their reading processes by using a Reading Process Checklist. Eight students further participated in a stimulated recall interview while viewing video footage of their gaze patterns on the test. The findings indicate (1) the range of cognitive processes elicited by different reading item types at the two levels; and (2) the differences between stronger and weaker test takers' reading patterns on each item type. The implications of this study to reflect on some fundamental questions regarding the use of eye-tracking in language research are discussed. The paper concludes with recommendations for future research in these areas.