• Establishing the validity of reading-into-writing test tasks for the UK academic context

      Chan, Sathena Hiu Chong (University of BedfordshireUniversity of Bedfordshire, 2013-11)
      The present study aimed to establish a test development and validation framework of reading-into-writing tests to improve the accountability of using the integrated task type to assess test takers' ability in Academic English. This study applied Weir's (2005) socio-cognitive framework to collect three components of test validity: context validity, cognitive validity and criterion-related validity of two common types of reading-into-writing test tasks (essay task with multiple verbal inputs and essay task with multiple verbal and non-verbal inputs). Through literature review and a series of pilot, a set of contextual and cognitive parameters that are useful to explicitly describe the features of the target academic writing tasks and the cognitive processes required to complete these tasks successfully was defined at the pilot phase of this study. A mixed-method approach was used in the main study to establish the context, cognitive and criterion-related validity of the reading-into-writing test tasks. First of all, for context validity, expert judgement and automated textual analysis were applied to examine the degree of correspondence of the contextual features (overall task setting and input text features) of the reading-into-writing test tasks to those of the target academic writing tasks. For cognitive validity, a cognitive process questionnaire was developed to assist participants to report the processes they employed on the two reading-into-writing test tasks and two real-life academic tasks. A total of 443 questionnaires from 219 participants were collected. The analysis of the cognitive validity included three stands: 1) the cognitive processes involved in real-life academic writing, 2) the extent to which these processes are elicited by the reading-into-writing test tasks, and 3) the underlying structure of the processes elicited by the reading-into-writing test tasks. A range of descriptive, inferential and factor analyses were performed on the questionnaire data. The participants' scores on these real-life academic and reading-into-writing test tasks were collected for correlational analyses to investigate the criterion-related validity of the test tasks. The findings of the study support the context, cognitive and criterion-related validity of the integrated reading-into-writing task type. In terms of context validity, the two reading-into-writing tasks largely resembled the overall task setting, the input text features and the linguistic complexity of the input texts of the real-life tasks in a number of important ways. Regarding cognitive validity, the results revealed 11 cognitive processes involved in 5 phases of real-life academic writing as well as the extent to which these processes were elicited by the test tasks. Both reading-into-writing test tasks were able to elicit from high-achieving and low-achieving participants most of these cognitive processes to a similar extent as the participants employed the processes on the real-life tasks. The medium-achieving participants tended to employ these processes more on the real-life tasks than on the test tasks. The results of explanatory factor analysis showed that both test tasks were largely able to elicit from the participants the same underlying cognitive processes as the real-life tasks did. Lastly, for criterion-related validity, the correlations between the two reading-into-writing test scores and academic performance reported in this study are apparently better than most previously reported figures in the literature. To the best of the researcher's knowledge, this study is the first study to validate two types of reading-into-writing test tasks in terms of three validity components. The results of the study proved with empirical evidence that reading-into-writing tests can successfully operationalise the appropriate contextual features of academic writing tasks and the cognitive processes required in real-life academic writing under test conditions, and the reading-into-writing test scores demonstrated a promising correlation to the target academic performance. The results have important implications for university admissions officers and other stakeholders; in particular they demonstrate that the integrated reading-into-writing task type is a valid option when considering language teaching and testing for academic purposes. The study also puts forward a test framework with explicit contextual and cognitive parameters for language teachers, test developers and future researchers who intend to develop valid reading-into-writing test tasks for assessing academic writing ability and to conduct validity studies in such integrated task type.
    • Investigating types of reading used by native and non-native English readers on academic reading: an eye tracking study

      Sheraz, Safia (University of BedfordshireUniversity of Bedfordshire, 2018-11-08)
      The purpose of this research was to investigate the types of reading used by native English readers on a sampled academic reading-into-writing task. Secondly, this study investigated the types of reading used by non-native English readers on the same academic reading-into-writing task. This study also compared the similarities and differences in the types of reading done by both native and non-native English readers. In particular, the research explored how the participants read the informative and non-informative paragraphs in the academic texts. The present study used Khalifa and Weir's model of reading (2009) as a framework to map the physical evidence from the eye tracking data to investigate the types of reading done by the students in relation to careful, selective and expeditious reading. A mixed method approach, which included eye-tracking technology, stimulated recalls, interviews and observations, was used in this study. Eye movements of the participants were tracked by Tobii X2 60 eye tracker while they read the three academic texts on a computer screen to prepare for a piece of academic writing. Immediately after they had completed reading each text, stimulated recalls and interviews were held and the purpose of these stimulated recalls was to triangulate the data obtained from eye tracking. A total of 32 participants were used in this study which included 16 native English readers and 16 non-native English readers. The eye tracking data was analysed quantitatively through the eye tracking measures that are the words read per minute, the number of fixations, the mean fixation duration and the proportion of regressive movements (same line regressions and regressions two lines or above). Qualitatively, eye tracking data was analysed through the heat maps and gaze plots. The results from the quantitative and qualitative eye tracking data suggested that the native readers read the informative paragraphs slower than the non-informative paragraphs, but different reasons were reported by them in the stimulated recalls for adopting a specific reading type. For the non-native readers, the results from the quantitative eye tracking data suggested that they read the informative paragraphs slower than the non-informative paragraphs. On the contrary, the qualitative eye tracking data suggested that they focused on both informative and non-informative paragraphs and there were similarities in the reasons reported by non-native readers for reading the informative and the non-informative paragraphs slowly. A comparison in the types reading used by L1 and L2 readers suggested that L2 readers read the informative and the non-informative paragraphs slower than L1 readers as suggested by the words read per minute and the mean fixation duration of the participants. Native readers made more same line regressions on informative and non-informative paragraphs than the non-native readers and the proportion of regressions two lines or above made by both groups was also negligible. The qualitative eye tracking data suggested that both groups read informative and non-informative paragraphs slowly, but native readers fixated more on both types of paragraphs than non-natives, as suggested by the heat maps and gaze plots. The data from the stimulated recalls and observations also suggested that both groups read the informative and the non-informative paragraphs with attention. To the best of the researcher's knowledge, this is the first study that evaluated the types of reading of L2 participants in an academic context. It contributes a methodology for investigating the types of reading used by the L2 students by employing different eye tracking measures and the stimulated recalls. According to the researcher's knowledge, this study contributes new knowledge about the reading speed (in terms of words read per minute) and other eye tracking measures of L2 readers on an academic reading-into-writing task. This study holds implications for the universities as the findings suggested that academic literacy skills training is required for both the native and non-native readers. The findings would also help to design preparatory tasks for the first-year undergraduates as a part of pre-sessional courses in the international universities or back in their home countries where they are trained for admission in international universities.
    • Reading during an academic reading-into-writing task: an eye-tracking study

      Latimer, Nicola (University of BedfordshireUniversity of Bedfordshire, 2018-09)
      The study aimed to establish the types of reading university students engaged on an academic reading-into-writing task through a mixed-methods approach. To achieve this, eye-tracking technology was used to record 30 students’ eye-movements as they engaged in a one-hour computer-based academic reading into writing test task. After the test events, stimulated recall interviews and a cognitive process questionnaire were used to collect more comprehensive data on these students' academic reading processes. The study also investigated whether there were differences in the reading patterns of students with more experience of performing academic reading into writing tasks and those students with less experience. Differences in the way high and low scoring students tackled the task were also investigated. 30 participants (15 first year undergraduates and 15 third year undergraduates or postgraduates) were recruited from a range of UK universities. The participants were observed and their eye movements were recorded whilst they completed the reading into writing task. After the task, participants watched a replay of their reading and writing activity and were prompted to recollect their thought processes during the task. Finally, participants completed a short cognitive processing questionnaire. This research made five key findings relating to university students' academic reading processes in the context of a reading into writing task. Participants spent almost half their time (47 per cent) fixating on their own emerging text and about a third of their time reading the source texts. The task instructions were relatively poorly attended to. The fixations on the written source text were more homogeneous than fixations on the participant emerging work. Fixations on the written source texts reported a shorter mean v fixation duration and contained much higher rates of regression than for reading reported in the literature. Careful reading accounted for less than 30 per cent of reading of the written source texts. Other forms of selective reading accounted for the remaining 70 per cent of reading. The predominance of selective reading appears to result from participants targeting their reading to spend more time on the more relevant sentences, although several factors seem to interact to determine total time on each sentence. When differences between the Year 1 (Y1) and Year 3+ (Y3+) groups were examined, it emerged that the Y3+ group spent much more time fixating on their own work than the Y1 group. It also emerged that the Y3+ group engaged in more selective reading than the Y1 .The increased levels of selective reading may have contributed to the greater attention that the Y3+ group devoted to the more relevant sentences. When the results for the five highest and five lowest scoring participants were compared it emerged that the low scoring participants were much less effective at focusing their attention on the most relevant sentences. In short, these findings suggest that reading during an academic reading into writing task is different to the careful reading described in the literature. It demands a wide range of selective reading skills and strategies in addition to careful reading skills. Task representation can influence the way the writer interacts with the source text(s). The findings imply that development of selective reading skills, in conjunction with developing task representation skills, could help inexperienced students produce better written work earlier in their courses.
    • Validating aspects of a model of academic reading

      Krishnan, Sarojani Devi (University of BedfordshireUniversity of Bedfordshire, 2010-09)
      In the past, the focus in language testing, teaching and research has largely been on careful reading while expeditious (quick, efficient and selective) reading has been largely ignored. However, some research suggests that careful reading ability alone is inadequate for students to meet the demands of undergraduate academic reading. In the main English for Academic Purposes (EAP), test instruments have been previously based on careful reading models which assume reading to be unicomponential. If this is not the case, the issue for language testing is whether the construct of academic reading can be validly measured by a focus on careful reading alone. The aims of this study were to investigate the types of academic reading required of firstyear undergraduates based on Urquhart and Weir's (1998) four-cell matrix of reading types which also forms an important part of Khalifa and Weir's (2009) reading model. Based on this, a valid academic reading test battery for undergraduate students was developed and used to examine the divisibility of the academic reading construct. The literature review on reading models suggested that current models were nearly all premised on careful reading and expeditious reading had in the main been ignored. The findings of a pilot and main questionnaire survey with undergraduates suggested that both careful and expeditious reading were important in accomplishing academic reading tasks at the undergraduate level. Accordingly, the empirical data generated by these surveys validated Urquhart and Weir's (2009) reading matrix and aspects of the reading model by Khalifa and Weir (2009). Based on this matrix and aspects of the model, a valid reading test was developed and administered to first-year undergraduate students. The performance of undergraduates across the different parts of the reading test confirmed that academic reading was a divisible construct. The findings of this study add to the literature on EAL academic reading by lending empirical support to a componential approach to the teaching and testing of reading. The componential model and the test design methodology employed should help test designers develop valid academic reading tests embracing both careful and expeditious reading types. The results from such tests might usefully inform pedagogical practice leading to more efficient reading practice at undergraduate level.