• General Language Proficiency (GLP): reflections on the "issues revisited" from the perspective of a UK examination board

      Taylor, Lynda (Taylor & Francis, 2014-05-21)
      Looking back to the language testing world of the 1980s in the United Kingdom, we need to be aware that how we perceive or remember ourselves to have been then—whether as individual language testing academics or as corporate language testing organisations—will be shaped by multiple influences. Although we may have been present at and shared in the 1980 discussions, our recollections of how things were then and our views on how they have (or have not) changed will vary. What follows in this article offers a predominantly personal perspective. It is the view as I perceive it, in light of my own journey as a UK-based language teacher and tester over the past 30 years, seen from where I stand now as a consultant to a large international examining board in the United Kingdom. It is also therefore an institutional perspective, drawing on a long association with one particular language testing organisation. Just as my perspective is from the position of only one language testing institution, I am also only one individual from within that institution. There will undoubtedly be other stances, voices, and perspectives that are equally valid and relevant from within the same institution.
    • The historical frame of reference

      Weir, Cyril J.; University of Bedfordshire (2011-07-09)
    • How much does test-takers’ listening proficiency matter in oral interview tests?

      Nakatsuhara, Fumiyo; University of Bedfordshire (2011-05-08)
    • The IELTS Speaking Test: what can we learn from examiner voices?

      Inoue, Chihiro; Khabbazbashi, Nahal; Lam, Daniel M. K.; Nakatsuhara, Fumiyo; University of Bedfordshire (2018-11-25)
    • IELTS washback in context: preparation for academic writing in higher education

      Green, Anthony (Cambridge University Press, 2007-12-01)
      The International English Language Testing System (IELTS) plays a key role in international student access to universities around the world. Although IELTS includes a direct test of writing, it has been suggested that test preparation may hinder international students from acquiring academic literacy skills required for university study. This study investigates the washback of the IELTS Writing test on English for Academic Purposes (EAP) provision.
    • The impact on candidates of examiner interventions in oral interview tests

      Nakatsuhara, Fumiyo; Field, John; University of Bedfordshire (2012-06-03)
    • Implementing a learning-oriented approach within English Language assessment in Hong Kong schools: practices, issues and complexities.

      Hamp-Lyons, Liz (Palgrave Macmillan, 2016-12-16)
      This paper provides an overview of the multiple studies carried out between 2005 and 2011 on the Hong Kong School-based assessment (SBA), which was designed to implement an assessment for learning philosophy, and places the work within a learning-oriented language assessment (LOLA) paradigm (Hamp-Lyons & Green 2014) which is growing worldwide. The Hong Kong SBA continues to be used Hong Kong-wide to formatively assess the English as a second language speaking of all students in secondary years 4, 5 and 6. After discussing the structure and goals of this innovative assessment and its teacher language assessment literacy aims and processes, the chapter then discusses some of the constraints and issues, which have inhibited the degree to which the intended consequences have transpired. It points to compulsory ‘statistical moderation’, which undermines teachers’ trust in the new system; and to local contextual issues such as heavy reliance on ‘cram schools’, competition among school and teachers’ perceptions of fairness as being ‘the same for everyone’.
    • Interactional competence in the workplace: challenges and opportunities

      Galaczi, Evelina D.; Taylor, Lynda; Cambridge Assessment English; University of Bedfordshire (2018-11-25)
    • Interactional competence with and without extended planning time in a group oral assessment

      Lam, Daniel M. K. (Routledge, Taylor & Francis Group, 2019-05-02)
      Linking one’s contribution to those of others’ is a salient feature demonstrating interactional competence in paired/group speaking assessments. While such responses are to be constructed spontaneously while engaging in real-time interaction, the amount and nature of pre-task preparation in paired/group speaking assessments may have an influence on how such an ability (or lack thereof) could manifest in learners’ interactional performance. Little previous research has examined the effect of planning time on interactional aspects of paired/group speaking task performance. Within the context of school-based assessment in Hong Kong, this paper analyzes the discourse of two group interactions performed by the same four student-candidates under two conditions: (a) with extended planning time (4–5 hours), and (b) without extended planning time (10 minutes), with the aim of exploring any differences in student-candidates’ performance of interactional competence in this assessment task. The analysis provides qualitative discourse evidence that extended planning time may impede the assessment task’s capacity to discriminate between stronger and weaker candidates’ ability to spontaneously produce responses contingent on previous speaker contribution. Implications for the implementation of preparation time for the group interaction task are discussed.
    • Interactional competence: conceptualisations, operationalisations, and outstanding questions

      Galaczi, Evelina D.; Taylor, Lynda (Routledge, 2018-04-30)
      This article on interactional competence provides an overview of the historical influences that have shaped theoretical conceptualisations of this construct as it relates to spoken language use, leading to the current view of it as involving both cognitive and social dimensions, and then describes its operationalisation in tests and assessment scales, and the challenges associated with this activity. Looking into the future, issues that need to be dealt with include developing a fuller representation of the construct and of more contextually relevant assessments, deciding upon additional assessment criteria and the appropriate interpretation thereof, and determining how technology can be applied in assessment practice and the extent to which technology fundamentally changes the construct itself. These all have implications for testing if it is to be relevant and fit for purpose.
    • International assessment and local contexts: a case study of an English language initiative in higher education institutes in Egypt

      Khalifa, Hanan; Khabbazbashi, Nahal; Abdelsalam, Samar; Said, Mohsen Elmahdy; Cambridge English Language Assessment; Cairo University (Association for Language Testing and Assessment of Australia and New Zealand, 2015-11-07)
      Within the long-term objectives of English language reform in higher education (HE) institutes across Egypt and increasing employability in the global job market, the Center for Advancement of Postgraduate Studies and Research in Cairo University (CAPSCU), Cambridge English Language Assessment and the British Council (Egypt) have implemented a multi-phase upskilling program aimed at enhancing the workplace language skills of socially disadvantaged undergraduates, developing teachers’ pedagogical knowledge and application, providing both students and teachers with a competitive edge in the job markets through internationally recognised certification and the introduction of 21st century skills such as digital-age literacy and effective communication in HE, and, lastly, integrating international standards for teaching, learning and assessment within the local context. This paper reports on a mixed methods research study aimed at evaluating the effectiveness of this initiative and its impact at the micro and macro levels. The research focused on language progression, learner autonomy, motivation towards digital learning and assessment, improvements in pedagogical knowledge and teaching practices. Standardised assessment, attitudinal and perceptions surveys, and observational data were used. Findings suggested a positive impact of the upskilling program, illustrated how international collaborations can provide the necessary skills for today’s global job market, and highlighted areas for consideration for upscaling the initiative.
    • Introduction of statistical analyses for language testing/learning research (Part 1)

      Chan, Sathena Hiu Chong (University of Bedfordshire, 2014-02-19)
    • Investigating examiner interventions in relation to the listening demands they make on candidates in oral interview tests

      Nakatsuhara, Fumiyo (John Benjamins, 2018-08-08)
      Examiners intervene in second language oral interviews in order to elicit intended language functions, to probe a candidate’s proficiency level or to keep the interaction going. Interventions of this kind can affect the candidate’s output language and score, since the candidate is obliged to process them as a listener and respond to them as a speaker. This chapter reports on a study that examined forty audio-recorded interviews of the oral test of a major European examination board, with a view to examining examiner interventions (i.e., questions, comments) in relation to the listening demands they make upon candidates. Half of the interviews involved candidates who scored highly on the test while the other half featured low-scoring candidates. This enabled a comparison of the language and behaviour of the same examiner across candidate proficiency levels, to see how they were modified in response to the communicative competence of the candidate. The recordings were transcribed and analyzed with regard to a) types of examiner intervention in terms of linguistic and pragmatic features and b) the extent to which the interventions varied in response to the proficiency level of the candidate. The study provides a new insight into examiner-examinee interactions, by identifying how examiners are differentiating listening demands according to the task types and the perceived proficiency level of the candidate. It offers several implications about the ways in which examiner interventions engage candidates’ listening skills, and the ways in which listening skills can be more validly and reliably measured when using a format based on examiner-candidate interaction.
    • Investigating the cognitive constructs measured by the Aptis writing test in the Japanese context: a case study

      Moore, Yumiko; Chan, Sathena Hiu Chong; British Council (the British Council, 2018-11-30)
      This study investigates the context and cognitive validity of the Aptis General Writing Part 4 Tasks. An online survey with almost 50 Japanese universities was conducted to investigate the nature of the predominant academic writing in the wider context. Twenty-five Year 1 academic writing tasks were then sampled from a single Japanese university. Regarding the context validity of the Aptis test, online survey and expert judgement were used to examine the degree of correspondence between the task features of the Aptis task and those of the target academic writing tasks in real life. Regarding its cognitive validity, this study examined the cognitive processes elicited by the Aptis task as compared to the Year 1 writing tasks through a cognitive process questionnaire (n=35) and interviews with seven students and two lecturers. The overall resemblance between the test and the real-life tasks reported in this study supports the context and cognitive validity of the Aptis Writing test Part 4 in the Japanese context. The overall task setting (topic domain, cognitive demands and language function to be performed) of the Aptis test resembles that of the real-life tasks. Aptis Writing test Part 4 tasks, on the other hand, outperformed the sampled real-life tasks in terms of clarity of writing purpose, knowledge of criteria and intended readerships. However, when considering the wider Japanese academic context, a wider range of academic genres, such as summary and report, and some more demanding language functions such as synthesis, should also be represented in the Aptis Writing test. The results show that all target processes in each cognitive phase (conceptualisation, meaning and discourse construction, organising, low-level monitoring and revising, and high-level monitoring and revising) were reported by a reasonable percentage of the participants. Considering the comparatively lower proficiency in English of Japanese students and their unfamiliarity of direct writing assessment, the results are encouraging. However, some sub-processes such as linking important ideas and revising appear to be under-represented in Aptis. In addition, the lack of time management and typing skills of some participants appear to hinder them from spending appropriate time planning, organising, and revising at low and high levels. Recommendations are provided to address these issues.
    • Investigating the cognitive validity of EAP reading-into-writing test tasks: a pilot study

      Chan, Sathena Hiu Chong; University of Bedfordshire (2011-07-07)
    • An investigation into double-marking methods: comparing live, audio and video rating of performance on the IELTS Speaking Test

      Nakatsuhara, Fumiyo; Inoue, Chihiro; Taylor, Lynda (The IELTS Partners: British Council, IDP: IELTS Australia and Cambridge English Language Assessment, 2017-03-01)
      This study compared IELTS examiners’ scores when they assessed test-takers’ spoken performance under live and two non-live rating conditions using audio and video recordings. It also explored examiners’ perceptions towards test-takers’ performance in the two non-live rating modes.  This was a mixed-methods study that involved both existing and newly collected datasets. A total of six trained IELTS examiners assessed 36 test-takers’ performance under the live, audio and video rating conditions. Their scores in the three modes of rating were calibrated using the multifaceted Rasch model analysis.  In all modes of rating, the examiners were asked to make notes on why they awarded the scores that they did on each analytical category. The comments were quantitatively analysed in terms of the volume of positive and negative features of test-takers’ performance that examiners reported noticing when awarding scores under the three rating conditions.  Using selected test-takers’ audio and video recordings, examiners’ verbal reports were also collected to gain insights into their perceptions towards test-takers’ performance under the two non-live conditions.  The results showed that audio ratings were significantly lower than live and video ratings for all rating categories. Examiners noticed more negative performance features of test-takers under the two non-live rating conditions than the live rating condition. The verbal report data demonstrated how having visual information in the video-rating mode helped examiners to understand test-takers’ utterances, to see what was happening beyond what the test-takers were saying and to understand with more confidence the source of test-takers’ hesitation, pauses and awkwardness in their performance.  The results of this study have, therefore, offered a better understanding of the three modes of rating, and a recommendation was made regarding enhanced double-marking methods that could be introduced to the IELTS Speaking Test.
    • An investigation into double-marking methods: comparing live, audio and video rating of performance on the IELTS Speaking Test

      Nakatsuhara, Fumiyo; Inoue, Chihiro; Taylor, Lynda; University of Bedfordshire (IELTS Partners, 2017-03-01)
      This study compared IELTS examiners’ scores when they assessed test-takers’ spoken performance under live and two non-live rating conditions using audio and video recordings. It also explored examiners’ perceptions towards test-takers’ performance in the two non-live rating modes.  This was a mixed-methods study that involved both existing and newly collected datasets. A total of six trained IELTS examiners assessed 36 test-takers’ performance under the live, audio and video rating conditions. Their scores in the three modes of rating were calibrated using the multifaceted Rasch model analysis.  In all modes of rating, the examiners were asked to make notes on why they awarded the scores that they did on each analytical category. The comments were quantitatively analysed in terms of the volume of positive and negative features of test-takers’ performance that examiners reported noticing when awarding scores under the three rating conditions.  Using selected test-takers’ audio and video recordings, examiners’ verbal reports were also collected to gain insights into their perceptions towards test-takers’ performance under the two non-live conditions.  The results showed that audio ratings were significantly lower than live and video ratings for all rating categories. Examiners noticed more negative performance features of test-takers under the two non-live rating conditions than the live rating condition. The verbal report data demonstrated how having visual information in the video-rating mode helped examiners to understand test-takers’ utterances, to see what was happening beyond what the test-takers were saying and to understand with more confidence the source of test-takers’ hesitation, pauses and awkwardness in their performance.  The results of this study have, therefore, offered a better understanding of the three modes of rating, and a recommendation was made regarding enhanced double-marking methods that could be introduced to the IELTS Speaking Test.
    • Language assessment literacy for learning-oriented language assessment

      Hamp-Lyons, Liz (Australian Association of Applied Linguistics, 2017-12-16)
       A small-scale and exploratory study explored a set of authentic speaking test video samples from the Cambridge: First (First Certificate of English) speaking test, in order to learn whether, and where, opportunities might be revealed in, or inserted into formal speaking tests, order to provide language assessment literacy opportunities for language teachers teaching in test preparation courses as well as teachers training to become speaking test raters. By paying particular attention to some basic components of effective interaction that we would want an examiner or interlocutor to exhibit if they seek to encourage interactive responses from test candidates. Looking closely at body language (in particular eye contact; intonation, pacing and pausing), management of turn-taking, and elicitation of candidate-candidate interaction we saw ways in which a shift in focus to view tests as learning opportunities is possible: we call this new focus learning-oriented language assessment (LOLA).