• CEFR and ACTFL crosswalk: a text based approach

      Green, Anthony (Stauffenburg, 2012-01-01)
    • A cognitive pragmatic perspective on communication and culture

      Žegarac, Vladimir (De Gruyter Mouton, 2008-01-01)
    • Cognitive validity

      Field, John; University of Bedfordshire (2012-03-15)
    • Cognitive validity in language testing: theory and practice

      Field, John; University of Bedfordshire (2012-07-05)
    • Cognitive validity in the testing of speaking

      Field, John; University of Bedfordshire (2013-11-17)
    • The cognitive validity of reading and writing tests designed for young learners

      Field, John (Cambridge University Press, 2018-06-01)
      The notion of cognitive validity becomes considerably more complicated when one extends it to  tests designed for Young Learners. It then becomes necessary to take full account of the level of cognitive development of the target population (their ability to handle certain mental operations and not others). It may also be necessary to include some consideration of their level of linguistic development in L1: in particular, the degree of proficiency they may have achieved in reading and writing. This chapter examines the extent to which awareness of the cognitive development of young learners up to the age of 12 should and does influence the decisions made by those designing tests of second language reading  and writing. The limitations and strengths of young learners of this age range are matched against the various processing demands entailed in second language reading and writing and are then related to characteristics of the Young Learners tests offered by the Cambridge English examinations.
    • The cognitive validity of tests of listening and speaking designed for young learners

      Field, John (Cambridge University Press, 2018-06)
      The notion of cognitive validity becomes considerably more complicated when one extends it to  tests designed for Young Learners. It then becomes necessary to take full account of the level of cognitive development of the target population (their ability to handle certain mental operations and not others). It may also be necessary to include some consideration of their level of linguistic development in L1: in particular, the degree of proficiency they may have achieved in each of the four skills. This chapter examines the extent to which awareness of the cognitive development of young learners up to the age of 12 should and does influence the decisions made by those designing tests of second language listening and speaking. The limitations and strengths of young learners of this age range are matched against the various processing demands entailed in second language listening and speaking and are then related to characteristics of the Young Learners tests offered by the Cambridge English examinations.
    • Communication and core conditions in rapport building: a case study

      Zegarac, Vladimir; Bhatti, Joanna; Caley, Joy; University of Bedfordshire; English Language Centre, York (Brill Academic Publishers, 2015-01-01)
      The main aim of this article is to show how difficulties in communication across cultural boundaries can be addressed effectively by taking account of the complex interplay between individual, culture-specific and universal aspects of social interaction. The article considers an unconventional, creative and effective approach to dealing with a critical incident situation that arose in an intercultural efl classroom. The description and analysis of the problem situation draw on Carl Rogers' (see Kirschenbaum and Henderson, 1989) core conditions for facilitative educational practice and the key concepts of Relevance-Theoretic pragmatics (Sperber and Wilson, 1986, 1995), showing how the mechanisms of communication can be used in building positive rapport between the interactants as whole integrated individuals.
    • A comparative study of the variables used to measure syntactic complexity and accuracy in task-based research

      Inoue, Chihiro; University of Bedfordshire (Taylor & Francis (Routledge): SSH Titles, 2016-04-12)
      The constructs of complexity, accuracy and fluency (CAF) have been used extensively to investigate learner performance on second language tasks. However, a serious concern is that the variables used to measure these constructs are sometimes used conventionally without any empirical justification. It is crucial for researchers to understand how results might be different depending on which measurements are used, and accordingly, choose the most appropriate variables for their research aims. The first strand of this article examines the variables conventionally used to measure syntactic complexity in order to identify which may be the best indicators of different proficiency levels, following suggestions by Norris and Ortega. The second strand compares the three variables used to measure accuracy in order to identify which one is most valid. The data analysed were spoken performances by 64 Japanese EFL students on two picture-based narrative tasks, which were rated at Common European Framework of Reference for Languages (CEFR) A2 to B2 according to Rasch-adjusted ratings by seven human judges. The tasks performed were very similar, but had different degrees of what Loschky and Bley-Vroman term ‘task-essentialness’ for subordinate clauses. It was found that the variables used to measure syntactic complexity yielded results that were not consistent with suggestions by Norris and Ortega. The variable found to be the most valid for measuring accuracy was errors per 100 words. Analysis of transcripts revealed that results were strongly influenced by the differing degrees of task-essentialness for subordination between the two tasks, as well as the spread of errors across different units of analysis. This implies that the characteristics of test tasks need to be carefully scrutinised, followed by careful piloting, in order to ensure greater validity and reliability in task-based research.
    • Comparing rating modes: analysing live, audio, and video ratings of IELTS Speaking Test performances

      Nakatsuhara, Fumiyo; Inoue, Chihiro; Taylor, Lynda; (Taylor & Francis, 2020-08-26)
      This mixed methods study compared IELTS examiners’ scores when assessing spoken performances under live and two ‘non-live’ testing conditions using audio and video recordings. Six IELTS examiners assessed 36 test-takers’ performances under the live, audio, and video rating conditions. Scores in the three rating modes were calibrated using the many-facet Rasch model (MFRM). For all three modes, examiners provided written justifications for their ratings, and verbal reports were also collected to gain insights into examiner perceptions towards performance under the audio and video conditions. Results showed that, for all rating criteria, audio ratings were significantly lower than live and video ratings. Examiners noticed more negative performance features under the two non-live rating conditions, compared to the live condition. However, richer information about test-taker performance in the video mode appeared to cause raters to rely less on such negative evidence than audio raters when awarding scores. Verbal report data showed that having visual information in the video-rating mode helped examiners to understand what the test-takers were saying, to comprehend better what test-takers were communicating using non-verbal means, and to understand with greater confidence the source of test-takers’ hesitation, pauses, and awkwardness.
    • Comparing writing proficiency assessments used in professional medical registration: a methodology to inform policy and practice

      Chan, Sathena Hiu Chong; Taylor, Lynda; University of Bedfordshire (Elsevier, 2020-10-13)
      Internationally trained doctors wishing to register and practise in an English-speaking country typically have to demonstrate that they can communicate effectively in English, including writing proficiency. Various English language proficiency (ELP) tests are available worldwide and are used for such licensing purposes. This means that medical registration bodies face the question of which test(s) will meet their needs, ideally reflecting the demands of their professional environment. This article reports a mixed-methods study to survey the policy and practice of health-care registration organisations in the UK and worldwide. The study aimed to identify ELP tests that were, or could be, considered as suitable for medical registration purposes and to understand the differences between them. The paper discusses what the study revealed about the function and comparability of different writing tests used in professional registration as well as the complex criteria a professional body may prioritise when selecting a test. Although the original study was completed in 2015, the paper takes account of subsequent changes in policy and practice. It offers a practical methodology and worked example which may be of interest and value to other researchers, language testers and policymakers as they face challenges in selecting and making comparisons across tests.
    • A comparison of holistic, analytic, and part marking models in speaking assessment

      Khabbazbashi, Nahal; Galaczi, Evelina D. (SAGE, 2020-01-24)
      This mixed methods study examined holistic, analytic, and part marking models (MMs) in terms of their measurement properties and impact on candidate CEFR classifications in a semi-direct online speaking test. Speaking performances of 240 candidates were first marked holistically and by part (phase 1). On the basis of phase 1 findings – which suggested stronger measurement properties for the part MM – phase 2 focused on a comparison of part and analytic MMs. Speaking performances of 400 candidates were rated analytically and by part during that phase. Raters provided open comments on their marking experiences. Results suggested a significant impact of MM; approximately 30% and 50% of candidates in phases 1 and 2 respectively were awarded different (adjacent) CEFR levels depending on the choice of MM used to assign scores. There was a trend of higher CEFR levels with the holistic MM and lower CEFR levels with the part MM. While strong correlations were found between all pairings of MMs, further analyses revealed important differences. The part MM was shown to display superior measurement qualities particularly in allowing raters to make finer distinctions between different speaking ability levels. These findings have implications for the scoring validity of speaking tests.
    • Computer delivered listening tests: a sad necessity or an opportunity?

      Field, John; University of Bedfordshire (2017-07-06)
    • Conceptualizing mindfulness-mindlessness in intercultural interaction

      Žegarac, Vladimir; Spencer-Oatey, Helen; Ushioda, Ema; University of Bedfordshire; University of Warwick (John Benjamins Publishing Company, 2014-01-01)
      The concept of 'mindfulness' is increasingly used in the intercultural literature and yet so far it is largely just a heterogeneous construct with underspecified theoretical content. In this paper we draw on multidisciplinary perspectives to address this shortcoming and develop an integrated analysis of this important construct. We relate 'mindfulness' explicitly to the Relevance-theoretic concept of “manifestness”, and we incorporate insights from the psychology of motivation. We use extracts of authentic intercultural interactions to help explain and illustrate our arguments.
    • Continuity and innovation: a history of the Cambridge Proficiency in English examination 1913-2002

      Weir, Cyril J.; Milanovic, Michael (Cambridge University Press, 2003-01-01)
      This volume documents in some detail the most recent revision of Cambridge English: Proficiency, also known as Certificate of Proficiency in English (CPE), which took place from 1991 to 2002. CPE is the oldest of the Cambridge suite of English as a Foreign Language (EFL) examinations and was originally introduced in 1913. Since that time the test has been regularly revised and updated to bring it into line with current thinking in language teaching, applied linguistics and language testing theory and practice. The volume provides a full account of the revision process, the questions and problems faced by the revision teams, and the solutions they came up with. It is also an attempt to encourage in the public domain greater understanding of the complex thinking, processes and procedures which underpin the development and revision of all the Cambridge English tests, and as such it will be of interest and relevance to a wide variety of readers.
    • Contriving authentic interaction: task implementation and engagement in school-based speaking assessment in Hong Kong

      Lam, Daniel M. K.; Yu, Guoxing; Jin, Yan; University of Bedfordshire; University of Bristol; Shanghai Jiaotong University (Palgrave Macmillan, 2016-01-01)
      This chapter examines the validity of the Group Interaction task in a school-based speaking assessment in Hong Kong from the perspectives of task implementation and authenticity of engagement. The new format is intended to offer a more valid assessment than the external examination by eliciting ‘authentic oral language use’ (HKEAA, 2009, p.7) in ‘low-stress conditions’ (p.3), and emphasizes the importance of flexibility and sensitivity to students’ needs in its implementation. Such a policy has then been translated into diverse assessment practices, with considerable variation in the amount of preparation time given to students. The present study draws on three types of data, namely 1) students’ discourse in the assessed interactions, 2) stimulated recall with students and teachers, and 3) a mock assessment, where the group interaction task, the preparation time, and the post-interview were all video-recorded. Results show that while the test discourse exhibits some features that ostensibly suggest authentic interaction, a closer examination of students’ pre-task planning activities reveals the contrived and pre-scripted nature of the interaction. Implications for the assessment of students’ interactional competence and recommendations for task implementation are discussed.
    • CRELLA and the socio-cognitive approach to test validation

      Chan, Sathena Hiu Chong; University of Bedfordshire (2013-10-31)
    • CRELLA: its socio-cognitive approach to validating tests

      Nakatsuhara, Fumiyo; University of Bedfordshire (2013-05-19)
    • Defining integrated reading-into-writing constructs: evidence at the B2 C1 interface

      Chan, Sathena Hiu Chong (Cambridge University Press, 2018-06-01)
    • Demonstrating the cognitive validity and face validity of PTE Academic Writing items Summarize Written Text and Write Essay

      Chan, Sathena Hiu Chong (Pearson, 2011-07-01)
      This study examines the cognitive validity of two item types of the Writing Section of the PTE Academic test – Summarize Written Text and Write Essay - within Weir’s (2005) socio-cognitive framework for test validation. The study focuses on cognitive validity by investigating and comparing the cognitive processes of a group of ESL test takers undertaking Summarize Written Text (an integrated writing item) and Write Essay (an independent writing item). Cognitive validity is a ‘measure of how closely it [a writing task] represents the cognitive processing involved in writing contexts beyond the test itself’ (Shaw and Weir, 2007:34). In addition, the study investigates test takers’ opinions regarding the two different writing item types: independent and integrated. Test takers’ scores on both items are compared to investigate if the two performances correlate. The study uses screen capture technique to record test takers’ successive writing processes on both items, followed by retrospective stimulated recalls. The findings demonstrate that Summarize Written Text and Write Essay engage different cognitive processes that are essential in academic writing contexts. In particular, macro-planning and discourse synthesis processes such as selecting relevant ideas from source text are elicited by the Summarize Written Text item whereas processes in micro-planning, monitoring and revising at low levels are activated on the Write Essay item. In terms of test performances, the results show that test takers in this study performed significantly better on Write Essay than on Summarize Written Text.