Now showing items 1-20 of 157

    • Applying the socio-cognitive framework: gathering validity evidence during the development of a speaking test

      Nakatsuhara, Fumiyo; Dunlea, Jamie; University of Bedfordshire; British Council (UCLES/Cambridge University Press, 2020-06-18)
      This chapter describes how Weir’s (2005; further elaborated in Taylor (Ed) 2011) socio-cognitive framework for validating speaking tests guided two a priori validation studies of the speaking component of the Test of English for Academic Purposes (TEAP) in Japan. In this chapter, we particularly reflect upon the academic achievements of Professor Cyril J Weir, in terms of: • the effectiveness and value of the socio-cognitive framework underpinning the development of the TEAP Speaking Test while gathering empirical evidence of the construct underlying a speaking test for the target context • his contribution to developing early career researchers and extending language testing expertise in the TEAP development team.
    • Three current, interconnected concerns for writing assessment

      Hamp-Lyons, Liz (Elsevier Ltd, 2014-09-26)
      Editorial
    • The need for EAP teacher knowledge in assessment

      Schmitt, Diane; Hamp-Lyons, Liz; Nottingham Trent University; University of Bedfordshire (Elsevier Ltd, 2015-05-08)
    • What is a John Swales?

      Hamp-Lyons, Liz (Elsevier Ltd, 2015-09-03)
      Editorial
    • Opposing tensions of local and international standards for EAP writing programmes: who are we assessing for?

      Bruce, Emma; Hamp-Lyons, Liz; City University of Hong Kong; University of Bedfordshire (Elsevier Ltd, 2015-04-24)
      In response to recent curriculum changes in secondary schools in Hong Kong including the implementation of the 3-3-4 education structure, with one year less at high school and one year more at university and the introduction of a new school leavers' exam, the Hong Kong Diploma of Secondary Education (HKDSE), universities in the territory have revisited their English language curriculums. At City University a new EAP curriculum and assessment framework was developed to fit the re-defined needs of the new cohort of students.In this paper we describe the development and benchmarking process of a scoring instrument for EAP writing assessment at City University. We discuss the opposing tensions of local (HKDSE) and international (CEFR and IELTS) standards, the problems of aligning EAP needs-based domain scales and standards with the CEFR and the issues associated with attempting to fulfil the institutional expectation that the EAP programme would raise students' scores by a whole CEFR scale step. Finally, we consider the political tensions created by the use of external, even international, reference points for specific levels of writing performance from all our students and suggest the benefits of a specific, locally-designed, fit-for-purpose tool over one aligned with universal standards.
    • The future of JEAP and EAP

      Hamp-Lyons, Liz (Elsevier Ltd, 2015-12-12)
      Editorial
    • Farewell to holistic scoring?

      Hamp-Lyons, Liz (Elsevier, 2016-01-26)
      Editorial
    • Using assessment to promote learning: clarifying constructs, theories, and practices

      Leung, Cyril; Davison, C.; East, M.; Evans, M.; Liu, Y.; Hamp-Lyons, Liz; Purpura, J.E. (Georgetown University Press, 2017-11-22)
    • Why researching EAP practice?

      Hamp-Lyons, Liz (Elsevier Ltd, 2018-01-08)
    • Comparing rating modes: analysing live, audio, and video ratings of IELTS Speaking Test performances

      Nakatsuhara, Fumiyo; Inoue, Chihiro; Taylor, Lynda; (Taylor & Francis, 2020-08-26)
      This mixed methods study compared IELTS examiners’ scores when assessing spoken performances under live and two ‘non-live’ testing conditions using audio and video recordings. Six IELTS examiners assessed 36 test-takers’ performances under the live, audio, and video rating conditions. Scores in the three rating modes were calibrated using the many-facet Rasch model (MFRM). For all three modes, examiners provided written justifications for their ratings, and verbal reports were also collected to gain insights into examiner perceptions towards performance under the audio and video conditions. Results showed that, for all rating criteria, audio ratings were significantly lower than live and video ratings. Examiners noticed more negative performance features under the two non-live rating conditions, compared to the live condition. However, richer information about test-taker performance in the video mode appeared to cause raters to rely less on such negative evidence than audio raters when awarding scores. Verbal report data showed that having visual information in the video-rating mode helped examiners to understand what the test-takers were saying, to comprehend better what test-takers were communicating using non-verbal means, and to understand with greater confidence the source of test-takers’ hesitation, pauses, and awkwardness.
    • Assessment for learning in language education

      Green, Anthony (Urmia University, 2018-10-01)
      This paper describes the growing interest in assessment for learning (AfL) approaches in language education. It explains the term, traces the origins of AfL in developments in general education and considers the evidence for its claimed impact on learning outcomes. The paper sets out some of the challenges involved in researching, implementing and evaluating AfL initiatives in the context of language teaching and learning and considers how this may impact on our field in the future.
    • Writing: the re-construction of language

      Davidson, Andrew (Elsevier, 2018-09-13)
      This paper takes as its point of departure David Olson’s contention (as expressed in The Mind on Paper, (2016) CUP, Cambridge) that writing affords a meta-representation of language through allowing linguistic elements to become explicit objects of awareness. In so doing, a tradition of suspicion of writing (e.g. Rousseau and Saussure) that sees it as a detour from and contamination of language is disarmed: writing becomes innocent, becomes naturalised. Also disarmed are some of the concerns given rise to by the observation made in the title of Per Linell’s book of a ‘written language bias in linguistics’ (2005, Routledge, London) with its attendant criticisms of approaches (e.g. Chomsky’s) that assume written language to be transparent to the putative underlying natural object. Taking Chomsky’s position (an unaware scriptism) as a representative point of orientation and target of critique, the paper assembles evidence that problematises the first-order, natural reality of cardinal linguistic constructs: phonemes, words and sentences. It is argued that the facticity of these constructs is artefactual and that that facticity is achieved by way of the introjection of ideal objects which the mind constructs as denotations of elements of an alphabetic writing system: the mental representation of language is transformed by engagement with writing and it is this non-natural artefact to which Structuralist/Generativist linguistics has been answering. Evidence for this position from the psycholinguistic and neurolinguistic literature is presented and discussed. The conclusion arrived at is that the cultural practice of literacy re-configures the cognitive realisation of language. Olson takes writing to be a map of the territory; however, it is suggested that the literate mind re-constructs the territory to answer to the features of the map.
    • Reflecting on the past, embracing the future

      Hamp-Lyons, Liz; University of Bedfordshire (Elsevier, 2019-10-14)
      In the Call for Papers for this anniversary volume of Assessing Writing, the Editors described the goal as “to trace the evolution of ideas, questions, and concerns that are key to our field, to explain their relevance in the present, and to look forward by exploring how these might be addressed in the future” and they asked me to contribute my thoughts. As the Editor of Assessing Writing between 2002 and 2017—a fifteen-year period—I realised from the outset that this was a very ambitious goal, l, one that no single paper could accomplish. Nevertheless, it seemed to me an opportunity to reflect on my own experiences as Editor, and through some of those experiences, offer a small insight into what this journal has done (and not done) to contribute to the debate about the “ideas, questions and concerns”; but also, to suggest some areas that would benefit from more questioning and thinking in the future. Despite the challenges of the task, I am very grateful to current Editors Martin East and David Slomp for the opportunity to reflect on these 25 years and to view them, in part, through the lens provided by the five articles appearing in this anniversary volume.
    • Research and practice in assessing academic English: the case of IELTS

      Taylor, Lynda; Saville, N. (Cambridge University Press, 2019-12-01)
      Test developers need to demonstrate they have premised their measurement tools on a sound theoretical framework which guides their coverage of appropriate language ability constructs in the tests they offer to the public. This is essential for supporting claims about the validity and usefulness of the scores generated by the test.  This volume describes differing approaches to understanding academic reading ability that have emerged in recent decades and goes on to develop an empirically grounded framework for validating tests of academic reading ability.  The framework is then applied to the IELTS Academic reading module to investigate a number of different validity perspectives that reflect the socio-cognitive nature of any assessment event.  The authors demonstrate how a systematic understanding and application of the framework and its components can help test developers to operationalise their tests so as to fulfill the validity requirements for an academic reading test.  The book provides:   An up to date review of the relevant literature on assessing academic reading  A clear and detailed specification of the construct of academic reading  An evaluation of what constitutes an adequate representation of the construct of academic reading for assessment purposes  A consideration of the nature of academic reading in a digital age and its implications for assessment research and test development  The volume is a rich source of information on all aspects of testing academic reading ability.  Examination boards and other institutions who need to validate their own academic reading tests in a systematic and coherent manner, or who wish to develop new instruments for measuring academic reading, will find it of interest, as will researchers and graduate students in the field of language assessment, and those teachers preparing students for IELTS (and similar tests) or involved in English for Academic Purpose programmes. 
    • A comparison of holistic, analytic, and part marking models in speaking assessment

      Khabbazbashi, Nahal; Galaczi, Evelina D. (SAGE, 2020-01-24)
      This mixed methods study examined holistic, analytic, and part marking models (MMs) in terms of their measurement properties and impact on candidate CEFR classifications in a semi-direct online speaking test. Speaking performances of 240 candidates were first marked holistically and by part (phase 1). On the basis of phase 1 findings – which suggested stronger measurement properties for the part MM – phase 2 focused on a comparison of part and analytic MMs. Speaking performances of 400 candidates were rated analytically and by part during that phase. Raters provided open comments on their marking experiences. Results suggested a significant impact of MM; approximately 30% and 50% of candidates in phases 1 and 2 respectively were awarded different (adjacent) CEFR levels depending on the choice of MM used to assign scores. There was a trend of higher CEFR levels with the holistic MM and lower CEFR levels with the part MM. While strong correlations were found between all pairings of MMs, further analyses revealed important differences. The part MM was shown to display superior measurement qualities particularly in allowing raters to make finer distinctions between different speaking ability levels. These findings have implications for the scoring validity of speaking tests.
    • Cognitive validity in language testing: theory and practice

      Field, John; University of Bedfordshire (2012-07-05)
    • Working for washback from university entrance tests in Japan

      Green, Anthony; University of Bedfordshire (2013-07-11)