Recent Submissions

  • Use of keystroke logging to collect cognitive validity evidence for integrated writing tests

    Chan, Sathena Hiu Chong; University of Bedfordshire (Cambridge University Press, 2024-08-01)
    Integrated writing tasks are commonly used for teaching, learning and assessment purposes in most higher education contexts. These tasks are cognitively demanding as they require students to transform knowledge by engaging in processes of discourse synthesis, i.e. selecting, organising, and connecting information from multiple source texts into a new or synthesis text. The purpose of the present exploratory study was to investigate L2 writers’ discourse synthesis processes underlying the performance of an integrated reading-writing task. The participants were three university students who completed an integrated reading-writing task as part of a postadmission academic literacy test at a British university. Data were collected using a variety of qualitative research techniques: analysis of keystroke logs, retrospective interviews, and text quality analysis. Data analysis revealed distinct engagement in discourse synthesis processes among L2 writers. The study proposes a qualitative approach to analysing keystroke logging data to collect cognitive validity evidence (i.e. test takers’ engagement in discourse synthesis) underlying integrated writing test performance. The other major implications of the findings are the need for explicit teaching and assessment of these discourse synthesis processes, i.e. selecting, connecting and organising relevant ideas from multiple reading stimuli to produce a text, and the need to construct specific rating descriptors which reflect skills of discourse synthesis for integrated writing tasks.
  • Differences between L2 listening and reading

    Chan, Sathena Hiu Chong (Routledge, 2024-07-31)
    With technology being an increasingly important presence in modern life, children, adolescents and L2 learners are exposed to more and more digital materials, such as audio books, interactive posters with sound files and videos, and TED talks, in classrooms and daily life. These digital audio-visual materials are increasingly becoming a major source of information and learning (Khabbazbashi, Chan, & Clark, 2022). Herring (2019) argues that education is now operating within a communication paradigm that is “fundamentally multimodal”. The affordances of new digital platforms (e.g., Google classroom, Zoom, Microsoft Teams) mean that L2 learners can now more easily collaborate with their peers to complete group work at home. Such a shift means that L2 listening comprehension is playing a more prominent role in social and educational contexts. Nevertheless, it has not received as much attention as reading comprehension has in second language acquisition, assessment, and pedagogical research, especially in relation to the processes involved in L2 listening (Field, 2008, 2013). Furthermore, listening comprehension is often conflated with reading comprehension and operationalized in a similar way in pedagogical and assessment practice (van Zeeland & Schmitt, 2013). To contribute to the discussion of how the processes of L2 reading and listening comprehension differ, this chapter provides an overview of cognitive models of L2 reading and listening, and discusses how input modality may affect the process of comprehension, followed by a discussion of the differences between L2 reading and listening. Based on the account of the nature of L2 reading and listening, the chapter will discuss the implications for task design by contrasting some key characteristics in reading and listening texts and their impact on comprehension.
  • The one with all the sarcasm: Pauline Madella discusses the pragmatics of Chandler Bing’s sarcastic humour

    Madella, Pauline; University of Bedfordshire (2024-08-26)
    It is precisely this subjectivity – what Perry called Chandler’s “world-weary yet witty view of life”, his sharing of impressions, emotions, feelings, attitudes and moods – that is “descriptively ineffable”. In those instances where Chandler does not commit to one single interpretation, but rather offers a diffuse impression, it is hard to pin down exactly what it is that is being expressed. And yet we understand his intentions. In order to understand, we attend to a number of nonverbal as well as verbal cues and do a great amount of inferential reconstruction. In my research, I urge language practitioners (specifically second language practitioners) to expose language learners to instances of ineffable communication, where attention to nonverbal stimuli is key to understanding the speaker’s intended effects. When I am asked how I learned English as a second language, I often say “By watching Friends”. The tragic passing of Matthew Perry on 28 October 2023 was followed by an outpouring of tributes from his friends and fans across the globe. They remembered him as a comedy legend who was able to nail a complex mix of timing, pace, cadence and emphasis, while bringing joy – and belly-aching laughter – to millions. Beyond its contribution to Linguistics and my sharing of a long-held passion for Pragmatics with language lovers, this article is meant as a tribute to Matty Perry and the iconic Chandler Muriel Bing.
  • Ask a linguist: experts answer your questions: "What exactly is contrastive stress in English?”

    Madella, Pauline; University of Bedfordshire (2024-08-26)
    It is not what you say but how you say it. In spoken English, the intonation contours of an utterance (also called prosody) can greatly affect the meaning that the speaker conveys. Contrastive stress is often described as the most conspicuous and ubiquitous prosodic phenomenon in English (you may also see it called contrastive focus, contrastive accent or prosodic contrastive focus). Contrastive stress is used to draw the addressee’s attention to a particular constituent in an utterance – one that is not typically accented – and, in doing so, it triggers a particular interpretation of the utterance. Its acoustic salience or extra ‘oomph’ is characterised by greater auditory prominence and articulatory care, loudness, and increased intensity.
  • Exploring the dynamic relationship between Dr. GEPT feedback and learners’ L2 motivation

    Nakatsuhara, Fumiyo; Lam, Daniel M. K.; Jones, Johnathan; Chan, Sathena Hiu Chong; Chen, Sean; Wu, Rachel; The Language Training and Testing Center, Taiwan (The Language Training and Testing Center, Taiwan, 2023-11-03)
    Feedback is an important means to bridge assessment and learning, but its usefulness ultimately depends on whether and how learners engage with and act on the feedback. Learners’ L2 learning motivation may interact with feedback in meaningful and consequential ways, yet there is relatively little research to date that explores such a dynamic relationship, particularly among language learners in secondary education. This study aimed to fill this gap by exploring the relationship between learners’ motivation and assessment feedback offered by Dr. GEPT – automated personalised feedback provided to GEPT each test-taker alongside their test scores, including an analysis of their strengths and weaknesses, learning advice, and vocabulary and sentence patterns for further study. Taking a mixed-methods approach, Phase 1 of this study involved a large-scale questionnaire survey (n = 635) to explore L2 motivation among senior high school learners of English in Taiwan and their general perceptions towards assessment feedback. The questionnaire was developed based on the L2 Motivational Self System model (Dörnyei, 2005, 2009). Phase 2 used learning logs (n = 14) and interviews (n = 10) for an in-depth qualitative inquiry into how learners engaged with Dr. GEPT feedback and how the feedback might have shaped the developments in learners’ learning journeys. The report concludes with a discussion of how Dr. GEPT helps learners develop a positive orientation towards assessments and cultivates learner autonomy, as well as making some suggestions for enhancing the effectiveness of Dr. GEPT feedback.
  • Exploring the speaking construct in academic settings in a digital age

    Nakatsuhara, Fumiyo; May, Lyn; Khabbazbashi, Nahal; British Council; Cambridge Assessment English; IDP: IELTS Australia (British Council, Cambridge Assessment English and IDP: IELTS Australia, 2023-08-16)
    This study explored language functions and skills utilised in technology-mediated academic speaking contexts, which is timely given the increasing prevalence of digitally-mediated communication in higher education settings and the recent introduction of IELTS Indicator featuring a video-call mode in the Speaking Test. Using an embedded mixed-methods approach, the research involved: 1. language function analysis of spoken communication and simultaneous written chat contributions in online taught classes and supervision meetings 2. thematic analysis of students’ and lecturers’ understandings of distinctive features of online academic speaking and what constitutes successful online speaking interaction in those contexts. We analysed a total of over 40 hours of recordings, consisting of 17 video-recorded classes from four undergraduate and postgraduate units in an Australian University, and 23 video/audio recordings of online PhD supervision meetings from a UK university. This was followed by the administration of a questionnaire and semi-structured interviews with selected participants. In order to examine the construct of online academic communication, we adapted O’Sullivan et al.’s (2002) language function checklist for our purposes. Following the identification of language functions and skills observed in real-life online academic settings, we explored the synergy between the functions observed in online teaching and learning contexts and those elicited in the video-call IELTS Speaking Test (Nakatsuhara et al., 2021). Analyses of questionnaire and interview data helped us understand the skills perceived to be important for successful online interaction. The report concludes with a discussion on the multimodal construct of speaking in digitally-mediated academic contexts and the ways in which the findings of this study can be useful in informing the future development of IELTS Speaking Test tasks so that they remain representative of the reality of academic speaking in the digital age.
  • Accommodations in language testing and assessment: Safeguarding equity, access, and inclusion [editorial[

    Taylor, Lynda; Banerjee, Jayanti; University of Bedfordshire; Trinity College, London (SAGE Publications Ltd, 2023-10-07)
    editorial
  • Language assessment accommodations: issues and challenges for the future [editorial]

    Taylor, Lynda; Banerjee, Jayanti; University of Bedfordshire; Trinity College, London (SAGE Publications Ltd, 2023-10-07)
    In this concluding piece to the special issue, we attempt to tease out and comment on some themes that have emerged from the six published papers. Some of these themes highlight potential avenues for further theoretical and empirical investigation, and may assist in mapping out a coherent research agenda on the topic for language testers and assessment specialists in the future.
  • Relevance and multimodal prosody: implications for L2 teaching and learning

    Madella, Pauline; ; University of Bedfordshire (Frontiers, 2023-12-01)
    In this paper, I build on Scott's relevance-theoretic account of contrastive stress (2021). Contrastive stress works as an extra cue to ostension in altering the salience of a particular constituent in an utterance and, as a result, the salience of one particular interpretation of that utterance. I draw on Scott’s argument that contrastive stress does not encode procedural meaning. Contrastive stress is unpredictable and, as such, it is in confounding the hearer’s expectations that it draws his attention to the accented word and prompt his search for different interpretive effects. I argue that contrastive stress is interpreted purely inferentially precisely because it is one of many pointing devices. It is to be interpreted by virtue of its interaction with other paralinguistic behaviours, all of which being different aspects of the same ostensive act of communication. This leads me to focus on the gestural nature of contrastive stress working as an act of pointing, which, as an ostensive communicative behaviour, conveys that if you look over there, you’ll know what I mean (Tomasello et al., 2007). Finally, I present the implications of analysing contrastive stress in its multimodal context – as prosodic pointing – for the teaching and learning of L2 prosodic pragmatics and the development of interpretive abilities in the L2 hearer’s mind.
  • Exploring open consonantal environments for at-home testing of vowel perception in advanced L2 speakers

    Jones, Johnathan; (Applied Linguistics Review, 2022-11-29)
    Recent work has called for increased investigation into methods used to explore second language (L2) speech perception (Flege 2021). The present study attends to this call, examining a common practice for developing listening prompts in the context of at-home administrations. Vowel perception studies have historically used fixed consonantal frames to determine how well participants can discriminate between target L2 vowels, and the present study compares the effects of employing a fixed consonant-vowel-consonant frame (h-vowel-d) with open (phonologically diverse) consonantal environments using real and nonce words. Thirty-eight Mandarin (n = 31) and English (n = 8) first language speakers participated in a listening experiment and a post-experiment question. Data were framed within Best and Tyler’s (2007) Perceptual Assimilation Model-L2. Internal consistency and proportion correct were calculated and a generalised linear mixed model design was used to investigate how well performance with h-vowel-d prompts predicts performance with the more diverse prompt types. Results suggest an inflation of scores for the fixed frame prompt and support the use of diverse words for listening prompt designs. Findings have implications for vowel perception researchers as well as computer (and mobile) assisted language learning developers wishing to inform their designs with relevant empirical evidence.
  • Non-verbal communication and context: multi-modality in interaction

    Madella, Pauline; Wharton, Tim; Romero-Trillo, Jesús; University of Bedfordshire; University of Brighton; Universidad Autónoma de Madrid (Cambridge University Press, 2023-11-01)
    Traditionally, the study of linguistics has focussed on verbal communication. In the sense that linguistics is the scientific study of language, the approach is perfectly justified. Those working in the sub-discipline of linguistic pragmatics, however, are faced with something of a dilemma. The aim of a pragmatic theory is to explain how utterances are understood, and utterances, of course, have both linguistic and non-linguistic properties. As well as this, current work in pragmatics emphasizes that the affective dimension of a speaker’s meaning is at least as important as the cognitive one and it is often the non-linguistic properties of utterances that convey information relating to this dimension. This paper highlights the major role of non-verbal ‘modes’ of communication (‘multi-modality’) in accounting for how meaning is achieved and explores in particular how the quasi-musical contours we impose on the words we say, as well as the movements of our face and hands that accompany speech, constrain the context and guide the hearer to our intended meaning. We build on previous exploration of the relevance of prosody (Wilson and Wharton 2006) and, crucially, looks at prosody in relation to other non-verbal communicative behaviours from the perspective of relevance theory. In-so-doing, we also hope to shed light on the role of multimodality in both context construction and utterance interpretation and suggest prosody needs to be analysed as one tool in a set of broader gestural ones (Bolinger 1983). Relevance theory is an inferential model, in which human communication revolves around the expression and recognition of the speaker’s intentions in the performance of an ostensive stimulus: an act accompanied by the appropriate combination of intentions. This inferential model is proposed as a replacement for the traditional code-model of communication, according to which a speaker simply encodes into a signal the thought they wish to communicate and the hearer retrieves their meaning by decoding the signal they have provided. We will argue that much existing work on multimodality remains rooted in a code model and show how adopting an inferential model enables us to integrate multimodal behaviours more completely within a theory of utterance interpretation. As ostensive stimuli, utterances are composites of a range of different behaviours, each working together to form a range of contextual cues.
  • L2 writing assessment: an evolutionary perspective

    Green, Anthony (Palgrave Macmillan, 2022-12-08)
    This book tackles three choices that face developers of L2 writing assessments: defining L2 writing abilities; collecting evidence of those abilities (usually by getting L2 writers to write something); and judging their performance (usually by assigning a score or grade to it). It takes a historical view of how assessment developers have made those choices, how contemporary practices emerged, and of alternative techniques that have risen and fallen over time. The three sections each tackle one of these choices. The first considers the social functions that define L2 writing and assessment; the second relates how assessment tasks have adapted to changing conceptions of languages, writing, and assessment; and the third explores how scoring systems have evolved. Each section brings the reader up to date with current issues confronting writing assessment (both in large-scale testing and in language classrooms) before considering the new opportunities and challenges of the digital age. This book will be of interest to students, scholars and practitioners in language assessment, language education, and applied linguistics.
  • The impact of input format on written performance in a listening-into-writing assessment

    Westbrook, Carolyn; British Council; University of Bedfordshire (Elsevier Ltd, 2022-12-06)
    Over the last five decades, research in teaching and testing (academic) listening has investigated different foci. Initially, teaching listening involved bottom-up approaches (Dirven and Oakeshott-Taylor, 1984) then both higher- and lower-level processes were integrated (Voss, 1984). In the early 2000s, different input formats (Read, 2002) and discourse features of lectures (Thompson, 2003) were the subjects of academic listening research. More recently, EAP tests have increasingly taken an integrated approach to reflect real-world tasks, yet few studies have looked at integrated listening-into-writing tasks (Cubilo and Winke, 2013). This counter-balanced measures design study investigates how test taker performance differs on an integrated EAP listening-into-writing task when lecture input is presented as audio only in one half and video in the other half of the input. Two groups of test takers took part in the current study. A Hotelling's T2 test revealed a statistically significant effect on scores when test takers were presented with the audio only input first but there was no significant effect on scores when the video input was presented first. Data on test taker preferences revealed that more people preferred the video input to audio only.
  • Assessing second language pronunciation: a reference guide

    Jones, Johnathan; Isaacs, Talia (Springer, 2022-01-14)
    Pronunciation assessment (PA) is a resurgent subfield within applied linguistics that traverses the domains of psycholinguistics, second language acquisition (SLA), speech sciences, sociolinguistics, and more recently, computational linguistics. Though the terms ‘pronunciation’ and ‘assessment’ are sometimes defined in different ways by different authors, here we regard pronunciation as the vocal articulation of consonants and vowels (segmentals) combined with aspects of oral speech that extend beyond individual sounds, including stress, rhythm and intonation (suprasegmentals).
  • Towards more valid scoring criteria for integrated reading-writing and listening-writing summary tasks

    Chan, Sathena Hiu Chong; May, Lyn; (SAGE, 2022-12-12)
    Despite the increased use of integrated tasks in high-stakes academic writing assessment, research on rating criteria which reflect the unique construct of integrated summary writing skills is comparatively rare. Using a mixed-method approach of expert judgement, text analysis and statistical analysis, the current study examines writing features that discriminate summaries produced by 150 candidates at five levels of proficiency on integrated reading-writing (R-W) and listening-writing (LW) tasks. The expert judgement revealed a wide range of features which discriminated R-W and L-W responses. When responses at five proficiency levels were coded by these features, significant differences were obtained in seven features, including relevance of ideas, paraphrasing skills, accuracy of source information, academic style, language control, coherence and cohesion and task fulfilment across proficiency levels on the R-W task. The same features did not yield significant differences in L-W responses across proficiency levels. The findings have important implications for clarifying the construct of integrated summary writing in different modalities, indicating the possibility of expanding integrated rating categories with some potential for translating the identified criteria into automated rating systems. The results on the L-W indicate the need for developing descriptors which can more effectively discriminate L-W responses.
  • Assessing interactional competence: exploring ratability challenges

    Lam, Daniel M. K.; Galaczi, Evelina D.; Nakatsuhara, Fumiyo; May, Lyn; University of Glasgow; Cambridge University Press and Assessment; University of Bedfordshire; Queensland University of Technology (John Benjamins, 2023-02-13)
    This paper is positioned at the interface of second/foreign language (L2) assessment and Conversation Analysis-Second Language Acquisition (CA-SLA). It explores challenges of ratability in assessing interactional competence (IC) from three dimensions: an overview of the conceptual and terminological convergence/divergence in the CA-SLA and L2 assessment literature, a micro-analytic Conversation Analysis of test-taker interactions, and the operationalisation of IC construct features in rating scales across assessment contexts. It draws insights from these dimensions into a discussion of the nature of the IC construct and the challenges of IC ratability, and concludes with suggestions on ways in which insights from CA research can contribute to addressing these issues.
  • Report to the Nursing and Midwifery Council on language testing policy

    Green, Anthony; Chan, Sathena Hiu Chong; University of Bedfordshire (Nursing and Midwifery Council, 2022-09-28)
    Responding to the NMC’s review of its language testing policy, our project involved: • A review of the extent to which the approach to language testing currently adopted by the NMC is proportionate and appropriate, and • Recommendations for a methodology to investigate whether language tests of interest should be accepted by the NMC to be met.
  • Integrated writing and its correlates: a meta-analysis

    Chan, Sathena Hiu Chong; Yamashita, J. (Elsevier, 2022-07-26)
    Integrated tasks are increasing in popularity, either replacing or complementing writing- only independent tasks in writing assessments. This shift has generated many research interests to investigate the underlying construct and features of integrated writing (IW) performances. However, due to the complexity of the IW construct, there are conflicting findings about whether and the extent to which various language skills and IW text features correlate to IW scores. To understand the construct of IW, we conducted a meta-analysis to synthesize correlation coefficients between scores of IW performances and (1) other language skills and (2) text quality features of IW. We also examined factors that may moderate the correlation of IW scores with these two groups of correlates. Consequently, (1)reading and writing skills showed stronger correlations than listening to IW scores; and (2) text length had a strongest correlation, followed by source integration, organization and syntactic complexity, with a smallest correlation of lexical complexity. Several IW task features affected the magnitude of correlations. The results supported the view that IW is an independent construct, albeit related, from other language skills and IW task features may affect the construct of IW.
  • Book review: Assessing speaking in context: expanding the construct and its applications

    Taylor, Lynda (SAGE, 2022-02-16)
    review of Salaberry MR, Burch AR (2021) Assessing speaking in context: expanding the construct and its applications, Bristol: Multilingual Matters, ISBN 9781788923804
  • Validation of a large-scale task-based test: functional progression in dialogic speaking performance

    Inoue, Chihiro; Nakatsuhara, Fumiyo (Springer Nature, 2022-02-07)
    A list of language functions is usually included in task-based speaking test specifications as a useful tool to describe target output language of test-takers, to define TLU domains, and to specify task demands. Such lists are, however, often constructed intuitively and they also tend to focus solely on the types of function to be elicited and ignore the ways in which each function is realised across different levels of proficiency (Green, 2012). The study reported in this chapter is a part of a larger-scale test revision project for Trinity’s Integrated Skills in English (ISE) spoken examinations. Analysing audio-recordings of 32 performances on the ISE spoken examination both quantitatively and qualitatively, the aims of this study are (a) to empirically validate lists of language functions in the test specifications of the operational, large-scale, task-based examinations, (b) to explore the usefulness and potential of function analysis as a test task validation method, and (c) to contribute to a better understanding of varied test-taker language that is used to generate language functions.

View more