• Evaluation of complex integrated care programmes: the approach in North West London

      Greaves, Felix; Pappas, Yannis; Bardsley, Martin; Harris, Matthew; Curry, Natasha; Holder, Holly; Blunt, Ian; Soljak, Michael; Gunn, Laura H.; Majeed, Azeem; et al. (Igitur Publishing, 2013)
      Background: Several local attempts to introduce integrated care in the English National Health Service have been tried, with limited success. The Northwest London Integrated Care Pilot attempts to improve the quality of care of the elderly and people with diabetes by providing a novel integration process across primary, secondary and social care organisations. It involves predictive risk modelling, care planning, multidisciplinary management of complex cases and an information technology tool to support information sharing. This paper sets out the evaluation approach adopted to measure its effect. Study design: We present a mixed methods evaluation methodology. It includes a quantitative approach measuring changes in service utilization, costs, clinical outcomes and quality of care using routine primary and secondary data sources. It also contains a qualitative component, involving observations, interviews and focus groups with patients and professionals, to understand participant experiences and to understand the pilot within the national policy context. Theory and discussion: This study considers the complexity of evaluating a large, multi-organisational intervention in a changing healthcare economy. We locate the evaluation within the theory of evaluation of complex interventions. We present the specific challenges faced by evaluating an intervention of this sort, and the responses made to mitigate against them. Conclusions: We hope this broad, dynamic and responsive evaluation will allow us to clarify the contribution of the pilot, and provide a potential model for evaluation of other similar interventions. Because of the priority given to the integrated agenda by governments internationally, the need to develop and improve strong evaluation methodologies remains strikingly important.
    • Integrated care pilot in North West London: a mixed methods evaluation

      Curry, Natasha; Harris, Matthew; Gunn, Laura H.; Pappas, Yannis; Blunt, Ian; Soljak, Michael; Mastellos, Nikolaos; Holder, Holly; Smith, Judith; Majeed, Azeem; et al. (Igitur Publishing, 2013)
      Introduction: This paper provides the results of a year-long evaluation of a large-scale integrated care pilot in north-west London. The pilot aimed to integrate care across primary, acute, community, mental health and social care for people with diabetes and/or those aged 75+ through care planning, multidisciplinary case reviews, information sharing and project management support. Methods: The evaluation team conducted qualitative studies of change at organisational, clinician and patient levels (using interviews, focus groups and a survey); and quantitative analysis of change in service use and patient-level clinical outcomes (using patient-level datasets and a matched control study). Results: The pilot had successfully engaged provider organisations, created a shared strategic vision and established governance structures. However, the engagement of clinicians was variable and there was no evidence to date of significant reductions in emergency admissions. There was some evidence of changes in care processes. Conclusion: Although the pilot has demonstrated the beginnings of large-scale change, it remains in the early stages and faces significant challenges as it seeks to become sustainable for the longer term. It is critical that National Health Service managers and clinicians have realistic expectations of what can be achieved in a relatively short period of time.
    • Multidisciplinary group performance—measuring integration intensity in the context of the North West London Integrated Care Pilot

      Harris, Matthew; Greaves, Felix; Gunn, Laura H.; Patterson, Susan; Greenfield, Geva; Car, Josip; Majeed, Azeem; Pappas, Yannis; Imperial College London; Royal Brisbane and Women’s Hospital (Igitur Publishing, 2013)
      Introduction: Multidisciplinary Group meetings (MDGs) are seen as key facilitators of integration, moving from individual to multidisciplinary decision-making, and from a focus on individual patients to a focus on patient groups. We have developed a method for coding MDG transcripts to identify whether they are or are not vehicles for delivering the anticipated efficiency improvements across various providers and apply it to a test case in the North West London Integrated Care Pilot. Methods: We defined ‘integrating’ as the process within the MDG meeting that enables or promotes an improved collaboration, improved understanding, and improved awareness of self and others within the local healthcare economy such that efficiency improvements could be identified and action taken. Utterances within the MDGs are coded according to three distinct domains grounded in concepts from communication, group decision-making, and integrated care literatures—the Valence, the Focus, and the Level. Standardized weighted integrative intensity scores are calculated across ten time deciles in the Case Discussion providing a graphical representation of its integrative intensity. Results: Intra- and Inter-rater reliability of the coding scheme was very good as measured by the Prevalence and Bias-adjusted Kappa Score. Standardized Weighted Integrative Intensity graph mirrored closely the verbatim transcript and is a convenient representation of complex communication dynamics. Trend in integrative intensity can be calculated and the characteristics of the MDG can be pragmatically described. Conclusion: This is a novel and potentially useful method for researchers, managers and practitioners to better understand MDG dynamics and to identify whether participants are integrating. The degree to which participants use MDG meetings to develop an integrated way of working is likely to require management, leadership and shared values.
    • Multidisciplinary integration in the context of integrated care - results from the North West London Integrated Care Pilot

      Harris, Matthew; Greaves, Felix; Gunn, Laura H.; Patterson, Sue; Greenfield, Geva; Car, Josip; Majeed, Azeem; Pappas, Yannis; Imperial College London; Stetson University; et al. (Igitur Publishing, 2013-10)
      Background: In the context of integrated care, Multidisciplinary Group meetings involve participants from diverse professional groups and organisations and are potential vehicles to advance efficiency improvements within the local health economy. We advance a novel method to characterise the communication within Multidisciplinary Group meetings measuring the extent to which participants integrate and whether this integration leads to improved working. Methods: We purposively selected four Multidisciplinary Group meetings and conducted a content analysis of audio-recorded and transcribed Case Discussions. Two coders independently coded utterances according to their ‘integrative intensity’ which was defined against three a-priori independent domains - the Level (i.e. Individual, Collective and Systems); the Valence (Problem, Information and Solution); the Focus (Concrete and Abstract). Inter- and intra-rater reliability was tested with Kappa scores on one randomly selected Case Discussion. Standardised weighted mean integration scores were calculated for Case Discussions across utterance deciles, indicating how integrative intensity changed during the conversations. Results: Twenty-three Case Discussions in four different Multidisciplinary Groups were transcribed and coded. Inter- and intra-rater reliability was good as shown by the Prevalence and Bias-Adjusted Kappa Scores for one randomly selected Case Discussion. There were differences in the proportion of utterances per participant type (Consultant 14.6%; presenting general practitioner 38.75%; Chair 7.8%; nonpresenting general practitioner 2.25%; Allied Health Professional 4.8%). Utterances were predominantly coded at low levels of integrative intensity; however, there was a gradual increase (R2 = 0.71) in integrative intensity during the Case Discussions. Based on the analysis of the minutes and action points arising from the Case Discussions, this improved integration did not translate into actions moving forward. Interpretation: We characterise the Multidisciplinary Groups as having consultative characteristics with some trend towards collaboration, but that best resemble Community-Based Ward Rounds. Although integration scores do increase from the beginning to the end of the Case Discussions, this does not tend to translate into actions for the groups to take forward. The role of the Chair and the improved participation of non-presenting general practitioners and Allied Health Professionals seems important, particularly as the latter contribute well to higher integrative scores. Traditional communication patterns of medical dominance seem to be being perpetuated in the Multidisciplinary Groups. This suggests that more could be done to sensitise participants to the value of full participation from all the members of the group. The method we have developed could be used for ongoing and future evaluations of integrated care projects.