Three overarching themes arose from the data, and they are discussed in turn below. The first theme is concerned with how participants viewed integrated care and its benefits. The second theme focuses on potential measures for integrated care. The final theme explores some broader contextual and methodological considerations surrounding assessing benefits of integration initiatives.
Integrated care and its benefits
When asked to define integrated care, the majority of stakeholders first and foremost defined it from the perspective of the patient or service user as an approach that coordinates services around the individual to fulfil their needs. Most went on to state that a key purpose of taking a person-centred approach to integrated care is to achieve better outcomes for patients and service users, for example, keeping people in their homes for longer, maintaining independence, reducing unnecessary admissions, improving health outcomes, and quality of life. In addition to improved outcomes, the potential for integrated care to provide patients with a better experience of care was also raised by stakeholders. Examples given of what a better patient experience may look like included: improved access to services, smoother transitions between services, increased involvement in decision-making, receiving the right care at the right time in the right place, and telling ‘their story’ less frequently.
Secondary to the person-focused definition offered above, some stakeholders provided a systems-focused perspective of integrated care, as an activity that coordinates or brings services together to provide ‘joined-up’ or ‘seamless’ care. Some outlined this vision in greater detail, specifying the services involved in integration (e.g. health, community, voluntary, and social care), as well as detailing the mechanisms by which integrated care can be achieved, for example, vertical or horizontal integration, and multi-disciplinary teams (MDTs). It is important to note that most stakeholders provided a systems-focused perspective in addition to the person-centred definitions given, outlining the structures and processes necessary in order to achieve better outcomes and experiences for patients. For example, one stakeholder spoke about how improvements to the structure of the health and care systems, specifically the integration of IT systems, could benefit patients:
“We don’t have a single joined up system. We don’t yet really have shared information systems which are incredibly important. And they're incredibly important enabler for integration. And so that’s what it's about for me – joining up information, making sure that care professionals have access to the right levels of information that they can use to make joint decisions that are with the patient as well, to improve the patient and service user outcomes.” (S10, Programme Lead, Clinical Commissioning Group)
Although structures and processes were discussed, the benefits of integrated care were generally coached in terms of the patient or service user. A few stakeholders further emphasised a person-centred approach by specifically stating that the end goal of integrated care is not the integration of structures, but rather the outcomes that can be achieved:
“It's [integrated care] about the experience and the improved outcomes that it brings to people, that is what it's all about; it's not about structures … it's about improving people's health and well-being, it's about closing health and inequalities, it's about keeping people well at home, giving them a better, more co-ordinated experience of care, about helping them to help themselves, about using, all of the assets in a place to support the community to live as happily and independently and safely as they can. And so that’s what integrated care is to us. It's not a thing in itself; it's the outcomes that it achieves.” (S9, Senior Advisor, Local Government Association)
The benefits of integrated care to service providers were also discussed, albeit less than the benefits to patients and service users, with two key areas of benefit raised. Firstly, many stated improved service efficiencies as a key benefit, achieved through more efficient pathways, reduced duplication of work (e.g. reduced number of assessments), reduced admissions, and an improved ability to make decisions, plan, and co-ordinate care. Secondly, the potential for integrated care to reduce costs was often discussed, although there were discrepancies between stakeholders regarding whether savings could be realised, with several acknowledging that this was not a given and that current evidence to support this was limited. In addition to these two key areas, a few stakeholders also highlighted the potential for integrated care to improve the experience and retention of staff.
Providing a definition of integrated care was more challenging for patient representatives. However, of those who were able, they provided similar albeit less detailed definitions than professional stakeholders, focused on bringing services together for the benefit of the patient:
“different organisations getting together … clinicians, carers, community people, researchers, research scientists, sharing knowledge and sharing anything that might be relevant to the patient that might improve the outcome to a patient” (Patient Representative 3)
Furthermore, patient representatives also provided similar descriptions of the benefits of integrated care, with a particular focus on streamlining processes to benefit the patient; for example, having one point of contact and being asked fewer times for the same information.
Measures of integrated care
All professional stakeholders discussed how they thought integrated care should be evaluated, both in terms of the aspects to evaluate and how best to measure those aspects. Several professional stakeholders spoke about the difficulty of measuring the impact of integrated care, particularly with regards to the challenges faced when selecting appropriate measures, and assessing the benefits to patients and service users. Under current arrangements, with diverse measures employed across the system, a few stakeholders mentioned that it was impossible to draw general conclusions about the effectiveness of alternative models of integration. As one might expect, patient representatives discussed the measurement of integrated care in much less detail than professional stakeholders. While various system metrics were raised as possible measures of integrated care, patient representatives generally thought evaluation should focus on patients’ experiences of care, and any improvements in patient outcomes.
Three broadly distinct types of measure of the benefit of integration were discussed: system effects, patient experience and patient outcomes.
System effects
System metrics were commonly referred to by stakeholders when discussing measures of integrated care, of which, delayed transfers of care (DTOC), number of emergency admissions, and length of stay, all of which relate to hospital care, were among those most frequently mentioned. While there was great awareness among stakeholders of metrics used to assess system outcomes, and their value acknowledged, many felt that evaluations of integrated care focus too heavily on system effects. Concerns were raised that such an approach is too narrow and does not necessarily effectively evaluate the broader impact of integrated care and the benefits to patients. In particular, some regarded these metrics as ‘blunt instruments’ that do not capture the true impact and benefits of integrated care as defined by stakeholders, i.e. whether system changes are improving patient experiences and outcomes:
“The outcomes often associated with integrated care are hardcore metrics like reductions of emergency admissions, unplanned admissions, length of stay and that sort of thing … these are the metrics that they use to measure integrated care against, don’t actually demonstrate an effective connection, and that’s because those metrics in themselves aren't particularly sensitive to what is the aspiration of integrated care.” (S11, Senior Academic, University)
Vast awareness and use of system metrics is, in part, likely explained by the requirement of regulators to collect data on certain system outcomes, for example, Delayed Transfers of Care (DTOC). An Assistant Director at a local authority questioned this focus:
“The sign of a good performing system is that you have low levels of DTOC … yes, it's nice to have a low level of DTOC, but that still means that the person's been in hospital. They could have been in hospital for three or four weeks. Is that necessarily a good experience for them? … Sometimes we latch onto a particular measurement and then say, "Oh yeah, your system's performing well because you’ve got a really low DTOC." Well, not if you’ve got really high non-elective admissions and really high readmissions and really high residential admissions … that tells me that the system's not operating as well as it could.” (S4, Assistant Director, Local Authority)
Patient experiences
The majority of participants spoke about the need to measure patients’ experiences, with a few specifically extending this to include the experiences of carers and / or family members. Furthermore, several thought it also important to measure the experiences of the workforce.
Patient representatives frequently referred to experiences of care, talking about their own experiences from a patient or carer perspective, as well as referring to the experiences of others (e.g. family, friends and neighbours). One patient representative described the experience of their neighbour:
“She suffered a mental health breakdown with two young children, and the [city] social services department took six months to assess her. And even then they didn’t complete the assessment because the lady was homeless. And they moved her to temporary accommodation into the neighbouring borough of [location], where they had to start the whole process again because they said they'd lost the papers … So, there's no communication between one department and another department.” (Patient Representative 1)
This focus reinforces the stakeholders’ views of the importance of measuring patients’ experiences of care.
Some difficulties around measuring patients’ experiences were raised by stakeholders, for example, often patients and service users provide a positive report of their care despite negative aspects occurring, which often only come to light during subsequent discussions with the patient.
Patient outcomes
In general, stakeholders discussed patient-related outcomes in broader terms than they had discussed system effects. For example, some stakeholders advocated the importance of patient outcomes but generally without specificity or detail. This may be an effect of system outcomes being measured more frequently, and therefore, system metrics being better known to stakeholders than patient outcome measures. This was not true of all, with a few participants able to discuss dimensions to be measured (e.g. wellbeing, functionality, and pain), and tools that could be used to elicit such data, in much greater detail.
Several stakeholders called for a wider “basket of measures” to broaden measurement beyond system effects when evaluating the success of integrated care programmes. A few felt that measuring patient outcomes should be central to evaluations, for example, a Senior Associate of a charity stressed that “fundamentally, the best judge of a service or a package of services, is from the perspective of the people experiencing care” (S1, Senior Associate, Charity).
Overall, very few specific health indicators were referred to, with life expectancy, functionality, and dental decay among those mentioned. The majority of discussion was about patient- or carer- reported outcomes. Quality of life, well-being, or specific aspects of quality of life, such as pain, anxiety, and depression were raised by the majority of stakeholders as aspects of integrated care that should be evaluated. The value of measuring aspects such as these was highlighted in an example shared by a Senior Manager, who described an instance where a GP visited an elderly man with multiple conditions nearing end of life:
“And the GP thought, 'Well what can I do for this guy?' and he... so he just asked him. He said, "I'd really like my beard trimmed." So, he kind of got a barber to come round and trim his beard and he said it kind of transformed this guy's kind of, you know, you know, made him feel so much better. And his blood pressure was lower and things like that. We don’t... but we don’t kind of pick that up. I think all you'd pick up was the fact that whether the GP went there and how long he was there for, or she was there for, you know.” (S6, Senior Manager, County Council)
However, a few held some reservations about collecting such data due to its subjective nature, and problems with measurement (e.g. comparability):
“I might say my quality of life's really high and I enjoy my life; I could live in absolute squalor. And other people would say my quality of life is very poor. So, I think it's more difficult around quality of life … I think it is right that we ask the question, but I think it's one of those ones that you need to think about that is subjective.” (S4, Assistant Director, County Council)
Patient-reported measurement instruments
When asked about available patient-reported measures for assessing integrated care, broadly speaking, stakeholders working at a national level (e.g. national charities) and in academia were far more familiar with specific measures than those working at a local level (e.g. County Councils and Clinical Commissioning Groups). Measures named, included, the Patient-Centred Coordinated Care Experience Questionnaire (P3CEQ), the Friends and Family Test, the Patient Activation Measure (PAM-13), the Warwick-Edinburgh Mental Well-being Scale (WEMWBS), and the EuroQol 5 Dimensions (EQ-5D). None of the measures named were strongly advocated by the professional stakeholders. Although patient representatives did not specifically name any patient-reported measurement instruments, many had previously seen or completed patient-reported outcome or experience questionnaires as part of their care. One patient representative described their experience:
“I have endless review questionnaires following interventions about, you know, pain scores and all that sort of stuff. Can we do this, can we do that more, you know, was it better, was it worse, was it just the same. But no, I personally, have a bit of trouble with those as a patient because ticking boxes and being a human being don’t seem very compatible to me, but I do understand that it's the way we measure things. And I know you get something from it which translates into something which will help me.” (Patient Representative 4)
Contextual considerations
In addition to discussing potential measures for integrated care, stakeholders also raised some additional factors for consideration when evaluating integrated care programmes. Both professional stakeholders and patient representatives raised queries around the purpose of data collection in health and care settings. One patient representative discussed the lack of transparency with patients regarding the purpose for which data is collected, referring to “endless surveys” that go “into a dark hole and you assume that nobody ever reads” (Patient Representative 4). Several professional stakeholders called for greater clarity around why data is being collected and how that data will be used:
“The other thing that worries me is are we using that data? … it's very fashionable to talk about, you know, people's mental health and all this sort of stuff, so people feel that they ought to collect it, but people should only be collecting it if they're going to use it, you know. If, you should only be asking these people these questions if they’ve got a plan for that data. If they just want to kind of want to put it on a database somewhere and forget about it then it’s unethical.” (S6, Senior Manager, Council)
A further issue, raised by a few stakeholders, was around the tensions between collecting data for use at a local or national level. While stakeholders recognised the need to collect data at a national level to allow comparisons to be drawn between providers, there were concerns that national level data lacked local relevance, and necessitated a loss of local data ownership. An Assistant Director at a Council questioned the value of nationally imposed metrics:
“We are supposed to work in an integrated way and cooperate, but I think the culture that underpins that has been one of intimidation from the top. An insistence of further solutions by certain providers being delivered, and whether you’ve got a top driven set of metrics that have come from NHS England and don’t reflect the reality of local government funding. Those metrics become either as just a stick to beat people with. And it doesn’t help the integration of human beings working together on the ground; it gets in the way.” (S15, Assistant Director, Council)
Qualitative methods
Qualitative methods of evaluating integrated care were increasingly being used or advocated by many stakeholders in order to overcome difficulties they had experienced with quantitative evaluation of integrated care initiatives. A senior academic discussed their qualitative approach to evaluation:
“We get people to define what is it you wanted to get out of it and did integrated care actually help you? So, it's impossible to plonk it on a politician's desk and say, 'You know, with this integrated care initiative, X number of outcomes were achieved,' because it's not like that. Integrated care is much more evaluation of a process. And individually, did the processes actually facilitate some kind of improvement?” (S11, Senior Academic, University)
Stakeholders often discussed the use of qualitative methods to capture data about the benefits of integrated care from the patients’ perspective. Some advocate these methods as a way of hearing the stories of patients. Despite discussing the benefits of a qualitative approach, it was acknowledged that there is still a need for good metrics with which to evaluate and report on integrated care.
“Asking the person at the end of it how integrated their experience has been and then acting on what they tell us. It's experiential, it's qualitative and it's not going to be easily captured and quantified and reported upon.” (S15, Assistant Director, Council)
It has to be noted that while many stakeholders referred to “qualitative” approaches, it was evident that a few were in fact referring to patient experience surveys rather than qualitative methodological approaches, such as interviews and focus groups.