Table 3 Themes and associated outcomes
ORGANISATIONAL PERFORMANCE | |||
|---|---|---|---|
Theme | Sub-themes and associated outcomes | Sources | Exemplar quotes |
Patient outcomes | ● Clinical outcomes ● Patient safety ● Patient engagement ● Patient empowerment ● Patient experience ● Socio-economic benefits ● Service user recruitment | [33, 68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135] | Clinical outcomes |
“The adverse event rate increased from 2.9 to 4.8 per 100 patients in control hospitals and declined from 6.2 to 3.7 among SPI1 hospitals”. Authors; Benning et al. (2011, p. 11) | |||
Patient experience | |||
“…improving process performance, including waiting time reduction and patient flow with the subsequent impact of increasing patient satisfaction”. Authors; Honda et al. (2018, p. 70) | |||
Social impacts | |||
“…the list of possible social returns … became quite long, and each social impact (for example, less patient time spent in hospitals) could cascade into broader social impacts (for example, increased productivity, increased efficiency at hospitals, benefits of expenditures in other areas …)”. Authors, Moody et al. (2015, p. 30) | |||
Financial outcomes | ● Cost saving ● Revenue generation ● Cost-management ● Cost reduction ● Cost avoidance ● Financial stability | [33, 68,69,70,71, 74, 76,77,78,79, 81, 84, 86, 88, 92, 93, 108, 110, 116, 119, 124, 130, 134] | Legal costs reduction |
“In the last 6 years our professional liability exposure has decreased. It is possible that this resulted from higher quality care”. Authors; Swensen et al. (2013, p. 47) | |||
Cost reduction and revenue generation | |||
“The large-scale QI …has the potential for ROI at multiple levels… opportunity to improve efficiency, remove waste, lower cost, and increase revenue.” Authors; O’Sullivan et al. (2020, p. 3) | |||
ORGANISATIONAL DEVELOPMENT | |||
Strategic goals | ● Achievement of organisational strategies ● Improved alignment with strategies: refinement and clarification ● Generation of organisational mission, objectives, and priorities ● Improvement in organisational ethical, moral, legal, and value obligations ● Creating new personal and meaningful operating models ● Patient-centredness ● Staff-centredness ● Decision-making and problem-solving improvement ● Overall organisational performance improvement | [69, 73, 77,78,79, 81, 82, 87, 94, 97, 102, 104, 105, 111, 117, 121] | Increased market share |
“Significant improvements in waiting time and number of new patients were identified for two of the interventions”. Authors; de la Perrelle et al. (2020, p. 5) | |||
Strategy to engage service users | |||
“…to improve the total quality of every service user’s journey throughout the mental health system… by developing the capacity and skills of local care communities in order to make fundamental improvements in the way services are provided” Participant; Worrall et al. (2008, p.13) | |||
“At a policy level, patient safety is now articulated as a clear priority and has become more closely linked with the national drive to improve quality of care while increasing productivity and efficiency” Authors; The Health Foundation (2011, p. 27) | |||
Governance | Improve organisational transparency, accountability ● Improving clinical effectiveness and patient safety ● Improving human resource effectiveness ● Risk management ● Compliance with performance criteria ● Performance management and measurement beyond clinical governance to organisational governance | “We are currently exploring, through early pilot projects, a range of board development interventions and improvement approaches, to enable better governance of patient safety within organisation”. Authors, The Health Foundation (2011, p. 26) | |
“This flexibility and enabling grassroots practitioners to become the problem solvers is the key to changing over to a lean management or governance system”. Authors; MacVane (2019, p. 84) | |||
Human resource development | Improved staff capabilities ● Raising awareness on QI methods, patient safety, inefficiencies, and costs, ● Increase staff ability to assess which problems were best suited to QI ● Improved personal and career development and job security ● Staff engagement ● Staff empowered Improved staff experience ● Improved motivation, and enthusiasm, Improved staff capacity ● Supporting recruitment and retention, ● Improved job security, and reduced staff sickness ● Developing new QI roles ● Role clarification | [73, 74, 79, 81, 89, 95, 97, 99, 101, 103, 104, 106, 107, 113, 117, 124, 125, 129, 134] | Staff capabilities |
“Ninety-one per cent felt the Collaborative had empowered them to make a difference in reducing the number of pressure ulcers. Feedback given from one of the two people who did not answer this way stated that it was ‘already part of job role.’” Authors, Wood et al. (2014, p. 6) | |||
“…staff reported benefits to the social and work environment, but perhaps most significantly working on the programme was described by some staff as a long awaited opportunity for personal or career development” Authors; Morrow et al. (2012, p. 248) | |||
Staff experience | |||
“Greater knowledge tended to produce greater enthusiasm” Authors, The Health Foundation (2011, p. 11) | |||
“As great as the financial impact of purchasing safety devices and of a needlestick injury may be, the nonfinancial impact can be even greater. We desire the work environment to be as safe as possible for our staff”. Authors; Hatcher (2002, p. 413) | |||
Staff capacity | |||
“The apparent improvement in staff sickness rates; or the recorded decrease in bed numbers apparently associated with the trust’s analyses showing reduced length of stay on the targeted wards”. Authors; Hunter et al. (2014, p. 64) | |||
“…the programme appeals to the intrinsic values of frontline (particularly nursing) staff and has had a positive impact (key themes were: equipping staff with new skills, more time for better care, improved patient experiences, cost savings, and higher staff satisfaction and retention”. Authors; NHS Institute (2011, p. 17) | |||
Process, structural, and systems | Efficiency and productivity ● Team efficiency ● Systems efficiency ● Processes efficiency Resource management ● Optimisation, or leveraging of existing systems ● Facilitating effective resource allocation ● Spreading of costs and benefits or off-setting other organisational benefits Structural changes ● Guiding patient safety infrastructure development ● Reduction of incidences of violence | [33, 71, 73, 75, 79, 81, 84, 87, 91, 93, 95, 110, 114, 119, 121, 132, 135] | Process improvement |
“Process mapping the care of patients with sepsis, presenting key issues visually and as a gap analysis were essential to identify the core elements of the clinical pathway, to introduce structural changes”. Authors; Thursky et al. (2018, p. 7) | |||
Resource management | |||
“…this made it possible to revise the procedure for filing and monitoring patient files by nurses, thus reducing the time allocated to this activity by one hour per week.” Authors; Comtois et al. (2013, p. 174) | |||
“The collaborative learning process during audit and feedback, to enable self-monitoring and provision of action plans, resulted in various institutional changes…” Authors; Brink et al. (2017, p. 1232) | |||
Structural improvements | |||
“Benefits included better organised working environments, fewer patient safety incidents, and cash savings in terms of returned excess stock”. Authors; Morrow et al. (2012, p. 246) | |||
Culture and climate | Developing a QI safety culture ● Culture aligned to people ● An organisational learning culture ● Change from performance and regulation to continuous improvement ● Change from project orientation to capacity and capability building ● Change from top-down to bottom-up development ● Culture of shared leadership models ● Culture of collaboration ● Flexible and inclusive culture ● Challenging of existing mental models Improved organisational climate | Culture | |
“… I don’t think you can buy the attitude and mental approach that needed to happen. And I truly think money and resources wouldn’t have helped. …I think that is the level at which the intervention to change the system should have been, right at a deeper level. Not resource, not environment, but more the deep cultural partnership interpersonal level” (p. 103). Participant; Worrall et al. (2008, p. 103) | |||
“In those trusts we have rated as outstanding; we have found a culture of quality improvement embedded throughout the organisation.”. Authors; CQC (2018, p. 2) | |||
Climate | |||
“There were also significant improvements in secondary outcomes: patients’ overall rating of ward quality; nurses’ positive affect and team climate”. Authors; Williams et al., 2020, (p. 45) | |||
Leadership development | ● Leadership development ● Leadership effectiveness | [74, 92, 96, 102, 103, 107, 108, 116, 117, 120, 121, 127, 128] | Leadership development |
“…relatively junior staff with limited practical experience are now running the collaboratives. Without the right leaders, there is a risk that collaboratives are pale imitations of effective programmes”. Authors; Collins and Fenney (2019, p. 18) | |||
Leadership effectiveness | |||
“Having been involved in some major NHS improvement collaboratives, including one looking at adverse drug events, I initiated an internal collaborative on medication error”. Participant; The Health Foundation article (2011, p. 20) | |||
Internal collaboration | ● Intra-organisational learning networks ● Team-working ● Team cohesion ● Enhanced communication | Team-working | |
“…the process successfully facilitated a welcome shift from a ‘parent–child’ relationship where the pharmacists are always seeking the junior doctors and pointing out mistakes that need to be amended to a more effective and efficient ‘team work’ approach where junior doctors and clinical pharmacists work together to generate a safe discharge…” Authors; Botros and Dunn (2019, p.8) | |||
Research development | ● Increased awareness of QI evidence-base enhancement ● Stimulating ideas on innovative research methods development ● Evidence dissemination ● Increased focus on financial outcomes | “The three strands of evaluation of the Safer Patients Initiative have surfaced some important reflections on research and evaluation of complex, organisational interventions”. Authors, The Health Foundation (2011, p. 23) | |
“A program called “Measurement for Management,” offered by Qulturum with IHI input28and open to teams from across Sweden, was created following the 2006 study, to help participants build system-level capacity for measurement, data collection, and interpretation”. Authors; Staines et al. (2015, p. 26) | |||
Innovation | ● Development of new ways of working ● Development of new tools and methods | “NHS Safety Thermometer data collection tool was developed by the national programme team during the design period of phase I and refined iteratively thereafter”. Authors; Power et al. (2016, p. 9) | |
IT development & data management | ● Improved data management ● local ownership of data monitoring and reporting, ● Data transparency and sharing, ● Data used to guide improvements | [33, 79, 93, 96, 102, 103, 107, 108, 115, 121, 127, 128, 135] | “... the data collection before and what we collected data on afterwards were different things really in a way. So they had to be retrospective to get some of the baseline stuff, because we didn’t know what was going to come out and the changes that were going to happen.” Participant; Hunter et al. (2014, p. 62) |
The QI activities often resulted in an improved understanding that measurement was an important part of any Method adopted. In addition, staff often also realised that suitable metrics were not available, or that the data were of poor quality”. Authors; Hunter et al. (2014, p. 81) | |||
“Ownership of our data and ownership, that’s one of the things that’s really improved the clinical team I think”. Authors; Worrall (2008, p. 120) | |||
QI legacy | ● Sustainable benefits from previous programmes ● Created new standards and expectations of care ● Increased collective QI knowledge and skills ● Financial sustainability ● Performance sustainability ● Sustained organisational capabilities ● QI legacy through implementation outcomes spread or scale-up ● Built foundations for bigger more complex programmes ● Increased capacity to learn from challenges, failures and successes of self and others | [71, 74, 75, 81, 92, 94, 102, 104, 107, 114, 119, 121, 123, 124, 127, 128, 131, 133] | “Throughout five years since implementation of MEWS‐Sepsis tool patient screening, the organization has realized a sustained decline in sepsis mortality of 24%” Authors; Roney et al. (2016, p.3) |
They also provide the bedrock for future improvement in the quality, safety and efficiency of integrated hospital and community services, as well as between adult social care, mental and physical health care, and acute and long-term services.” Authors; Pearson et al. (2017, p. 5) | |||
“…we found that staff continued to apply these principles to their QI work even as organisational contexts changed over time”. Authors; Robert et al. (2020, p. 38) | |||
“I think that the legacy of MHIP and the restructuring has meant that we really have taken a much more defined systems approach, and I think much better clarity about roles and responsibilities and accountability in the system”. Participant; Worrall et al. (2008, p. 118) | |||
Organisational resilience | ● Achievement of a high reliability, high performing, and self-sustaining organisation ● Coping with changing and unstable contexts ● Organisational learning | “Projects can fail to show improvement or fail to sustain themselves. ELFT are interested in such cases too, and the considerable learning they can yield. This interest in failed projects, and difficult to improve areas, sends the message to staff that all is not lost if results are limited” Authors; O’Sullivan et al. (2020, p. 6) | |
EXTERNAL OUTCOMES (MACRO) | |||
Incentives | ● Recognition as a leader and influencer ● Financial incentives, awards, accreditation, ● Improved competitiveness, ● Improved influence and power ● Positive reputation ● Pride for the organisation and staff ● Improved bargaining power ● Accreditation ● Reduced regulation and oversight | [33, 77,78,79, 85, 91,92,93, 96, 100, 103, 107, 108, 116, 127, 128] | Influence |
“Although the Safer Patients Initiative did not achieve the level of organisational impact hoped for within the timeframe of the programme, it did have a significant effect and influence on participating hospitals and their staff, on patient care and on the wider NHS system”. Authors; The Health Foundation (2011, p. 14) | |||
Awards | |||
“This RPIW was frequently mentioned by interviewees as an exemplar that demonstrated the positive benefits of the NETS programme. It received national recognition through the Health Service Journal awards”. Authors; Hunter et al., (2014, p. 63) | |||
External obligations | ● Compliance with oversight, accreditation, regulation | “Holding providers accountable for blood product wastage contributed to the waste reduction and could be used as a component of the provider’s ongoing performance profile, which has recently become a Joint Commission requirement”. Authors; Heitmiller et al. (2010, p. 1895) | |
“Most of the NHS trusts in England that have been given an outstanding CQC rating have implemented an organisation-wide improvement programme”. Authors; Jones et al. (2019, p. 6) | |||
Community and society benefits | ● Community engagement ● Improved community resources ● Support for carers, children, and families ● Socio-economic benefits | External benefits | |
“The greatest benefit from these 6- to 9-month QI projects was internal, yet the communities also reaped significant external benefits”. Authors; Crawley-Stout et al. (2016, p. E35) | |||
External collaboration | ● Data sharing ● Shared governance ● Multi-stakeholder engagement and alignment ● Foundations and maintenance of strategic relationships ● Long-term learning networks ● Improved multi-organisational relations ● Development of deeper awareness of collective issues. | [69, 72, 84, 87, 92, 93, 98, 103, 107, 112, 115, 119, 126,127,128] | Improved organisational relations |
“There was also a local history of difficult relations between hospital and community services. Service reconfigurations that maintain stability against such a backdrop and which lead to important signals of improvement are a success. They also provide the bedrock for future improvement in the quality, safety and efficiency of integrated” Authors; Pearson et al. (2017, p. 5) | |||
Shared Governance | |||
“Opportunities to train with other NHS NE organisations, to jointly redesign pathways and to speak the same language of improvement, were highly valued”. Authors; Hunter et al. (2014, p. 74) | |||
UNINTENDED OUTCOMES ((MICRO, MESO) | |||
Positive unintended outcomes | ● Gaining new insights on related organisational needs ● Improvements in untargeted departments or patients ● Incidental innovations ● Enabling communication ● Enabling targeted recruitment of QI staff and leaders ● Academic development through creation of patient safety or QI training ● Learning from failure and negative outcomes | Incidental innovations | |
“A multidisciplinary team with existing expertise in tracheostomy care commenced detailed tracheostomy ward rounds, providing a different context to the other sites. Local MDT oversight teams were established at all sites…” Authors; McGrath et al. (2017, p. 7) | |||
Enabling communication | |||
“The attention paid to Patient Safety had been a door opener. Patient Safety made it possible for hospital CEOs to discuss accountability with physician”. Authors; Staines et al. (2015, p. 25) | |||
Negative unintended outcomes | ‘Top-down distortions’ ● Scepticism about focusing too narrowly on managerial goals ● QI exhaustion; QI ‘constant hammering’ demands ● Perceived loss of autonomy ● Feeling bullied or intimidated ● System gaming or manipulation ● “top-down distortions” (Robert et al. 2020, p. 39) External imposition ● Side-lined local goals in favour or external goals High resource demands ● Increased need for support for staff and leadership ● Increased financial resource needs ● Data burden due to the data demands from multiple stakeholders ● Increased resource and staff costs Duplication ● Duplication of resource needs, QI tools, and methods due to top-down and or external QI goals Loss of revenue ● Reduced patient enrolment as a service was no longer needed or needed less Loss of buy-in ● Disengagement, ● Loss of enthusiasm and motivation ● Staff disillusionment | [79, 80, 82, 88, 94, 95, 110, 112, 113, 116,117,118,119,120,121, 124, 127, 128] | ‘Top-down distortions’ |
“…top-down distortions through performance management systems”, which caused a shift away from a longer- term vision of empowering ward teams to take ownership, potentially limiting positive long- term legacies. Participant; Robert et al., (2020, p. 39) | |||
“There’s a constant hammering, it’s almost like a squeaky wheel. I wouldn’t call that performance management, in effect it comes down to a set of KPIs for the system and everything that is perceived to improve that, gets pushed”. Participant; Masso et al. (2010, p.356) | |||
External imposition | |||
“…they also expressed concern about aspects of the oversight of collaboratives, including the pressure to deliver complex programmes and demonstrate benefits within very short timescales, and the amount of time small teams needed to dedicate to the process of justifying their collaborative’s effectiveness”. Authors; Collin & Fenney (2011, p. 22) | |||
High resource demands | |||
“To bring about large-scale improvement is far costlier than anybody ever envisaged and unless you really are willing to make the true investment you often don’t get the sustained change improvement that you require”. Participant; Worral et al. (2008, p. 101) | |||
“Only one quarter of all respondents believed that training had been sufficient and 18% felt that resources had been sufficient to implement Lean”. Authors; Goodridge et al. (2018, p. 18) | |||
The challenges this posed for QI leads must not be underestimated, with the burden of collecting data (for NELA and ostensibly for use as part of the EPOCH improvement work) may have overwhelmed many”. Authors; Stephen’s et al. (2018, p. 11) | |||
Duplication | |||
“…interviews revealed some anger among informants. The disappointment was the highest in Primary care, as this sector already had a clinical information system in the past”. Authors; Staines et al., (2015, p. 25) | |||
Loss of revenue | |||
“a negative return on investment for the program in the short run. In the longer run, a positive return would occur through avoidance of increased morbidity, but, because of enrolee turnover, both organizations might not be able to realize that return”. Leatherman et al. (2003, p. 21) | |||
Loss of buy-in | |||
“…ward staff generally did not feel as engaged in the work and medical engagement remained one of the programme’s biggest barriers”. Authors; NHS Foundation (2011, p. 20) | |||
“Nurses who have been previously captured by the panacea of being ‘productive’ and ‘releasing time to care’ may simply have escaped the captivity and control of that dreamlike desire, and are just refusing to engage with the dance of efficiency (Rudge 2013) in White et al. (2014, p. 2420) | |||