The development of the concept of return-on-investment from large-scale quality improvement programmes in healthcare: an integrative systematic literature review
BMC Health Services Research volume 22, Article number: 1492 (2022)
Return on Investment (ROI) is increasingly being used to evaluate financial benefits from healthcare Quality Improvement (QI). ROI is traditionally used to evaluate investment performance in the commercial field. Little is known about ROI in healthcare. The aim of this systematic review was to analyse and develop ROI as a concept and develop a ROI conceptual framework for large-scale healthcare QI programmes.
We searched Medline, Embase, Global health, PsycInfo, EconLit, NHS EED, Web of Science, Google Scholar using ROI or returns-on-investment concepts (e.g., cost–benefit, cost-effectiveness, value). We combined this terms with healthcare and QI. Included articles discussed at least three organisational QI benefits, including financial or patient benefits. We synthesised the different ways in which ROI or return-on-investment concepts were used and discussed by the QI literature; first the economically focused, then the non-economically focused QI literature. We then integrated these literatures to summarise their combined views.
We retrieved 10 428 articles. One hundred and two (102) articles were selected for full text screening. Of these 34 were excluded and 68 included. The included articles were QI economic, effectiveness, process, and impact evaluations as well as reports and conceptual literature. Fifteen of 68 articles were directly focused on QI programme economic outcomes. Of these, only four focused on ROI. ROI related concepts in this group included cost-effectiveness, cost–benefit, ROI, cost-saving, cost-reduction, and cost-avoidance. The remaining articles mainly mentioned efficiency, productivity, value, or benefits. Financial outcomes were not the main goal of QI programmes. We found that the ROI concept in healthcare QI aligned with the concepts of value and benefit, both monetary and non-monetary.
Our analysis of the reviewed literature indicates that ROI in QI is conceptualised as value or benefit as demonstrated through a combination of significant outcomes for one or more stakeholders in healthcare organisations. As such, organisations at different developmental stages can deduce benefits that are relevant and legitimate as per their contextual needs.
Review registration: PROSPERO; CRD42021236948.
Quality Improvement (QI) programmes are being increasingly used to improve care quality in healthcare organisations . QI is defined as a methodical approach to making improvements in a number of healthcare service aspects [1–3]. In particular, large-scale QI programmes are used to effect organisational level outcomes e.g., financial, and patient outcomes or even health system level outcomes e.g., population health [4, 5]. Large-scale QI combines various strategic elements into a coherent improvement process to improve safety, quality, capability, and capacity of organisations [4, 5]. Some QI programmes are delivered through local, national, or international collaboratives. A QI collaborative (QIC) combines multidisciplinary teams from different organisations to test solutions, and share learning in a specific clinical or operational area . Whatever the type, QI programmes can have significant cost implications .
Investing in a QI programme may redirect money from other healthcare initiatives. In economic terms, this is called an “opportunity cost”, roughly meaning a lost opportunity for alternative investment . For this reason, healthcare leaders must justify investments made or proposed for particular programmes. This is called making a QI business case. Return on Investment (ROI) is one such justification tool, used to decide how best to allocate limited healthcare funds [9, 10]. Investment allocation decisions have ethical, moral, political, and equity implications [11, 12]. Thus, the need to understand the meaning of ROI in the context of healthcare QI.
ROI is a financial tool that forecasts financial returns or profit from an investment [13, 14]. The forecasting process is called ROI analysis. This uses a methodology to convert (monetise) costs and benefits into ROI [15–17]. ROI is reported as metric (percentage or a ratio), e.g., ROI = 1:1 means a 100% return was made. ROI is one of many financial metrics used to judge efficiency of an investment . ROI can be viewed independently, in comparison to other programmes, or against the counterfactual (doing nothing) . In healthcare, ROI has been used to evaluate financial value of a programme post implementation . ROI has also been used commercially as an economic performance measure for meeting product quality specifications . ROI is sometimes used as a performance management tool, that is to ensure that organisations achieve their desired strategic goals . These traditional definitions and uses of ROI are not disputed here. However, as ROI moved from commerce to healthcare frontlines, it became more than a metric. It became a concept of returns or gains from an investment.
ROI’s introduction into healthcare has caused concern [11, 21]. As well as the rationale for ROI being to justify investment business cases , the language also used is to “defend” against disinvestment [11, 23]. In many industries, including healthcare, there are several ROI technical and philosophical challenges [21, 24–28]. The major concerns are ROI’s de-emphasizing of wider organisational benefits. Modifications of ROI methodology have been attempted, for example detailing non-monetisable programme benefits as additional (not primary) benefits [9, 16]. This was in recognition that only a small fraction of QI benefits are actually monetisable . However, there is still a general belief that only monetisable benefits should be seen as ROI [9, 16, 29]. This is in-spite of the recognition that non-monetisable benefits are highly valued by most organisations [9, 16]. This has created scepticism as to the extent of the influence of non-monetisable benefits on investment decision-making [11, 30, 31].
Unsurprisingly, some deem the current ROI approach as aesthetic and synthetic , an insincere “placebo” and an oversimplification [27, 32]. As such, some industries appear to have rebranded ROI. In marketing and commercial service industries there is return-on-quality , and value-on-investment . In healthcare, Leggat , called for a return-to-care, and Fischer & Duncan , a return-to-value. Healthcare researchers have also been slow to embrace ROI. Currently, many published QI-ROI studies involve small projects, often in health promotion, public health, or back-office services like laboratories (e.g., [21, 37–40]. This calls for an understanding and reconciliation of healthcare views on ROI. This endeavour must be based on a logical assessment of ROI as a concept, not as a metric. Further, this must be driven by a logic of contextual appropriateness .
Studying concepts such as ROI in context invites the understanding of institutional logics. Logics are socially constructed sets of assumptions, values, and beliefs that are used to ascribe meaning, as well as frame reasoning and legitimise choices . As such, they reflect embedded cultures. Healthcare is a complex social environment, filled with sub-cultures . Thus here, complexity constitutes multiple actions and interactions of not only humans, but technologies, processes, and systems [44, 45]. Healthcare has various stakeholders or groups and individuals that affect and are affected by healthcare . These stakeholders have multiple, at times conflicting objectives and values . Therefore, contextual interactions also entail multiple embedded theories, cognitive or symbolic systems [41, 47, 48]. For this reason, QI in healthcare is a complex intervention, with varied emergent and unpredictable outcomes .
Healthcare is at the juncture of many logics, primarily scientific, clinical, social, and economic logics [41, 47, 48]. Medical professionals may use science logic by emphasising a curing by focusing on evidence-based medicine, and or a care logic by focusing on interpersonal aspects. Managers may use an economic logic and focus on competition, markets, and financial outcomes. Further, some may use societal logics and focus on population health and socio-economic outcomes [41, 47, 48]. The presence of multiple logics explains the multiple ways used to define healthcare quality . For example, some may describe quality as that which save costs; economic logic , is evidence based; science logic, or prioritises positive patient experience; care logic .
In the current study, the interest is the conceptualisation of ROI as a concept that is meaningful for healthcare stakeholders, particularly healthcare leaders as decision-makers. Concepts are mental abstractions which package complex meaning , and must be unpacked (or analysed) for effective application . In this endeavour, concept analysis must be part of development of testable and practical theories . Through concept analysis, scholars can produce evidence of their best estimate of the ‘probable truth’ about concepts . This is a complex entangled task of concept analysis and development. In modern philosophy such as Critical Realism , this undertaking assumes that concepts are contextual and changeable [54, 55]. Further, this presupposes moderate philosophies about the nature (Ontology) of concepts .
Moderate philosophies are different from traditional philosophies where the ‘truth’ is seen as absolute and or residing on one end of the spectrum. In Realism, reality exists regardless of human perception, whilst Relativism views reality as based on human perception and socially constructed. Lack of clarity about concepts can lead to poor communication, poor application in research and in practice [52, 57]. As healthcare organisations are complex dynamic contexts, modern philosophy insights could support relations between QI implementers and investors. A scientific study of ROI as a concept grounded on a moderate philosophy may help engage QI researchers, improve practical application for practitioners, and improve communication amongst improvement stakeholders.
The aim of this study was to learn how the concept of a return-on-investment for healthcare large-scale QI is understood, and how this differs from related concepts. We first analysed, then developed the large-scale QI-ROI concept for healthcare based on the systematic literature review. We then proposed a framework for analysis of return-on-investment from QI programmes.
This paper is part of a larger integrative systematic literature review on the conceptualisation of ROI in healthcare QI. Our review is registered with PROSPERO, CRD42021236948. A link to our PRISMA reporting checklist  can be found in the supplementary files. We followed review guidance by Whittemore and Knafl  and conceptual analysis and development by Hupcey and Penrod  and Jabareen . This led to 8 separate review stages. Stage 1; clarifying research question, involved background reading as discussed in our protocol on PROSPERO. The remainder of the stages are reported here. Stages 2–3 involved searching and selecting literature. In stage 4 we assessed the quality of research studies, stages 5–8 are reported in the synthesis, analysis, and results sections below.
The identification of suitable search terms was an iterative processes. To compile a list of ROI-like terms, we referred to the National Institute of Health and Care Excellence (NICE) ROI guide. NICE  views ROI as a term for various economic evaluation tools and processes used to evaluate value-for-money of healthcare programmes. Economic evaluation is the comparative analysis of alternative courses of action in relation to both their costs and consequences . Economic evaluation methods include cost–benefit analysis (CBA), cost-effectiveness analysis (CEA), and cost consequence analysis (CCA). Search terms were also derived from background reading in healthcare and commercial literature. The final search terms were in three categories (Table 1): (i) context, (ii) QI methods, and (iii) QI outcomes. Category 2 terms were the most frequently mentioned QI methods in literature. Category 3 terms denote some form of outcome (return, benefit) derived from some form of input (investment, cost, resource).
We searched Medline, Embase, Global health, PsycInfo, EconLit, NHS EED, Web of Science, Google, Google scholar, organisational journals, as well hand-searched citations. No language and date limits were set to enable us to note any changes in QI-ROI conceptualisation over time. The search ended January 30, 2021. An example of the search strategy for Web of Science has been provided as Supplementary Table 1. A link to more of our search strategies can be found in the Supplementary files. The main search terms are defined in Table 2.
As ROI is an investment allocation decision tool, our stakeholder of interest were the healthcare leaders, and level of analysis the organisation, where decision-making outcomes are assessed. During our initial search, many articles identified themselves as large-scale QI programmes. However, at closer inspection, some of these only impacted a small part of an organisation and were therefore equivalent to a small organisational unit intervention. To focus our selection criteria, we developed a preliminary ROI conceptual framework ( Fig. 1). The preliminary framework contained various needs and obligations of healthcare organisations [71, 72]. We assumed these to signal desired organisational outcomes.
The Framework had four criteria: 1) organisational performance (patients and financial outcomes), 2) organisational capacity and capability, 3) external relations (e.g., accreditation), and 4) unintended consequences (positive/negative). Organisational performance is a marker of how well organisations perform on delivering value for its stakeholders . Thus, in a way it includes external relations, e.g., population health. However, external outcomes were isolated here to deduce some unique external outcomes and obligations towards external stakeholders. We included negative outcomes as potential indicators of the lack of ROI positive returns. We then used this framework to decide on eligibility.
To be included, the literature that had to mention at least three QI organisational goals or benefits, two of which had to be patient or financial outcomes. Through this, we sought to isolate articles that discussed a range of QI outcomes, with patient and financial outcomes as basic organisational goals. In addition, articles had to mention use of at least one QI method, and involvement of various stakeholders, in at least two organisational units. Altogether, this denoted a three-dimensional criteria: depth, breath, and complexity of QI programmes per organisation. We included literature on discussions of large-scale QI programmes outcomes across healthcare globally. Table 3 has included/excluded article types.
Screening and selection of articles
Data were managed using Endnote citation manager  and Ryann systematic review app . Screening and selection were performed by two independent reviewers, ST, and MM. To refine our selection criteria, five articles were initially selected and discussed to clarify any uncertainties. The two reviewers then completed the screening and selection of the remaining articles independently: ST 100%, MM 5%. Overall agreement was over 90%. Disagreements were discussed and settled by ST and MM, as well as with co-authors CH and K-CC.
For researchers of integrative reviews and conceptual development, quality assessment is optional as the quality of studies has little or no bearing on concept development [53, 59]. As such, there was no intention to exclude articles based on their quality. However, to understand the scientific context in which QI benefits are discussed, we assessed all empirical studies using specific quality assessment and reporting tools. For reviews, we used the Critical Appraisal Skills Programme (CASP) tool , for mixed methods, we used the Mixed Methods Appraisal Tool (MMAT) , for implementation studies, we used Standards for Reporting Implementation Studies (STaRI) . For economic evaluations, we used the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) , and for QI, we used the Standards for QUality Improvement Reporting Excellence (SQUIRE) tool . As these are different tools, there was no single criteria to judge collective study quality. We therefore assessed the number of appropriate items reported or addressed as per respective study’s tool. We assigned good if 80–100% items were addressed, moderate if 50–79% of items were addressed, and poor if less than 50%.
Data extraction was performed using words and phrases in the preliminary conceptual framework as well as outcomes in the reviews search terms. We searched for these from all parts of an article where QI benefits, outcomes, and goals may be discussed. Articles were tabulated according to type of article, type of focus, country, setting, programme type, and outcomes discussed. The data collection tool can be found as Supplementary Table 2.
Data synthesis and analysis
The synthesis and analysis section forms stages 5–7 of the integrative review process (integrate, synthesise, analyse). The synthesis, analysis, and framework development were performed iteratively by ST. All steps in the synthesis and analysis were discussed with co-authors CH and K-CC. We used the principle-based analysis method , to assess the maturity of the QI-ROI concept in healthcare literature. This involved asking four principle-based questions: 1) Epistemological principle: is the concept clearly defined and well differentiated from other concepts? 2) Pragmatic principle: is the concept applicable and useful within the scientific realm of inquiry? Has it been operationalised? 3) Linguistic principle: is the concept used consistently and appropriately within context? 4) Logical principle: does the concept hold its boundaries through theoretical integration with other concepts?
Once the ROI concept maturity was established, we followed Jabareen’s  conceptual framework development process. Jabareen describes a conceptual framework as “a network…of interlinked concepts that…provide a comprehensive understanding of a phenomenon or phenomena” (p.51). A framework was developed by identifying and naming concepts, describing concepts. Concepts were then categorised according to their ontological, epistemological, or methodological role. This was followed by synthesising, sense-making, and integration of similar concepts into one new concept, the QI-ROI. We also contextualise the ROI concept by highlighting how the concept is defined in the healthcare context, the alternative explanations afforded by the new concept which are not normally enabled by similar concepts, and the patterns in which the QI-ROI concept appear in the healthcare context .
A total of 10 428 articles were retrieved, 10 327 were excluded for various reason as seen in Fig. 2. One hundred and two (102) articles were eligible, 34 were excluded and 68 included. Included articles were: Conceptual n = 24, Quantitative n = 19, including three economic evaluations (CEA n = 1, Economic Impact n = 1, ROI n = 1), Qualitative n = 3, Mixed-Methods n = 8, Systematic Reviews n = 8 (2 economic; 1 SROI), Literature reviews n = 2, Brief Report n = 4, Thirty three of the excluded articles engaged a single department and/or discussed two or less QI outcomes/goals. Thirteen of these were collaboratives. There was one pre-print. A link to the excluded studies document is available on the supplementary files.
Included articles covered different healthcare levels and disciplines globally. Primary care included public health, child and maternal health, and mental health. Secondary or tertiary healthcare included mental health, medical and surgical care, critical care, accident and emergency and acute care services, paediatrics and neonatal care, outpatients, pharmacy, and laboratories. One article covered both health and social care, and another was about a charitable organisation. Global regions were Africa, Asia, Europe, Australia, and Canada, with the US and UK the mostly represented. The summary of included studies can be found in supplementary files (Supplementary Table 3).
Quality of studies
From the 68 articles, 30 were not subject to quality assessment. This included conceptual articles, unsystematic literature reviews, brief reports. Thirty eight articles were subject to quality assessment: 19 quantitative studies, three qualitative studies, eight mixed-methods studies, and eight systematic reviews. Of the 38, 39% reported or addressed 80%-100% all items required, 43% reported on 50%-79% the data required, and 18% reported below 50% of items by their respective reporting tool. Thirty out of thirty eight studies were primary studies. In these, there were three main areas of poor reporting and or poor rigour on all types of studies: ethics (29%), statistical analysis methods (75%), discussion of study limitations and their management (42%), integration of quantitative and qualitative data unclear (29%). Reporting of funding and affiliations missing on three of all 35 studies. Therefore, the quality of the studies was summed up as moderate. The quality assessment summary is available as Supplementary Table 4.
QI Economically-focused literature
Economically-focused literature were those articles whose specific focus was on either studying or discussing QI economic benefits. This made up 15 articles of the 68 QI literature. Amongst these were four conceptual literature, (three of which were business case discussions) [22, 36, 81, 82], four QI evaluation frameworks [83–86], two systematic reviews of economic evaluations [87, 88], one article discussed cost effectiveness of QI programmes , one article calculated an organisation’s QI related cost-savings , and two were economic evaluations [91, 92]. One article discussed cost–benefit analysis . Of the total 15 articles, ROI was a specific subject of only four [81, 84, 87, 91].
Business case conceptual literature
Conceptual literature authors discussed QI business case development [22, 36, 81, 82]. Their views were based on literature reviews, expert opinions, case studies, or all three. This literature were mostly sources of information for how to develop business cases that justify QI programme from a financial benefit perspective. However, they also highlighted multiple QI objectives and stakeholders. As such, there was a requirement to present QI outcomes as a monetised ratio, and or use of ROI as a financial performance measurement method . However, this literature also highlighted multiple QI objectives as well as multiple stakeholders as QI beneficiaries.
These authors advanced on the conceptual literature by developing QI business case frameworks that incorporate monetary and non-monetary benefits [83–86, 93]. QI was seen to serve various organisational interests, for various internal and external stakeholders. As such, business case frameworks centred around these principles. Swensen et al. focused on four organisational interests: patient’s needs, organisational reputation, pride, and financial returns. There was also an interest in organisational productivity and efficiency. Bailit and Dyer , described 10 business case arguments that combine financial, strategic, and internal organisational objectives.
Shah and course  had a six category framework containing three as financial measures (revenue, cost-reduction cost-avoidance) and one measure for patients, family and carers experience, and one for staff experience, and lastly one for productivity and efficiency. Financial objectives and outcomes included cost-avoidance such future legal costs . Internal outcomes also included capacity building, whilst external outcomes examples included market share [84–86].
QI economic evaluation literature
There were three economic evaluations [89, 91, 92], and two systemic reviews of QI economic evaluations [87, 88]. By the virtue of their study foci, their measure of ROI was the monetary. These authors saw savings as an important QI outcome, however also discussed a wide range of QI benefits. For example, Crawley-stout et al.  considered internal outcomes (e.g., cost-reduction, productivity, and time savings) and external benefits (e.g., patient costs and carer time). Crawley-stout et al. described ROI as a performance measure used to evaluate investment efficiency in financial terms. de la Perrele et al. reported a lack of QI economic evaluations in their review. They concluded that collaboratives are potentially cost-saving. However, they found that studies used variable methods to assess cost and effectiveness, and that studies did not report negative findings. They recommended that future research should include societal perspectives of costs and savings . Banke-Thomas et al. stated that SROI (a societal version of ROI) can be used across healthcare . However, there were challenges with inadequate skills for ROI evaluation, lack of credible financial proxies, a lack of consensus on; who to include as beneficiaries, how to account for counterfactual and appropriate study-time horizon .
QI non-economically focused literature
These made up 53 [94–147] of the selected 68 articles. These articles included QI effectiveness, process, and impact evaluations as well as discussions of QI achievements over time e.g., [94, 110, 116, 117, 130, 138]. These articles did not focus on ROI or economic measures, but nonetheless highlighted financial outcomes as important benefits for consideration. Some QI implementation studies assessed their implementation costs [109, 140, 145], as part of their study reporting guidelines [78, 148]. Authors here also discussed improving QI effectiveness determinants such as staff and safety culture development. Authors discussed or mentioned financial value or benefit, financial returns or outcomes, cost savings, reduction, containment, and economic impact, as well as productivity, efficiency, value, and benefits. Of these, cost-saving was the most frequently used term. These articles considered ROI as one of many organisational outcomes [103, 130, 135].
There were three QI evaluation frameworks [103, 104, 124]. These frameworks also considered various elements of organisational benefits. Chow-Chua and Goh  combined existing organisational performance tools; the Singapore Quality Award (SQA) model (modelled after Baldrige Award) and balanced scorecard (BSC) to develop a performance and quality improvement evaluation framework for hospitals. Four strategic components were seen as crucial: the drivers of QI (e.g., leadership), internal performance, knowledge management, and QI outcomes. McLees et al.’s  framework for QI in public health was described as a performance management tool, and focused on two key constructs: efficiency and effectiveness. Ciarniene et al.’s  framework envisioned broad value creation through QI.
Morganti et al.  remarked that there is a lack of an agreed concept of QI success, and by extension, ROI. This was seen in how authors gave priority to certain outcomes. Van den Heuwel et al.  for an example, referred to quality improvement as business improvement, viewed ROI quantitatively, and expected quality improvement to be a valuable “side effect” of value improvement (often a euphemism financial improvement). Alternatively, others proclaimed to value patient safety and quality first, and saw financial matters as the valuable “side-effect” of QI . Hunter et al.  considered “cost savings or increased efficiency “helpful by-products” (p. 129).
Swensen et al.  QI business case discussion stated that their QI investment decisions were never based purely on positive ROIs but on broader qualitative considerations. A similar view was held by O’Sullivan et al.  and Shah and Course (2020). Bailit and Dyer , advocated for broad business cases that embrace different rationales for QI investment. Fischer and Duncan  stated that some interventions are purely designed to produce health outcomes. They also called for a broader QI outcome definition that acknowledges the utility of differing projects and value for all stakeholders. The review also indicated that even failed goal attainment can be useful in providing insights and legacies like building capacities and safety cultures [94, 110, 116, 117, 130, 138, 146].
Overall, financial outcomes were not the primary or commonly sought goal or addressed outcome. However, it was seen as directly or indirectly significant by the majority of the authors. The perception that QI is an expense used for revenue generation was seen to be due to faulty assumptions by some authors . For example, authors suggested that profit-seeking through QI first emerged as an optional strategy to increase revenue and market-shares by for-profit healthcare organisations [22, 83, 86]. However, grey areas on views existed and views appear to have shifted towards integrating or emphasising non-monetary outcomes over time. The literature also agreed that QI does not always save cost, and financial outcomes are not the only organisational objectives [22, 36, 82, 83, 89, 93]. These literatures portrayed ROI as any value or benefit from QI for various stakeholders.
The reviewed literature illustrated five main ROI uses related to QI: ROI as 1) a strategic business case development tool, 2) an investment performance measure, 3) a comparative evaluation tool, 4) a cost management tool, and 5) a performance management tool. ROI was also used to create fiscal awareness . Some of this ROI use was similar to commercial ROI use. Various concepts were used to denote a return-on-investment. These concepts were used in relation to changes and improvements in various organisational outcomes including patients, staff, financial, and overall organisational development. Both economic and non-economic focused literature used almost identical concepts to denote an investment and a return as seen in Fig. 3.
However, profit, revenue, and market share were mainly found in the few economic focused literature. The majority of literature put greater emphasis on non-monetary QI benefits. This indicates that although different logics were applied in the conceptualisation of healthcare QI-ROI, the dominant logic was that of health and social care and not economics or markets.
At face value, there were two broad QI-ROI philosophies: the economic and the healthcare ROI philosophies. However, our synthesis indicated a merging meaning of ROI from different schools of thought. Through markets logic, the economic philosophy perspective views outcomes in terms of their tangible, quantitative, or financial offerings. The economic philosophy is related to managerial logics as managers’ roles require them to monitor organisational financial performance. Alternatively, the healthcare perspective suggested a more dulled though increasing financial focus. Healthcare logics (medical, societal) tended to view ROI qualitatively, primarily from a patient and staff perspectives but also wider internal and external stakeholders. The integrated literature indicated that healthcare leaders value these perspectives too. Table 4 illustrates this finding.
Although concept analysis, concept development, and conceptual framework development are traditionally separate steps , they have been performed concurrently in this review. Firstly, we sought to understand the nature of the ROI concept in healthcare QI. Using the Hupcey and Penrod  principle-based concept analysis method, we found that the QI-ROI concept is not clearly defined or developed (principle 1), the ROI application method is unclear (principle 2), its appropriate use is unestablished (principle 3), and conceptual boundaries are unclear (principle 4). These determinations answered the first part of our research question; the nature of the ROI concept in healthcare QI.
We then sought to answer the second part of our research question; the definition of the QI-ROI concept and its relationship with similar concepts. We did this by following the concept development and conceptual framework development guide described earlier by Hupcey and Penrod , and Jabareen . Our review confirmed that various concepts and terms are used to denote returns or investment vs returns in healthcare QI. Some of these concepts (e.g., CEA, CBA, value) were identified in the background literature and used as the review’s search terms. These concepts, including the differences between costs, investments, and revenue have been discussed in more details in economic literature (e.g., [61, 149–151]. We refer interested readers to such text for nuanced descriptions.
QI-ROI concept development
To develop the QI-ROI concept, we first eliminated similar concepts by differentiating QI-ROI from other ROI-like concepts and terms. To start with, financial or economic returns are alternative ways of saying ROI . Other terms used to a lesser extent in the review were economic impact, which denotes only the cost of illness , and cost management which is a process of managing and controlling costs of a programme to fit desired criteria, e.g., to reduce costs . Cost-reduction is described as resulting from providing the same or better quality for a lower cost, through new ways of working that eliminate waste (, p.4). Cost avoidance is cost-reduction related to preventing future costs . Very few articles mentioned profit or revenue. Publicly funded healthcare such as the UK NHS does not customarily engage in profit-making. Net profit is what remains after all expenses have been subtracted from net revenue (income generated minus costs). The remaining concepts: SROI, cost–benefit, cost-effectiveness, productivity, efficiency, value, and benefit were seen as the most mimicking ROI. These concepts have varying degrees of association with ROI depending on the ultimate objective sought through a QI programme.
(1a) ROI vs other economic evaluation methods
In health economics, cost-effectiveness is assessed using CEA and whilst costs-benefits are assessed using CBA. CEA and CBA goals are to ensure that fixed allocated quantity of healthcare resources result in most health outcomes improvement (CEA) or maximum social advantage (CBA) . CEA and CBA denote cost vs goal achievement. Although conceptually different, in practice, researchers do conflate the different types of evaluation [154, 155] or see them as interchangeable . Comparative ROI mimics CEA, but ROI reports an aggregated cost–benefit metric similar to CBA’s cost–benefit ratio (CBR). Alternatively, CEA reports an incremental cost-effectiveness ratio (ICER) per health outcomes. Incremental benefits using ROI of new QI programmes have been compared by some researchers [156, 157]. However, given the multiple healthcare objectives, these metrics represent only a fraction of programme benefits or consequences . This point is supported by the current literature review.
CBA is the basis of ROI and SROI. SROI and CBA are monetisation of broad programme benefits and costs such as societal costs and benefits. SROI extends CBA by including environmental and other stakeholder benefits . Alternatively, ROI generally focuses on programme specific costs and benefits from a managerial perspective [9, 16]. As returns-on-investments evaluation methods, CEA, CBA, SROI, and traditional ROI are too narrowly focused as they all ultimately only emphasise monetary focus. According to Bridges , CBA, does not account for how care is produced, and thus excludes many crucial organisational outcomes. Bridges suggested that what is needed is a systematic value assessment approach.
(1b) ROI vs input–output based measures
CEA, CBA, productivity, and efficiency are similar as their goals are using resources without waste. However, they are all a single focus outcomes. CEA/CBA are an input vs goal measures, efficiency and productivity are input vs output measures. Productivity and efficiency are ROI-like as they denote a return (output) of an investment (input) . Inputs and outputs may be both monetary and non-monetary. Productivity is the quantity of outputs per investment/input. Efficiency is achieving those outputs with least or no waste (e.g., in time, money, effort). Therefore, unlike CBA and CEA, efficiency and productivity are related to exactly how care is produced. For example, increasing productivity by increasing patients seen (output) per clinician (input), whilst providing quality care without wasting resources (efficiency).
Efficiency is divided into allocative, productive, and technical efficiency . Simply put, allocative efficiency refers to allocation of healthcare resources such that the most benefits are delivered . Productive efficiency is increasing output per given resource/input/investment (e.g., seeing more patients by same staff member). If this is done such that more is obtained from the same resource, or less resource is required for the same output, it is technically efficient . This description also fits CEA, with outputs being effectiveness. It also mimics the concept of value-for-money (VfM), used to describe the optimal balance between efficiency, economy (lowest cost), and effectiveness . Efficiency and productivity are crucial in healthcare as profit-based ROI is deemed improbable . Efficiency can translate to both monetary ROI (e.g., savings), and non-monetary benefits (e.g., improved staff work experience).
Productivity and efficiency are often used to measure performance of healthcare organisations [150, 158]. Productivity may enable allocative efficiency of funds or better time allocation for tasks by staff. Productivity can be an efficiency measure (input/output) . It can also be a combined effectiveness and efficiency measure (goal/input/output), or of all that makes an organisation function better . The latter is what the reviewed literature indicated QI-ROI to be. Effectiveness through attainment of goals alone is therefore also insufficient to describe QI-ROI. Goals may be achieved, but inefficiently. In a balanced productivity-efficiency-effectiveness relationship, all three contribute to the overall QI-ROI . This may then mean avoiding, reducing, and containing costs, and thus saving costs.
(1c) ROI vs cost saving
Cost-saving is also a more likely outcome than hard-cash profit in healthcare QI . Cost-saving was a particularly prevalent term in the reviewed literature and ROI in healthcare has been called savings . The current desire to save costs is thought to have driven the change in focus from cost-effectiveness studies to ROI . Cost-saving means saving money that would have otherwise been spent. Savings (time/money) often result from better efficiency and productivity. Similar terms such as cost-containment, cost-minimisation, cost-avoidance, cost-reduction are also not seen here as complete representations of QI-ROI. Here, these terms are seen as representing outputs, initial or intermediate outcomes that lead to savings. These terms (including cost-management) can also be processes or abilities that enable cost-saving or profit-making. Together, these terms refer to mechanisms (or processes that enable an outcome)  through which long-term financial ROI may be achieved. Alternatively, some may see these initial outcomes as benefits themselves.
QI-ROI framework development
For some organisations, initial outputs and intermediate outcomes may be the intended outcomes and therefore may represent a form of ROI. In Phillips et al.  for example, productivity and efficiency were viewed as final intended outcomes of improvements. In other instances, cost-effectiveness may be the intended goal. Often in healthcare the ultimate objective is to achieve higher goals, such as financial stability. In such cases, implementing QI leads to change and development and possibly improvement in desired outcomes. Improvement may result in improved productivity and efficiency. This in turn improves abilities to better avoid, reduce, or manage costs, leading to savings, and potentially monetary ROI. All of this is a benefit in and of itself. Monetary ROI depends on each output, ability, or outcome, most of which is non-monetary. This conceptualisation of QI-ROI is illustrated in Fig. 4 below.
The QI-ROI conceptualisation can be translated as follows: value is any outcome seen to be of importance, utility, or usefulness ; attaining a return-on-investment whatever that/those are, is valued and therefore of benefit. A benefit is any outcome that produces useful, helpful, or advantageous outcomes . Any benefit is of value in of itself. Based on this review, a fuller description of QI-ROI is suggested as follows: QI-ROI is any value or benefit (or any valued benefit) derived from or contributed to by QI programmes. This value or benefit maybe in a form of an improved output, process, ability, outcome, and or overall impacts, depending on an organisation’s values and objectives.This description reflects the Healthcare Financial Management Association (HFMA), NHS England, and NHS Improvement (NHSE&I)'s view. The NHSE&I differentiates from cash-releasing benefits that enable resource allocation, and non-cash-releasing benefits that provide economic benefit, without releasing cash .
This definition is different from that which supports the economic logics. For example, Phillip et al. , and Solid  who discussed value and ROI as separate (presumed to mean non-monetary and monetary value). Here, only monetary value is seen as ROI. This assertion is based on viewing ROI as a purely quantitative metric. However, Solid’s writing also refers to value as being subjective (p. 5). The definition of ROI concept as any benefit may be deemed an abuse of ROI [16, 29]. However, this definition was based on the review of numerous healthcare stakeholders’ views. This may be an indication that new constructive conversations about how to integrate economic and accounting concepts in healthcare are needed [57, 165]. After-all, value was not always seen as a ratio between perceived benefits and perceived costs . It was initially viewed as multi-dimensional by earlier economic scholars [57, 166–168]. Therefore, other QI programme effects that are regarded as valuable must not only be considered , but prioritised where appropriate.
The view of QI-ROI in healthcare as a broad and no-specific concept, encompassing both monetary and non-monetary outcomes, opens ROI to being a context-specific and dynamic concept. This is in-line with the moderate ontological expectations of modern concepts [54, 55]. This suggests abilities to compromise and accommodate varied logics that govern healthcare . The concepts of ROI, value, and benefit, denote the beliefs about what is a true return, value, or benefit (Ontology) as well as what seen as a moral good (Axiology) in healthcare QI. As such, the QI-ROI can be seen as a value-based ROI. Traditional ROI is a monetary measure (Realism), benefits in general tend to be described as non-monetary (Relativism), and value can denote either a monetary or a non-monetary outcome (Critical Realism). These beliefs then influence how evidence is created, viewed, and studied (Epistemology and Methodology) .
The lack of convincing vocabulary to argue against the logics of the markets in healthcare was seen as the limiting factor by Bozeman . In the current review this has also been demonstrated. In non-economic literature, financial outcomes mentions appeared to be nebulous, or in general use of terms as in everyday language. Authors here focused on non-monetary outcomes. They discussed aspirations to raise fiscal awareness and encourage financial outcomes focus on QI evaluations. This can be contrasted with economic focussed literature where for example, economic evaluations referred to ROI as a specific scientific quantitative measure. Traditional ROI is portrayed as a rational measure of objectively assessed inputs leading to objective outputs . This suggests that the scientific language of healthcare stakeholders for ROI is currently underdeveloped. It reflects general challenges of legitimising and aligning qualitative benefits with specific scientific measures that are seen as valid and trusted .
The QI literature discussed the use of ROI in several ways, including to create fiscal awareness. Such a use for ROI was noted by Botchkarev & Andru  in their analysis of ROI definitions. Their typology included the use of ROI as a persuasive device to gain credibility for a desired programme . Healthcare leaders need credible recourse or language to articulate large-scale QI benefits [85, 170]. If we accept that reality is socially constructed, then we can view various logics as both coercive and emancipating . That is, although political and market logics may constrain freedoms of local expressions, the mere tendency for humans to create their own meanings has potential to liberate from such constraints. Logics ‘in situ’ provide symbolic systems and vocabularies for expression. Hence, the prevailing logic both shapes and is shaped by contexts. Scientifically developing concepts for healthcare is essential to support this.
Establishing ways of expressing QI-ROI from healthcare programmes is crucial to avoid missing opportunities for essential healthcare improvements . Additionally, insisting on inflexible use of a certain (ROI) policy may lead to data manipulation in bids to increase credibility [172, 173]. The view of ROI as both monetary and non-monetary benefits reflects the multi-stakeholder healthcare context. The lean towards non-monetary benefits is influenced by persistent healthcare and societal logics [11, 23, 47]. These logics emphasise relief of suffering and ethical principles such as beneficence (benefiting others) and non-maleficence (do no harm) . It is therefore important to differentiate ROI as concept and as a metric. A concept is more than a metric, it encapsulates mental abstractions about how it is perceived by those using it and influences the decisions that then may follow .
Strengths and limitations
This review has a few strengths and limitations. Concept analysis, concept development, and conceptual framework development are traditionally separate steps , unlike in this review. It is however accepted that these processes are intertwined . However, we based our analysis on intensive background reading as well as a large review of different QI literature. This enabled us to gain some understanding of the current “state of the science”  surrounding the ROI concept as used in healthcare QI. We then followed a well-recognised Hupcey and Penrod , and Jabareen  development process to start to develop the concept QI-ROI in healthcare.
Secondly, productivity and efficiency proved to be crucial parts of the QI-ROI concept. These concepts were not included as search terms, however the large amount of literature retrieved means that it is unlikely that this made a significant difference in the review. Alternatively, it could be argued that our inclusion of specific ROI-like concepts in our search terms constitutes sampling bias. However, this strategy helped identify relevant literature for a more in-depth review. Thirdly, a significant amount of the literature reviewed was non-empirical in nature. Although this literature lacks a scientific focus, it was nonetheless very insightful in understanding the nature of the QI-ROI concept. Fourthly, some of the literature is quite dated, however newer literature suggest continuance of some trends and issues in QI-ROI and business case development. Lastly, subjectivity in the synthesis and analysis cannot be ruled out. As Parkinson et al.  put it “…findings are a consequence of intersubjective meaning-making through imagination, interpretation, and conceptual input…” (p15).
Implications for research and practice
Implementation and Improvement Sciences are faced with the challenge of developing the ROI concept that is theoretically sound, and scientifically valid. This means a QI-ROI framework must clearly isolate constructs that can and should be included in an evaluation tool. The development of the QI-ROI concept and its conceptual framework must also ensure it is fit for purpose by incorporating both monetary and non-monetary benefits. This means finding more innovative and accessible ways for evaluating the QI-ROI aspects that are hard to measure and or monetise. Developing the QI-ROI concept in this way will enable the field to progress and take ownership in QI fiscal matters, and leaders to justify investments. This is crucial as justification for investment is unavoidable and necessary in the current economic climate.
The review indicated that the use of reporting tools is having a positive effect on the quality of QI studies. However, there remains room for improvement. QI researchers have a responsibility to show more transparency on ethical aspects of their studies. QI studies may not require ethical permissions, and if so, it must be stated as such. Current QI reporting tools allow for this [78, 148]. QI studies must also be clear about their statistical analysis methods use. Another area of improvement is the integration of qualitative and quantitative data in their analysis. This is important in strengthening research findings . Further, reporting of study limitations was limited in the reviewed literature. The knowledge of QI implementation or research challenges can help arm other researchers and field practitioners in their QI initiatives. This is crucial for developing a stronger evidence-base as we develop the QI-ROI concept.
Return-on-investment is an important tool with great potential to communicate QI benefits not covered by CEA and CBA. However, in its traditional form, ROI does not take advantage of this potential use. Ignoring the paradoxes contained within the traditional ROI use in healthcare may continue to keep ROI in the fringes of QI evaluation or cause conflict amongst stakeholders if enforced. Therefore, continued application of ROI must be based on its conceptualisation within healthcare QI and must be grounded on scientific inquiry that considers relevance to practice and policy. If QI-ROI is developed in this way, its legitimacy within healthcare stakeholders may be established and increased. In this review, we have begun to unpack what ROI is and means for healthcare stakeholders in QI at the organisational level. We hope to continue to develop this framework into a practical tool that is meaningful to its users: the QI teams and healthcare leaders, and QI investors.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request. Some data has been included in this article as its supplementary information files.
Return on Investment
Social Return on Investment
Return on Investment from healthcare quality improvement
Cost Effectiveness Analysis
Cost Utility Analysis
Cost Benefit Analysis
Ross S, Naylor C. Quality improvement in mental health. King's Fund. 2017.
Braithwaite J, Marks D, Taylor N. Harnessing implementation science to improve care quality and patient safety: a systematic review of targeted literature. Int J Qual Health Care. 2014;26(3):321–9.
Storkholm MH, Mazzocato P, Savage M, Savage C. Money’s (not) on my mind: a qualitative study of how staff and managers understand health care’s triple Aim. BMC Health Serv Res. 2017;17(1):98.
Benn J, Burnett S, Par A, Pinto A, Isk S, et al. Studying large-scale programmes to improve patient safety in whole care systems: challenges for research. Soc Sci Med. 2009;69(12):1767–76.
Øvretveit J, Klazinga N. Learning from large-scale quality improvement through comparisons. Int J Qual Health Care. 2012;24(5):463–9.
Schouten LMT, Grol RPTM, Hulscher MEJL. Factors influencing success in quality-improvement collaboratives: development and psychometric testing of an instrument. Implement Sci. 2010;5(1):84.
Dopp AR, Narcisse M-R, Mundey P, Silovsky JF, Smith AB, Mandell D, et al. A scoping review of strategies for financing the implementation of evidence-based practices in behavioral health systems: State of the literature and future directions. Implementation Research and Practice. 2020;1:2633489520939980.
Danzon PM, Drummond MF, Towse A, Pauly MV. Objectives, budgets, thresholds, and opportunity costs—a health economics approach: an ISPOR Special Task Force report . Value in Health. 2018;21(2):140–5.
Solid CA. Return on investment for healthcare quality improvement. Springer International Publishing; 2020.
World Health Organization. Making the investment case for mental health: a WHO/UND guidance note. Bahrain: 2019.
Masters R, Anwar E, Collins B, Cookson R, Capewell S. Return on investment of public health interventions: a systematic review. J Epidemiol Community Health. 2017;71(8):827.
Nystrand C, Gebreslassie M, Ssegonja R, Feldman I, Sampaio F. A systematic review of economic evaluations of public health interventions targeting alcohol, tobacco, illicit drug use and problematic gambling: Using a case study to assess transferability. Health Pol. 2021;125(1):54–74.
Botchkarev A. Estimating the accuracy of the Return on Investment (ROI) performance evaluations. Int J Inform, Knowledge, Manage. 2015;10;217–33.
Gosselin V, Boccanfuso D, Laberge S. Social return on investment (SROI) method to evaluate physical activity and sport interventions: a systematic review. The international journal of behavioral nutrition and physical activity. 2020;17(1):26.
Bukhari H, Andreatta P, Goldiez B, Rabelo L. A framework for determining the return on investment of simulation-based training in health care. INQUIRY: The Journal of Health Care Organization, Provision, and Financing. 2017;54. https://doi.org/10.1177/0046958016687176.
Phillips PP, Phillips JJ, Edwards LA. Measuring the success of coaching: a step-by-step guide for measuring impact and calculating ROI. American Society for Training and Development; 2012.
Pokhrel S. Return on investment (ROI) modelling in public health: strengths and limitations. Eur J Pub Health. 2015;25(6):908–9.
Parand A, Dopson S, Renz A, Vincent C. The role of hospital managers in quality and patient safety: a systematic review. BMJ Open. 2014;4(9): e005055.
Otake T, Min KJ. Inventory and investment in quality improvement under return on investment maximization. Comput Oper Res. 2001;28(10):997–1012.
Hewko SJ, Cummings GG. Performance management in healthcare: a critical analysis. Leadersh Health Serv. 2016;29(1):52–68.
Price CP, McGinley P, John AS. What is the return on investment for laboratory medicine? The antidote to silo budgeting in diagnostics. Br J Healthc Manag. 2020;26(6):1–8.
Leatherman S, Berwick D, Iles D, Lewin LS, Davidoff F, Nolan T, et al. The business case for quality: case studies and an analysis. Health affairs (Project Hope). 2003;22(2):17.
Brousselle A, Benmarhnia T, Benhadj L. What are the benefits and risks of using return on investment to defend public health programs? Prev Med Rep. 2016;3:135–8.
Bontis N, Fitz‐enz J. Intellectual Capital ROI: A causal map of human capital antecedents and consequents. J Intellect Cap. 2002;3(3):223–47.
Boonstra A, Broekhuis M. Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC Health Serv Res. 2010;10(1):231.
Boyd J, Epanchin-Niell R, Siikamäki J. Conservation Planning: A Review of Return on Investment Analysis. Review of Environmental Economics and Policy. 2015;9(1):23–42.
Dearden J. CASE AGAINST ROI CONTROL. Harvard Business Review. 1969. [Available https://hbr.org/1969/05/the-case-against-roi-control]
Ozminkowski RJ, Serxner S, Marlo K, Kichlu R, Ratelis E, Van de Meulebroecke J. Beyond ROI: Using Value of Investment to Measure Employee Health and Wellness. Popul Health Manag. 2016;19(4):227–9.
Botchkarev A, Andru P. A Return on Investment as a Metric for Evaluating Information Systems: Taxonomy and Application. Interdiscip J Inf Knowl Manag. 2011;6:245–69.
Anderson JA. Evolution of the health care quality journey: from cost reduction to facilitating patient safety. J Leg Med. 2010;31(1):59–72.
Lurie N, Somers SA, Fremont A, Angeles J, Murphy EK, Hamblin A. Challenges to using a business case for addressing health disparities. Health Aff. 2008;27(2):334–8.
Andru P, Botchkarev A. Return on investment: A placebo for the Chief Financial Officer… and other paradoxes. Journal of MultiDisciplinary Evaluation. 2011;7(16):201–6.
Rust RT, Zahorik AJ, Keiningham TL. Return on quality (ROQ): Making service quality financially accountable. J Mark. 1995;59(2):58–70.
Phelps CE, Madhavan G. Using Multicriteria Approaches to Assess the Value of Health Care. Value Health. 2017;20(2):251–5.
Leggat SG. Information management: the limitations of ROI. Australian Health Review. 2007;31:488.
Fischer HR, Duncan SD. The business case for quality improvement. J Perinatol. 2020;40(6):972–9.
Arvidson M, Battye F, Salisbury D. The social return on investment in community befriending. Int J Pub Sector Manage. 2014;27(3):225–40.
Astrella JA. Return on Investment: Evaluating the Evidence Regarding Financial Outcomes of Workplace Wellness Programs. J Nurs Adm. 2017;47(7/8):379–83.
Baxter S, Sanderson K, Venn AJ, Blizzard CL, Palmer AJ. The relationship between return on investment and quality of study methodology in workplace health promotion programs. Am J Health Promot. 2014;28(6):347–63.
Chalutz B-G. An ROI-based review of HR analytics: practical implementation tools. Pers Rev. 2019;48(6):1429–48.
Thornton PH, Ocasio W. Institutional logics. The Sage handbook of organizational institutionalism. 2008;2008(840):99–128.
Thornton PH, Ocasio W, Lounsbury M. The institutional logics perspective: A new approach to culture, structure and process. Oxford: Oxford University Press on Demand; 2012.
Fulop N, Robert G. Context for successful quality improvement. London: The Health Foundation; 2015.
Braithwaite J, Churruca K, Ellis LA, Long J, Clay-Williams R, Damen N, et al. Complexity science in healthcare. Sydney: Australian Institute of Health Innovation, Macquarie; 2017.
Cresswell KM, Worth A, Sheikh A. Actor-Network Theory and its role in understanding the implementation of information technology developments in healthcare. BMC Med Inform Decis Mak. 2010;10(1):1–11.
Gasparski WW. The stakeholder organisation theory and its systemic foundation. Int J Gen Syst. 2009;38(6):659–65.
Besharov ML, Smith WK. Multiple institutional logics in organizations: Explaining their varied nature and implications. Acad Manag Rev. 2014;39(3):364–81.
Dunn MB, Jones C. Institutional logics and institutional pluralism: The contestation of care and science logics in medical education, 1967–2005. Adm Sci Q. 2010;55(1):114–49.
Braithwaite J, Herkes J, Ludlow K, Testa L, Lamprell G. Association between organisational and workplace cultures, and patient outcomes: systematic review. BMJ Open. 2017;7(11): e017708.
Øvretveit J. Does improving quality save money. A review of evidence of which improvements to quality reduce costs to health service providers London: The Health Foundation. 2009;95.
Secretary of State for Health, editor. High quality care for all: NHS next stage review final report. Volume 7432. 2008.
Morse JM, Hupcey JE, Mitcham C, Lenz ER. Concept analysis in nursing research: a critical appraisal. Sch Inq Nurs Pract. 1996;10(3):253–77.
Jabareen Y. Building a Conceptual Framework: Philosophy, Definitions, and Procedure. Int J Qual Methods. 2009;8(4):49–62.
Risjord M. Rethinking concept analysis. J Adv Nurs. 2009;65(3):684–91.
Hupcey J, Penrod J. Concept Analysis: Examining the State of the Science. Res Theory Nurs Pract. 2005;19:197–208.
Heeks R, Ospina AV, Wall P, editors. Combining pragmatism and critical realism in ICT4D research: an e-Resilience Case Example. International Conference on Social Implications of Computers in Developing Countries. Cham: Springer; 2019. p. 14–25
Svoboda F. In search of value: Vienna School of Art History, Austrian value theory and the others. J Socio-Econ. 2011;40(4):428–35.
Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, The PRISMA, et al. statement: an updated guideline for reporting systematic reviews. BMJ. 2020;2021:372.
Whittemore R, Knafl K. The integrative review: updated methodology. J Adv Nurs. 2005;52(5):546–53.
National Institute for Health and Care Excellence (NICE). Supporting investment in public health: Review of methods for assessing cost effectiveness, cost impact and return on investment. Proof of Concept Report. London; 2011.
Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW. Methods for the Economic Evaluation of Health Care Programmes. Oxford, UNITED KINGDOM: Oxford University Press; 2015.
Berdot S, Korb-Savoldelli V, Jaccoulet E, Zaugg V, Prognon P, Lê LMM, et al. A centralized automated-dispensing system in a French teaching hospital: return on investment and quality improvement. Int J Qual Health Care. 2019;31(3):219–24.
Mason J, Freemantle N, Nazareth I, Eccles M, Haines A, Drummond M. When Is It Cost-effective to Change the Behavior of Health Professionals? JAMA. 2001;286(23):2988–92.
Viner J. The utility concept in value theory and its critics. J Polit Econ. 1925;33(6):638–59.
Cambridge Dictionary. Benefit definition 2022 [Available from: https://dictionary.cambridge.org/dictionary/english/benefit].
Merriam-Webster Dictionary. Outcome Definition 2022 [Available from https://www.merriam-webster.com/dictionary/outcome?utm_campaign=sd&utm_medium=serp&utm_source=jsonld]
Chartier LB, Cheng AH, Stang AS, Vaillancourt S. Quality improvement primer part 1: preparing for a quality improvement project in the emergency department. Canadian Journal of Emergency Medicine. 2018;20(1):104–11.
Jones B, Vaux E, Olsson-Brown A. How to get started in quality improvement. BMJ. 2019;364: k5408.
Healthcare Quality Improvement Partnership (HQIP). A guide to quality improvement methods. Det Norske Veritas and Germanischer Lloyd (DNV-GL); 2015.
Department of Health and Social Care. The health and care system explained. U.K; 2013.[Available https://www.gov.uk/government/publications/the-health-and-care-system-explained/the-health-and-care-system-explained]
Gartner J-B, Lemaire C. Dimensions of performance and related key performance indicators addressed in healthcare organisations: A literature review. Int J Health Plan Manage. 2022;37(4):1941–52.
Kruk ME, Gage AD, Arsenault C, Jordan K, Leslie HH, Roder-DeWan S, et al. High-quality health systems in the Sustainable Development Goals era: time for a revolution. Lancet Glob Health. 2018;6(11):e1196–252.
Elg M, Broryd KP, Kollberg B. Performance measurement to drive improvements in healthcare practice. Int J Oper Prod Manage. 2013;33(11/12):1623–51.
The EndNote Team, Clarivate. Endnote. Version: Endonote X9. Type: 64bt. Philadelphia; 2013.
Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web and mobile app for systematic reviews. Syst Rev. 2016;5(1):1–10.
Critical Appraisal Skills Programme (CASP) . CASP 2019 [https://casp-uk.net/]
Hong QN, Pluye P, Fàbregues S, Bartlett G, Boardman F, Cargo M, et al. Mixed methods appraisal tool (MMAT), version 2018. Registration of copyright. 2018;1148552:10.
Pinnock H, Barwick M, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ, et al. Standards for Reporting Implementation Studies (StaRI) Statement. BMJ. 2017;356: i6795.
Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement. BMJ : British Medical Journal. 2013;346: f1049.
Ogrinc G, Mooney SE, Estrada C, Foster T, Goldmann D, Hall LW, et al. The SQUIRE (Standards for QUality Improvement Reporting Excellence) guidelines for quality improvement reporting: explanation and elaboration. Qual Saf Health Care. 2008;17(Suppl 1):i13-32.
Moody M, Littlepage L, Paydar N. Measuring Social Return on Investment. Nonprofit Manag Leadersh. 2015;26(1):19–37.
Perencevich EN, Stone PW, Wright SB, Carmeli Y, Fisman DN, Cosgrove SE, et al. Raising standards while watching the bottom line: making a business case for infection control. Infect Control Hosp Epidemiol. 2007;28(10):1121–33.
Bailit M, Dyer MB. Beyond bankable dollars: establishing a business case for improving health care. Issue Brief (Commonw Fund). 2004;754:1–12.
Mery G, Dobrow MJ, Baker GR, Im J, Brown A. Evaluating investment in quality improvement capacity building: A systematic review. BMJ Open. 2017;7(2): e012431.
Shah A, Course S. Building the business case for quality improvement: a framework for evaluating return on investment. Future healthcare journal. 2018;5(2):132–7.
Swensen SJ, Dilling JA, Mc Carty PM, Bolton JW, Harper CM Jr. The business case for health-care quality improvement. J Patient Saf. 2013;9(1):44–52.
Banke-Thomas AO, Madaj B, Charles A, van den Broek N. Social Return on Investment (SROI) methodology to account for value for money of public health interventions: a systematic review. BMC Public Health. 2015;15(1):582.
de la Perrelle L, Radisic G, Cations M, Kaambwa B, Barbery G, Laver K. Costs and economic evaluations of Quality Improvement Collaboratives in healthcare: a systematic review. BMC Health Serv Res. 2020;20(1):155.
Gandjour A, Lauterbach KW. Cost-effectiveness of quality improvement programs in health care. Med Klin. 2002;97(8):499–502.
Comtois J, Paris Y, Poder TG, Chausse S. [The organizational benefits of the Kaizen approach at the Centre Hospitalier Universitaire de Sherbrooke (CHUS)]. L'approche Kaizen au Centre Hospitalier Universitaire de Sherbrooke (CHUS) : un avantage organisationnel significatif. 2013;25 2:169–77.
Crawley-Stout LA, Ward KA, See CH, olph G. Lessons Learned From Measuring Return on Investment in Public Health Quality Improvement Initiatives. JPHMP. 2016;22(2):E28–37.
Schouten LMT, Niessen LW, van de Pas JWAM, Grol RPTM, Hulscher MEJL. Cost-effectiveness of a quality improvement collaborative focusing on patients with diabetes. Med Care. 2010;48(10):884–91.
Rogers PJ, Stevens K, Boymal J. Qualitative cost–benefit evaluation of complex, emergent programs. Eval Program Plann. 2009;32(1):83–90.
Care Quality Commission (CQC), editor. Quality improvement in hospital trusts: Sharing learning from trusts on a journey of QI. In: Care Quality Commission, Newcastle Upon Tyne; 2018.
Beers LS, Godoy L, John T, Long M, Biel MG, et al. Mental Health Screening Quality Improvement Learning Collaborative in Pediatric Primary Care. Pediatrics. 2017;140(6):e20162966.
Benning A, Ghaleb M, Suokas A, Dixon-Woods M, Dawson J, Barber N, et al. Large scale organisational intervention to improve patient safety in four UK hospitals: mixed method evaluation. BMJ. 2011;342(7793):369.
Bevan H, Plsek P, Winstanley L. Part 1: Leading large scale change: A practical guide What the NHS Academy for Large Scale Change learnt and how you can apply these principles within your own health and healthcare setting. NHS Institute for Innovation and Improvement. In: Improvement. NHS Academy for Large Scale Change. Coventry. 2011.
Bielaszka-DuVernay C. Innovation profile: Redesigning acute care processes in Wisconsin. Health Aff. 2011;30(3):422–5.
Bosse G, Abels W, Mtatifikolo F, Ngoli B, Neuner B, Wernecke K-D, et al. Perioperative Care and the Importance of Continuous Quality Improvement-A Controlled Intervention Study in Three Tanzanian Hospitals. Plos One. 2015;10(9).
Botros S, Dunn J. Implementation and spread of a simple and effective way to improve the accuracy of medicines reconciliation on discharge: a hospital-based quality improvement project and success story. BMJ. 2019;8(3):e000363-e.
Bridges JFP. Lean systems approaches to health technology assessment: a patient-focused alternative to cost-effectiveness analysis. Pharmacoeconomics. 2006;24:101–9.
Brink AJ, Messina AP, Feldman C, Richards GA, van den Bergh D, Netcare AS. From guidelines to practice: a pharmacist-driven prospective audit and feedback improvement model for peri-operative antibiotic prophylaxis in 34 South African hospitals. J Antimicrob Chemother. 2017;72(4):1227–34.
Chow-Chua C, Goh M. Framework for evaluating performance and quality improvement in hospitals. Managing Service Quality: An International Journal. 2002;12(1):54–66.
Ciarniene R, Vienazindiene M, Vojtovic S. Process Improvement for Value Creation: a Case of Health Care Organization. Inzinerine Ekonomika-Engineering Economics. 2017;28(1):79–87.
Collins B, Fenney D. Improving patient safety through collaboration A rapid review of the academic health science networks’ patient safety collaboratives. King's Fund: 2019.
Crema M, Verbano C. Lean Management to support Choosing Wisely in healthcare: the first evidence from a systematic literature review. International journal for quality in health care : journal of the International Society for Quality in Health Care. 2017;29(7):889–95.
de Miranda Costa MM, Santana HT, Hern S, ez PJ, Carvalho AA, da Silva Gama ZA, et al. Results of a national system-wide quality improvement initiative for the implementation of evidence-based infection prevention practices in Brazilian hospitals. J Hosp Infect. 2020;105(1):24–34.
DelliFraine JL, Langabeer JR 2nd, Nembhard IM. Assessing the evidence of Six Sigma and Lean in the health care industry. Qual Manag Health Care. 2010;19(3):211–25.
Fortney J, Enderle M, McDougall S, Clothier J, Otero J, Altman L, et al. Implementation outcomes of evidence-based quality improvement for depression in VA community based outpatient clinics. Implement Sci. 2012;7(1):30.
The Health Foundation. Learning Report: Safer Patients Initiative: Lessons from the first major improvement programme addressing patient safety in the UK. The Health Foundation, editor. London; 2011.
Furukawa PdO, Cunha ICKO, Pedreira MdLG. Avaliação de ações ecologicamente sustentáveis no processo de medicação. Rev Bras Enferm. 2016;69(1):23–9.
Goodridge D, Rana M, Harrison EL, Rotter T, Dobson R, Groot G, et al. Assessing the implementation processes of a large-scale, multi-year quality improvement initiative: survey of health care providers. BMC Health Services Res. 2018;18(237).
Hatcher IB. Reducing sharps injuries among health care workers: a sharps container quality improvement project. Jt Comm J Qual Improv. 2002;28(7):410–4.
Heitmiller ES, Hill RB, Marshall CE, Parsons BJ, Berkow LC, Barrasso CA, et al. Blood wastage reduction using Lean Sigma methodology. Transfusion. 2010;50(9):1887–96.
Honda AC, Bernardo VZ, Gerolamo MC, Davis MM. How Lean Six Sigma Principles Improve Hospital Performance. Qual Manag J. 2018;25(2):70–82.
Hunter DJ, Erskine J, Hicks C, McGovern T, Small A, Lugsden E, Whitty P, Steen IN, Eccles M. A mixed-methods evaluation of transformational change in NHS North East. Southampton (UK): NIHR J Library; 2014.
NHS Institute for Innovation and Improvement. The Productive Ward: Releasing time to careTM Learning and Impact Review Final report. In: Institute IfIaIN, editor. 2011.
Jones B, Horton T, Warburton W. The improvement journey: Why organisation-wide improvement in health care matters, and how to get started. The Health Foundation: London; 2019.
Kanamori S, Sow S, Castro MC, Matsuno R, Tsuru A, Jimba M. Implementation of 5S management method for lean healthcare at a health center in Senegal: a qualitative study of staff perception. Glob Health Action. 2015;8:27256.
Lavoie-Tremblay M, O’Connor P, Lavigne GL, Frechette J, Briand A, et al. The Effects of the Transforming Care at the Bedside Program on Perceived Team Effectiveness and Patient Outcomes. Health Care Manag. 2017;36(1):10–20.
MacVane PF. Chasing the golden fleece: Increasing healthcare quality, efficiency and patient satisfaction while reducing costs. International Journal of Health Governance. 2019;24(3):182–6.
Masso M, Robert G, McCarthy G, Eagar K. The Clinical Services Redesign Program in New South Wales: Perceptions of senior health managers. Aust Health Rev. 2010;34(3):352–9.
McGrath BA, Lynch J, Bonvento B, Wallace S, Poole V, Farrell A, Diaz C, Khwaja S, Robertson DW. Evaluating the quality improvement impact of the Global Tracheostomy Collaborative in four diverse NHS hospitals. BMJ Open Qual Improve Rep. 2017;6(1):bmjqir-u220636.
McLees AW, Nawaz S, Thomas C, Young A. Defining and Assessing Quality Improvement Outcomes: A Framework for Public Health. Am J Public Health. 2015;105:S167–73.
Moraros J, Lemstra M, Nwankwo C. Lean interventions in healthcare: do they actually work? A systematic literature review. Int J Qual Health Care. 2016;28(2):150–65.
Morganti KG, Lovejoy S, Haviland AM, Haas AC, Farley DO. Measuring success for health care quality improvement interventions. Med Care. 2012;50(12):1086–92.
Morrow E, Robert G, Maben J, Griffiths P. Implementing large-scale quality improvement: lessons from The Productive Ward: Releasing Time to Care. Int J Health Care Qual Assur. 2012;25(4):237–53.
Neri RA, Mason CE, Demko LA. Application of Six Sigma/CAP methodology: controlling blood-product utilization and costs. Journal of healthcare management / American College of Healthcare Executives. 2008;53(3):183–6.
Niemeijer GC, Trip A, de Jong LJ, Wendt KW, Does RJ. Impact of 5 years of lean six sigma in a University Medical Center. Qual Manag Health Care. 2012;21(4):262–8.
O'Sullivan Owen P, Chang Nynn H, Baker P, Shah A. Quality improvement at East London NHS Foundation Trust: the pathway to embedding lasting change. Governance IJoH, editor: International Journal of Health Governance; 2020.
Pearson M, Hemsley A, Blackwell R, Pegg L, Custerson L. Improving Hospital at Home for frail older people: insights from a quality improvement project to achieve change across regional health and social care sectors. BMC Health Services Res. 2017;17(387).
Power M, Brewster L, Parry G, Brotherton A, Minion J, Ozieranski P, McNicol S, Harrison A, Dixon-Woods M. Multimethod study of a large-scale programme to improve patient safety using a harm-free care approach. BMJ Open. 2016;6(9):e011886.
Robert G, Sarre S, Maben J, Griffiths P, Chable R. Exploring the sustainability of quality improvement interventions in healthcare organisations: a multiple methods study of the 10-year impact of the ‘Productive Ward: Releasing Time to Care’ programme in English acute hospitals. BMJ Qual Saf. 2020;29(1):31.
Roney JK, Whitley BE, Long JD. Implementation of a MEWS-Sepsis screening tool: Transformational outcomes of a nurse-led evidence-based practice project. Nurs Forum. 2016;55(2):144–8.
Sermersheim ER, Moon MC, Streelman M, McCullum-Smith D, Fromm J, Yohannan S, et al. Improving Patient Throughput With an Electronic Nursing Handoff Process in an Academic Medical Center A Rapid Improvement Event Approach. J Nurs Adm. 2020;50(3):174–81.
Sibthorpe B, Gardner K, Chan M, Dowden M, Sargent G, McAullay D. Impacts of continuous quality improvement in Aboriginal and Torres Strait islander primary health care in Australia: A scoping systematic review. J Health Organ Manag. 2018;32(4):545–71.
Staines A, Thor J, Robert G. Sustaining Improvement? The 20-Year Jonkoping Quality Improvement Program Revisited. Qual Manag Health Care. 2015;24(1):21–37.
Stephens TJ, Peden CJ, Pearse RM, Shaw SE, Abbott TEF, Jones EL, et al. Improving care at scale: process evaluation of a multi-component quality improvement intervention to reduce mortality after emergency abdominal surgery (EPOCH trial). Implement Sci. 2018;13(1):142.
Strauss R, Cressman A, Cheung M, Weinerman A, Waldman S, Etchells E, et al. Major reductions in unnecessary aspartate aminotransferase and blood urea nitrogen tests with a quality improvement initiative. BMJ Qual Saf. 2019;28(10):809–16.
Thursky K, Lingaratnam S, Jayarajan J, Haeusler GM, Teh B, Tew M, et al. Implementation of a whole of hospital sepsis clinical pathway in a cancer hospital: impact on sepsis management, outcomes and costs. BMJ. 2018;7(3):e000355-e.
van den Heuvel J, Does RJMM, Bogers AJJC, Berg M. Implementing Six Sigma in The Netherlands. Jt Comm J Qual Patient Saf. 2006;32(7):393–9.
Wells S, Tamir O, Gray J, Naidoo D, Bekhit M, Goldmann D. Are quality improvement collaboratives effective? A systematic review. BMJ Qual Saf. 2018;27(3):226–40.
White M, Wells JS, Butterworth T. The transition of a large-scale quality improvement initiative: a bibliometric analysis of the Productive Ward: Releasing Time to Care Programme. J Clin Nurs. 2014;23(17–18):2414–23.
Williams B, Hibberd C, Baldie D, Duncan EAS, Elders A, Maxwell M, Rattray JE, Cowrie J, Strachan H, Jones MC. Evaluation of the impact of an augmented model of The Productive Ward: Releasing Time to Care on staff and patient outcomes: a naturalistic stepped-wedge trial. BMJ Qual Safety. 2020;30(1):27–37.
Wood J, Brown B, Bartley A, Margarida Batista Custodio Cavaco A, Roberts AP, Santon K, et al. Reducing pressure ulcers across multiple care settings using a collaborative approach. BMJ. 2019;8(3):e000409.
Worrall A, Ramsay A, Gordon K, Maltby S, Beecham J, King S, Shaw I, Walshe K, Lelliot P, Worrall A. Evaluation of the Mental Health Improvement Partnerships programme. National Co-ordinating Centre for NHS Service Delivery and Organisation R&D (NCCSDO); 2008.
Yamamoto J, Abraham D, Malatestinic B. Improving insulin distribution and administration safety using lean six sigma methodologies. Hosp Pharm. 2010;45(3):212–24.
Ogrinc G, Davies L, Goodman D, Batalden P, Davidoff F, Stevens D. Standards for QUality Improvement Reporting Excellence 2.0: revised publication guidelines from a detailed consensus process. J Surg Res. 2016;200(2):676–82.
Hollingsworth B. The measurement of efficiency and productivity of health care delivery. Health Econ. 2008;17(10):1107–28.
Linna P, Pekkola S, Ukko J, Melkas H. Defining and measuring productivity in the public sector: managerial perceptions. Int J Public Sect Manag. 2010;23(5):479–99.
Palmer S, Torgerson DJ. Definitions of efficiency Bmj. 1999;318(7191):1136.
Byford S, Torgerson DJ, Raftery J. Cost of illness studies. BMJ. 2000;320(7245):1335.
Goldberg AJ, Fleming WP. Cost-containment and cost-management strategies. J Healthc Manag. 2010;55(5):308–11.
Evans RW, Pagani FD. Economic evaluation in health care: A modern day quagmire. J Heart Lung Transplant. 2014;33(4):344–6.
Bridges JFP. Future challenges for the economic evaluation of healthcare. Pharmacoeconomics. 2005;23(4):317–21.
Chisholm D, Sweeny K, Sheehan P, Rasmussen B, Smit F, Cuijpers P, et al. Scaling-up treatment of depression and anxiety: a global return on investment analysis. The Lancet Psychiatry. 2016;3(5):415–24.
Rost K, Smith JL, Dickinson M. The effect of improving primary care depression management on employee absenteeism and productivity. A randomized trial Med Care. 2004;42(12):1202–10.
Sheiner L, Malinovskaya A. Measuring productivity in healthcare: an analysis of the literature. Hutchins center on fiscal and monetary policy at Brookings. 2016.
Consortium YHE. 2016 [Available from: https://yhec.co.uk/glossary/efficiency/.
Russ-Eft D, Preskill H. In Search of the Holy Grail: Return on Investment Evaluation in Human Resource Development. Adv Dev Hum Resour. 2005;7(1):71–85.
Hjeltnes TA, Hansson B. Cost effectiveness and cost efficiency in e-learning. Norway: QUIS-Quality, Interoperability and Standards in e-learning; 2005.
Edwards RT, Charles JM, Lloyd-Williams H. Public health economics: a systematic review of guidance for the economic evaluation of public health interventions and discussion of key methodological issues. BMC Public Health. 2013;13(1):1001.
Lewis CC, Klasnja P, Powell BJ, Lyon AR, Tuzzio L, Jones S, et al. From Classification to Causality: Advancing Understanding of Mechanisms of Change in Implementation Science. Frontiers in public health. 2018;6:136-.
Cambridge Dictionary. Defition of benefit 2021 [Available from: https://dictionary.cambridge.org/dictionary/english/benefit].
O’Flynn J. From New Public Management to Public Value: Paradigmatic Change and Managerial Implications. Aust J Public Adm. 2007;66(3):353–66.
Grassl W. Toward a Unified Theory of Value: From Austrian Economics to Austrian Philosophy. Axiomathes. 2017;27(5):531–59.
Eabrasu M. A Praxeological Assessment of Subjective Value. Q J Austrian Econ. 2011;14(2):216–41.
Wieser F. The Austrian School and the Theory of Value. Econ J. 1891;1(1):108–21.
Andersson T, Liff R. Co-optation as a response to competing institutional logics: Professionals and managers in healthcare. Journal of Professions and Organization. 2018;5(2):71–87.
Bozeman B. Public-Value Failure: When Efficient Markets May Not Do. Public Adm Rev. 2002;62(2):145–61.
Millar R, Hall K. Social Return on Investment (SROI) and Performance Measurement. Public Manag Rev. 2013;15(6):923–41.
Anderson JE, Chang DC. Using electronic health records for surgical quality improvement in the era of big data. JAMA Surg. 2015;150(1):24–9.
Gao J, Chan HS, Yang K. Gaming in performance management reforms and its countermeasures: Symposium introduction. Public Perform Manag Rev. 2021;44(2):243–9.
Hem MH, Gjerberg E, Husum TL, Pedersen R. Ethical challenges when using coercion in mental healthcare: a systematic literature review. Nurs Ethics. 2018;25(1):92–110.
Gelman SA, Kalish CW. Conceptual development. Handbook Child Psychol. 2007;1(2).
Parkinson S, Eatough V, Holmes J, Stapley E, Midgley N. Framework analysis: a worked example of a study exploring young people’s experiences of depression. Qual Res Psychol. 2016;13(2):109–29.
Moffatt S, White M, Mackintosh J, Howel D. Using quantitative and qualitative data in health services research–what happens when mixed method findings conflict?[ISRCTN61522618]. BMC Health Serv Res. 2006;6(1):1–10.
The Healthcare Financial Management Association, NHS England, and NHS Improvement (NHSE&I). Bristol: NHS value and efficiency map; 2022.
The authors would like to acknowledge Professor Judith E. Hupcey for her advice on concept development. The authors would also like to thank Mr Andy Healy for his advice on economic evaluation.
This work is supported by the Economic and Social Research Council, grant number ES/P000703/1. TS is supported by the Welcome Trust (219425/Z/19/Z) and Diabetes UK (19/0006055).
Ethics approval and consent to participate
Consent for publication
TS received funding from Cancer Alliance and Health Education England for training cancer multidisciplinary teams (MDTs) in assessment and quality improvement methods in the United Kingdom. TS received consultancy fees from Roche Diagnostics. The other authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The original version of this article was revised: errors in the Funding section and the first author's name were corrected.
. Example search strategy. Supplemental Table 2. Data extraction tool. Supplementary Table 3. Included studies. Supplementary Table 4. Summary of Quality assessment.
About this article
Cite this article
Thusini, S., Milenova, M., Nahabedian, N. et al. The development of the concept of return-on-investment from large-scale quality improvement programmes in healthcare: an integrative systematic literature review. BMC Health Serv Res 22, 1492 (2022). https://doi.org/10.1186/s12913-022-08832-3