Skip to main content

Sustainability in Health care by Allocating Resources Effectively (SHARE) 10: operationalising disinvestment in a conceptual framework for resource allocation



This is the tenth in a series of papers reporting a program of Sustainability in Health care by Allocating Resources Effectively (SHARE) in a local healthcare setting. After more than a decade of research, there is little published evidence of active and successful disinvestment. The paucity of frameworks, methods and tools is reported to be a factor in the lack of success. However there are clear and consistent messages in the literature that can be used to inform development of a framework for operationalising disinvestment. This paper, along with the conceptual review of disinvestment in Paper 9 of this series, aims to integrate the findings of the SHARE Program with the existing disinvestment literature to address the lack of information regarding systematic organisation-wide approaches to disinvestment at the local health service level.


A framework for disinvestment in a local healthcare setting is proposed. Definitions for essential terms and key concepts underpinning the framework have been made explicit to address the lack of consistent terminology. Given the negative connotations of the word ‘disinvestment’ and the problems inherent in considering disinvestment in isolation, the basis for the proposed framework is ‘resource allocation’ to address the spectrum of decision-making from investment to disinvestment. The focus is positive: optimising healthcare, improving health outcomes, using resources effectively.

The framework is based on three components: a program for decision-making, projects to implement decisions and evaluate outcomes, and research to understand and improve the program and project activities. The program consists of principles for decision-making and settings that provide opportunities to introduce systematic prompts and triggers to initiate disinvestment. The projects follow the steps in the disinvestment process. Potential methods and tools are presented, however the framework does not stipulate project design or conduct; allowing application of any theories, methods or tools at each step. Barriers are discussed and examples illustrating constituent elements are provided.


The framework can be employed at network, institutional, departmental, ward or committee level. It is proposed as an organisation-wide application, embedded within existing systems and processes, which can be responsive to needs and priorities at the level of implementation. It can be used in policy, management or clinical contexts.

Peer Review reports

About share

This is the tenth in a series of papers reporting Sustainability in Health care by Allocating Resources Effectively (SHARE). The SHARE Program is an investigation of concepts, opportunities, methods and implications for evidence-based investment and disinvestment in health technologies and clinical practices in a local healthcare setting. The papers in this series are targeted at clinicians, managers, policy makers, health service researchers and implementation scientists working in this context. This paper proposes a framework for operationalising disinvestment in the context of resource allocation in the local healthcare setting.


Although there is no clear single definition, disinvestment is generally understood to be removal, reduction or restriction of technologies and clinical practices (TCPs) that are unsafe or of little benefit, in order to improve patient outcomes and use available resources more efficiently [1]. Three main areas of opportunity for disinvestment have been identified: 1) TCPs in current use that were not evaluated rigorously prior to their introduction and have subsequently been identified as harmful, ineffective or not cost-effective for all patients or certain subgroups, 2) existing TCPs that are safe, effective and cost-effective but which have alternatives offering greater benefit, and 3) TCPs that are overused or misused [1].

Following successful implementation of a systematic, integrated, transparent, evidence-based program to assess new TCPs prior to their introduction within the health service [2], Monash Health, a large health service network in Melbourne Australia, sought to develop a similar program for disinvestment. The ‘Sustainability in Health care by Allocating Resources Effectively’ (SHARE) Program was established to investigate this. An overview of the program and a guide to the SHARE publications are provided in the first paper in this series [3] and a summary of the findings are in the final paper [4].

It is common for healthcare networks and individual facilities to make decisions within organisation-wide frameworks; for example introduction of new TCPs and models of care, delivery of programs and services, development and authorisation of policies and procedures, capital expenditure and clinical purchasing. Although disinvestment can be considered in all these contexts, it is frequently reported in individual standalone projects, isolated from other decision-making settings. Monash Health chose to explore disinvestment in the context of organisation-wide systems and processes for all resource allocation decisions.

There was little published information available to guide development of a systematic organisation-wide local approach to disinvestment at Monash Health. In the absence of guidance from the literature, a two-phased process was proposed to identify and then evaluate potential opportunities for disinvestment (Fig. 1). The aim of Phase One was to understand concepts and practices related to disinvestment and the implications for a local health service and, based on this information, to identify potential settings and methods for decision-making. The aim of Phase Two was to develop, implement and evaluate the proposed settings and methods to determine which were sustainable, effective and appropriate at Monash Health.

Fig. 1
figure 1

Overview of SHARE Program

The outcomes of Phase One provide information regarding decision-making settings, decision-makers, scope and type of decisions, strengths and weaknesses in current processes, barriers and enablers, and criteria used for allocating resources within a local health service which, to our knowledge, has not previously been documented to this level of detail in this context [5,6,7,8]. While the program had limited success in achieving the aims of Phase Two, the investigation provides in-depth insight into the experience of disinvestment in one local health service and reports the process of disinvestment from identification, through prioritisation and decision-making, to implementation and evaluation, and finally explication of the processes and outcomes [9,10,11]. These detailed findings enabled development of several frameworks and models for a range of purposes related to disinvestment and resource allocation in the local healthcare setting.

At the completion of these activities, a third phase was undertaken to review the current literature from the perspective of a local health service, and combine it with the published findings from the SHARE Program to address some of the gaps in information about disinvestment in this setting. This review focuses on the practical and operational aspects of disinvestment at the local level. It is a companion to the ninth paper of the SHARE series which provides a conceptual description; disinvestment is introduced and discussed in relation to terminology and concepts, motivation and purpose, relationships with other health improvement paradigms, challenges, and implications for policy, practice and research [1]. The methods of the literature review are included in Paper 9 and the contents of both reviews are summarised in Table 1.

Table 1 Contents of the literature reviews

Although research and debate has broadened considerably over the past decade, there remains a lack of information to guide healthcare networks or individual facilities in how they might take a systematic, integrated, organisation-wide approach to disinvestment in the context of all resource allocation decisions [1]. Despite the paucity of evidence in this context, there are clear and consistent messages regarding principles for decision-making, settings and opportunities to identify disinvestment targets, steps in the disinvestment process, barriers and enablers to successful implementation, and some frameworks and models for elements of the disinvestment process. This practical information can be used to develop an organisation-wide framework for operationalising disinvestment in the local healthcare setting.


The aims of this paper are to discuss the current literature on disinvestment from an operational perspective, combine it with the experiences of the SHARE Program, and propose a framework for disinvestment in the context of resource allocation in the local healthcare setting.

Existing theories, frameworks and models


Theories are based on concepts or ideas that characterise a particular phenomenon and propositions or relationships that link the concepts [12]. No specific theories of disinvestment have been proposed, however resource allocation theory, prioritisation theories, and decision-making theories have been applied in disinvestment projects; examples are listed in Table 2 [13,14,15,16,17,18].

Table 2 Examples of theories proposed or applied in disinvestment-related projects

Perhaps the most relevant to disinvestment is the theory of discontinuance, defined by Rogers in his discussion of the theory of diffusion as “a decision to reject an innovation after having previously adopted it” [19]. In their review of diffusion of innovations in health care, Greenhalgh et al. note the importance of research into discontinuance and the lack of studies in this area [20]. Hollingworth et al. propose a schema of health technology adoption and withdrawal which includes both discontinuance and disinvestment [21] and Niven et al. use the definition of discontinuance for the term ‘de-adoption’ in their review of low-value clinical practices [22].


Frameworks use concepts and relationships to provide a frame of reference, organise and focus thinking and assist interpretation. Frameworks are descriptive, tend to be high-level and can apply to a wide variety of situations [12, 23]. No frameworks for systematic, integrated, organisation-wide approaches to disinvestment were identified, however there are several frameworks for specific aspects of the disinvestment process. These are summarised by setting, aims, method of development and components in Table 3. Those applicable to the local healthcare setting are discussed in more detail under the relevant steps in the disinvestment process below.

Table 3 Examples of frameworks and models related to disinvestment

Polisena and colleagues [24] identified three frameworks in their review of disinvestment projects: Health Technology Assessment (HTA) [25], Accountability for Reasonableness (A4R) [26] and Program Budgeting and Marginal Analysis (PBMA) [27]. To distinguish between evaluation of new TCPs and those in current practice, the term Health Technology Reassessment (HTR) has been introduced for methods aiming to identify potential targets for disinvestment [28, 29]. HTA and A4R are frameworks by definition and are valuable tools for decision-making; however, although their use may lead to disinvestment, they are not frameworks specifically for disinvestment. Like A4R and HTA, PBMA and other priority setting frameworks [30,31,32] can play a key role in certain approaches to disinvestment, but do not address all potential aspects of the disinvestment process or all opportunities to drive change. However they would all integrate readily into a wider framework for disinvestment, as aspired to with the trialing of the Australian Medicare Benefits Schedule Review initiative [33]. Recently Elshaug et al. provided a comprehensive inventory of disinvestment policy and practice levers that could flow from HTA/HTR and other priority setting processes [34].

Sources of synthesised evidence such as HTAs, systematic reviews and evidence-based guidelines, can underpin disinvestment decisions in two ways. Firstly, the process of evidence synthesis can be undertaken reactively to address policy, management or clinical questions as they arise and inform the resultant decisions. Secondly, dissemination of the findings of published HTAs, systematic reviews or guidelines can be a proactive method of initiating decision-making to ensure policy and practice is consistent with the best available evidence.

The ‘Disinvestment framework to guide resource allocation decisions in health service delivery’ [16] and the ‘Guideline for Not Funding Health Technologies’ (GuNFT) [35] are examples of frameworks to identify and disinvest individual TCPs. They are very similar to the process outlined in the Workflow Diagram of the New Zealand National Health Committee for introduction of new and removal of old technologies [36]. All three are systematic, transparent and based on a series of steps to identify suitable TCPs, engage relevant stakeholders, make the appropriate decisions, implement and evaluate change.

The New Zealand National Health Committee also includes a framework for wider stakeholder engagement in their Business Plan [36].

Joshi and colleagues use both framework and model when referring to the outcome of their narrative review ‘Reassessment of Health Technologies: Obsolescence and Waste’ [37]. Based on the definitions used herein, it is classified as a framework. It includes the role of reassessment in the life cycle of a health technology and triggers, structures and outcomes for health technology reassessment and decommissioning.

Bhatia et al. present an ‘Integrative framework for measuring overuse’ as an evaluation tool to be implemented within initiatives that aim to reduce ‘low value care’ [38] and Barasa and colleagues propose a framework for evaluation of priority setting processes which considers both procedure aspects and outcomes in a range of contexts [39].

Conceptual frameworks developed in the SHARE Program for a range of purposes within the disinvestment process include potential settings and methods to integrate disinvestment into health service systems and processes [5], components in the resource allocation process [6], an evaluation framework and plan for the overall SHARE program [40] and an algorithm to facilitate decision-making for selecting projects from an evidence-based catalogue of potential opportunities for disinvestment [9]. An existing framework for evaluation and explication of implementation of an evidence-based innovation was adapted for use in disinvestment projects [9] and health information products and services [11].


Models are more precise and more prescriptive than frameworks. They are narrower in scope, the concepts are well defined and the relationships between them are specific. Models are representations of the real thing [12, 23].

The SHARE Program produced three models: integrating consumer values and preferences into decision-making for resource allocation in a local healthcare setting [7], exploring Sustainability in Health care by Allocating Resources Effectively in this context [8] and facilitating use of recently published synthesised evidence in organisational decision-making through an Evidence Dissemination Service [11]. These are summarised in Table 3. No other models for disinvestment were identified in the literature.

New Framework

Information pertaining to the practical and operational aspects of disinvestment in the local healthcare setting is presented and discussed in the context of a new framework (Fig. 2). The framework proposes a systematic approach that is integrated within organisational infrastructure. It brings together the definitions, concepts, principles, decision-making settings, potential prompts and triggers to consider disinvestment, and steps in the disinvestment process identified from the literature. It also seeks to remove barriers when it is possible to do so through establishment of new or adjustment of existing operational mechanisms. The details of each of the framework components are clearly articulated in the literature; many are derived from extensive work with stakeholder groups including decision-makers, policy-makers, health service staff, patients and members of the public.

Fig. 2
figure 2

Framework for an organisation-wide approach to disinvestment in the local healthcare setting

The proposed framework builds on the work of others. While incorporating all the messages from the literature, it draws heavily on the three noted frameworks which identify steps in the disinvestment process [16, 35, 36]; the SHARE frameworks and models [5,6,7,8,9]; and other frameworks for introduction of new TCPs [2] and evidence-based change [41].


The framework is aimed at health service decision-makers considering disinvestment and resource allocation, and health service researchers and implementation scientists working in this context.

The setting for this initiative was Monash Health, a large health service network in Melbourne Australia operating within a state-allocated fixed-budget model of financing. We anticipate results of this work and elements of the framework to have broader applicability and transferability, including to fee-for-service environments.


Decision-making in healthcare is described at three levels: macro (national, state/provincial and regional), meso (institutional) and micro (individuals) [42, 43].

The proposed framework was developed for use in policy, management and/or clinical decision-making at the meso level. It was designed to be embedded within existing systems and processes where it can be responsive to local needs and priorities at the level of implementation; for example health service networks, individual facilities, departments, wards or committees.


The lack of standardised terminology is a barrier to development of systematic approaches to operationalise disinvestment [1]. To address this, definitions and key concepts underpinning the framework are made explicit. The proposed framework provides a common language for researchers and decision-makers within and between programs, institutions and health systems making it easier to build and share a body of knowledge.

There are multiple definitions for disinvestment in the literature based on a range of different concepts [1, 44]. Numerous alternative terms conveying the same concepts are also in common use. Disinvestment is focused on the use of ‘health technologies’ but there is also a range of definitions for this term. To compound the difficulties arising from multiple definitions, the terms ‘disinvestment’ and ‘health technologies’ are frequently used in one way by researchers and in another by health service decision-makers [1]. Definitions relevant to the local healthcare setting are provided in Table 4.

Table 4 Definitions

We use the term disinvestment in the broadest sense, ‘removal, reduction or restriction of any aspect of the health system for any reason’. This can be applied to products, devices and equipment; clinical practices and procedures; health services and programs; information technology and corporate systems. Unlike most of the research definitions for disinvestment, this version is not constrained by a specified purpose (eg withdrawing practices of low value), defined criteria (eg effectiveness or cost-effectiveness) or anticipated outcome (eg reallocation of resources) which do not address cessation or limitation of TCPs for other purposes, based on other criteria, for different outcomes, which are likely to arise in local health services [1].

In contrast, we define health technologies in the narrowest sense; as products, devices and equipment used to deliver health care (eg prostheses, implantable devices, vaccines, pharmaceuticals, surgical instruments, telehealth, interactive IT and diagnostic tools) which reflects common use by health service decision-makers and consumers [1]. Clinical practices, health programs and services, information technologies, support systems, and organisational and managerial systems are not included in this definition. Although contained in many research definitions, they are not included in general references to health technologies in the local healthcare setting [1].

The terms ‘principles’ and ‘criteria’ are often used interchangeably; definitions for use in this review are included in Table 4.


The proposed framework is underpinned by several key concepts (Table 5). While disinvestment is the aim, it is not considered in isolation but in the context of resource allocation, addressing the spectrum of decision-making covering investment in new, continuation of existing, and disinvestment from current activities. The focus of the framework is positive: optimising healthcare, improving health outcomes, using resources effectively and efficiently. The components of the framework are integrated within current systems and processes and within existing health improvement paradigms such as evidence-based practice (EBP), quality improvement (QI) and system redesign.

Table 5 Concepts

Level of detail

Many of the elements within the proposed framework should be self-evident and be applied routinely as good practice, making it unnecessary to stipulate their requirement. However strong and consistent messages in the literature confirm that they are not standard practice and authors felt the need to state that they should be made explicit. Incorporating them all into a detailed framework achieves this.

Another reason for including all the elements in detail is to address potential ethical dilemmas [1]. In some circumstances it may be difficult to accommodate the principles of beneficence and utilitarian justice; clinicians advocate for the best interests of individual patients but resource allocation aims for the greatest benefit for the most people [45,46,47]. Similarly, arguments for equity may conflict with those for efficiency when the most efficient outcome is not the most equitable [48,49,50]. A systematic, transparent approach acknowledging these issues may facilitate difficult discussions and create potential for some efficiency to be traded away for equity maintenance or gain.

Some elements may be more important than others in individual situations. However, because they are all defined in the framework, the decision to exclude or reduce the role of some elements in extenuating circumstances becomes explicit. This strengthens the process and empowers those who have previously participated in suboptimal decision-making due to lack of resources, hidden agendas or organisational politics [6, 51,52,53,54,55,56,57].


The proposed framework is composed of three interconnected and interdependent components: 1) a program for organisation-wide decision-making, 2) projects to implement decisions and evaluate outcomes, and 3) research to understand and improve the program and project activities. Each component has a number of elements which are outlined in detail below.


The framework is primarily descriptive to enable application in a local healthcare service and allow adaptation, replication and testing. It was developed using both deductive and inductive methods. Although not based on a specific theory, it has potential to facilitate future theory development and/or testing. Specific characteristics of the framework and potential for its use are summarised in Table 6 using domains and criteria developed to assess the robustness and utility of proposed models and frameworks [12]. This assessment enables potential users to identify whether the framework will meet their aims and be applicable to their situation.

Table 6 Characteristics of a framework for organisation-wide approach to disinvestment in the local setting


Principles for decision-making

Forty-two principles were identified from the existing literature and the SHARE publications and grouped into eight categories that emerged from these findings: Boundaries, Ethics, Governance, Structures, Processes, Stakeholder involvement, Resources and Preconditions. These are presented in the framework as two groups (Fig. 2).

The first group have a hierarchical relationship depicted as a series of nested boxes. The whole program is defined by explicit boundaries, ethical principles underpin good governance, governance directs and controls structure, and structure enables and accommodates process. The decision-making settings, prompts and triggers all sit within the scaffold of these five categories.

The second group, represented as three vertical bars, are required across all of the other elements. For example, stakeholders need to be involved in defining the boundaries and establishing the ethical parameters and methods of governance; they should be included in the structures and processes and participate in the projects and research. Adequate and appropriate resources and the noted preconditions will be required to establish, maintain and improve all aspects of the framework.

The intersection of the two groups of principles also demonstrates that ethics, governance, structures and processes also apply to stakeholder engagement, resources and preconditions. For example, stakeholder engagement should be systematic and integrated, funding should be sourced ethically and influence should be transparent.

These principles and their relationships also apply to the project and research components.

Further details of the categories, full descriptions of individual principles, and related citations are outlined in Additional file 1.


Nine settings for decision-making are described in three categories: Decision-making infrastructure, Specific initiatives and Individual decision-makers.

While the framework is proposed for organisation-wide application, any of the nine settings could be considered individually. A framework for a single setting would be underpinned by the same principles, decisions would lead to projects with the same steps and research could be conducted on all elements.

Decision-making infrastructure

Each sector of the health system has an organisational infrastructure of decision-making settings where committees, designated panels or individuals with delegated authority make decisions on behalf of the jurisdiction or individual facility. A classification system and descriptors for decision-making settings, decision-makers, scope and type of decisions in the local health service setting was developed in the SHARE Program [6].

Decisions can be categorised as routine, reactive and proactive [6, 58]. Routine decisions are made on a regular basis; reactive decisions are made in response to situations as they arise; and proactive decisions are driven by information that was actively sought for the purpose of healthcare improvement. Examples are included in Table 4.

A range of potential decision-making activities are outlined in Table 7 [1, 5, 6, 8, 59,60,61]. Most of these occur in more than one of the three categories of decision-making and can be used for more than one aspect of the disinvestment process. Development or revision of guidance documents is a good example. Guideline and protocol development can occur routinely, particularly when existing documents are updated at regular intervals; in reactive situations such as a critical incident which highlights lack of guidance in a specific area; or when proactive use of research identifies that current documents do not reflect the best available evidence. Disinvestment opportunities can be identified if the systematic review process undertaken when initiating or revising a guidance document determines that a TCP, service or program should be removed or replaced [5, 17, 60,61,62,63]. Guidance documents can also be used to implement disinvestment decisions and audit of guideline adherence can measure the results [59, 60, 64,65,66]. Manuals for guideline or protocol production could include prompts to note and follow up opportunities for disinvestment as part of the document development process [5].

Table 7 Examples of activities and settings for disinvestment within decision-making infrastructure

Formal priority setting exercises may also be built into the decision-making infrastructure. These determine which TCPs, programs or services to introduce, maintain or remove based on a pre-determined set of criteria. An example might be annual capital expenditure decisions. In this situation, priority setting could be classified as ‘routine’, however it is noted separately in the framework as it also commonly arises in the context of individual initiatives described below.

Specific initiatives

In addition to the decision-making settings outlined, specific initiatives to improve practice are undertaken by health services, many of which involve disinvestment. These may be instigated by government, management or health practitioners, and although there is considerable diversity, most are related to EBP, QI, system redesign or economic approaches to priority setting such as PBMA [1, 6, 34]. Some projects may set out to disinvest, others may have quite different initial aims but the need for disinvestment becomes apparent during the project.

An EBP approach might be to remove or reduce use of inferior practices identified from systematic reviews, HTAs, evidence-based guidelines or ‘low value’ lists, or reduce their use to levels deemed clinically appropriate [9]. Clinical audit, QI and system redesign methods may be used to tackle inappropriate use of TCPs or organisational waste. Priority setting exercises like PBMA consider the costs and benefits of relevant alternatives in an aspect of healthcare delivery to determine the maximum outcome from the available resources.

There are several examples of disinvestment-related initiatives with relevance at the local health service level. Therapeutic equivalence or drug substitution programs involving replacement of expensive drugs with equally effective but lower cost alternatives from the same drug family has demonstrated considerable cost saving in macro and meso programs [67, 68]. Generic prescribing, substituting brand name drugs with generic alternatives, has been addressed at international, national, institutional and individual levels with mixed outcomes [69,70,71,72]. Benchmarking the results from individual interventions or programs across different health providers aims to ascertain best practice which others can aspire to and which can be applied at all levels; but by simultaneously identifying inferior practices it can also be used as “a tool to start a disinvestment dialogue” [21, 73, 74].

Individual decision-makers

At the micro level, the term ‘disinvestment’ is not generally applied to changes initiated by individuals; however the principle is the same. Individuals cease or restrict practices when they become aware of new evidence or to address local needs and priorities.

Much of the literature on decision-making focuses on how money is spent, however there are considerable opportunities for disinvestment in allocation of non-monetary resources. Although clinical encounters do not usually involve funding decisions, they offer opportunities to consider disinvestment in use of other resources such as ordering tests, referring to other practitioners, using drugs and other therapies, or undertaking procedures. An example is the Choosing Wisely program being replicated in national campaigns across the world which highlights potentially ‘low value’ treatments and tests so that clinicians and consumers can consider the relative benefits in their specific situations [75].

Prompts and triggers

Prompts and triggers are proposed to initiate and facilitate identification of disinvestment opportunities. Prompts are informal reminders or encouragement for thought or action and triggers are formal mechanisms that initiate or activate a reaction, process or chain of events (Table 4). The settings above provide opportunities to introduce systematic prompts and triggers to use evidence from research, data and stakeholder feedback to drive decision-making.

Prompts, triggers and potentially even mandatory requirements to consider disinvestment could be built into existing decision-making infrastructure [5, 37]. Using expenditure decisions as an example, prompts and triggers could be incorporated into meeting agendas of finance committees, budgeting processes, application forms, algorithms, protocols or checklists. Mandatory requirements to consider disinvestment could be implemented as specific directions within purchase orders, explicit decision-making criteria for committees, or steps in application processes that require authorisation. Additional examples of prompts and triggers at the organisational level are outlined in Table 8.

In specific initiatives to implement health service improvements, prompts and triggers to consider disinvestment could be introduced into project management templates or training programs for project management, change management, quality improvement processes, etc.

Prompts, triggers and mandatory requirements could also be used to guide the decisions of individual practitioners in clinical encounters; these could be included in local guidelines and protocols to steer practice away from unsafe, ineffective or inefficient use of TCPs.

Table 8 Examples of systematic prompts and triggers to initiate disinvestment decisions

Steps in the disinvestment process

The disinvestment process begins when opportunities for disinvestment are identified from the activities in the settings above. Eight steps in the disinvestment process were ascertained from existing frameworks [6, 16, 35, 36]: Identification of opportunities; Prioritisation (if required) and Decision-making; Development of a proposal; Implementation; Monitoring, Evaluation and Reporting; Reinvestment (if required); Dissemination and Diffusion; and Maintenance. Two additional elements are included: some projects may require development of local criteria for prioritisation and decision-making and projects that aim to reinvest will need to measure the resources released as part of the evaluation process.

The first two steps are part of the decision-making program, the following six are undertaken in projects arising from the decisions.


Once a decision has been made, a project to implement it can be initiated. While individual projects will have specific characteristics and requirements such as aims, objectives, timelines, budgets, deliverables, roles and responsibilities, the principles outlined in the framework apply to all project activities.

Examples of methods and tools for disinvestment are discussed below; however the proposed framework does not stipulate project design or conduct, allowing application of any theories, methods or tools at each step.


Research is required to understand and improve the program and project activities. It is overlaid across all elements in the diagram to represent the potential for research in each aspect of the framework.

Methods and tools

There are many definitions for the terms theory, framework, model, method, tool, strategy and related concepts. Some definitions note specific features that make the terms mutually exclusive, others allow the terms to be used interchangeably, and some overlap. In this review, the label ‘methods and tools’ is used pragmatically to assist health service staff in disinvestment and includes approaches, instruments or other resources that identify ‘what’ tasks are needed at each step and/or ‘how’ to undertake them. This broad definition allows frameworks and models to be included if they meet these criteria.

Appropriate, valid and reliable methods and tools are required for effective decision-making, implementation and evaluation. The resources identified are described briefly but no evaluation was undertaken due to lack of relevant data; some have been piloted and refined, but most have no published reports of their effectiveness or impact. The availability of validated materials is noted where appropriate. Hence users will need to consider the validity and applicability of these resources in their individual contexts.

There are many sources of generic advice for ascertaining and utilising evidence, undertaking and applying health economic analyses, making decisions, implementing change and evaluating outcomes including, but not limited to, The Cochrane Library, Canadian National Coordinating Centre for Methods and Tools, UK National Institute for Health and Care Excellence (NICE), US Institute for Healthcare Improvement, US Centers for Disease Control and Prevention, and US Agency for Healthcare Research and Quality.

There are also many methods and tools from other areas of health research and practice that are relevant to disinvestment which could be employed within this framework; knowledge translation, EBP, QI, system redesign and other improvement methodologies all have well-developed validated processes that are familiar to health service staff [1]. While there are few published examples of successful initiatives labelled as ‘disinvestment’ within local health services, there are many examples in the EBP and quality and safety literature of disinvestment-type activities where TCPs that are unsafe or ineffective have been discontinued. A review of ‘de-adoption’ summarises 39 such interventions that provide information on several steps in the disinvestment process [22].

Two publications provide advice in a range of areas relevant to disinvestment. A book on rationing, priority setting and resource allocation in health care discusses multiple generic and specific methods and tools suitable for disinvestment including stakeholder participation, leadership, economic evaluation and several of the steps in the disinvestment process [76]. A toolkit for decommissioning and disinvestment, defined as withdrawal of funding from the provider organisation, provides high-level guidance on governance and administrative matters for removal of health services, not individual TCPs, and some tools for assessing service performance against UK data [77].

The GuNFT guideline provides guidance on establishment of a decision-making program and recommendations, templates and other tools for several steps in the disinvestment process [78]

Several products from the SHARE Program also address a range of principles and steps in the disinvestment process.

  • Summaries of issues to consider in development of an organisational program for disinvestment [5] and implications for disinvestment in the local setting [8] were compiled.

  • An investigation of the resource allocation process in a local health service generated a framework of eight components, the relationships between them, and features of structure and practice for each component [6]. Structure is described as ‘who’ and ‘what’ and includes people, systems, policies, requirements, relationships and coordination. Practice addresses ‘how’ through processes, procedures, rules, methods, criteria and customs.

  • A classification of decision-making settings, decision-makers, and scope and type of decisions was developed and strengths, weaknesses, barriers and enablers to resource allocation in a local health service were ascertained [6].

  • A model for exploring Sustainability in Health care by Allocating Resources Effectively (SHARE) in the local healthcare setting brings together systems and processes for decision-making; identifying and undertaking disinvestment projects; support services to facilitate making, implementing and evaluating decisions; evaluation and research to measure and understand the processes and outcomes of these disinvestment-related activities; and principles and preconditions for success and sustainability [8].

Methods and tools for the principles are presented in Additional file 1.

1. Identification of opportunities

Potential disinvestment opportunities can be derived from all of the decision-making settings discussed above, either incidentally or systematically from prompts or triggers embedded in local systems and processes. However, at the health service level, it is more common for disinvestment opportunities to be identified through ad hoc proposals based on individual’s observations or local knowledge than through a systematic evidence-based approach [9, 21, 79, 80].

The sources of information noted in the literature that could be used in these settings to identify disinvestment opportunities include research, health service data, expert opinion and stakeholder consultation. While any one of these sources could identify a potential target for disinvestment, ideally information from all four would be combined in confirming the appropriateness of the choice [5]. Evidence from research would be considered in light of local data. For example, if a systematic review or HTA identified a more cost-effective intervention to one in current use, decision-makers could use local data to assess whether the burden of disease, volume of use, likely impact and potential cost of change warrant the required disinvestment activities. Similarly, evidence from local data would be enhanced by using the literature to identify best practice. For example, if an audit of prescribing rates of a high cost drug finds variation between departments, a review of the appropriate research would confirm whether the higher rate is overuse and should be reduced or the lower rate is underuse and should be increased. Expert opinion and stakeholder consultation add clarification and important perspectives to these decisions and may also reveal examples of inappropriate use of TCPs not identified by other methods. The SHARE Program used the SEAchange model [41], a formal evidence-based approach to change, to ensure that evidence from research and local data, experience and expertise of health service staff, and values and perspectives of consumers were considered at each step (Fig. 1) [3].

1.1 Research

Reactive decisions can be informed by synthesised evidence and relevant primary studies; the type of research design and level of evidence required depends on the context of the decision and the nature of the question being addressed. Rigorous evaluation of new TCPs prior to inclusion in nationally funded health schemes has been standard practice for the past two decades and high quality HTAs, systematic reviews, evidence based guidelines and clinical effectiveness research reports have been developed to determine other national health policies. There is also a long history of locally-developed HTAs for use in decisions about introduction of new TCPs at health service level [2, 81]. Health technology reassessment of existing TCPs with view to identifying potential targets for disinvestment has been undertaken at both national and local level [28, 29, 82, 83].

Systematic use of research in routine decisions is evident in reassessment of new TCPs at specified time periods after their introduction at national [72, 84] and local level [2]. At the other end of the TCP lifespan, “obsolescence forecasting” has also been proposed as a systematic approach to initiate HTR when it is anticipated that “a new, more functional product or technology supersedes the old or when the cost of maintenance or repair of old technology outpaces the benefits of a new piece of technology” [37].

Examples of proactive use of research for disinvestment at national level include a review of all listed drugs conducted in France resulting in removal of 525 drugs considered to have “insufficient medical value” [72] and commissioning of a complete review of the Australian Medicare Benefits Schedule (fee-for-service) to ensure that all funded items are safe, effective and cost-effective [33]. There are other examples of systematic and ad hoc use of research to drive disinvestment at national level [60, 72, 85].

Similar approaches have been used at local level where organisations have reassessed all of the TCPs related to a specific clinical issue or area, or reassessed one particular TCP at a time [83]. The SHARE Program implemented an Evidence Dissemination Service to proactively retrieve, appraise, summarise and categorise synthesised evidence from high-quality sources soon after publication and deliver it directly to the relevant designated groups and individuals responsible for organisational decision-making related to resource allocation [11]. The SHARE Program also proposed a framework for consumer involvement that included proactive use of sources of published consumer evidence [7].

Lessons from these national and local examples may be useful to those undertaking local disinvestment initiatives.

High quality sources of research evidence are available and readily accessible through online resources, however there are some challenges to their use in the local health service setting.

Health service staff report lack of time, knowledge, skills and resources as barriers to searching for, accessing and appraising research; and that evidence is not used systematically or proactively to inform decisions [6, 10, 86,87,88,89,90,91,92,93,94,95,96]. Reports of HTAs undertaken by local health services [81, 97] and decision-making for use of TCPs [2, 98,99,100] note limitations in local processes, resources and expertise resulting in decision-making with varying degrees of rigour, structure and transparency. In addition to expertise, training and support, systematic prompts and triggers to use research evidence in all three types of decision-making are needed at the local level and could also be used to identify relevant TCPs for disinvestment or initiate discussions on potential disinvestment topics.

There are also limitations in coverage and applicability of currently available synthesised evidence to address all the needs of local decision-makers. The topics reviewed by national agencies are most frequently medical interventions, pharmaceuticals and diagnostic tests that have a high profile and are expensive as individual items. While these are obviously important in local health services, lower profile areas such as nursing and allied health practices, service delivery options, models of care and clinical consumable items, all of which have potential for considerable improvement in patient outcomes and reduction in costs and resource utilisation, are less commonly addressed in these formats, leading to locally-conducted HTA/HTR with the shortcomings noted above.

These limitations have additional implications for local health services given the lack of standardised methods for HTR [37, 82, 83]. Further research in this area has been proposed to develop consistent methods which will increase rigour, enable replication, enable comparison with others, facilitate application in equivalent situations to reduce duplication, engender familiarity and understanding to increase uptake and use of content, and build on existing work [28, 29, 83].

1.2 Health service data

Routine, reactive or proactive investigation of available data can identify potential opportunities for disinvestment. There are many generic tools like dashboards, statistical process control or balanced scorecards available to analyse health service data, however none were identified in this review of the disinvestment literature. These tools, plus simple data interrogation methods, can identify factors associated with TCPs that might be worthy of further exploration as candidates for disinvestment; for example high volume, high cost, long length of stay and high rates of mortality, adverse events, readmission or reoperation, and geographic variation [5].

Searching routinely-collected datasets for known ‘low value’ practices is a direct and potentially productive method of identifying disinvestment opportunities [38, 101, 102]. With initiatives such as Choosing Wisely proliferating, it is now less a case of list-making as list-taking and prioritising. An algorithm developed in the SHARE Program for selection and prioritisation of disinvestment projects from a catalogue of potential targets derived from the research literature using locally-developed criteria could be adapted for use with a collection of potential targets identified from investigation of local data [9].

There is a large body of literature on examination of practice variation [103]. Reporting on variations in healthcare practice has been done at national and regional levels and atlases of healthcare variation have been produced [104,105,106,107,108]. Similar processes could be undertaken at local level. Comparisons can be made between regions, facilities, departments and individual practitioners, or over time; but should only be done when the population demographics, socio-economic factors and particularly patient acuity are similar [5, 21, 73, 105, 109, 110].

Recent studies have investigated practice variation specifically to identify ineffective practices; they note the potential to do so within local health services or for health services to benchmark against their counterparts [21, 105, 110]. Examination of health service utilisation and patient outcomes data, as well as differences in rates of prescribing, ordering diagnostic tests or use of specific interventions, could indicate inappropriate or suboptimal practices suitable for disinvestment. Procedures with high variability are often not on the ‘low value’ lists, indicating additional possibilities to identify disinvestment opportunities from this approach [21].

Use of local data clearly has potential but problems with data validity, reliability, comprehensiveness and degree of sensitivity to disinvestment requirements remain significant barriers [21, 24, 46, 48, 58, 60].

There are many methods for analysis, synthesis and interpretation of data however, like research evidence, there is a lack of systematic prompts or triggers to use them [5, 21]. While not specifically directed at disinvestment or resource allocation, a conceptual framework and logic model developed by Nutley and colleagues for improving data use in health system decision-making could facilitate a more proactive, systematic approach [111, 112].

The aims of the SHARE Data Service were 1) to interrogate routinely-collected data to identify potential disinvestment opportunities and communicate this information to appropriate decision-makers and 2) to respond to requests from decision-makers to assess local data related to potential disinvestment opportunities that had been identified from the research literature [10]. Although the Data Service was not implemented due to unanticipated local factors, the decisions underpinning the design and the models proposed may be helpful to local health services wishing to establish similar resources to support use of data in the disinvestment process.

1.3 Stakeholder nominations

Stakeholder engagement is noted as a fundamental principle of the decision-making process and involvement of stakeholders and local ownership of decisions and projects are noted as facilitators of change in general [113,114,115] and in relation to disinvestment [21, 58, 72, 82].

The Ontario Reassessment Framework gives priority to potential candidates for disinvestment if nominated by a clinical expert [85] and four frameworks for disinvestment employ applications from stakeholders in the identification process [9, 16, 35, 36].

Participants in the SHARE Program noted that, while formal prompts and triggers could be built into existing decision-making infrastructure, there are also informal yet systematic approaches that could be integrated into other systems and processes to facilitate identification of opportunities for disinvestment by health service staff [9]. Examples are included in Table 9.

Table 9 Additional systematic methods to facilitate identification of disinvestment opportunities in a local health service

Stakeholder nomination can be a powerful contribution to the process, providing the nominated items are objectively scrutinised against additional identification and prioritisation criteria [109], however there are some considerations in the actual implementation.

Although evaluation of the applications in these frameworks is rigorous, based on explicit local criteria and health technology assessment, the process of how the topic was raised initially is not systematic or transparent. Applications can be received from any stakeholder for any reason. In this context they are likely to be driven by non-systematic factors such as clinician’s interests, information obtained from conferences or journal articles, or awareness of practice elsewhere [2, 6]. “Understanding how the technology got on the agenda, where it came from and who was pushing for it” and the potential for “gaming by industry” are concerns reported by senior health decision-makers [116], but are often unclear in a stakeholder application process.

When invited to nominate candidates for disinvestment, clinicians have been found to be more likely to identify the practices of other professional groups than their own, practices that do not affect their revenue-generating services and practices of low impact [9, 21, 117].

Clarity of aims and objectives at the start of a project and clear rationale for change were in the top 10 considerations for successful disinvestment and one of three best practice recommendations arising from a Delphi study of international experts [52]. However lack of clarity and rationale have been noted as problems in identifying disinvestment opportunities [63], particularly from stakeholder applications [9, 10].

These issues may create systematic biases in the choice of investment targets and miss some key opportunities. Unnecessary duplication of effort may also result, with individual facilities or regions undertaking extensive evaluations of the same topics.

1.4 ‘Low value’ lists

‘Low value’ lists are compilations of practices that have been demonstrated to have little or no benefit or potential to cause harm. They have been developed by governments and health agencies [118,119,120], commissioners of health services [121], professional bodies [65, 122, 123] and researchers [124,125,126]. Some of these lists are derived from research evidence, some are based on expert opinion and others from a combination of the two.

Duckett and colleagues separate them into ‘top down’ and ‘bottom up’ approaches, noting that each has benefits and drawbacks [73]. The ‘top down’ approaches, such as the UK National Institute for Health and Clinical Excellence ‘Do Not Do’ Recommendations [54], are described as providing the most consistent, objective, transparent and relevant evaluations. The ‘bottom up’ approaches, such as the Choosing Wisely program [74], highlight potentially ‘low value’ treatments and tests that should be carefully considered at the point of care.

Removing, reducing or restricting practices of little or no value clearly has merit, and ‘low value’ lists are likely to be very useful to health service decision-makers if they are based on sound evidence backed by expert consensus. However the definition of ‘low value’ is not always explicit and the validity and appropriateness of some of the lists and the ethics of their application have been questioned [117, 125, 127,128,129,130]. Potential users of ‘low value’ lists may wish to confirm the basis for claims made, in particular the definition being used and the use of systematic review evidence in the inclusion process [9].

The SHARE algorithm described earlier could also be applied to ‘low value’ lists to assess local applicability and facilitate prioritisation [9].

1.5 Economic approaches to priority setting

These priority setting approaches combine evidence from local data, expert opinion and stakeholder consultation [27, 32].

PBMA applies the economic principles of opportunity cost and marginal analysis to determine priorities for health program budgets in the context of limited resources [131]. This method approaches disinvestment from the relative perspective, with decision-makers weighing up options for investment and disinvestment and reaching their preferred balance using locally-relevant criteria established by the stakeholders. The process is well-tested and guidance is available [27]. Although decision-makers acknowledge the usefulness of PBMA, it remains quite difficult to achieve in practice [24, 48, 131]. Another criticism is that it fragments the health sector into ‘program budget silos’ resulting in allocation and re-allocation of resources within, rather than between, programs which fails to identify more cost-effective options outside the program area [31, 48, 131, 132].

In contrast to PBMA, the Health-sector Wide model is designed to shift the focus of priority setting away from program budgets towards well-defined target populations with particular health problems [31]. The condition-specific silos created here may be an improvement on program budget silos, but the model is more difficult to apply in local health services where funding decisions are not based on condition-specific populations.

The major limitations for all priority setting approaches include idiosyncrasies in cost-accounting, lack of sufficient high quality data to inform decision-making, and lack of time and appropriate skills of decision-makers to undertake the process and implement the decisions [24, 27, 46, 48, 55, 131]. Lack of in-house expertise in health economics is a particular barrier at the local level [9].

2. Prioritisation and decision-making

Priority setting exercises clearly include a prioritisation process, however initiatives that identify their disinvestment targets by other means may need a specific prioritisation process to choose between the options available. Methods and tools for systematic, transparent and equitable decision-making may be used if prioritisation is not required or to complement the prioritisation process.

Prioritisation tools primarily focus on characteristics intrinsic to the TCP; however additional criteria may influence the decision to proceed with a disinvestment project in the local healthcare setting [9]. These might be pragmatic features that enhance initiatives chosen specifically as pilot or demonstration projects, such as opportunities for ‘quick wins’, or factors that affect the outcome of a project, such as likelihood of success and sustainability or potential usefulness of the evaluation.

There is a huge range of potentially relevant criteria for resource allocation decisions. Most authors emphasise that a list of criteria should be developed with input from all stakeholders to meet the objectives of individual situations. The commonly cited basic requirements include clinical parameters such as safety and effectiveness, economic measures such as cost-effectiveness and affordability, and social factors such as local values and priorities. Additional criteria will depend on the setting and context. Methods and tools to assist in assessment of safety and effectiveness [133,134,135,136] and use of economic measures [137,138,139] are available. Similar resources for consumer and community engagement are addressed in Additional file 1.

Deciding between several alternatives is a complex process requiring consideration of multiple factors. Health service decision-makers are often not good at this, relying on heuristic or intuitive approaches which ignore potentially important information [140]. Methods such as burden of disease analyses, cost-effectiveness analyses and equity analyses focus on some but not all of the available information [140]. Multi-criteria decision analysis (MCDA) allows consideration of all factors simultaneously, and although used widely in other scientific disciplines, it has only been used in health care relatively recently [76, 140].

The Star model, a “socio-technical allocation of resources” based on MCDA and health economic theory, has been piloted successfully in two settings, revised and developed into a toolkit [141,142,143]. MCDA is also the foundation of the Evidence and Value: Impact on DEcision Making (EVIDEM) framework, which is being investigated further through research conducted by the international EVIDEM Collaboration [144].

While the components of the A4R framework are included within several principles in the new framework, policy makers, managers and clinicians may also wish to use the A4R terminology specifically in their decision-making processes.

A4R is also the basis for the Systematic Tool for Evaluating Pharmaceutical Products for Public Funding Decisions (6-STEPPPs) [145] and A4R and MCDA have been combined in other decision-making applications [146, 147].

Lists of criteria for consideration in prioritisation and decision-making have been published for disinvestment [78, 82, 85, 109, 148], including many who have applied or adapted the criteria framework proposed by Elshaug et al. [72]; resource allocation [6, 149,150,151]; and general decision-making [42]. A tool to analyse gaps in priority setting has also been developed [152].

Many health service decision-makers use a prioritisation matrix, but most of these are developed locally and based on simple spreadsheets or business case templates [9, 48, 55, 153]. This variety of tools makes it difficult to compare costs and outcomes more broadly and there is some scepticism amongst decision-makers about the lack of rigour, transparency and skills involved in their local programs [21, 48].

There are also software applications to facilitate PBMA and generic prioritisation processes [27, 154, 155].

3. Development of a proposal

Once a decision has been made that there is a need for change, a proposal to meet that need and implement the decision is developed. When the proposal is drafted, the time and other resources required to implement and evaluate it can be assessed to determine if the benefits outweigh the costs of the exercise and to inform planning.

The range of potential disinvestment activities is broad and disparate. A proposal to remove a drug from a hospital formulary is likely to be very different to a proposal to close down an inpatient facility. No specific methods and tools were identified for developing disinvestment proposals, but generic materials for developing the program theory or rationale and defining the program logic would be useful [156,157,158,159,160,161,162,163,164], as would business case proformas and communication templates.

Proposals are more likely to be successful if they have certain favourable characteristics and new initiatives are more likely to be sustainable if there is appropriate availability and adequate provision of critical factors to achieve and maintain the proposed components and activities [20, 165,166,167]. A checklist of the factors influencing likelihood of success and sustainability is available [8].

4. Implementation

Some successes with national approaches to disinvestment have been reported and may have elements that are generalisable to local circumstances [72, 85, 102]. However in some circumstances national approaches are not applicable at state/provincial, regional or institutional levels; for example removing or refining indications for reimbursement for ‘low value’ TCPs in national fee-for-service schemes for doctors in private practice may not apply to doctors working in state funded hospitals.

As noted above, there are also many examples in the EBP and quality and safety literature of successful projects at local level to remove unsafe or ineffective TCPs which are not labelled as disinvestment.

Many articles about disinvestment do not address implementation at all and some note that there are difficulties related to implementation but offer no solutions. Of those that do consider implementation, many of the comments are principles, captured in the section above, or barriers and enablers, captured below.

One recommendation for successful implementation is that “we could create conditions that make it easy for people to avoid using low-value health care services” [128]. Environmental changes such as closing services, physically removing products from storerooms and work areas, and eliminating items from formularies and purchasing catalogues should achieve this aim and result in complete cessation. In addition, if providers or recipients of a TCP, program or service receive financial reimbursement, removal of funding is likely to reduce use considerably, although not necessarily completely [64, 72, 117, 168, 169]. But not all disinvestment decisions can be managed with structural changes.

The need for an implementation strategy for each disinvestment activity is widely acknowledged. One disinvestment guideline details eight generic steps in their Action Plan [35], the SHARE Program used the SEAchange model for evidence-based change [41] to implement disinvestment pilot projects and support services [9, 10], and a model for ‘de-adoption’ utilises the ‘Knowledge to Action’ framework [22, 170].

A range of approaches to facilitate implementation of disinvestment decisions has been proposed. These include communication and educational materials [58, 72, 78, 117, 121, 171]; financial incentives and pay-for-performance [59, 64, 72, 117, 168]; reinvestment of resources saved [29, 78, 80, 172]; clinical champions [48, 80]; clinical pharmacists to monitor and advise prescribers [68]; quality standards [59, 117]; professional standards, maintenance-of-certification activities and practice audit [117]; prompts through guidelines, protocols, clinical pathways and decision support systems [5, 58,59,60, 72, 82, 168, 171]; requirements to report variations from mandatory guidelines [59, 72]; monitoring and reporting of outcomes [72, 78, 168]; public reporting of provider performance [59, 117, 168]; training and re-organisation of staffing and equipment [10, 78]; and “picking low hanging fruit” before tackling more difficult projects [80]. These proposals have arisen from qualitative work with stakeholders or been derived from an understanding of implementation science; the papers offering these suggestions for implementation do not report application or evaluation of these strategies in the disinvestment context.

Several authors note that implementation is more likely to be successful if decisions are made at the local level, integrated into everyday decision-making and central to local planning [55, 59, 60, 80]. A well-resourced and well-designed formal priority setting entity is reported to improve implementation of decisions [27, 37, 55, 173]. It provides a recognised vehicle to consider information such as new evidence or local performance concerns, one which has transparent processes and appropriate authority for decision-making and action, where local expertise can be built up and local knowledge utilised. It is thought to “make contentious decisions more palatable and defensible” [55].

The SHARE Program used the Technology/Clinical Practice Committee (TCPC) as a formal decision-making structure [2]. After piloting several approaches, the Evidence Dissemination Service mentioned above as a method of identification, was finally implemented within a governance model to ensure maximum adherence [11]. Recently-published, high-quality synthesised evidence was identified and publications reporting evidence of harm, lack of effect or findings of a more cost-effective alternative to current practice were prioritised for dissemination. An Evidence Bulletin summarising an individual publication was provided to the TCPC, which then forwarded it to the department head or committee chair responsible for practice in the specific topic area. A response was required to confirm whether current practice was consistent with the evidence, and if not, what measures were being taken to address this or an explanation of why change was not required. When there was evidence of harm, responses to the TCPC were required within 1 month and the responses, or lack thereof, were reported to the Chief Executive the following month. Responses to other Evidence Bulletins were required in three or 6 months. A total of 175 publications were disseminated, 55 of the Evidence Bulletins required responses. Of the 43 responses received during the evaluation period, 32 reported that local practice was consistent with the evidence, six reported that the evidence was not applicable at Monash Health, three noted that local practice was not consistent with the evidence but provided a justifiable reason, and two reported that remedial action was planned to bring local practice into line with the evidence [11].

Although there are some particular challenges to asking people to stop doing things they believe in [1], the general principles of implementation should apply to disinvestment as they do for any practice change. These are summarised in the SEAchange model and the Knowledge to Action framework: engaging all stakeholders, identifying what is already known about practice change in the topic area from the literature and local knowledge, undertaking an analysis of local barriers and enablers, developing an implementation plan including strategies to minimise barriers and build on enablers, piloting and revising as required, and finally implementing in full [41, 170].

5. Monitoring, evaluation and reporting

The Schmidt ‘Framework for disinvestment’ notes that both process and outcome evaluations should be undertaken but provides no details [16]. In their framework for evaluation of priority setting processes, Barasa and colleagues propose measures for both procedure aspects and outcomes [39] and a systematic review summarises a range of performance measures to assess use of ‘low value’ TCPs [174]. The ‘Integrative framework for measuring overuse’ lists measurement tools linked to specific project/program goals and discusses advantages and disadvantages of each approach [38].

The SHARE Evaluation Framework and Plan was created for an organisation-wide program of disinvestment in a local health service network [40]. It was developed in consultation with stakeholders and included evaluation domains, audience, scope, evaluation questions, sources of data, methods of collection and analysis, reporting and timelines. A theoretical framework and taxonomy adapted for evaluation and explication of disinvestment projects was also used to understand the process of disinvestment in the SHARE Program [9].

The deficiencies in available economic and usage data and lack of methods for quantifying savings are considered to be significant limitations to evaluation [46, 60, 82, 175, 176].

There are many generic guidance documents for monitoring and evaluation of health programs and projects in a range of settings. Like implementation, the principles, methods and tools for evaluation should be as appropriate for disinvestment as they are for any healthcare improvement project.

Findings from monitoring and evaluation activities should be reported on a regular and/or scheduled basis to the appropriate stakeholders in accordance with project terms of reference, governance protocols and other local requirements.

6. Reinvestment

This step will only apply in certain projects when it is anticipated that firstly resources will be released and secondly that they will be available for use elsewhere. Although there is considerable discussion about the potential for reinvestment or reallocation, there is little information about how to do it [1]. Resource release and reallocation are built into prioritisation processes for budget-setting but are not integral to other methods of disinvestment. One proposal for a “sensible, well-managed reinvestment program” describes “a cost-accounting process to capture, and a financial strategy and analysis to return, a pre-agreed portion of real savings” [172]. However the comments by other authors regarding inconsistencies in accounting practices, insufficient valid and reliable data, lack of methods and tools and absence of reported examples suggest that this may not be currently achievable [1, 21, 48, 60, 83, 175,176,177].

7. Dissemination and diffusion

These terms have been used with specific, but inconsistent, meanings in the disinvestment literature. For example, diffusion has been used to refer to uptake of ‘new’ technologies where disinvestment is used for removal of ‘old’ technologies [178]. In contrast, diffusion and discontinuation have been used to represent ‘spontaneous’ uptake and removal of technologies where dissemination and disinvestment are their counterparts for ‘managed’ uptake and removal [21]. The former links disinvestment with diffusion, the latter with dissemination.

Since the focus of this framework is on implementation of change, and does not differentiate between implementation of investment or disinvestment decisions, the definitions of dissemination and diffusion are taken from the knowledge translation literature (Table 4) [20, 170, 179]. Dissemination involves planned, active processes to share and spread information; diffusion is unplanned and passive.

Outcomes of disinvestment projects should be disseminated to others working in this area to fill gaps in knowledge, avoid duplication, build on successes and learn from mistakes and misfortune. However no guidance for systematic dissemination or facilitation of diffusion of successful disinvestment initiatives at the local health service level was identified. Guidance from the knowledge translation, EBP, QI and implementation science literature for dissemination and diffusion of new TCPs may be a useful starting point, however the specific challenges of disinvestment may influence the generalisability of these methods [1, 180].

8. Maintenance

Maintenance is the final step in the change process. It involves strategies to sustain recently implemented change after project support is removed; to integrate the change into organisational systems, processes and practices; and to attain long-term viability of the change (Table 4). Several terms are used in the broader health literature to capture this concept; examples include adoption, assimilation, sustainability and institutionalisation. Sustainability has been used in the context of disinvestment [3, 8, 22, 169, 181]. Maintenance is used in this framework to avoid confusion with use of the term ‘sustainability’ in a different context in the title of the SHARE Program. Maintenance is also used in the evaluation literature to assess “the extent to which a program becomes institutionalized or part of the routine organizational practices and policies” and can be applied to both the population targeted for behaviour change and the organisation that enacted or adopted the policy [182].

Montini and Graham propose that the disciplines of “Science and technology studies, the History and philosophy of science, the Sociology of health and illness, and Medical Anthropology” be explored to understand the factors relating to sustaining change related to ‘de-implementation’ [169]. Niven and colleagues recommend that ‘de-adoption’ interventions include a sustainability plan to prevent healthcare providers knowingly or unknowingly reverting to old practices [22].

The SHARE Program applied, adapted and developed methods and tools to facilitate sustainability of disinvestment-related initiatives at both the program and project level.

  • SHARE projects were assessed against a framework for sustainability based on five categories: structure, skills, resources, commitment and leadership [8].

  • The SEAchange model for sustainable, effective, appropriate evidence-based change in health services applied in SHARE projects includes formal assessment of sustainability at each step in the change process [41].

  • The determinants of effectiveness outlined in a framework and taxonomy adapted for evaluation and explication of SHARE disinvestment projects could be considered in developing strategies for sustainability of new disinvestment interventions [9].

  • The preconditions and underlying principles derived from the literature and local research in development of the SHARE model for exploring sustainability in health care by allocating resources effectively in the local health service setting were identified as factors related to success and sustainability of the whole SHARE Program [8].

Barriers and enablers

The terms barrier and enabler are commonly used to describe factors influencing the success of change initiatives in health care, but interestingly they are less frequent in the disinvestment literature. Most authors refer to the ‘challenges’ related to disinvestment, few refer to specific ‘barriers’. ‘Enablers’ or existing factors that could facilitate desired change are rarely mentioned, however many authors describe favourable conditions that represent the absence of specific negative factors or strategies that seek to remove them. The challenges and negative factors identified are interpreted as barriers and summarised in Table 10.

Table 10 Examples of potential barriers to disinvestment

Some barriers impact on all aspects of disinvestment across each level of influence [15, 16, 21, 24, 29, 48, 58, 78,79,80, 83, 116, 120, 129, 175, 178, 183,184,185,186,187]. Barriers to establishment and delivery of a program for decision-making are noted [8, 9, 24, 31, 55, 58, 64, 79, 82, 120, 131, 132, 153, 175, 183] and other barriers are categorised using the steps of the disinvestment process: stakeholder engagement [2, 58, 78,79,80, 82, 120, 153], identification of disinvestment targets [8, 9, 16, 21, 24, 46, 48, 58, 60, 63, 72, 79, 82, 120, 129, 175, 183, 188,189,190], prioritisation and decision-making [2, 21, 24, 31, 46, 48, 55, 58, 60, 63, 64, 72, 79, 82, 120, 129, 132, 175, 183, 188, 190, 191], implementation [2, 8, 21, 46, 58, 64, 79, 82, 120, 132, 153, 169], monitoring and evaluation [8, 46, 48, 60, 82, 175], reinvestment [55, 64, 153, 175, 176] and research [58, 183, 189]. There is some overlap where the same barriers affect more than one aspect of the process.

This summary only captures barriers to disinvestment activities. Barriers and enablers to investment in new TCPs and strategies to address them are summarised elsewhere [2]. Programs for disinvestment may require system reform, so the barriers inherent in large-scale change will also be applicable. The body of literature on barriers and enablers to using evidence in decision-making and implementing practice change will also be relevant to disinvestment activities.

In addition to the list summarised here and the wider literature, an analysis of local barriers and enablers should be undertaken for every project to identify crucial contextual factors.



Although a rigorous systematic approach was taken, it is impossible to be comprehensive in ascertaining all the relevant literature on disinvestment; the reasons are outlined in the conceptual review [1]. As a result, some relevant publications may not have been identified and some information may not have been published. Despite these limitations, the messages about operationalising disinvestment are generally clear and consistent and provide strong underpinnings for the framework.

The literature has been reviewed from the perspective of a local health service, however the authors’ experience is based in the Australian health system; hence differences with other health systems may not have been recognised and additional decision-making settings or methods and tools may have been missed.

The specific details of the ‘where, who and how’ of decision-making is likely to differ between organisations but the underlying premises should be the same: individuals and groups make decisions under certain conditions. The classifications of decisions and decision-makers might be useful starting points to elucidate local particulars.

The proposed framework is conceptual and untested. Naming of categories, determination of their constituent elements and the relationships between components has not been piloted or refined with stakeholder input. It is large, complex and all-encompassing and may prove too daunting or complicated to be achieved in this format. Testing and research may establish if it is feasible in the current overarching format or if it should be renamed, redefined or reformulated for implementation in smaller sections.

The framework is proposed at the ‘big picture’ level and requires supplementation with detail for all the components. There are some existing frameworks, models, methods and tools that can be applied in several areas but not for all elements within the framework.

There are many barriers that cannot be addressed by generic system changes and must be tackled when implementing the framework in individual situations. Many of these may be successfully overcome with local strategies; however some aspects of the framework involve potentially insurmountable barriers in the current environment. The main example is lack of valid, reliable, timely, appropriate and sufficiently specific data to identify disinvestment targets and monitor and evaluate disinvestment initiatives.

Implications for policy and practice

As the focus of this review is operationalisation of disinvestment, the implications for policy and practice have been integrated throughout the paper.

Implications for research

The implications for research in operationalising disinvestment are enormous. Placing the research component of the proposed framework across all the constituent elements illustrates that there is a need for research in each of them. Some topics stand out as priorities.

Many authors report a lack of frameworks, models, methods or tools for disinvestment. However there are some frameworks and models for disinvestment, although not tested; and plenty of methods and tools, many of which are tested, frequently from other research disciplines but which are relevant for disinvestment projects. Perhaps a more important factor is the lack of proactive mechanisms, prompts and triggers [9, 11, 16, 21, 24, 27, 192]. There are rigorous methods for HTA and analysis of health service data but no systematic methods to initiate these processes or draw the results to the attention of health service decision-makers. It is also not clear who is, or should be, responsible for instigating and making decisions and taking action. Research in these areas is a priority.

Investigation of data requirements, data collection methods and skills of decision-makers to use data for disinvestment is another priority [21, 24, 27, 46, 48, 55, 58, 60, 131]. Support for data collection after a TCP has been introduced is low and research into methods and resources required for post-market surveillance and “coverage with evidence development” is required [24, 132].

Some authors have highlighted other aspects of disinvestment for research such as exploring disinvestment at local health service and individual practitioner level [16, 55, 56, 80, 188, 193], taking a longitudinal approach from inception through implementation to outcomes that cross organisational boundaries [80, 188], identifying determinants for disinvestment [15, 80, 129], implementing change management [56, 58], and drafting and refining frameworks, methods and tools [15, 24, 29, 58, 129, 175, 184, 185]. Mechanisms to develop, implement and evaluate disinvestment activities can be built on existing theoretical frameworks from other research paradigms such as HTA, knowledge translation and implementation science [28, 83]. Measures of impact, potential unintended consequences and factors contributing to success or failure also need to be captured [24, 83, 193]. The SHARE Program provides some early work to build on by reporting disinvestment projects from inception to implementation [9]; identifying determinants for disinvestment, potential unintended consequences and factors contributing to success or failure [9]; and developing frameworks, models and algorithms [5,6,7,8,9, 11] and evaluation frameworks and plans [10, 11, 40]. These outputs of the SHARE Program are summarised in Paper 1 [3].

Research could also include testing the proposed framework in different contexts.


There is no agreed definition or common understanding of disinvestment, yet the concept is widely discussed and disinvestment initiatives and research are called for. Although there are only a few, largely untested, frameworks and models and little practical guidance in the literature, there are clear and consistent messages regarding principles for decision-making, settings and opportunities to identify disinvestment targets, steps in the disinvestment process, methods and tools, and barriers and enablers. This information has been drawn together into a framework for operationalising disinvestment in a systematic, integrated, organisation-wide approach in the local healthcare setting.

Definitions for essential terms are proposed and key concepts underpinning the framework have been made explicit. The term disinvestment is used in the broadest sense, ‘removal, reduction or restriction of any aspect of the health system for any reason’, and can be applied to products, devices and equipment; clinical practices and procedures; health services and programs; information technology and corporate systems. Given the negative connotations of the word and the problems inherent in considering disinvestment in isolation, the basis for the framework is ‘resource allocation’ addressing the spectrum of decision-making from investment to disinvestment.

The framework is based on three components: the program consists of principles for decision-making and settings that provide opportunities to introduce systematic prompts and triggers to initiate consideration of disinvestment; projects follow the steps of the disinvestment process; and research is needed across all aspects of the framework.

The proposed framework can be employed at network, institutional, departmental, ward or committee level. It is proposed as an organisation-wide application, embedded within existing systems and processes, which can be responsive to needs and priorities at the level of implementation. It can be used in policy, management or clinical contexts, for resource allocation and potentially other decision-making processes.

There are many theories, frameworks, models, methods and tools from other areas of health research and practice that are relevant to disinvestment which could be employed within this framework.

Multiple barriers to establishing a decision-making framework and implementing disinvestment initiatives were identified. Some of these relate to the lack of elements that form individual principles and are addressed in the framework, however many involve local factors that can only be tackled when implementing the framework in particular contexts.

The framework captures all the identified information from the literature about operationalisation of disinvestment in the context of resource allocation. This could be a strength, if all the elements are required for a robust effective program of decision-making and action, or a weakness, if it is too complex to be achieved in practice.



Accountability for Reasonableness


Evidence Based Practice


Evidence and Value: Impact on DEcision Making


Health Sector Wide


Health Technology Assessment


Health Technology Reassessment


Multi-criteria decision analysis


National Institute of Health and Clinical Excellence


Program Budgeting and Marginal Analysis


Quality Improvement


Sustainability in Health care by Allocating Resources Effectively


Systematic Tool for Evaluating Pharmaceutical Products for Public Funding Decisions


Technology/Clinical Practice Committee


Technologies and Clinical Practices


  1. Harris C, Green S, Ramsey W, Allen K, King R. Sustainability in Health care by Allocating Resources Effectively (SHARE) 9: Conceptualising disinvestment in the local healthcare setting BMC health services research. 2017;in press.

  2. Harris C, Garrubba M, Allen K, King R, Kelly C, Thiagarajan M, et al. Development, implementation and evaluation of an evidence-based program for introduction of new health technologies and clinical practices in a local healthcare setting. BMC Health Serv Res. 2015;15(1):575. doi:10.1186/s12913-015-1178-4.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Harris C, Green S, Ramsey W, Allen K, King R. Sustainability in health care by allocating resources effectively (SHARE) 1: introducing a series of papers reporting an investigation of disinvestment in a local healthcare setting BMC health services research; 2017. doi:10.1186/s12913-017-2210-7.

    Google Scholar 

  4. Harris C, Allen K, King R, Ramsey W, Green S. Sustainability in Health care by Allocating Resources Effectively (SHARE) 11: Reporting outcomes of an evidence-driven approach to disinvestment in a local healthcare setting. BMC health services research. 2017;in press.

  5. Harris C, Allen K, King R, Ramsey W, Kelly C, Thiagarajan M. Sustainability in health care by allocating resources effectively (SHARE) 2: identifying opportunities for disinvestment in a local healthcare setting BMC health services research; 2017. doi:10.1186/s12913-017-2211-6.

    Google Scholar 

  6. Harris C, Allen K, Waller C, Brooke V. Sustainability in health care by allocating resources effectively (SHARE) 3: examining how resource allocation decisions are made, implemented and evaluated in a local healthcare setting BMC health services research; 2017. doi:10.1186/s12913-017-2207-2.

    Google Scholar 

  7. Harris C, Ko H, Waller C, Sloss P, Williams P. Sustainability in health care by allocating resources effectively (SHARE) 4: exploring opportunities and methods for consumer engagement in resource allocation in a local healthcare setting BMC health services research; 2017. doi:10.1186/s12913-017-2212-5.

    Google Scholar 

  8. Harris C, Allen K, Waller C, Green S, King R, Ramsey W, et al. Sustainability in health care by allocating resources effectively (SHARE) 5: developing a model for evidence-driven resource allocation in the local healthcare setting BMC health services research; 2017. doi:10.1186/s12913-017-2208-1.

    Google Scholar 

  9. Harris C, Allen K, Brooke V, Dyer T, Waller C, King R, et al. Sustainability in health care by allocating resources effectively (SHARE) 6: investigating methods to identify, prioritise, implement and evaluate disinvestment projects in a local healthcare setting. BMC Health Serv Res. 2017; doi:10.1186/s12913-017-2269-1.

  10. Harris C, Allen K, Waller C, Dyer T, Brooke V, Garrubba M, et al. Sustainability in health care by allocating resources effectively (SHARE) 7: supporting staff in evidence-based decision-making, implementation and evaluation in a local healthcare setting BMC health services research; 2017. doi:10.1186/s12913-017-2388-8.

    Google Scholar 

  11. Harris C, Garrubba M, Melder A, Voutier C, Waller C, King R et al. Sustainability in Health care by Allocating Resources Effectively (SHARE) 8: Developing, implementing and evaluating an Evidence Dissemination Service in a local healthcare setting. BMC health services research. 2017;in press.

  12. Rycroft-Malone J, Bucknall TK, editors. Models and frameworks for implementing evidence-based practice: linking evidence to action. Evidence-based nursing. Chichester: UK Wiley-Blackwell; 2010.

    Google Scholar 

  13. Boivin A, Lehoux P, Lacombe R, Burgers J, Grol R. Involving patients in setting priorities for healthcare improvement: a cluster randomized trial. Implement Sci. 2014;9:24. doi:10.1186/1748-5908-9-24.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Carter D, Watt AM, Braunack-Mayer A, Elshaug AG, Moss JR, Hiller JE. Should there be a female age limit on public funding for assisted reproductive technology? J Bioethical Inq. 2013;10(1):79–91. doi:10.1007/s11673-012-9415-6.

    Article  Google Scholar 

  15. Watt AM, Hiller JE, Braunack-Mayer AJ, Moss JR, Buchan H, Wale J, et al. The ASTUTE health study protocol: deliberative stakeholder engagements to inform implementation approaches to healthcare disinvestment. Implement Sci. 2012;7:101. doi:10.1186/1748-5908-7-101.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Schmidt DE. The development of a disinvestment framework to guide resource allocation decisions in health service delivery organizations. The University of British Columbia 2010. Available from: Accessed 9 Aug 2017.

  17. Lindstrom H, Waldau S. Ethically acceptable prioritisation of childless couples and treatment rationing: “accountability for reasonableness”. Eur J Obstet Gynecol Reprod Biol. 2008;139(2):176–86. doi:10.1016/j.ejogrb.2008.02.018.

    Article  PubMed  Google Scholar 

  18. Hodgetts K, Hiller JE, Street JM, Carter D, Braunack-Mayer AJ, Watt AM, et al. Disinvestment policy and the public funding of assisted reproductive technologies: outcomes of deliberative engagements with three key stakeholder groups. BMC Health Serv Res. 2014;14:204. doi:10.1186/1472-6963-14-204.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Rogers E. Diffusion of innovations. 5th ed. New York: Free Press; 2003.

    Google Scholar 

  20. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629. doi:10.1111/j.0887-378X.2004.00325.x.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Hollingworth W, Rooshenas L, Busby J, Hine CE, Badrinath P, Whiting PF et al. Using clinical practice variations as a method for commissioners and clinicians to identify and prioritise opportunities for disinvestment in health care: a cross-sectional study, systematic reviews and qualitative study. Southampton UK: Queen's Printer and Controller of HMSO 2015; 2015.

  22. Niven DJ, Mrklas KJ, Holodinsky JK, Straus SE, Hemmelgarn BR, Jeffs LP, et al. Towards understanding the de-adoption of low-value clinical practices: a scoping review. BMC Med. 2015;13:255. doi:10.1186/s12916-015-0488-z.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Ilott I, Gerrish K, Laker S, Bray K. Naming and framing the problem: using theories, models and conceptual frameworks. Bridging the gap between knowledge and practice. Your starter for 10: no.2. Sheffield teaching hospital NHS foundation trust, National Institute for Health Research collaboration for leadership in applied Health Research and Care for South Yorkshire. 2013.

  24. Polisena J, Clifford T, Elshaug AG, Mitton C, Russell E, Skidmore B. Case studies that illustrate disinvestment and resource allocation decision-making processes in health care: a systematic review. Int J Technol Assess Health Care. 2013;29(2):174–84. doi:10.1017/s0266462313000068.

    Article  PubMed  Google Scholar 

  25. Giovagnoni A, Bartolucci L, Manna A, Morbiducci J, Ascoli G. Health technology assessment: principles, methods and current status. Radiol Medica. 2009;114(5):673–91. doi:10.1007/s11547-009-00387-5.

    Article  CAS  Google Scholar 

  26. Daniels N, Sabin J. Accountability for reasonableness: an update. BMJ. 2008;337 doi:10.1136/bmj.a1850.

  27. Mitton C, Dionne F, Donaldson C. Managing healthcare budgets in times of austerity: the role of program budgeting and marginal analysis. Appl Health Econ Health Policy. 2014;12(2):95–102. doi:10.1007/s40258-013-0074-5.

    Article  PubMed  PubMed Central  Google Scholar 

  28. MacKean G, Noseworthy T, Elshaug AG, Leggett L, Littlejohns P, Berezanski J, et al. Health technology reassessment: the art of the possible. Int J Technol Assess Health Care. 2013;29(04):418–23. doi:doi:10.1017/S0266462313000494

    Article  PubMed  PubMed Central  Google Scholar 

  29. Leggett L, Noseworthy TW, Zarrabi M, Lorenzetti D, Sutherland LR, Clement FM. Health technology reassessment of non-drug technologies: current practices. Int J Technol Assess Health Care. 2012;28(3):220–7. doi:10.1017/S0266462312000438.

    Article  PubMed  Google Scholar 

  30. Sibbald SL, Singer PA, Upshur R, Martin DK. Priority setting: what constitutes success? A conceptual framework for successful priority setting. BMC Health Serv Res. 2009;9:43. doi:10.1186/1472-6963-9-43.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Segal L, Mortimer D. A population-based model for priority setting across the care continuum and across modalities. Cost Effectiveness & Resource Allocation. 2006;4:6.

    Article  Google Scholar 

  32. Hauck K, Smith P, Goddard M. The Economics of Priority Setting for Health Care: A Literature Review. Health, Nutrition and Population Discussion Paper World Bank 2004. Available from: Accessed 9 Aug 2017.

  33. Department of Health. Medicare Benefits Schedule Review. Accessed 9 Aug 2017.

  34. Elshaug AG, Rosenthal MB, Lavis JN, Brownlee S, Schmidt H, Nagpal S, et al. Levers for addressing medical underuse and overuse: achieving high-value health care. Lancet. 2017; doi:10.1016/s0140-6736(16)32586-7.

  35. Ibargoyen-Roteta N, Gutierrez-Ibarluzea I, Asua J. Guiding the process of health technology disinvestment. Health policy. 2010;98(2–3):218–26. doi:10.1016/j.healthpol.2010.06.018.

    Article  PubMed  Google Scholar 

  36. National Health Committee NZ. Business plan 2013/14–2015/16. Wellington: New Zealand Ministry of Health; 2013.

    Google Scholar 

  37. Joshi NP SF, Noseworthy TW. Reassessment of health technologies: obsolescence and waste. 2009.

    Google Scholar 

  38. Bhatia RS, Levinson W, Shortt S, Pendrith C, Fric-Shamji E, Kallewaard M, et al. Measuring the effect of choosing wisely: an integrated framework to assess campaign impact on low-value care. BMJ quality & safety. 2015;24(8):523–31. doi:10.1136/bmjqs-2015-004070.

    Article  Google Scholar 

  39. Barasa EW, Molyneux S, English M, Cleary S. Setting healthcare priorities at the macro and Meso levels: a framework for evaluation. International journal of health policy and management. 2015;4(11):719–32. doi:10.15171/ijhpm.2015.167.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Centre for Clinical Effectiveness. Sustainability in Health care by Allocating Resources Effectively (SHARE): Evaluation and Research Plan. Southern Health 2009. Available from: Accessed 9 Aug 2017.

  41. Harris C, Turner T, Wilkinson F. SEAchange: Guide to a pragmatic evidence-based approach to Sustainable, Effective and Appropriate change in health services. 2015. Available from: Accessed 9 Aug 2017.

  42. Tanios N, Wagner M, Tony M, Baltussen R, van Til J, Rindress D, et al. Which criteria are considered in healthcare decisions? Insights from an international survey of policy and clinical decision makers. Int J Technol Assess Health Care. 2013;29(04):456–65. doi:10.1017/S0266462313000573

  43. Rubinstein A, Belizan M, Discacciati V. Are economic evaluations and health technology assessments increasingly demanded in times of rationing health services? The case of the argentine financial crisis. Int J Technol Assess Health Care. 2007;23(2):169–76. doi:10.1017/s0266462307070274.

    Article  PubMed  Google Scholar 

  44. Gnjidic D, Elshaug AG. De-adoption and its 43 related terms: harmonizing low-value care terminology. BMC Med. 2015;13:273. doi:10.1186/s12916-015-0511-4.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Wolfson D, Santa J, Slass L. Engaging physicians and consumers in conversations about treatment overuse and waste: a short history of the choosing wisely campaign. Acad Med. 2014;89(7):990–5. doi:10.1097/acm.0000000000000270.

    Article  PubMed  Google Scholar 

  46. Rubenfeld GD. Cost-effective critical care: cost containment and rationing. Semin Respir Crit Care Med. 2012;33(4):413–20. doi:10.1055/s-0032-1322411.

    Article  PubMed  Google Scholar 

  47. Brody H. From an ethics of rationing to an ethics of waste avoidance. N Engl J Med. 2012;366(21):1949–51. doi:doi:10.1056/NEJMp1203365

    Article  CAS  PubMed  Google Scholar 

  48. Marks L, Weatherly H, Mason A. Prioritizing investment in public health and health equity: what can commissioners do? Public Health. 2013;127(5):410–8. doi:10.1016/j.puhe.2013.01.027.

    Article  CAS  PubMed  Google Scholar 

  49. National Health and Medical Research Council. Ethical considerations relating to health care resource allocation decisions. Commonwealth of Australia 1993. Available from: Accessed 9 Aug 2017.

  50. Culyer A. Efficiency, equity and equality in health and health care. CHE Research Paper 120. Centre for Health Economics, University of York, UK, 2015.

  51. Yost J, Dobbins M, Traynor R, DeCorby K, Workentine S, Greco L. Tools to support evidence-informed public health decision making. BMC Public Health. 2014;14:728. doi:10.1186/1471-2458-14-728.

    Article  PubMed  PubMed Central  Google Scholar 

  52. Robert G, Harlock J, Williams I. Disentangling rhetoric and reality: an international Delphi study of factors and processes that facilitate the successful implementation of decisions to decommission healthcare services. Implement Sci. 2014;9(1):123. doi:10.1186/s13012-014-0123-y.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Smith N, Mitton C, Bryan S, Davidson A, Urquhart B, Gibson JL, et al. Decision maker perceptions of resource allocation processes in Canadian health care organizations: a national survey. BMC Health Serv Res. 2013;13:247. doi:10.1186/1472-6963-13-247.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Liverani M, Hawkins B, Parkhurst JO. Political and institutional influences on the use of evidence in public health policy. A systematic review. PLoS One. 2013;8(10):e77404. doi:10.1371/journal.pone.0077404.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  55. Robinson S, Williams I, Dickinson H, Freeman T, Rumbold B. Priority-setting and rationing in healthcare: evidence from the English experience. Soc Sci Med. 2012;75(12):2386–93. doi:10.1016/j.socscimed.2012.09.014.

    Article  PubMed  Google Scholar 

  56. Smith N, Mitton C, Peacock S, Cornelissen E, MacLeod S. Identifying research priorities for health care priority setting: a collaborative effort between managers and researchers. BMC Health Serv Res. 2009;9:165. doi:10.1186/1472-6963-9-165.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Bowen S, Erickson T, Martens PJ, Crockett S. More than “using research”: the real challenges in promoting evidence-informed decision-making. Healthc Policy. 2009;4(3):87–102.

    PubMed  PubMed Central  Google Scholar 

  58. Henshall C, Schuller T, Mardhani-Bayne L. Using health technology assessment to support optimal use of technologies in current practice: the challenge of “disinvestment”. Int J Technol Assess Health Care. 2012;28(3):203–10. doi:10.1017/s0266462312000372.

    Article  PubMed  Google Scholar 

  59. Rumbold B, Alakeson V, Smith P. Rationing health care. Quest for NHS Efficiency Series 2012. Available from: Accessed 9 Aug 2017.

  60. Garner S, Littlejohns P. Disinvestment from low value clinical interventions: NICEly done? BMJ. 2011;343:d4519. doi:10.1136/bmj.d4519.

  61. Pearson S, Littlejohns P. Reallocating resources: how should the National Institute for health and clinical excellence guide disinvestment efforts in the National Health Service? J Health Serv Res Policy. 2007;12(3):160–5.

    Article  PubMed  Google Scholar 

  62. Watt AM, Willis CD, Hodgetts K, Elshaug AG, Hiller JE. Engaging clinicians in evidence-based disinvestment: role and perceptions of evidence. Int J Technol Assess Health Care. 2012;28(03):211–9. doi:doi:10.1017/S0266462312000402

    Article  PubMed  Google Scholar 

  63. Garner S, Docherty M, Somner J, Sharma T, Choudhury M, Clarke M, et al. Reducing ineffective practice: challenges in identifying low-value health care using Cochrane systematic reviews. J Health Serv Res Policy. 2013;18(1):6–12. doi:10.1258/jhsrp.2012.012044.

    Article  PubMed  Google Scholar 

  64. Haas M, Hall J, Viney R, Gallego G. Breaking up is hard to do: why disinvestment in medical technology is harder than investment. Aust Health Review. 2012;36(2):148–52. doi:10.1071/AH11032.

    Article  Google Scholar 

  65. Owens DK, Qaseem A, Chou R, Shekelle P. High-value, cost-conscious health care: concepts for clinicians to evaluate the benefits, harms, and costs of medical interventions. Ann Intern Med. 2011;154(3):174–80. doi:10.7326/0003-4819-154-3-201102010-00007.

    Article  PubMed  Google Scholar 

  66. Chamberlain CA, Martin RM, Busby J, Gilbert R, Cahill DJ, Hollingworth W. Trends in procedures for infertility and caesarean sections: was NICE disinvestment guidance implemented? NICE recommendation reminders. BMC Public Health. 2013;13:112. doi:10.1186/1471-2458-13-112.

    Article  PubMed  PubMed Central  Google Scholar 

  67. Duru OK, Ettner SL, Turk N, Mangione CM, Brown AF, Fu J, et al. Potential savings associated with drug substitution in Medicare part D: the translating research into action for diabetes (TRIAD) study. J Gen Intern Med. 2014;29(1):230–6. doi:10.1007/s11606-013-2546-6.

    Article  PubMed  Google Scholar 

  68. Larmour I, Pignataro S, Barned KL, Mantas S, Korman MG. A therapeutic equivalence program: evidence-based promotion of more efficient use of medicines. Med J Aust. 2011;194(12):631–4.

    PubMed  Google Scholar 

  69. Oyetunde OO, Aina BA, Tayo F. Impact of generic substitution practice on care of diabetic patients. Int J Clin Pharm. 2014;36(3):623–9. doi:10.1007/s11096-014-9946-5.

    Article  PubMed  Google Scholar 

  70. Saverno K, Gothe H, Schuessel K, Biskupiak J, Schulz M, Siebert U, et al. Consideration of international generic distribution policies on patient outcomes in the United States and Germany. Pharmazie. 2014;69(3):238–40.

    CAS  PubMed  Google Scholar 

  71. Beshears J, Choi JJ, Laibson D, Madrian BC, Reynolds G. Testimonials do not convert patients from brand to generic medication. Am J Manag Care. 2013;19(9):e314–31.

    PubMed  PubMed Central  Google Scholar 

  72. Parkinson B, Sermet C, Clement F, Crausaz S, Godman B, Garner S, et al. Disinvestment and value-based purchasing strategies for pharmaceuticals: an international review. PharmacoEconomics. 2015;33(9):905–24. doi:10.1007/s40273-015-0293-8.

    Article  PubMed  Google Scholar 

  73. National Health Committee NZ. Think Piece. Towards a plan for better targeting of existing health services. 2012. Available from: Accessed 9 Aug 2017.

  74. Nuti S, Vainieri M, Bonini A. Disinvestment for re-allocation: a process to identify priorities in healthcare. Health Policy. 2010;95(2–3):137–43. doi:10.1016/j.healthpol.2009.11.011

    Article  PubMed  Google Scholar 

  75. Levinson W, Kallewaard M, Bhatia RS, Wolfson D, Shortt S, Kerr EA. ‘choosing wisely’: a growing international campaign. BMJ Qual Saf. 2015;24(2):167–74. doi:10.1136/bmjqs-2014-003821.

    Article  PubMed  Google Scholar 

  76. Williams I, Robinson S, Dickinson H. Rationing in health care. The theory and practice of priority setting. Bristol: The Policy Press; 2012.

    Google Scholar 

  77. Hughes E, McKenny K. Decommissioning and disinvestment toolkit 2013–2014 Rayleigh. Essex: Castle Point and Rochford Clinical Commissioning Group; 2013.

    Google Scholar 

  78. Ibargoyen-Roteta N, Gutiérrez-Ibarluzea I, Asua J. Report on the development of the GuNFT guideline. Guideline for not funding existing health technologies in health care systems. Quality plan for the NHS of the MHSP: Basque Office for Health Technology Assessment (Osteba), 2009.

    Google Scholar 

  79. Rooshenas L, Owen-Smith A, Hollingworth W, Badrinath P, Beynon C, Donovan JL. “I won't call it rationing...”: an ethnographic study of healthcare disinvestment in theory and practice. Soc Sci Med. 2015;128:273–81. doi:10.1016/j.socscimed.2015.01.020.

    Article  PubMed  Google Scholar 

  80. Daniels T, Williams I, Robinson S, Spence K. Tackling disinvestment in health care services. The views of resource allocators in the English NHS. J Health Organ Manag. 2013;27(6):762–80.

    Article  PubMed  Google Scholar 

  81. Gagnon MP, Desmartis M, Poder T, Witteman W. Effects and repercussions of local/hospital-based health technology assessment (HTA): a systematic review. Syst Rev. 2014;3:129. doi:10.1186/2046-4053-3-129.

    Article  PubMed  PubMed Central  Google Scholar 

  82. Mayer J, Nachtnebel A. Disinvesting from ineffective technologies: lessons learned from current programs. Int J Technol Assess Health Care. 2015:1–8. doi:10.1017/s0266462315000641.

  83. Leggett L, MacKean G, Noseworthy T, Sutherland L, Clement F. Current status of health technology reassessment of non-drug technologies: survey and key informant interviews. Health Res Policy Syst. 2012; doi:10.1186/1478-4505-10-38.

  84. Medical Benefits Reviews Task Group. Comprehensive Management Framework for the MBS. Commonwealth Department of Health, Australia. Accessed 9 Aug 2017.

  85. Paprica PA, Culyer AJ, Elshaug AG, Peffer J, Sandoval GA. From talk to action: policy stakeholders, appropriateness, and selective disinvestment. Int J Technol Assess Health Care. 2015;31(4):236–40. doi:10.1017/s0266462315000392.

    Article  PubMed  PubMed Central  Google Scholar 

  86. Ellen ME, Leon G, Bouchard G, Ouimet M, Grimshaw JM, Lavis JN. Barriers, facilitators and views about next steps to implementing supports for evidence-informed decision-making in health systems: a qualitative study. Implement Sci. 2014;9(1):179. doi:10.1186/s13012-014-0179-8.

    Article  PubMed  PubMed Central  Google Scholar 

  87. Oliver K, Innvar S, Lorenc T, Woodman J, Thomas J. A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Serv Res. 2014;14(1):2. doi:10.1186/1472-6963-14-2.

    Article  PubMed  PubMed Central  Google Scholar 

  88. Clarke MA, Belden JL, Koopman RJ, Steege LM, Moore JL, Canfield SM, et al. Information needs and information-seeking behaviour analysis of primary care physicians and nurses: a literature review. Health Inf Libr J. 2013;30(3):178–90. doi:10.1111/hir.12036.

    Article  Google Scholar 

  89. LaRocca R, Yost J, Dobbins M, Ciliska D, Butt M. The effectiveness of knowledge translation strategies used in public health: a systematic review. BMC Public Health. 2012;12:751. doi:10.1186/1471-2458-12-751.

    Article  PubMed  PubMed Central  Google Scholar 

  90. Wallace J, Nwosu B, Clarke M. Barriers to the uptake of evidence from systematic reviews and meta-analyses: a systematic review of decision makers’perceptions. BMJ Open. 2012;2(5) doi:10.1136/bmjopen-2012-001220.

  91. Moore G, Redman S, Haines M, Todd A. What works to increase the use of research in population health policy and programmes: a review. Evid Policy. 2011;7(3):277–305. doi:10.1332/174426411X579199.

    Article  Google Scholar 

  92. Solomons NM, Spross JA. Evidence-based practice barriers and facilitators from a continuous quality improvement perspective: an integrative review. J Nurs Manag. 2011;19(1):109–20. doi:10.1111/j.1365-2834.2010.01144.x.

    Article  PubMed  Google Scholar 

  93. Younger P. Internet-based information-seeking behaviour amongst doctors and nurses: a short review of the literature. Health Inf Libr J. 2010;27(1):2–10. doi:10.1111/j.1471-1842.2010.00883.x.

    Article  Google Scholar 

  94. Gifford W, Davies B, Edwards N, Griffin P, Lybanon V. Managerial leadership for nurses’use of research evidence: an integrative review of the literature. Worldviews Evid Based Nurs. 2007;4(3):126–45. doi:10.1111/j.1741-6787.2007.00095.x.

    Article  PubMed  Google Scholar 

  95. Gagliardi AR. “More bang for the buck”: exploring optimal approaches for guideline implementation through interviews with international developers. BMC Health Serv Res. 2012;12:404. doi:10.1186/1472-6963-12-404.

    Article  PubMed  PubMed Central  Google Scholar 

  96. Tricco AC, Cardoso R, Thomas SM, Motiwala S, Sullivan S, Kealey MR, et al. Barriers and facilitators to uptake of systematic reviews by policy makers and health care managers: a scoping review. Implement Sci. 2016;11(1):4. doi:10.1186/s13012-016-0370-1.

    Article  PubMed  PubMed Central  Google Scholar 

  97. Martelli N, Lelong AS, Prognon P, Pineau J. Hospital-based health technology assessment for innovative medical devices in university hospitals and the role of hospital pharmacists: learning from international experience. Int J Technol Assess Health Care. 2013;29(2):185–91. doi:10.1017/s0266462313000019.

    Article  PubMed  Google Scholar 

  98. Gallego G. Hospital drug and therapeutics committees in Australia: is there a role for economic evaluation at the institutional level? Int J Clin Pharm. 2011;33(6):895–7. doi:10.1007/s11096-011-9565-3.

    Article  PubMed  Google Scholar 

  99. Lourenco T, Grant A, Burr J, Vale L. Local decision-makers views’of national guidance on interventional procedures in the UK. J Health Serv Res Policy. 2010;15(Suppl 2):3–11. doi:10.1258/jhsrp.2009.009090.

    Article  PubMed  Google Scholar 

  100. Schumacher I, Zechmeister I. Assessing the impact of health technology assessment on the austrian healthcare system. Int J Technol Assess Health Care. 2013;29(1):84–91. doi:10.1017/s0266462312000748.

    Article  PubMed  Google Scholar 

  101. Schwartz AL, Landon BE, Elshaug AG, Chernew ME, McWilliams JM. Measuring low-value Care in Medicare. JAMA Intern Med. 2014; doi:10.1001/jamainternmed.2014.1541.

  102. Coronini-Cronberg S, Bixby H, Laverty AA, Wachter RM, Millett C. English National Health Service's savings plan may have helped reduce the use of three ‘low-value’procedures. Health Aff (Millwood). 2015;34(3):381–9. doi:10.1377/hlthaff.2014.0773.

    Article  PubMed  Google Scholar 

  103. Gallego G, Haas M, Hall J, Viney R. Reducing the use of ineffective health care interventions: an evidence check rapid review brokered by the sax institute for the NSW Treasury 2010. Available from: Accessed 9 Aug 2017.

  104. Organisation for Economic Co-operation and Development. Geographic variations in health care. What do we know and what can be done to improve health system performance? OECD Health Policy Studies. 2014. doi:10.1787/9789264216594-en.

  105. Australian Commission on Safety and Quality in Health Care and Australian Institute of Health and Welfare. Exploring Healthcare Variation in Australia: Analyses Resulting from an OECD Study. ACSQHC 2014. Available from: Accessed 9 Aug 2017.

  106. The Dartmouth Institute for Health Policy and Clinical Practice. Dartmouth Health Atlas. Accessed 9 Aug 2017.

  107. Health Quality and Safety Commission New Zealand. Atlas of Healthcare Variation. Accessed 9 Aug 2017.

  108. National Health Service. NHS Atlas of Variation. Accessed 9 Aug 2017.

  109. Elshaug A, Moss J, Littlejohns P, Karnon J, Merlin T, Hiller J. Identifying existing health care services that do not provide value for money. Med J Aust. 2009;190(5):269–73.

    PubMed  Google Scholar 

  110. Duckett S, Breadon P, Romanes D, Fennessy P, Nolan J. Questionable care: Stopping ineffective treatments. Grattan Institute 2015. Available from: Accessed 9 Aug 2017.

  111. Nutley T, Reynolds HW. Improving the use of health data for health system strengthening. Glob Health Action. 2013;6:20001. doi:10.3402/gha.v6i0.20001.

    Article  Google Scholar 

  112. Nutley T. Improving data use in decision making: an intervention to strengthen health systems. 2012.

    Google Scholar 

  113. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the expert recommendations for implementing change (ERIC) project. Implement Sci. 2015;10:21. doi:10.1186/s13012-015-0209-1.

    Article  PubMed  PubMed Central  Google Scholar 

  114. Lavis JN, Permanand G, Oxman AD, Lewin S, Fretheim A. SUPPORT tools for evidence-informed health policymaking (STP) 13: preparing and using policy briefs to support evidence-informed policymaking. Health Research Policy and Systems. 2009;7(1):1–9. doi:10.1186/1478-4505-7-s1-s13.

    Article  Google Scholar 

  115. Lomas JCT, McCutcheon C, McAuley L, Law S. Conceptualizing and combining evidence for health system guidance. Can Health Serv Res Found. 2005;

  116. Stafinski T, Menon D, McCabe C, Philippon DJ. To fund or not to fund: development of a decision-making framework for the coverage of new health technologies. PharmacoEconomics. 2011;29(9):771–80. doi:10.2165/11539840-000000000-00000.

    Article  PubMed  Google Scholar 

  117. Morden NE, Colla CH, Sequist TD, Rosenthal MB. Choosing wisely--the politics and economics of labeling low-value services. N Engl J Med. 2014;370(7):589–92. doi:10.1056/NEJMp1314965.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  118. National Institute of Health and Clinical Excellence. 'Do Not Do' Recommendations. National Institute of Health and Clinical Excellence, UK. Accessed 9 Aug 2017.

  119. U.S. Preventive Services Task Force. The Guide to Clinical Preventive Services Recommendations of the U.S. Preventive Services Task Force. Agency for Healthcare Research and Quality 2014. Available from: Accessed 9 Aug 2017.

  120. HealthPACT. Disinvestment in Australia and New Zealand. Health Policy Advisory Committee on Technology 2013. Available from: Accessed 9 Aug 2017.

  121. Audit Commission. Reducing spending on low clinical value treatments. London: Health Briefing; 2011.

    Google Scholar 

  122. Cassel CK, Guest JA. Choosing wisely: helping physicians and patients make smart decisions about their care. JAMA. 2012;307(17):1801–2. doi:10.1001/jama.2012.476.

    Article  CAS  PubMed  Google Scholar 

  123. Qaseem A, Alguire P, Dallas P, Feinberg LE, Fitzgerald FT, Horwitch C, et al. Appropriate use of screening and diagnostic tests to foster high-value, cost-conscious care. Ann Intern Med. 2012;156(2):147–9. doi:10.7326/0003-4819-156-2-201201170-00011.

    Article  PubMed  Google Scholar 

  124. Prasad V, Vandross A, Toomey C, Cheung M, Rho J, Quinn S, et al. A decade of reversal: an analysis of 146 contradicted medical practices. Mayo Clin Proc Mayo Clin. 2013; doi:10.1016/j.mayocp.2013.05.012.

  125. Elshaug AG, Watt AM, Mundy L, Willis CD. Over 150 potentially low-value health care practices: an Australian study. Med J Aust. 2012;197(10):556–60.

    Article  PubMed  Google Scholar 

  126. Ioannidis JP. Contradicted and initially stronger effects in highly cited clinical research. JAMA. 2005;294(2):218–28. doi:10.1001/jama.294.2.218.

    Article  CAS  PubMed  Google Scholar 

  127. Admon AJ, Cooke CR. Will choosing wisely(R) improve quality and lower costs of Care for Patients with critical illness? Ann Am Thor Soc. 2014; doi:10.1513/AnnalsATS.201403-093OI.

  128. Blumenthal-Barby JS. “Choosing wisely” to reduce low-value care: a conceptual and ethical analysis. J Med Philos. 2013;38(5):559–80. doi:10.1093/jmp/jht042.

    Article  CAS  PubMed  Google Scholar 

  129. Elshaug AG, McWilliams J, Landon BE. The value of low-value lists. JAMA. 2013;309(8):775–6. doi:10.1001/jama.2013.828.

    Article  CAS  PubMed  Google Scholar 

  130. Elshaug AG. Over 150 potentially low-value health care practices: an Australian study. Reply. Med J Aust. 2013;198(11):597–8.

    Article  PubMed  Google Scholar 

  131. Cornelissen E, Mitton C, Davidson A, Reid RC, Hole R, Visockas AM, et al. Changing priority setting practice: the role of implementation in practice change. Health Policy. 2014; doi:10.1016/j.healthpol.2014.04.010.

  132. Organisation for Economic Co-operation and Development. The OECD Health Project: Health Technologies and Decision Making. OECD Publishing 2005. Available from: Accessed 9 Aug 2017.

  133. National Collaborating Centre for Methods and Tools. Knowledge translation methods and tools. McMaster University, Hamilton. Accessed 9 Aug 2017.

  134. Harris C, Garrubba M. Finding the evidence: Guide to the best available evidence to support introduction of new technologies and clinical practices. Centre for Clinical Effectiveness, Southern Health 2011. Available from: Accessed 9 Aug 2017.

  135. CASP UK. Critical Appraisal Skills Programme (CASP). Oxford. Accessed 9 Aug 2017.

  136. Ciliska D, Thomas H, Buffett C. An Introduction to Evidence-Informed Public Health and A Compendium of Critical Appraisal Tools for Public Health Practice. National Collaborating Centre for Methods and Tools, Hamilton. Accessed 9 Aug 2017.

  137. National Collaborating Centre for Methods and Tools. Economics tools for public health. McMaster University, Hamilton. 2011. Accessed 9 Aug 2017.

  138. Centers for Disease Control and Prevention. Public Health Economics and Tools. Accessed 9 Aug 2017.

  139. Simoens S. Health economic assessment: a methodological primer. Int J Environ Res Public Health. 2009;6(12):2950–66. doi:10.3390/ijerph6122950.

    Article  PubMed  PubMed Central  Google Scholar 

  140. Baltussen R, Niessen L. Priority setting of health interventions: the need for multi-criteria decision analysis. Cost effectiveness and resource allocation : C/E. 2006;4:14. doi:10.1186/1478-7547-4-14.

    Article  PubMed  PubMed Central  Google Scholar 

  141. The Health Foundation. Star: Socio-Technical Allocation of Resources. Guidance document. 2013. Available from: Accessed 9 Aug 2017.

  142. The Health Foundation. Looking for value in hard times. How a new approach to priority setting can help improve patient care while making savings. 2012. Available from: Accessed 9 Aug 2017.

  143. Airoldi M, Morton A, Smith J, Bevan G. Healthcare prioritisation at the local level: a socio-technical approach. Priority Setting for Population Health 2011. Available from:,%20G_Bevan_Healthcare_prioritisation_local_2011_Bevan_Healthcare_prioritisation_local_author.pdf. Accessed 9 Aug 2017.

  144. Goetghebeur MM, Wagner M, Khoury H, Levitt RJ, Erickson LJ, Rindress D. Evidence and value: impact on DEcisionMaking--the EVIDEM framework and potential applications. BMC Health Serv Res. 2008;8:270. doi:10.1186/1472-6963-8-270.

    Article  PubMed  PubMed Central  Google Scholar 

  145. Browman GP, Manns B, Hagen N, Chambers CR, Simon A, Sinclair S. 6-STEPPPs: a modular tool to facilitate clinician participation in fair decisions for funding new cancer drugs. J Oncol Practice Am Soc Clin Oncol. 2008;4(1):2–7. doi:10.1200/jop.0812001.

    Article  Google Scholar 

  146. Baltussen R, Mikkelsen E, Tromp N, Hurtig A, Byskov J, Olsen O, et al. Balancing efficiency, equity and feasibility of HIV treatment in South Africa - development of programmatic guidance. Cost effectiveness and resource allocation : C/E. 2013;11(1):26. doi:10.1186/1478-7547-11-26.

    Article  PubMed  PubMed Central  Google Scholar 

  147. Youngkong S, Tromp N, Chitama D. The EVIDEM framework and its usefulness for priority setting across a broad range of health interventions. Cost effectiveness and resource allocation: C/E. 2011;9:8. doi:10.1186/1478-7547-9-8.

  148. Prasad V, Ioannidis JP. Evidence-based de-implementation for contradicted, unproven, and aspiring healthcare practices. Implement Sci. 2014;9:1. doi:10.1186/1748-5908-9-1.

    Article  PubMed  PubMed Central  Google Scholar 

  149. Guindo LA, Wagner M, Baltussen R, Rindress D, van Til J, Kind P, et al. From efficacy to equity: literature review of decision criteria for resource allocation and healthcare decisionmaking. Cost effectiveness and resource allocation: C/E. 2012;10(1):9. doi:10.1186/1478-7547-10-9.

  150. Tromp N, Baltussen R. Mapping of multiple criteria for priority setting of health interventions: an aid for decision makers. BMC Health Serv Res. 2012;12:454. doi:10.1186/1472-6963-12-454.

    Article  PubMed  PubMed Central  Google Scholar 

  151. Gibson JL, Martin DK, Singer PA. Setting priorities in health care organizations: criteria, processes, and parameters of success. BMC Health Serv Res. 2004;4(1):25. doi:10.1186/1472-6963-4-25.

    Article  PubMed  PubMed Central  Google Scholar 

  152. Leggat SG, Scheil W, Williams H, Kerin K. Tools for priority setting: lessons from South Australia. Aust Health Rev. 2006;30(1):65–72.

    PubMed  Google Scholar 

  153. Robinson S, Dickinson H, Williams I, Freeman T, Rumbold B, Spence K. Setting priorities in health: a study of English primary care trusts: health services management Centre, University of Birmingham and the Nuffield trust, 2011.

    Google Scholar 

  154. Golan O, Hansen P. Which health technologies should be funded? A prioritization framework based explicitly on value for money. Isr J Health Policy Res. 2012;1(1):44. doi:10.1186/2045-4015-1-44.

    Article  PubMed  PubMed Central  Google Scholar 

  155. Ruano-Ravina A, Velasco-Gonzalez M, Varela-Lema L, Cerda-Mota T, Ibargoyen-Roteta N, Gutierrez-Ibarluzea I et al. Identification, prioritisation and assessment of obsolete health technologies. A methodolgical guideline. HTA Reports: avalia-t No. 2007/01: Galician Health Technology Assessment Agency, 2009.

  156. Vachon B, Desorcy B, Camirand M, Rodrigue J, Quesnel L, Guimond C, et al. Engaging primary care practitioners in quality improvement: making explicit the program theory of an interprofessional education intervention. BMC Health Serv Res. 2013;13:106. doi:10.1186/1472-6963-13-106.

    Article  PubMed  PubMed Central  Google Scholar 

  157. Oosthuizen C, Louw J. Developing program theory for purveyor programs. Implement Sci. 2013;8:23. doi:10.1186/1748-5908-8-23.

    Article  PubMed  PubMed Central  Google Scholar 

  158. Huijbregts MP, Kay T, Klinck B. Theory-based programme development and evaluation in physiotherapy. Physiother Can. 2008;60(1):40–50. doi:10.3138/physio/60/1/40.

    Article  PubMed  PubMed Central  Google Scholar 

  159. Donaldson SI. Program theory-driven evaluation science: strategies and applications. New York: Lawrence Erlbaum Associates; 2007.

    Google Scholar 

  160. Funnell SC, Rogers PJ. Purposeful program theory; 2011.

    Google Scholar 

  161. Wilder Research. Program theory and logic models. 2009.

    Google Scholar 

  162. Knowlton LW, Phillips CC. The Logic Model Guidebook. Better Strategies for Great Results. Second ed. Thousand Oaks, California: SAGE Publications, Inc 2013.

  163. W.K. Kellogg Foundation. Logic model development guide. Using logic models to bring together planning, evaluation, and action. 2004.

    Google Scholar 

  164. McLaughlin JA, Jordan GB. Logic models: a tool for telling your programs performance story. Eval Program Plann. 1999;22(1):65–72. doi:

    Article  Google Scholar 

  165. Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’care. Lancet. 2003;362(9391):1225–30.

    Article  PubMed  Google Scholar 

  166. Grol R, Wensing M. Eccles Me. Improving practice: a systematic approach to implementation of change in patient care. Oxford: Elsevier Science; 2004.

    Google Scholar 

  167. Health NSW. A framework for building capacity to improve health. Better health Centre. Sydney: NSW Health Department; 2001.

    Google Scholar 

  168. Baker DW, Qaseem A, Reynolds PP, Gardner LA, Schneider EC. Design and use of performance measures to decrease low-value services and achieve cost-conscious care. Ann Intern Med. 2013;158(1):55–9. doi:10.7326/0003-4819-158-1-201301010-00560.

    Article  PubMed  Google Scholar 

  169. Montini T, Graham ID. “Entrenched practices and other biases”: unpacking the historical, economic, professional, and social resistance to de-implementation. Implementation science : IS. 2015;10:24. doi:10.1186/s13012-015-0211-7.

    Article  PubMed  PubMed Central  Google Scholar 

  170. Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, et al. Lost in knowledge translation: time for a map? J Contin Educ Heal Prof. 2006;26(1):13–24. doi:10.1002/chp.47.

    Article  Google Scholar 

  171. Robinson S, Glasby J, Allen K. It ain't what you do it's the way that you do it': lessons for health care from decommissioning of older people's services. Health Soc Care Community. 2013;21(6):614–22. doi:10.1111/hsc.12046.

    PubMed  Google Scholar 

  172. Noseworthy T, Clement F. Health technology reassessment: scope, methodology, & language. Int J Technol Assess Health Care. 2012;28(03):201–2. doi:doi:10.1017/S0266462312000359

    Article  PubMed  Google Scholar 

  173. Peacock SJ, Mitton C, Ruta D, Donaldson C, Bate A, Hedden L. Priority setting in healthcare: towards guidelines for the program budgeting and marginal analysis framework. Expert Rev Pharmacoecon Outcomes Res. 2010;10(5):539–52. doi:10.1586/erp.10.66.

    Article  PubMed  Google Scholar 

  174. de Vries EF, Struijs JN, Heijink R, Hendrikx RJ, Baan CA. Are low-value care measures up to the task? A systematic review of the literature. BMC Health Serv Res. 2016;16(1):405. doi:10.1186/s12913-016-1656-3.

    Article  PubMed  PubMed Central  Google Scholar 

  175. Gerdvilaite J, Nachtnebel A. Disinvestment: overview of disinvestment experiences and challenges in selected countries. HTA- Projektbericht., vol nr. 57. Vienna: Ludwig Boltzmann Institut für Health Technology Assessment; 2011.

    Google Scholar 

  176. Ovretveit J. Does improving quality save money? A review of evidence of which improvements to quality reduce costs to health service providers. London: The Health Foundation; 2009.

    Google Scholar 

  177. Malhotra N, Jacobson B. Save to invest. Developing criteria-based commissioning for planned health care in London, 2007.

    Google Scholar 

  178. Department of Human Services. Future directions for health technology uptake, diffusion and disinvestment in Victorian public health services. Victoria: Department of Human Services; 2007.

    Google Scholar 

  179. Rabin BA, Brownson RC, Haire-Joshu D, Kreuter MW, Weaver NL. A glossary for dissemination and implementation research in health. J Public Health Manag Pract. 2008;14(2):117–23. doi:10.1097/

    Article  PubMed  Google Scholar 

  180. van Bodegom-Vos L, Davidoff F, Marang-van de Mheen PJ. Implementation and de-implementation: two sides of the same coin? BMJ Qual Safety. 2016; doi:10.1136/bmjqs-2016-005473.

  181. Massatti RR, Sweeney HA, Panzano PC, Roth D. The de-adoption of innovative mental health practices (IMHP): why organizations choose not to sustain an IMHP. Admin Pol Ment Health. 2008;35(1–2):50–65. doi:10.1007/s10488-007-0141-z.

    Article  Google Scholar 

  182. Jilcott S, Ammerman A, Sommers J, Glasgow RE. Applying the RE-AIM framework to assess the public health impact of policy change. Ann Behav Med. 2007;34(2):105–14. doi:10.1080/08836610701564055.

    Article  PubMed  Google Scholar 

  183. Elshaug AG, Hiller JE, Tunis SR, Moss JR. Challenges in Australian policy processes for disinvestment from existing, ineffective health care practices. Aust New Zealand Health Policy. 2007;4:23. doi:10.1186/1743-8462-4-23.

    Article  PubMed  PubMed Central  Google Scholar 

  184. Healthcare Improvement Scotland. What approaches have been taken and efforts made to ensure public involvement in decision making relating to potential disinvestment in healthcare interventions and technologies?. In: Public involvement in decision making relating to potential: Technologies scoping report 16 2013. Accessed 9 Aug 2017.

  185. Garcia-Armesto S, Campillo-Artero C, Bernal-Delgado E. Disinvestment in the age of cost-cutting sound and fury. Tools for the Spanish National Health System. Health Policy. 2013;110(2–3):180–5. doi:10.1016/j.healthpol.2013.01.007.

    Article  PubMed  Google Scholar 

  186. Hughes D, Ferner R. New drugs for old: disinvestment and NICE. BMJ. 2010;340 doi:10.1136/bmj.c572.

  187. Centre for Health Economics Research and Evaluation. Reducing the use of ineffective health care interventions. Working paper 2010/5. Sydney: University of Technology Sydney; 2010.

    Google Scholar 

  188. Eddama O, Coast J. A systematic review of the use of economic evaluation in local decision-making. Health Policy. 2008;86(2–3):129–41. doi:10.1016/j.healthpol.2007.11.010.

    Article  PubMed  Google Scholar 

  189. Bryan S, Mitton C, Donaldson C. Breaking the addiction to technology adoption. Health Econ. 2014;23(4):379–83. doi:10.1002/hec.3034.

    Article  PubMed  Google Scholar 

  190. Baghbanian A, Torkfar G. Economics and resourcing of complex healthcare systems. Aust Health Rev. 2012;36(4):394–400. doi:10.1071/ah11041.

    PubMed  Google Scholar 

  191. Wulff KC, Miller FG, Pearson SD. Can coverage be rescinded when negative trial results threaten a popular procedure? The ongoing saga of vertebroplasty. Health Aff (Millwood). 2011;30(12):2269–76. doi:10.1377/hlthaff.2011.0159.

    Article  Google Scholar 

  192. Evans BA, Snooks H, Howson H, Davies M. How hard can it be to include research evidence and evaluation in local health policy implementation? Results from a mixed methods study. Implementation science: IS. 2013;8:17. doi:10.1186/1748-5908-8-17.

  193. Hamilton S, Huby G, Tierney A, Powell A, Kielmann T, Sheikh A, et al. Mind the gap between policy imperatives and service provision: a qualitative study of the process of respiratory service development in England and Wales. BMC Health Serv Res. 2008;8:248. doi:10.1186/1472-6963-8-248.

    Article  PubMed  PubMed Central  Google Scholar 

  194. National Health and Medical Research Council. Ethical considerations in quality assurance and evaluation activities. Canberra: Commonwealth of Australia; 2014.

    Google Scholar 

Download references


The authors would like to thank Violet Marion and Kelly Allen for assistance in development and revision of the framework, Tari Turner for critical comments on the manuscript, and Richard King for co-supervision of CH’s PhD.


The SHARE Program was funded by Monash Health and the Victorian Department of Human Services. No conditions related to the project or subsequent publications were imposed. This review was undertaken as part of an unfunded PhD.

Availability of data and materials

Many of the datasets supporting the conclusions of the articles in the SHARE series are included within the articles and/or the accompanying additional files. Some datasets provide information for more than one article and are only provided once; where they are not included within an article and/or the accompanying additional file, the relevant citations to the articles in which they are provided are included. Datasets have not been made available where it is impossible to de-identify individuals due to the nature of survey or interview responses or where the data is published in confidential internal reports.

Author information

Authors and Affiliations



CH and SG developed the conceptual approach. CH undertook the literature review and drafted the framework and initial manuscript. SG provided critical revisions to both. AGE provided critical revisions and proposed refinements and changes to scope. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Claire Harris.

Ethics declarations

Authors’ information

CH was the Director of the Centre for Clinical Effectiveness and the SHARE Program Director. CH undertook this review and completed the SHARE publications as part of an unfunded PhD. SG is Professorial Fellow in the Monash University School of Public Health and Preventive Medicine and co-supervisor of CH’s PhD. AGE is Professor of Health Policy, HCF Research Foundation Professorial Research Fellow, and Co-Director of the Menzies Centre for Health Policy at The University of Sydney and Senior Fellow with the Lown Institute in the USA.

Ethics approval and consent to participate

The Monash Health Human Research and Ethics Committee (HREC) approved the SHARE program as a Quality Assurance activity. Further ethical review was not required as the program met the following criteria [194]:

“The data being collected and analysed is coincidental to standard operating procedures with standard equipment and/or protocols;

The data is being collected and analysed expressly for the purpose of maintaining standards or identifying areas for improvement in the environment from which the data was obtained;

The data being collected and analysed is not linked to individuals; and

None of the triggers for consideration of ethical review are present.” [194]

Participation was based on the ‘opt-out approach’ [194]. “The opt-out approach is a method used in the recruitment of participants into an activity where information is provided to the potential participant regarding the activity and their involvement and where their participation is presumed unless they take action to decline to participate.” [194] Consent to participate was approved by the HREC based on the following criteria:

▪ Health care providers, managers, consumer representatives, and officers within government health departments will be informed about the project and the processes and invited to participate.

▪ Participation in interviews, workshops and/or surveys will be considered to be implied consent.

These conditions were met.

Consent for publication

Not applicable

Competing interests

AGE receives salary support as the HCF Research Foundation Professorial Research Fellow, and holds research grants from The Commonwealth Fund and Australia’s National Health and Medical Research Council (ID 1109626 and 1,104,136). AGE receives consulting/sitting fees from Cancer Australia, the Capital Markets Cooperative Research Centre-Health Quality Program, NPS MedicineWise (facilitator of Choosing Wisely Australia), The Royal Australasian College of Physicians (facilitator of the EVOLVE program) and the Australian Commission on Safety and Quality in Health Care, and as a member of the Australian Government Department of Health’s Medicare Benefits Schedule Review Taskforce.

CH and SG declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1:

Principles for resource allocation. (PDF 697 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Harris, C., Green, S. & Elshaug, A.G. Sustainability in Health care by Allocating Resources Effectively (SHARE) 10: operationalising disinvestment in a conceptual framework for resource allocation. BMC Health Serv Res 17, 632 (2017).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: