Sustainability of healthcare innovations (SUSHI): long term effects of two implemented surgical care programmes (protocol)
- Stephanie M C Ament†1, 4Email author,
- Freek Gillissen†2, 3, 4Email author,
- José M C Maessen1, 5,
- Carmen D Dirksen4,
- Trudy van der Weijden1 and
- Maarten F von Meyenfeldt2, 3
© Ament et al.; licensee BioMed Central Ltd. 2012
Received: 3 October 2012
Accepted: 24 October 2012
Published: 23 November 2012
Two healthcare innovations were successfully implemented using different implementation strategies. First, a Short Stay Programme for breast cancer surgery (MaDO) was implemented in four early adopter hospitals, using a hospital-tailored implementation strategy. Second, the Enhanced Recovery After Surgery (ERAS) programme for colonic surgery was implemented in 33 Dutch hospitals, using a generic breakthrough implementation strategy. Both strategies resulted in a shorter hospital length of stay without a decrease in quality of care. Currently, it is unclear to what extent these innovative programmes and their results have been sustained three to five years following implementation. The aim of the sustainability of healthcare innovations (SUSHI) study is to analyse sustainability and its determinants using two implementation cases.
This observational study uses a mixed methods approach. The study will be performed in 14 hospitals in the Netherlands, from November 2010. For both implementation cases, the programme aspects and the effects will be evaluated by means of a follow-up measurement in 160 patients who underwent breast cancer surgery and 300 patients who underwent colonic surgery. A policy cost-effectiveness analysis from a societal perspective will be performed prospectively for the Short Stay Programme for breast cancer surgery in 160 patients. To study determinants of sustainability key professionals in the multidisciplinary care processes and implementation change agents will be interviewed using semi-structured interviews.
The concept of sustainability is not commonly studied in implementation science. The SUSHI study will provide insight in to what extent the short-term implementation benefits have been maintained and in the determinants of long-term continuation of programme activities.
Quality improvement strategies are used to change and improve healthcare. However, often healthcare innovations barely get incorporated in every day practice [1–4]. Implementing an innovation is a complex process. Studies show that 30-40% of patients do not receive healthcare according to evidence-based knowledge and 20-25% of the healthcare provided is not needed or potentially harmful [5–7]. There is a high level of agreement between all parties involved in healthcare that the quality of care could be improved. Currently, there is no consensus on how to achieve better care. A diversity of quality improvement strategies to change and improve healthcare routines has been proposed. However, there is not one strategy that is superior in its effect above other strategies in improving healthcare processes . Although magic bullets are lacking among the prevailing implementation strategies, evidence suggests that implementation strategies which are systematically tailored to the actual and perceived barriers and facilitators for change increase the success of implementation [8, 9].
Less is known on the long term effectiveness of implementation strategies. Even after successful implementation, practice seems to indicate that there is a tendency to return to old routines after implementation activities have ended. It can be argued that without consolidation of the achieved benefits, implementation strategies are a waste of time and money. Implementation strategies’ ‘good value for money’ is determined by its ability to implement evidence based clinical guidelines into daily routines and by the implementation costs . The implication of discontinued programmes is however not just financial but can also result in less optimal care and service for patients. Moreover, it can also diminish community trust and support for future programmes. Therefore, it is important that achieved benefits of a proven effective intervention are sustained after implementation.
Sustainability is a relatively new term in healthcare research, but has become an issue of growing interest. There is no uniform definition of sustainability in the literature . In its simplest form, it can be seen as ‘holding the gains’ and ‘evolving as required’. Different definitions regarding sustainability are used, for example “maintaining the health benefits ”, “continuation of the programme activities within an organisational structure ” and “building the capacity of a recipient community ” [11, 15, 16]. Sustainability is generally seen as a dynamic process of continuous improvement. In the current study we use the following definition: “Sustainability of change exists when a newly implemented innovation continues to deliver the achieved benefits over a longer period of time, certainly does not return to the usual processes and becomes ‘the way things are done around here’, until a better innovation comes along, even after the implementation project is no longer actively carried out [11, 17]”
Sustainability is obtaining its position in healthcare research, but existing work on sustainability has mainly been undertaken from a pragmatic perspective and was impoverished by a lack of process information . Several determinants can have a facilitating or impeding effect on sustainability. Important questions that still need to be answered are by what strategies particular healthcare innovations are implemented and sustained (or not) in particular contexts and settings, and whether these strategies can be improved.
Overview implementation cases
No of participating hospitals
Ultra Short stay programme after breast cancer surgery
ERAS programme in colonic surgery
Hospital tailored strategy
The Breakthrough Series
Implementation time span
2005 – 2007
2006 – 2009
Two measurement periods of six months, with six months of actual implementation in between
Three runs of each one year
In the present study we will assess sustainability of a Short Stay Programme in breast cancer surgery and the Enhanced Recovery After Surgery programme in colonic surgery, three to five years after their implementation.
To what extent have the achieved benefits of the Short Stay Programme in breast cancer surgery and the Enhanced Recovery After Surgery programme in colonic surgery been sustained?
What is the current policy cost-effectiveness of the Short Stay Programme in breast cancer surgery?
Which are the determinants of sustainability as perceived by the professionals of the Short Stay Programme in breast cancer surgery and the Enhanced Recovery After Surgery programme in colonic surgery?
The present study is an observational study using a mixed methods approach, applied to the two implementation cases. First, by means of a retrospective analysis of patient files we will analyse process and outcomes of care to assess the extent of sustainability. Second, the policy cost-effectiveness will be calculated for the Short Stay Programme based on prospective data from breast cancer surgery patients. The cost-effectiveness analysis will use the same methodology as in the primary implementation study . Third, the determinants of sustainability will be explored by means of in-depth semi-structured interviews with relevant stakeholders and external change agents. This study will be executed between November 2010 and November 2013.
Ethical approval and informed consent
The Medical Ethical Committee of the University of Maastricht has granted approval, METC 11-4-015.10. The privacy of the included patients is protected, and all data will be coded and processed anonymously. Medical files with explicit patient statements that their medical information should not be used for clinical research will not be included. Breast cancer patients eligible for the second research question in this study will be asked for informed consent for prospective collection of quality of life and cost data.
Hospitals participating in the breast cancer surgery case are those four hospitals which participated in the primary implementation strategy, covering four main organisational hospital settings in the Netherlands. The uptake of the Short Stay Programme in breast cancer surgery was successful in all four hospitals; after implementation the average proportion of patients treated in short stay was increased with 36%.
Hospitals participating in the colonic surgery case were selected out of the 33 hospitals which participated in the primary implementation strategy. Selection was based on the criterion that the implementation strategy was successfully executed in these hospitals, such that the chance of finding information on sustainability and its determinants is increased. Success was defined as follows: (1) a median hospital length of stay of six days or lower, (2) an overall protocol compliance above seventy percent, and (3) at least forty patients treated according to the ERAS programme. Ten hospitals were selected for the SUSHI study. Nine hospitals fulfilled the criteria and one hospital was selected on top of that, based on being a successful early adopter hospital.
Research question one: To what extent have the achieved benefits of the Short Stay Programme in breast cancer surgery and the Enhanced Recovery After Surgery programme in colonic surgery been sustained?
Data collection will be conducted among the last 160 consecutive patients who had been scheduled for breast cancer surgery in short stay (40 per centre) and the last 300 consecutive patients who underwent colonic surgery (30 per centre), who met the same inclusion criteria as in the primary implementation studies. Patients in the breast cancer surgery case are all patients aged over 18 years, diagnosed with breast cancer, and who underwent surgery. Patients whose physiology at diagnosis impeded participation in the short stay programme as assessed by the breast surgeon, anaesthesiologist and nurse, will be excluded from the study. Patients who could not rely on sufficient informal home care during the first night after surgery and patients with complaints that necessitated postoperative monitoring and were scheduled for at least one overnight stay, will also be excluded. Patients included in the colonic surgery case were all patients aged over 18 undergoing an elective colonic resection above the peritoneal reflection for both malignant and benign diseases. Patients undergoing emergency surgery and requiring an ileo- or colostomy are excluded.
Variables and measurements
Outcome indicators and baseline characteristics scored in both cases
Treated surgically in day care
Postoperative hospital length of stay
Treated surgically in overnight stay
Day functional recovery was reached
Eligible for surgery in short stay
Laparoscopic or open approach
Receiving breast conserving surgery
Different types of operations
Process indicators in both cases
No preoperative bowel preparation
Preoperative PreOp carbohydrate drink
Not planned for short stay admission despite fulfilling the inclusion criteria for short stay admission
Being offered home care nursing after breast cancer surgery
Perioperative warming (Bair hugger)
No abdominal drains placed during surgery
Nasogastric tube removed after surgery
Nutritional supplements postoperatively
Mobilisation > 15 minutes at day 0
Use of oral fluids >500 ml at day 0
Mobilisation > 3 hours at day 1
The reasons for discrepancy between fulfilling the inclusion criteria for short stay admission and being scheduled for inpatient admission
Intravenous fluid infusion stopped at day
Resumption of solid food at day 1
Removal of epidural analgesia on day 2
Oral laxatives postoperatively
The reasons for not being treated in short stay despite being scheduled for short stay
The reasons for not adhering to these different protocol elements
Descriptive statistics (proportions, mean, median) will be performed for both outcome data sets. The percentage adherence to the protocol will be calculated per programme element for both cases and an overall protocol adherence score will be calculated per patient. All statistical analyses will be performed using SPSS 18.0 (SPSS Inc. Chicago, Illinois, USA).
Research question two: What is the current policy cost-effectiveness of the Short Stay Programme in breast cancer surgery?
In this prospective part of the study 40 consecutive patients operated per centre (n = 160 (4 × 40)) will be included. Again we will use the same inclusion and exclusion criteria as in the primary implementation study.
Variables and measurements
Policy cost analysis
The cost analysis will be performed from a societal perspective, using the Dutch guidelines for cost calculations in healthcare . Policy costs represent both the programme and implementation costs . The Short Stay Programme in breast cancer surgery related direct and indirect healthcare and non-healthcare costs will be measured and valued. We will use a bottom up micro costing method which identifies and measures the healthcare products per patient . Costs will be expressed in 2012 Euros. Direct healthcare related resource use will be obtained using Case Record Forms. Direct costs outside healthcare and out of pockets costs will be collected through a cost diary filled out by patients. One day before surgery, patients will fill out the retrospective part of the cost book to determine the healthcare related costs for a period of two weeks before surgery. Patients will fill out the prospective part of the cost book during six weeks from the moment of discharge. This time horizon will cover most costs related to surgical treatment and is also used in the primary implementation study. To perform the cost calculations, the volumes of (healthcare) resource use are multiplied with the cost prices per unit of resource use. Cost prices from the primary study will be actualised and used in this study. Indirect costs are the productivity losses due to sick leave; these will be calculated using the friction costs method. A friction cost method confines productivity losses to the period needed to replace a sick worker.
Historical implementation costs of the Short Stay Programme in breast cancer surgery will be used in this study, as these costs have been made in the primary implementation study. The implementation costs were assessed as mean costs per patient.
Effectiveness will be determined through a generic health-related quality of life (HRQoL) instrument. In this study the EuroQol (EQ-5D) will be used to calculate quality-adjusted life years (QALYs) . The EQ-5D consists of five different dimensions (mobility, self-care, daily activities, pain/discomfort, depression/anxiety). Every dimension has three answer possibilities (no problems, some problems, severe problems), which can lead to 243 different health states. The health states as measured in this study will be used for utility score calculation, based on the UK tariff. The utility scores will be determined at four consecutive time points: one day before first surgery (at baseline), one day after first surgery, one week after first surgery and six weeks after first surgery.
In this study the current incremental cost-effectiveness ratio (ICER) will be calculated for the Short Stay Programme in breast cancer surgery versus care as usual before primary implementation. For calculation of the ICER, mean incremental policy costs will be divided by the mean incremental QALYs. Multiple imputations will be used to replace missing values with plausible estimates. Bootstrapping will be performed to determine 95% confidence intervals around cost differences between the Short Stay Programme in breast cancer surgery and care as usual. Bootstrapping will also be used to quantify the uncertainty around the ICER and will be performed using Excel 2003 . The results will be presented in an incremental cost-effectiveness plane in which the vertical axis will represent the incremental effects and the horizontal axis will show the incremental costs between the Short Stay Programme and care as usual before implementation. This will result in four quadrants: 1) southeast quadrant (SE) 2) northwest quadrant (NW) 3) southwest quadrant (SW) 4) northeast quadrant (NE). The cost-effectiveness acceptability curve will be calculated to present the probability of the Short Stay Programme being cost-effective for a range of ceiling ratios using Excel 2003. These analyses will be performed using SPSS 18.0 (SPSS Inc. Chicago, Illinois, USA). Sensitivity analyses will be performed to test the robustness of the results for changing several parameters, as well as subgroup analyses. In addition, current policy costs and effects of the Short Stay programme will be compared to policy costs and effects as calculated in the primary implementation study, and potential differences will be related to sustainability issues when applicable.
Research question three: Which are the determinants of sustainability as perceived by the professionals of the Short Stay Programme in breast cancer surgery and the Enhanced Recovery After Surgery programme in colonic surgery?
Professionals within the participating hospitals will be interviewed. The first interview will be conducted with the responsible surgeon. Following this interview approximately 1-2 key persons in the actual care process per hospital will be selected. The number of interviews may differ per hospital based on the qualitative research process. Before the start of the interview, the interviewees will be formally informed about the relevance, expected duration, confidentiality of the personal data and the permission to audio tape the interviews.
External change agents
External change agents involved in the primary implementation strategies will also be invited for an interview to explore their perspective on sustainability in relation to the implementation processes and results.
Variables and measurements
The interviews will be transcribed and independently coded by both researchers (SA and FG). Differences in coding will be discussed until consensus is reached. Research software NVivo will be used to analyse the data and progress in coding and analysis will be discussed in the research team.
This paper describes the protocol of a mixed methods observational study to gain more insight into the concept of sustainability and its determinants. Two healthcare innovations were implemented in a selection of Dutch hospitals three to five years ago, using different implementation strategies. A Short Stay Programme was implemented in breast cancer surgery in 4 early adopter hospitals by means of a hospital tailored implementation strategy, and the Enhanced Recovery After Surgery programme in colonic surgery was implemented in 33 hospitals by means of a generic implementation strategy. As a result of these efforts, both cases showed improved short-term results in terms of hospital length of stay. Currently, it is unclear to what extent these innovative programmes and their results have been sustained. In the current study, the extent of consolidation of the implemented programmes will be assessed in fourteen hospitals in the Netherlands; four hospitals will participate in the breast cancer surgery case and ten hospitals will participate in the colonic surgery case.
Strengths and limitations of the study
This study will specifically and systematically address the sustainability of two healthcare innovations three to five years following two different implementation strategies. Regarding the breast cancer case, research on sustainability is not limited to long term effects of the implementation strategy, but also comprises policy cost-effectiveness. In the primary study, a policy cost effectiveness analysis on the Short Stay Programme was performed. As a consequence of e.g. changes in programme elements or case-mix, the current cost-effectiveness may be different. A recalculation of the cost-effectiveness will be conducted which will partly be based on the detailed cost- and effectiveness data collected during the primary study.
Another strength is the possibility to explore perceived determinants of sustainability from two different implementation strategies. Both programmes have been implemented in a multidisciplinary hospital setting. By studying these two different programmes, a large pool of medical specialists as well as other professionals can be approached. As such, this study will offer a multi-disciplinary perspective on sustainability and its determinants. We have chosen for the Consolidated Framework Implementation Research (CFIR) model to examine the determinants and the concept of sustainability. This framework is based on several scientific theories and existing models and considers a wide scope of possible determinants of sustainability.
In this study a mixed method approach will be applied, combining quantitative data with qualitative data. In the quantitative analyses we will examine whether the implementation results have been sustained, while in the qualitative analyses we will examine the professionals view on possible determinants of sustainability. The determinants of sustainability will be explored by means of semi-structured interviews with stakeholders of the healthcare processes and external change agents. This study is not aimed to analyse independent determinants in a quantitative manner by means of a multivariate regression model because we currently lack well-defined hypotheses on what are the most important determinants of sustainability. In this study we therefore also chose for qualitative methods.
Furthermore, the outcome and care process data will be extracted from patient files. The practical advantage is that the participating professionals are not claimed for intensive data collection by means of completing case record forms (CRFs). Also, by auditing the files of the last 40 breast cancer patients operated, and of the last 30 colonic surgery patients, a Hawthorne effect will be prevented. This is the phenomenon that a team is improving its performance due to awareness of being monitored.
This study has also some limitations. The generalisability of our quantitative results regarding sustainability will be limited due to the hospital selection. An analysis of sustainability of the ERAS programme in colonic surgery including all 33 participating implementation hospitals would have been optimal. Although the ten hospitals were selected based on carefully chosen criteria, it cannot be ruled out that some of the other hospitals that initially were not successful in achieving the ERAS goals, continued their activities and ultimately implemented the protocol successfully. The evaluation of the effects of the Short Stay Programme in breast cancer surgery after implementation will be conducted among the same hospitals as during the primary study. However, these early adopter hospitals may not fully represent Dutch practice regarding breast cancer surgery.
The analysis of sustainability is based on extracting data from existing data files. Although a Hawthorne effect is thus prevented, a potential limitation is that some data of interest for this study may not be properly recorded.
Not much is known about long term effects of implementation strategies and the determinants of sustainability. In this study, we perform an analysis on sustainability and we will try to shed light on potentially relevant attributes of sustainable change. We will build on previous work and aim to expand the knowledge regarding sustainability and its determinants. The results of this study will be relevant for researchers, implementation experts, healthcare practitioners and policy makers when making decisions about implementation and sustainability resource allocation.
The authors thank ZonMW, the Netherlands Organisation for Health Research and Development and NWO, The Netherlands Organisation for Scientific Research for their financial support.
- Grol R, Wensing M: What drives change? Barriers to and incentives for achieving evidence-based practice. Med J Aust. 2004, 180: S57-S60.PubMedGoogle Scholar
- Wilson KD, Kurz RS: Bridging implementation and institutionalization within organizations: proposed employment of continuous quality improvement to further dissemination. J Public Health Manag Pract. 2008, 14: 109-116.View ArticlePubMedGoogle Scholar
- Stetler CB, McQueen L, Demakis J, Mittman BS: An organizational framework and strategic implementation for system-level change to enhance research-based practice: QUERI Series. Implement Sci. 2008, 3: 30-10.1186/1748-5908-3-30.View ArticlePubMedPubMed CentralGoogle Scholar
- Grimshaw JM, Eccles MP, Walker AE, Thomas RE: Changing physicians’ behavior: what works and thoughts on getting more things to work. J Contin Educ Health Prof. 2002, 22: 237-243. 10.1002/chp.1340220408.View ArticlePubMedGoogle Scholar
- Schuster MA, McGlynn EA, Brook RH: How good is the quality of health care in the United States?. Milbank Q. 1998, 76: 563-509.View ArticleGoogle Scholar
- Grol R, Grimshaw J: From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003, 362: 1225-1230. 10.1016/S0140-6736(03)14546-1.View ArticlePubMedGoogle Scholar
- Grimshaw J, Eccles M, Tetroe J: Implementing clinical guidelines: current evidence and future implications. J Contin Educ Health Prof. 2004, 24 (Suppl 1): S31-S37.View ArticlePubMedGoogle Scholar
- Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, Robertson N: Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2010, CD005470-3
- van Bokhoven MA, Kok G, van der Weijden T: Designing a quality improvement intervention: a systematic approach. Qual Saf Health Care. 2003, 12: 215-220. 10.1136/qhc.12.3.215.View ArticlePubMedPubMed CentralGoogle Scholar
- Sculpher M: Evaluating the cost-effectiveness of interventions designed to increase the utilization of evidence-based guidelines. Fam Pract. 2000, 17 (Suppl 1): S26-S31.View ArticlePubMedGoogle Scholar
- Shediac-Rizkallah MC, Bone LR: Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Educ Res. 1998, 13: 87-108. 10.1093/her/13.1.87.View ArticlePubMedGoogle Scholar
- Scheirer M: Is sustainability possible? A review and commentary on empirical studies of program sustainability. Am J Eval. 2005, 26: 320-347. 10.1177/1098214005278752.View ArticleGoogle Scholar
- Pluye P, Potvin L, Denis JL, Pelletier J: Program sustainability: focus on organizational routines. Health Promot Int. 2004, 19: 489-500. 10.1093/heapro/dah411.View ArticlePubMedGoogle Scholar
- Duckers ML, Wagner C, Vos L, Groenewegen PP: Understanding organisational development, sustainability, and diffusion of innovations within hospitals participating in a multilevel quality collaborative. Implement Sci. 2011, 6: 18-10.1186/1748-5908-6-18.View ArticlePubMedPubMed CentralGoogle Scholar
- Gruen RL, Elliott JH, Nolan ML, Lawton PD, Parkhill A, McLaren CJ, Lavis JN: Sustainability science: an integrated approach for health-programme planning. Lancet. 2008, 372: 1579-1589. 10.1016/S0140-6736(08)61659-1.View ArticlePubMedGoogle Scholar
- Rabin BA, Brownson RC, Haire-Joshu D, Kreuter MW, Weaver NL: A glossary for dissemination and implementation research in health. J Public Health Manag Pract. 2008, 14: 117-123.View ArticlePubMedGoogle Scholar
- NHS: Sustainability and its relationship with spread and adoption, General improvement skills. 2007, Conventry, UK: Improvement NIfIa edGoogle Scholar
- Wiltsey S, Kimberly J, Cook N, Calloway A, Castro F, Charns M: The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012, 7: 17-10.1186/1748-5908-7-17.View ArticleGoogle Scholar
- de Kok M, Dirksen CD, Kessels AG, van der Weijden T, van de Velde CJ, Roukema JA, Bell AV, van der Ent FW, von Meyenfeldt MF: Cost-effectiveness of a short stay admission programme for breast cancer surgery. Acta Oncol. 2010, 49: 338-346. 10.3109/02841861003610192.View ArticlePubMedGoogle Scholar
- de Kok M, Frotscher CN, van der Weijden T, Kessels AG, Dirksen CD, van de Velde CJ, Roukema JA, Bell AV, van der Ent FW, von Meyenfeldt MF: Introduction of a breast cancer care programme including ultra short hospital stay in 4 early adopter centres: framework for an implementation study. BMC Cancer. 2007, 7: 117-10.1186/1471-2407-7-117.View ArticlePubMedPubMed CentralGoogle Scholar
- de Kok M, Scholte RW, Sixma HJ, van der Weijden T, Spijkers KF, van de Velde CJ, Roukema JA, van der Ent FW, Bell AV, von Meyenfeldt MF: The patient’s perspective of the quality of breast cancer care. The development of an instrument to measure quality of care through focus groups and concept mapping with breast cancer patients. Eur J Cancer. 2007, 43: 1257-1264. 10.1016/j.ejca.2007.03.012.View ArticlePubMedGoogle Scholar
- de Kok M, van der Weijden T, Kessels A, Dirksen C, van de Velde C, Roukema J, van der Ent F, Bell A, von Meyenfeldt M: Implementation of an ultra-short-stay program after breast cancer surgery in four hospitals: perceived barriers and facilitators. World J Surg. 2008, 32: 2541-2548. 10.1007/s00268-007-9357-z.View ArticlePubMedGoogle Scholar
- Fearon KC, Ljungqvist O, Von Meyenfeldt M, Revhaug A, Dejong CH, Lassen K, Nygren J, Hausel J, Soop M, Andersen J, Kehlet H: Enhanced recovery after surgery: a consensus review of clinical care for patients undergoing colonic resection. Clin Nutr. 2005, 24: 466-477. 10.1016/j.clnu.2005.02.002.View ArticlePubMedGoogle Scholar
- Maessen J, Dejong CH, Hausel J, Nygren J, Lassen K, Andersen J, Kessels AG, Revhaug A, Kehlet H, Ljungqvist O, et al: A protocol is not enough to implement an enhanced recovery programme for colorectal resection. Br J Surg. 2007, 94: 224-231. 10.1002/bjs.5468.View ArticlePubMedGoogle Scholar
- Maessen JM, Dejong CH, Kessels AG, von Meyenfeldt MF: Length of stay: an inappropriate readout of the success of enhanced recovery programs. World J Surg. 2008, 32: 971-975. 10.1007/s00268-007-9404-9.View ArticlePubMedPubMed CentralGoogle Scholar
- Oostenbrink JB, Koopmanschap MA, Rutten FF: Standardisation of costs: the Dutch Manual for Costing in economic evaluations. Pharmacoeconomics. 2002, 20 (7): 443-454. 10.2165/00019053-200220070-00002.View ArticlePubMedGoogle Scholar
- Severens JL: Value for money of changing healthcare services? Economic evaluation of quality improvement. Qual Saf Health Care. 2003, 12: 366-371. 10.1136/qhc.12.5.366.View ArticlePubMedPubMed CentralGoogle Scholar
- The EuroQol Group: EuroQol--a new facility for the measurement of health-related quality of life. Health Policy. 1990, 16: 199-208.View ArticleGoogle Scholar
- Efron B: Introduction to the bootstrap. 1993, New York, USA: Chapman & HallView ArticleGoogle Scholar
- Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC: Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009, 4: 50-10.1186/1748-5908-4-50.View ArticlePubMedPubMed CentralGoogle Scholar
- Rogers E: Diffusion on innovations. 2003, New York, NY: Free Press, 5thGoogle Scholar
- Damschroder LJ, Hagedorn HJ: A guiding framework and approach for implementation research in substance use disorders treatment. Psychol Addict Behav. 2011, 25 (2): 194-205.View ArticlePubMedGoogle Scholar
- Simpson DD, Flynn PM: Moving innovations into treatment: A stage-based approach to program change. J Subst Abuse Treat. 2007, 33: 111-120. 10.1016/j.jsat.2006.12.023.View ArticlePubMedPubMed CentralGoogle Scholar
- Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O: Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004, 82: 581-629. 10.1111/j.0887-378X.2004.00325.x.View ArticlePubMedPubMed CentralGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/12/423/prepub