The cirrhosis care Alberta (CCAB) protocol: implementing an evidence-based best practice order set for the management of liver cirrhosis - a hybrid type I effectiveness-implementation trial
BMC Health Services Research volume 20, Article number: 558 (2020)
Liver cirrhosis is a leading cause of morbidity, premature mortality and acute care utilization in patients with digestive disease. In the province of Alberta, hospital readmission rates for patients with cirrhosis are estimated at 44% at 90 days. For hospitalized patients, multiple care gaps exist, the most notable stemming from i) the lack of a structured approach to best practice care for cirrhosis complications, ii) the lack of a structured approach to broader health needs and iii) suboptimal preparation for transition of care into the community. Cirrhosis Care Alberta (CCAB) is a 4-year multi-component pragmatic trial which aims to address these gaps. The proposed intervention is initiated at the time of hospitalization through implementation of a clinical information system embedded electronic order set for delivering evidence-based best practices under real-world conditions. The overarching objective of the CCAB trial is to demonstrate effectiveness and implementation feasibility for use of the order set in routine patient care within eight hospital sites in Alberta.
A mixed methods hybrid type I effectiveness-implementation design will be used to evaluate the effectiveness of the order set intervention. The primary outcome is a reduction in 90-day cumulative length of stay. Implementation outcomes such as reach, adoption, fidelity and maintenance will also be evaluated alongside other patient and service outcomes such as readmission rates, quality of care and cost-effectiveness. This theory-based trial will be guided by Normalization Process Theory, Consolidated Framework on Implementation Research (CFIR) and the Reach-Effectiveness-Adoption-Implementation-Maintenance (RE-AIM) Framework.
The CCAB project is unique in its breadth, both in the comprehensiveness of the multi-component order set and also for the breadth of its roll-out. Lessons learned will ultimately inform the feasibility and effectiveness of this approach in “real-world” conditions as well as adoption and adaptation of these best practices within the rest of Alberta, other provinces in Canada, and beyond.
Liver cirrhosis is a chronic condition that results from vascular and hepatocellular injury, and leads to progressive hepatic fibrosis. It is a major cause of morbidity and premature mortality in patients with digestive disease [1, 2]. Caused by a range of potential insults including alcohol, hepatitis B and C, and non-alcoholic fatty liver disease, cirrhosis is characterized by liver related complications from portal hypertension and hepatic insufficiency (i.e. ascites, variceal bleeding, hepatic encephalopathy, infections) [3, 4]. Many patients also have broader health needs that complicate their presentation including frailty, addictions, psychosocial challenges, and financial and employment instability [5,6,7]. Not surprisingly, this progressive, largely incurable disease results in an exceedingly poor quality of life for patients and their caregivers [8, 9] as low as that reported for patients with advanced metastatic cancer .
A number of recent studies have brought attention to the high rates of acute care and resource utilization associated with cirrhosis [5, 11,12,13,14,15]. In a recent prospective cohort study of 14 centers across North America, 90-day readmission rates were reported at 53% . This data parallels provincial Alberta Health Services (AHS) administrative data for a 1-year period (2015–2016) which revealed readmission rates of 44% at 90 days  (Carbonneau M, Davyduke T, Tandon P, Ma M, Den Heyer V, Newnham K, et al: Impact of Specialized Multidisciplinary Care on Cirrhosis Outcomes and Acute Care Utilization, unpublished). AHS data analysis over that time period reported a mean hospital length of stay of 13.7 days and an annual inpatient cost of approximately $120 million . Notably, the costs for cirrhosis were comparable to the $131 million in costs for chronic obstructive pulmonary disease in the same year, a condition already identified as a priority area for many healthcare organizations including AHS . At $28,205, the mean cost per admission for Albertans with cirrhosis was 3.5 times the cost of a standard hospital stay.
In response to this “crisis in the making” , there has been much attention focused on strategies to reduce care gaps, improve quality of care and reduce readmissions in patients with cirrhosis [19,20,21]. For inpatients, these gaps can be summarized into three priority areas: i) management of cirrhosis complications, ii) management of broader health needs such as frailty, alcohol use disorder (AUD) and advance care planning and iii) preparation for transition of care into the community, including structured education and timely follow-up post discharge. There has been extensive data to support the presence of these care gaps in cirrhosis and emerging data to support the positive impact of solutions in each of these areas. For example, despite robust evidence that guideline-based care improves patient outcomes, including for those patients with cirrhosis, evidence-based guidelines are only utilized in 30–60% of patients [22,23,24,25]. An example of a successful solution in this area has been an electronic order set intervention implemented by Tapper et al. focused on inpatient management of hepatic encephalopathy and prophylaxis against spontaneous bacterial peritonitis. This intervention reduced 30-day readmission rates by 40% . With regards to gaps in broader health needs, although greater than 50% of cirrhosis-related admissions can be attributed to alcohol , and it is clear that abstinence reduces hepatic decompensation, readmissions and mortality, in two recent series < 15% of patients received treatment for AUD [27, 28]. Similarly, malnutrition, sarcopenia and frailty are strong predictors for morbidity and mortality in patients with cirrhosis . The management of malnutrition has been associated with lower hospital readmissions but structured care in this area is infrequent [30,31,32]. Lastly, there are gaps in in preparing patients for transition of care into the community. While it is clear that education improves patient knowledge, ability to self-manage their disease, and can even reduce the risk of re-hospitalization in populations such as heart failure, no hospital site in Alberta offers standardized discharge teaching to patients with cirrhosis [33, 34]. Moreover, the minority of patients have follow-up appointments arranged with primary and/or specialty care prior to discharge despite evidence that timely follow-up reduces mortality .
Standardized order sets represent a potential vehicle for care optimization. Clinical order sets are pre-defined templates that standardize and expedite the ordering process, guiding clinicians to facilitate guideline-based care [36, 37]. The use of standardized order sets, particularly those incorporated into an electronic clinical information system (CIS), have been associated with a reduction in in-hospital mortality, improvement in the delivery of care, and increased adherence to clinical practice guidelines [38,39,40,41,42]. While there have been several studies examining the impact of standardized order sets on the management of individual cirrhosis complications including variceal hemorrhage, hepatic encephalopathy and spontaneous bacterial peritonitis, to date we are unaware of a study that has covered a comprehensive list of cirrhosis complications and extended beyond these complications to address selected broader health needs and optimization of the transition of care into the community [43,44,45].
Aims and objectives
The primary aim of this study is to evaluate the effectiveness of the Cirrhosis Care Alberta (CCAB) order set intervention in reducing 90-day cumulative length of stay (LOS). As this is a hybrid study, we will also assess the contextual factors that affect implementation in a real-world clinical setting (Fig. 1). Our research questions address the effectiveness of the intervention in addition to implementation processes and outcomes.
As compared to usual care, will a CIS facilitated rollout of the CCAB order set:
… reduce the 90-day cumulative hospital length of stay for patients with cirrhosis?
… reduce the occurrence of other clinical outcomes including hospital admission rate, median length of stay, readmission rate at 30 and 90 days, time to readmission, emergency department (ED) visit rate, outpatient visit rate and mortality?
… increase the quality of care (i.e. proportion of pre-defined process and clinical outcome-based quality measures that are fulfilled)?
… improve the patient/caregiver experience with care?
… demonstrate cost-utility?
… have uptake/reach among eligible individuals at each hospital site?
What contextual factors will influence adoption, reach, implementation fidelity, implementation feasibility and maintenance of the CCAB order set rollout?
This 4-year pragmatic non-randomized study is guided by a hybrid type I effectiveness-implementation design. This study design evaluates the effects of the Cirrhosis Care Alberta (CCAB) order set intervention on relevant outcomes, while at the same time observing and collecting data on the implementation [46,47,48]. Pragmatic across multiple aspects of the Pragmatic-Explanatory Continuum Indicator Summary 2 (PRECIS-2) tool kit , and guided by various checklists from the EQUATOR network [50,51,52] (see Additional files 1, 2, 3), the intervention has been designed to be incorporated into clinical practice in a sustainable manner using existing infrastructure and the leadership of site champions (healthcare providers caring for patients with cirrhosis and site administrators). In view of contextual variations and other practical constraints, most notably the coincidental roll out of a unifying Epic® based CIS platform across the province, a staged implementation was chosen (Fig. 2) with three phases including engagement, preparation and implementation. These stages are further described in the “description of study phases and implementation strategies” section and in Fig. 1. The staged implementation and analysis allows comparison of the impact of the phases within and between study sites.
Guiding theoretical frameworks
The overarching evaluation of the order set implementation is guided by the Reach-Effectiveness-Adoption-Implementation-Maintenance framework (RE-AIM)  summarized in Table 2. The Normalization Process Theory (NPT) and Consolidated Framework for Implementation Research (CFIR) will be used to understand the implementation process in greater depth. NPT is a theory of implementation that aims to explain the behaviour of individuals and groups around embedding and sustaining a new practice, with specific focus given to explaining how the practice impacts relationships within the clinical setting, how it is integrated into normal workflow, and how it is understood by implementers . CFIR is a conceptual framework that provides a structured means of facilitating the design, implementation and evaluation of interventions across five domains: (i) intervention characteristics, (ii) outer setting, (iii) inner setting, (iv) characteristics of individuals, and (v) process of implementation . In conjunction with NPT, we will use CFIR to understand both individual and organizational contextual factors that influence implementation of the CCAB order set.
NPT allows exploration of the generative mechanisms (coherence, cognitive participation, collective action, reflexive monitoring), examples of self-organizing mechanisms in complex adaptive systems. It also incorporates an understanding of differences in context and how implementation variably happens in different settings, and over time .
Health care in the province of Alberta is publicly funded and includes access to medically necessary services for all residents. Alberta Health Services (AHS), is a single health authority responsible for decision-making and delivery of healthcare services at hundreds of sites in the province (hospitals, as well as various continuing care and community health programs). It is organized into five geographic zones (North, Edmonton, Central, Calgary and South) . In late 2019, AHS introduced a province-wide Epic® based CIS known as Connect Care to consolidate 1300 clinical information systems in use across the province. The implementation of Connect Care will facilitate consistent practices and access to patient medical records across hospital care sites in the province, and will be implemented in nine waves across all five provincial health zones from November 2019 until the Fall of 2022.
Eight hospital sites in Alberta have been selected to participate in the study. Implementation of the CCAB order set intervention at the sites will be facilitated by the Connect Care rollout. The selection of hospital sites has been guided by a consideration of (i) urban vs. rural communities, (ii) type of hospital (academic vs. community), and (iii) prevalence of cirrhosis. The selection has also considered the importance of having representation from the five health zones comprising AHS. Table 1 shows the selected hospitals in each zone as well as the 2017 administrative data estimates of the prevalence of cirrhosis in each zone.
Participant eligibility criteria
Adult patients (≥ 18 yrs.) admitted to a hospital site between January 2019 and January 2023, with a diagnosis of cirrhosis as determined by a validated ICD-10 based coding algorithm [58,59,60,61] will be included in the study cohort. After review by our local research ethics board, the need for direct patient consent has been waived for the development of the main study cohort.
Informed consent will be required for patients who participate in surveys, focus groups or interviews. Patient eligibility for these parts of the study will require a history of hospitalization during the study period with a diagnosis of cirrhosis based on compatible clinical presentation, FibroScan® and/or liver biopsy. Informed consent will also be required for healthcare providers (including physicians, nurses and other allied health) who participate in surveys, focus groups or interviews. Consenting healthcare providers will be eligible to participate if they are employed in Alberta and provide care to patients with cirrhosis.
Intervention overview – the CCAB order set
The CCAB intervention, a standardized CIS order set, was developed through an iterative process in collaboration with a team of over 100 stakeholders in Alberta, including clinical experts such as physicians, other health providers, and nursing staff, as well as operational leadership, and patient advisors. The order set, intended for use during hospitalization of any patient with cirrhosis, includes three core domains (Fig. 3) – i) cirrhosis complications including ascites, hepatic encephalopathy and variceal bleeding, ii) broader health needs such as frailty, alcohol use disorder management, and advance care planning and iii) preparation for transition of care into the community including orders for the provision of standardized education and booking timely follow-up prior to discharge.
Description of Study Phases & Implementation Strategies
The Expert Recommendations for Implementing Change (ERIC) includes a consensus-derived compilation of implementation strategies . Selected ERIC strategies will be utilized to support implementation of the CCAB order set into routine clinical practice throughout three distinct study phases: (i) engagement, (ii) preparation, and (iii) implementation.
Phase 1 - engagement
Phase 1 will take place from months 1 to 24, prior to the CCAB order set integration within the CIS at each site (see Fig. 2). During this engagement phase, baseline data will be collected, and readiness assessments will be performed at each of the eight study sites.
ERIC Strategies within this phase: Assess for readiness and identify barriers and facilitators, Identify and prepare champions, Develop educational materials
During planned site visits, readiness assessments will be carried out using qualitative methods including individual interviews and focus groups (guided by the constructs of NPT and CIFR), with the aim of understanding individual site context and identifying barriers and facilitators to implementation of the order set. Site visits will also be used to identify current practices in each of the three order set domains (cirrhosis complications, broader health needs, preparation for transition of care into the community). Site champions will be identified. These champions will come forward to support the work and facilitate implementation and ongoing communication between the project team and front-line staff. Finally, educational tools such as webinars and handouts focused on the three domains of the order set will be developed in collaboration with site champions and subject matter experts. Co-development of these tools will facilitate ongoing engagement at the sites and will support the implementation.
Phase 2 - preparation
The preparation phase of the study will commence within 6 months prior to the planned electronic order set implementation at each site (Fig. 2).
ERIC Strategies within this phase: Conduct education meetings with health care providers. Distribute educational materials.
Using the tools developed in Phase 1, the study team will facilitate pre-implementation education that considers the preferred timing and learning modalities of each study site, as well as the specific learning needs of individuals (e.g., physicians and nurses). Education will include review of the order set and its components. For example, education meetings for prescribers will involve teaching around AUD diagnosis and use of pharmacologic therapies to reduce relapse, while nurses will receive education about screening and having conversations with patients about AUD. At the request of our study sites, educational materials will be distributed in a variety of formats (videos, webinars, handouts) and will be hosted on easily accessible web-based platforms.
Phase 3 – implementation
The final phase of the study will commence with “go-live” of the CCAB order set in the provincial CIS at each site. The order set will be implemented using a staged approach, timing determined by timing of CIS implementation across the province (months 12–33). Outcome effectiveness measures will be collected during this phase.
ERIC Strategies within this phase: Audit and provide feedback. Capture and share local knowledge. Purposely re-examine the implementation. Tailor strategies.
As sites enter phase 3, site champions will be invited to participate in monthly learning sessions (conducted virtually with representatives from each site) to discuss their experience, including barriers, facilitators, and lessons learned. Collection of Key Performance Indicators (KPIs) will occur quarterly and be shared with sites on electronic scorecards representing audit and feedback. The scorecards will include a combination of standard KPIs (e.g. order set utilization, length of stay) and site prioritized KPIs (e.g. performing a screen for alcohol use disorder). The rationale for including site prioritized KPIs is to promote site engagement and ownership of practice change. In the learning session following the quarterly release of scorecards, the teams will share learnings with each other and develop action plans to tailor implementation strategies at their sites. These Plan-Do-Study-Act (PDSA) cycles will continue quarterly throughout the implementation phase.
This hybrid trial evaluation is guided by the RE-AIM framework as summarized in Table 2. The outcome measures and data analyses plans are classified into effectiveness and implementation outcomes in accordance with the research questions.
Primary: The cumulative 90-day hospital length of stay (LOS)
Clinical outcomes including hospital admission rate, median length of stay, readmission rate at 30 and 90 days, time to readmission, emergency department (ED) visit rate, outpatient visit rate and mortality
Health care utilization and cost utility
Quality of care as evaluated by quality measures from the three core order set domains
Patient and caregiver experience with the intervention
The proportion of cirrhosis patients being managed using the CCAB order set, measured as the number of order sets used/ the number of patients eligible for orders set use
The proportion of care providers who participate in education interventions for the order set, measured as the number of care providers who participate in education interventions for the order set / number invited
The proportion of hospital sites that use the order sets measured as the number of hospital sites that use the order sets/number of hospital sites enrolled
Qualitative evaluation of processes and determinants of adoption guided by the NPT and CFIR
Feasibility/ acceptability: Qualitative evaluation of processes and determinants of feasibility and acceptability of order set implementation from the perspectives of patients, caregivers and health care providers’ guided by the NPT and CFIR.
Fidelity: Proportion of prescribed orders that adhere to the three core domains of the standardized CCAB order set, measured as the number of order sets that use orders in each of the three core domains/number of patients eligible for order set use
Maintenance (Sustainability): Proportion of cirrhosis patients being managed using the CCAB order set 6 months after the research team has completed supported rollout (measurement similar to “reach”)
Sample size, data collection and analysis
Effectiveness outcomes - clinical outcomes, cost-utility
Consistent with the type 1 hybrid trial design, the overarching sample size calculation is based on the cumulative 90-day hospital length of stay (LOS) . Based on 2015/2016 AHS administrative data (4176 cirrhosis-attributed admissions for 2652 patients at study sites)  and CCAB implementation time per site, we estimate a pre-intervention cohort of 3975 patients and post-intervention cohort of 3975 patients (n = 7950). Our calculations tested our ability to detect a 3-day reduction in average 90-day cumulative LOS from our historically observed average of 13.7 days. Computer simulations re-sampled historic LOS data to assess power to detect proposed effects for plausible scenarios using interrupted time series models with random intercepts for sites and an over-dispersed Poisson distribution. A scenario that assumed that 50% of eligible patients would receive the pathway (our most conservative estimate) and that there would be a rapid and sustained uptake of the pathway in the two years following implementation had > 90% power to detect an effect in the three largest sites and the whole cohort .
Data collection and analysis
The index population will be identified at admission with a validated Canadian algorithm of ICD9 and ICD10 codes (overall accuracy 87%) . We will analyze administrative data (n = 7950) using linear effect modeling that accounts for patient factors and observation clustering within sites . We will use segmented regression analyses of interrupted time series to model 90-day cumulative LOS and hospitalization rate in each period , comparing post-intervention changes with pre-intervention secular trends . We will evaluate and account for autocorrelation or other serial dependencies in data. Each of five geographic zones and eight hospital sites will serve as its own control, enabling us to identify effectiveness within each jurisdiction. We will also combine site series into generalized linear mixed effects models , with a random intercept for each site, fixed effects for time, and an indicator variable for pre-and post-intervention periods for each site.
Cost-utility evaluation will estimate expected incremental cost per Quality Adjusted Life Year (QALY) gained by implementing CCAB, within the Net Benefit Regression framework . Analysis will control for differences in relevant patient cohort characteristics. The evaluation will adopt a health system perspective and within-study analysis (only study cohort costs and outcomes), comparing resource use and health outcomes (health-related quality of life, mortality) for CCAB and usual care cohorts. These will be combined to calculate within-study expected QALY for each cohort. Methods for economic evaluation (e.g., discount rate choice, uncertainty characterization, results presentation) will adhere to recent reference case recommendations (Canadian Agency for Drugs & Technologies in Healthcare) . Stochastic analysis will be implemented with non-parametric bootstraps. Results will be presented as Expected Net Health Benefit and Cost Effectiveness Acceptability Curve. We will also report Value of Information to characterize residual decision uncertainty on value to Alberta of CCAB spread and scale at study completion.
Effectiveness outcome - quality of care measures
We will randomly sample at least 50 charts per site. This will provide 95% confidence intervals with widths no greater than +/− 10% for percentages of patients meeting quality measures (QMs) within each site.
Data collection and analysis
Improvements in quality of care will be evaluated using QMs for each of the three CCAB order set domains  (see Table 3). The majority of QMs will be obtained from administrative and chart review data audits from randomly selected charts and will take place quarterly during the study. The data from these audits will be used to promote change at each site. The central data collection team will record survey results in a secure AHS database for analysis. We will use inverse sampling weights to estimate the percent of patients attaining the QMs for the province. Administrative data will be analyzed for clinical outcomes, with additional analyses to compare change in QMs based on patient-reported outcomes, using linear mixed effects models that account for patient factors and observation clustering within sites .
Effectiveness outcome - patient and caregiver experience with the intervention
Purposeful maximum variation sampling will be used to ensure recruitment of a diverse group of patients and caregivers (rural, urban, socioeconomic status) to explore their experiences with the intervention. We estimate needing 40–50 participants to achieve saturation, typical for this methodological approach and appropriate given the variation in cirrhosis disease experiences . This data will be supplemented by patient phone surveys done in the pre and post implementation periods where patients with cirrhosis admitted to study sites will be contacted 7–14 days post-discharge and administered a health-related quality of life measure (EQ-5D)  and care transitions survey (CTM-15)  to assess their experience with care. For patient phone surveys, we will take a convenience sample of patients pre and post implementation of the intervention based on the capacity of units to identify patients.
Data collection and analysis
The data collection will begin after 1–2 months post-implementation at each study site (to allow for intervention uptake). Patients and caregivers will be invited to participate in individual semi-structured qualitative interviews focusing on key areas such as self-management, self-efficacy for cirrhosis, and relationships with healthcare providers using qualitative description methods [70, 73, 74]. For comparable historical context, only patients with at least one hospital admission prior to implementation of the order set will be invited to participate. For participant convenience, interviews will be in-person or virtual, with these interviews being recorded and transcribed verbatim. Participant data will be coded into meaningful segments, then organized into areas of similar patterns or themes . We will examine areas of commonality and difference in thematic analysis based on factors such as demographics, socioeconomic status and cirrhosis/hospitalization experiences [76, 77]. Data collection and analysis will be concurrent and iterative to enable refinement of the recruitment process and semi-structured interview guide .
Over the course of the project we anticipate surveying approximately 280 service providers (physicians, nurses and other health professionals), estimates derived from the number of providers involved with the management of cirrhosis at the sites. The qualitative evaluation of implementation will use a similar sampling strategy with the evaluation of patients’ experience described previously.
Data collection and analyses
Implementation data will be collected from a variety of sources including AHS administrative database, chart reviews, provider surveys using validated questionnaires and interviews with physicians and nurses. The CCAB project repository and participant observations from throughout the project will be leveraged to obtain data on implementation context and processes. Implementation data will also be collected during monthly virtual learning session meetings and any on-site visits that occur during the study phases. The quantitative data will be analyzed descriptively to monitor reach, adoption and implementation fidelity following the PDSA cycles. Qualitative data will be analyzed using a framework analytic approach to evaluate the contextual factors impacting adoption, reach, implementation fidelity, implementation feasibility and maintenance of the CCAB order set rollout .
Across a range of populations, there is evidence to support that order set delivery via an electronic medical record reduces medical errors, improves adherence to clinical practice guidelines, and has a positive impact on patient outcomes including a reduction in patient mortality [40, 79,80,81]. There has also been promising data to support the impact of order set delivery in patients with cirrhosis. To date however, the majority of cirrhosis studies have focused on more of an explanatory research design, filling the gaps in only one or two cirrhosis associated complications at time [26, 43,44,45]. The current pragmatic study builds off of the excellent work that has already been done, adding several novel aspects in the implementation of a comprehensive and pragmatic best practice intervention for hospitalized patients with cirrhosis.
The CCAB order set is unique in that it includes a comprehensive set of best practice orders. In addition to including orders for nine of the top cirrhosis complications that patients are admitted for, uniquely, it also contains orders focused on broader health needs including enhanced diet and activity orders as well as a specific order panel for the management of alcohol use disorder. Moreover, it includes orders to optimize care coordination and transition of patients into the community with elements such timely notification of primary care providers and patient instructions for follow up appointments and home self-management. Additionally, the order set is supported by educational materials such as written and video materials developed for patients and providers with a goal of facilitating discharge education. It is anticipated that these educational materials will improve knowledge not only for patients, but also for nurses and will be valuable tools for additional scale and spread of the intervention once the current study is complete.
The scope of the study is broad, including all adult patients with cirrhosis admitted at eight hospital sites, with representation from each of the five zones across the province. Universal access to healthcare services within Alberta will reduce patient selection bias. Importantly, the study spans urban as well as suburban sites, allowing us to evaluate discrepancies in the patient populations, quality indicators and clinical outcomes that may be present across these sites. The CIS will be the vehicle for the order set intervention and the study timing is therefore aligned with the CIS roll-out. We anticipate that the qualitative interviews that will occur during the study period will provide a wealth of information about the acceptability and barriers that come up during such a large scale CIS roll-out. This will be potentially useful to other sites who are transitioning to a hospital CIS.
Another unique aspect from existing work is that the CCAB study uses a type I hybrid effectiveness-implementation design. A broad range of effectiveness outcomes will be evaluated including clinical outcomes, quality of care, health economic outcomes, health-related quality of life, and experience with the intervention. By using multi-methods quantitative and qualitative evaluation, the hybrid design allows for a concurrent evaluation of implementation strategies and contextual factors that impact the effectiveness of the intervention. Site champions will be crucial in supporting implementation and sustainability of the intervention into routine clinical practice. Diverse implementation strategies will be used to keep site champions and local teams engaged, and information about the success of these strategies will be captured using the hybrid study design. The multi-methods evaluation allows for patient and provider input to be prioritized throughout the project. The participatory, patient-focused approach to continuous evaluation and improvement of implementation strategies will ensure that the order set is optimally implemented to the benefit and satisfaction of patients.
As the study aims to implement in a complex real-world situation as opposed to a controlled setting, the Pragmatic-Explanatory Continuum Indicator Summary 2 (PRECIS-2) tool kit  has been applied to align the design towards a more pragmatic than an explanatory trial (Additional file 1). As with other pragmatic trials, several limitations to the study design exist, including the lack of randomization. The rollout of the CIS has been fixed by AHS and has by necessity determined when each site begins the intervention. Significant competing priorities within the organization such as the COVID-19 response will inevitably impact the implementation timelines. Moreover, although we are recording quality measures across all three domains of the order set, given the complexity of the intervention and implementation context, our ability to attribute causality to any one component of the order set will be limited.
In conclusion, the findings from this unique and ambitious project are expected to contribute to existing knowledge on the effectiveness and feasibility of implementation of best practices in cirrhosis care. It is anticipated that technical and allocative efficiency of care will be positively impacted, even within the 4 year study duration . Lessons learned and materials developed during the rollout of the CCAB intervention will serve as a framework for potential future spread of the intervention. With an aim to improve equitable access and enhance allocative efficiency of health care expenditure across the province of Alberta, we anticipate that the CCAB order set, supporting materials and implementation processes will represent an important and innovative step forward in closing the gaps in cirrhosis care not only in our province but also in various other contexts across Canada and the world [82, 83].
Availability of data and materials
The datasets analysed for the current study protocol are available from the corresponding author on reasonable request.
Alberta Health Services
Cirrhosis Care Alberta
Clinical Information System
Consolidated Framework for Implementation Research
EuroQol- 5 Dimension
Experts Recommendation Implementing Change
High Need, High Cost
International Classification of Diseases (10th Revision)
Key performance Indicators
Length of Stay (on admission)
Normalization Process Theory
Organizational Readiness to Implement Change
Pragmatic-Explanatory Continuum Indicator Summary 2
Tapper EB, Parikh ND, et al. BMJ (Clinical Research Ed). 2018;362:k2817 k.
McPherson S, Lucey MR, Moriarty KJ. Decompensated alcohol related liver disease: acute management. BMJ (Clinical Research Ed). 2016;352:i124 i.
Kim WR, Brown RS Jr, Terrault NA, El-Serag H. Burden of liver disease in the United States: summary of a workshop. Hepatology. 2002;36(1):227–42.
Durante A, Louis T, Meek J, Navarro V, Sofair A. The mortality burden of chronic liver disease may be substantially underestimated in the United States. Conn Med. 2008;72:389–92.
Stepanova M, De Avila L, Afendy M, Younossi I, Pham H, Cable R, et al. Direct and indirect economic burden of chronic liver disease in the United States. Clin Gastroenterol Hepatol. 2017;15(5):759–66 e5.
Nguyen NH, Khera R, Ohno-Machado L, Sandborn WJ, Singh S. Annual burden and costs of hospitalization for high-need, high-cost patients with chronic gastrointestinal and liver diseases. Clin Gastroenterol Hepatol. 2018;16(8):1284–92 e30.
Asrani SK, Larson JJ, Yawn B, Therneau TM, Kim WR. Underestimation of liver-related mortality in the United States. Gastroenterology. 2013;145(2):375–82 e1–2.
Chouaid C, Agulnik J, Goker E, Herder GJ, Lester JF, Vansteenkiste J, et al. Health-related quality of life and utility in patients with advanced non-small-cell lung cancer: a prospective cross-sectional patient survey in a real-world setting. J Thor Oncol. 2013;8(8):997–1003.
Stein D, Joulain F, Naoshy S, Iqbal U, Muszbek N, Payne KA, et al. Assessing health-state utility values in patients with metastatic colorectal cancer: a utility study in the United Kingdom and the Netherlands. Int J Color Dis. 2014;29(10):1203–10.
Peng JK, Hepgul N, Higginson IJ, Gao W. Symptom prevalence and quality of life of patients with end-stage liver disease: a systematic review and meta-analysis. Palliat Med. 2019;33(1):24–36.
Kim D, Cholankeril G, Li AA, Kim W, Tighe SP, Hameed B, et al. Trends in hospitalizations for chronic liver disease-related liver failure in the United States, 2005-2014. Liver Int. 2019;39(9):1661–71.
Neff GW, Duncan CW, Schiff ER. The current economic burden of cirrhosis. Gastroenterol Hepatol (N Y). 2011;7(10):661–71.
Mellinger JL, Shedden K, Winder GS, Tapper E, Adams M, Fontana RJ, et al. The high burden of alcoholic cirrhosis in privately insured persons in the United States. Hepatology. 2018;68(3):872–82.
Bajaj JS, Reddy KR, Tandon P, Wong F, Kamath PS, Garcia-Tsao G, et al. The 3-month readmission rate remains unacceptably high in a large north American cohort of patients with cirrhosis. Hepatology. 2016;64(1):200–8.
Asrani SK, Kouznetsova M, Ogola G, Taylor T, Masica A, Pope B, et al. Increasing health care burden of chronic liver disease compared with other chronic diseases, 2004-2013. Gastroenterology. 2018;155(3):719–29 e4.
CIHI. Data quality documentation, discharge abstract database-multi-year information. Canadian Institute for Health Information: Ottawa; 2018.
AHS Memorandum: Outcomes Improvement Initiative - COPD & Heart Failure [press release]. AHS, 2018.
Sherman M. Liver disease in Canada: a crisis in the making. Markham: Canadian Liver Foundation; 2013.
Tapper EB, Volk M. Strategies to reduce 30-day readmissions in patients with cirrhosis. Curr Gastroenterol Rep. 2017;19(1):1.
Tapper EB. Building effective quality improvement programs for liver disease: a systematic review of quality improvement initiatives. Clin Gastroenterol Hepatol. 2016;14(9):1256–65 e3.
Tapper EB, Halbert B, Mellinger J. Rates of and reasons for hospital readmissions in patients with cirrhosis: a multistate population-based cohort study. Clin Gastroenterol Hepatol. 2016;14(8):1181–8 e2.
Volk ML, Kanwal F. Quality of Care in the Cirrhotic Patient. Clin Transl Gastroenterol. 2016;7:e166.
Kredo T, Bernhardsson S, Machingaidze S, Young T, Louw Q, Ochodo E, et al. Guide to clinical practice guidelines: the current state of play. Int J Qual Health Care. 2016;28(1):122–8.
Ghaoui R, Friderici J, Visintainer P, Lindenauer PK, Lagu T, Desilets D. Measurement of the quality of care of patients admitted with decompensated cirrhosis. Liver Int. 2014;34(2):204–10.
Vogeli C, Kang R, Landrum MB, Hasnain-Wynia R, Weissman JS. Quality of care provided to individual patients in US hospitals: results from an analysis of national hospital quality Alliance data. Med Care. 2009;47(5):591–9.
Tapper EB, Finkelstein D, Mittleman MA, Piatkowski G, Chang M, Lai M. A quality improvement initiative reduces 30-day rate of readmission for patients with cirrhosis. Clin Gastroenterol Hepatol. 2016;14(5):753–9.
Rogal S, Youk A, Zhang H, Gellad WF, Fine MJ, Good CB, et al. Impact of Alcohol Use Disorder Treatment on Clinical Outcomes among Patients with Cirrhosis. Hepatology. 2019;71(6):2080-92.
Peeraphatdit T, Kamath PS, Karpyak VM, Davis B, Desai V, Liangpunsakul S, et al. Alcohol Rehabilitation Within 30 Days of Hospital Discharge Is Associated With Reduced Readmission, Relapse, and Death in Patients With Alcoholic Hepatitis. Clin Gastroenterol H. 2020;18(2):477.
Montano-Loza AJ. Clinical relevance of sarcopenia in patients with cirrhosis. World J Gastroenterol. 2014;20(25):8061–71.
Ney M, Abraldes JG, Ma M, Belland D, Harvey A, Robbins S, et al. Insufficient protein intake is associated with increased mortality in 630 patients with cirrhosis awaiting liver transplantation. Nutr Clin Pract. 2015;30(4):530–6.
Tandon P, Tangri N, Thomas L, Zenith L, Shaikh T, Carbonneau M, et al. A rapid bedside screen to predict unplanned hospitalization and death in outpatients with cirrhosis: a prospective evaluation of the clinical frailty scale. Am J Gastroenterol. 2016;111(12):1759–67.
Reuter B, Shaw J, Hanson J, Tate V, Acharya C, Bajaj JS. Nutritional assessment in inpatients with cirrhosis can be improved after training and is associated with lower readmissions. Liver Transpl. 2019;25(12):1790–9.
Volk ML, Fisher N, Fontana RJ. Patient knowledge about disease self-Management in Cirrhosis. Am J Gastroenterol. 2013;108(3):302–5.
Beg S, Curtis S, Shariff M. Patient education and its effect on self-management in cirrhosis: a pilot study. Eur J Gastroen Hepat. 2016;28(5):582–7.
Kanwal F, Asch SM, Kramer JR, Cao YM, Asrani S, El-Serag HB. Early outpatient follow-up and 30-day outcomes in patients hospitalized with cirrhosis. Hepatology. 2016;64(2):569–81.
Grissinger M. Guidelines for standard order sets. P T. 2014;39(1):10–50.
Ahmad A, Teater P, Bentley TD, Kuehn L, Kumar RR, Thomas A, et al. Key attributes of a successful physician order entry system implementation in a multi-hospital environment. J Am Med Inform Assoc. 2002;9(1):16–24.
Bates DW, Leape LL, Cullen DJ, Laird N, Petersen LA, Teich JM, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. Jama-J Am Med Assoc. 1998;280(15):1311–6.
Garg AX, Adhikari NKJ, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes - a systematic review. Jama-J Am Med Assoc. 2005;293(10):1223–38.
Ballard DJ, Ogola G, Fleming NS, Stauffer BD, Leonard BM, Khetan R, et al. Impact of a standardized heart failure order set on mortality, readmission, and quality and costs of care. Int J Qual Health C. 2010;22(6):437–44.
Haynes K, Linkin DR, Fishman NO, Bilker WB, Strom BL, Pifer EA, et al. Effectiveness of an information technology intervention to improve prophylactic antibacterial use in the postoperative period. J Am Med Inform Assn. 2011;18(2):164–8.
Schnipper JL, Liang CL, Ndumele CD, Pendergrass ML. Effects of a computerized order set on the inpatient Management of Hyperglycemia: a cluster-randomized controlled trial. Endocr Pract. 2010;16(2):209–18.
Johnson EA, Spier BJ, Leff JA, Lucey MR, Said A. Optimising the care of patients with cirrhosis and gastrointestinal haemorrhage: a quality improvement study. Aliment Pharm Ther. 2011;34(1):76–82.
Kijsirichareanchai K, Ngamruengphong S, Rakvit A, Nugent K, Parupudi S. The utilization of standardized order sets using AASLD guidelines for patients with suspected cirrhosis and acute gastrointestinal bleeding. Quality management in health care. 2013;22:146–51.
Mayorga CA, Rockey DC. Clinical utility of a standardized electronic order set for the Management of Acute Upper Gastrointestinal Hemorrhage in patients with cirrhosis. Clin Gastroenterol H. 2013;11(10):1342–8.
Riley WT. Rapid, Relevant, and Responsive Research. Ann Behav Med. 2015;49:S160.
Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217–26.
Landes SJ, McBain SA, Curran GM. Reprint of: an introduction to effectiveness-implementation hybrid designs. Psychiatry Res. 2020;283:112630.
Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. Bmj. 2015;350:h2147.
Chan AW, Tetzlaff JM, Gotzsche PC, Altman DG, Mann H, Berlin JA, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013;346:e7586.
Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;348:g1687.
Richards D. The EQUATOR network and website. Evid Based Dent. 2007;8(4):117.
Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–7.
May CR, Mair F, Finch T, MacFarlane A, Dowrick C, Treweek S, et al. Development of a theory of implementation and integration: normalization process theory. Implement Sci. 2009;4.
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4.
May CR, Johnson M, Finch T. Implementation, context and complexity. Implement Sci. 2016;11(1):141.
AHS map and Zone Overview. Alberta Health Services; 2017.
Lapointe-Shaw L, Georgie F, Carlone D, Cerocchi O, Chung H, Dewit Y, et al. Identifying cirrhosis, decompensated cirrhosis and hepatocellular carcinoma in health administrative data: A validation study. PLoS One. 2018;13(8):e0201120.
Niu BL, Forde KA, Goldberg DS. Coding algorithms for identifying patients with cirrhosis and hepatitis B or C virus using administrative data. Pharmacoepidem Dr S. 2015;24(1):107–11.
Goldberg D, Lewis JD, Halpern SD, Weiner M, Lo RV. Validation of three coding algorithms to identify patients with end-stage liver disease in an administrative database. Pharmacoepidem Dr S. 2012;21(7):765–9.
Nehra MS, Ma Y, Clark C, Amarasingham R, Rockey DC, Singal AG. Use of administrative claims data for identifying patients with cirrhosis. J Clin Gastroenterol. 2013;47(5):E50–E4.
Jelinski S. DHSCN supported data analysis for PRIHS IV CCAP submission; 2018.
Faris P. Power and sample size for CCAP. Director, Research Facilitation Analytics (DIMR), vol. 2018; 2018.
Bates D, Machler M, Bolker BM, Walker SC. Fitting linear mixed-effects models using lme4. J Stat Softw. 2015;67(1):1–48.
Biglan A, Ary D, Wagenaar AC. The value of interrupted time-series experiments for community intervention research. Prev Sci. 2000;1(1):31–49.
Penfold RB, Zhang F. Use of interrupted time series analysis in evaluating health care quality improvements. Acad Pediatr. 2013;13(6 Suppl):S38–44.
Hoch JS, Briggs AH, Willan AR. Something old, something new, something borrowed, something blue: a framework for the marriage of health econometrics and cost-effectiveness analysis. Health Econ. 2002;11(5):415–30.
Donnan PT, Dorward DWT, Mutch B, Morris AD. Development and validation of a model for predicting emergency admissions over the next year (PEONY) - a UK historical cohort study. Arch Intern Med. 2008;168(13):1416–22.
Kanwal F, Tapper EB, Ho C, Asrani SK, Ovchinsky N, Poterucha J, et al. Development of quality measures in cirrhosis by the practice metrics Committee of the American Association for the study of liver diseases. Hepatology. 2019;69(4):1787–97.
Kim H, Sefcik JS, Bradway C. Characteristics of qualitative descriptive studies: a systematic review. Res Nurs Health. 2017;40(1):23–42.
Nord E. Euroqol - health-related quality-of-life measurement - valuations of health states by the general public in Norway. Health Policy. 1991;18(1):25–36.
Coleman EA, Eilertsen TB, Smith JD, Frank JC, Thiare JN, Ward A, et al. Development and testing of a measure designed to assess the quality of care transitions. J Am Geriatr Soc. 2002;50(4):S7–S.
Sandelowski M. Whatever happened to qualitative description? Res Nurs Health. 2000;23(4):334–40.
Sullivan-Bolyai S, Bova C, Harper D. Developing and refining interventions in persons with health disparities: the use of qualitative description. Nurs Outlook. 2005;53(3):127–33.
Miles MB, Huberman AM. Qualitative data analysis: an expanded sourcebook. Thousand Oaks: SAGE Publications; 1994.
Neergaard MA, Olesen F, Andersen RS, Sondergaard J. Qualitative description - the poor cousin of health research? BMC Med Res Methodol. 2009;9:52.
Bradshaw C, Atkinson S, Doody O. Employing a qualitative description approach in health care research. Glob Qual Nurs Res. 2017;4:2333393617742282.
Palinkas LA, Aarons GA, Horwitz S, Chamberlain P, Hurlburt M, Landsverk J. Mixed method designs in implementation research. Adm Policy Ment Hlth. 2011;38(1):44–53.
Ansari S, Fung K, MacNeil SD, Nichols AC, Yoo J, Sowerby LJ. The use of standardized order sets to improve adherence to evidence-based postoperative management in major head and neck surgery. Eur Ann Otorhinolaryngol Head Neck Dis. 2018;135(5S):S107–S11.
Chan J, Shojania KG, Easty AC, Etchells EE. Does user-centred design affect the efficiency, usability and safety of CPOE order sets? J Am Med Inform Assoc. 2011;18(3):276–81.
Fleming NS, Ogola G, Ballard DJ. Implementing a standardized order set for community-acquired pneumonia: impact on mortality and cost. Joint Commission J Qual Patient Saf. 2009;35(8):414–21.
Cylus J, Papanicolas I, Smith PC, editors. Health system efficiency: How to make measurement matter for policy and management [Internet]. Copenhagen (Denmark): European Observatory on Health Systems and Policies; 2016. (Health Policy Series, No. 46.) Available from: https://www.ncbi.nlm.nih.gov/books/NBK436888/.
Palmer S, Torgerson DJ. Economics notes - Definitions of efficiency. Brit Med J. 1999;318(7191):1136.
The Digestive Health Strategic Clinical Network (L. Morrin, L. Reeb and Dr. S.VeldhuyzenVanZanten), the Enhancing Alberta Primary Care Research Network (Dr. L. Green), the Physician Learning Programs in Edmonton and Calgary (Dr. D. Campbell-Scherer, Dr. KW Burak), CCAB Patient advisory team (under the guidance of Mr. Derek Drager), Alberta SPOR unit (Consultation and Research Services Platform) and the large CCAB team of collaborators across Alberta and Internationally (Drs. J. Bajaj, E. Tapper, M. Volk) for guidance on initial process and material development.
This trial is funded through a grant by Alberta Innovates Health Solutions. Grant award number: RES0043017. Additional funding support from Alberta Health Services and the Alberta SPOR unit Pragmatic Clinical Trials Platform (RES0048995). The content reported herein are those of the authors. The funders had no role in the design of this study, decision to publish, nor in the writing of this protocol.
Ethics approval and consent to participate
Ethics approval for this study has been obtained from the Health Research Ethics Board of the University of Alberta (ID Pro00089501- April 9, 2019; and ID Pro00094054- September 24, 2019). Permission has also been obtained from the AHS Health Systems Access and the Northern Alberta Clinical Trials and Research Centre to carry out the study at the various study sites. Written and signed informed consent to participate will be obtained from all study participants prior to survey and interview data collection. Waiver of consent for administrative and chart review data has been obtained.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
PRagmatic Explanatory Continuum Indicator Summary (PRECIS-2). PRECIS-2 is a tool to help trialists designing clinical trials consider where they would like their trial to be on the pragmatic/explanatory continuum
The TIDieR (Template for Intervention Description and Replication) Checklist. TIDieR reports details of the intervention elements of the study
Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) Checklist. SPIRIT reports on items to address in a clinical trial protocol
About this article
Cite this article
Carbonneau, M., Eboreime, E.A., Hyde, A. et al. The cirrhosis care Alberta (CCAB) protocol: implementing an evidence-based best practice order set for the management of liver cirrhosis - a hybrid type I effectiveness-implementation trial. BMC Health Serv Res 20, 558 (2020). https://doi.org/10.1186/s12913-020-05427-8
- Order set
- Normalization process theory (NPT)
- Reach-effectiveness-adoption-implementation-maintenance (RE-AIM)
- Hybrid trial
- Consolidated framework on implementation research (CFIR)