Skip to content

Advertisement

  • Research article
  • Open Access
  • Open Peer Review

Health outcomes measurement and organizational readiness support quality improvement: a systematic review

Contributed equally
BMC Health Services Research201818:1005

https://doi.org/10.1186/s12913-018-3828-9

  • Received: 29 June 2018
  • Accepted: 17 December 2018
  • Published:
Open Peer Review reports

Abstract

Background

Using outcome measures to advance healthcare continues to be of widespread interest. The goal is to summarize the results of studies which use outcome measures from clinical registries to implement and monitor QI initiatives. The second objective is to identify a) facilitators and/or barriers that contribute to the realization of QI efforts, and b) how outcomes are being used as a catalyst to change outcomes over time.

Methods

We searched the PubMed, EMBASE and Cochrane databases for relevant articles published between January 1995 and March 2017. We used a standardized data abstraction form. Studies were included when the following three criteria were fulfilled: 1) they relied on structural data collection, 2) when a structural and comprehensive QI intervention had been implemented and evaluated, and 3) impact on improving clinical and/or patient-reported outcomes was described. Data on QI strategies, QI initiatives and the impact on outcomes was extracted using standardized assessment tools.

Results

We included 21 articles, of which eight showed statistically significant improvements on outcomes using data from clinical registries. Out of these eight studies, the Chronic Care Model, IT application as feedback, benchmarking and the Collaborative Care Model were used as QI methods. Encouraging trends in realizing improved outcomes through QI initiatives were observed, ranging from improving teamwork, implementation of clinical guidelines, implementation of physician alerts and development of a decision support system. Facilitators for implementing QI initiatives included a high quality database, audits, frequent reporting and feedback, patient involvement, communication, standardization, engagement, and leadership.

Conclusion

This review suggests that outcomes collected in clinical registries are supportive to realize QI initiatives. Organizational readiness and an active approach are key in achieving improved outcomes.

Keywords

  • Clinical registries
  • Quality improvement
  • Value-based healthcare
  • Improvement science

Background

The use of clinical registries is considered crucial to systematically measure clinical outcomes in achieving better value for patients [1]. A clinical or patient registry is defined as “an organized system that uses observational study methods to collect uniform data (clinical data as structure, process and outcome measures) to evaluate specified outcomes for a population defined by a particular disease, condition, or exposure” [2]. Registries that are used for evaluating patient outcomes are used for the purpose of this review. The importance of clinical registries has been widely recognized as a tool to realize quality improvement (QI) and public accountability [1, 38]. Medical associations use clinical registries for collecting data using pre-defined measures in patients undergoing a certain procedure or for a specific disease [9]. In particular, feedback based on clinical registry data is used to identify and monitor improvement initiatives [10]. Therefore, registries are seen as a promising tool to achieve improvements in value for the patient by measuring outcomes [1]. A previous review on the structure, use and limitations of current clinical registries showed that registries and their respective measures are used for monitoring the work of health care providers, discussion platforms for QI, improving risk adjustment modelling and for improving preoperative risk profiling [11]. However, the current body of literature lacks insights into the extent to which the use of outcome measures from clinical registries, either when identifying, selecting or monitoring QI initiatives, can impact health outcomes.

With rising healthcare costs, service restrictions, differences in quality and costs, there is an increasing need for reform to improve value of healthcare [12]. Value in healthcare is defined as outcomes relative to costs [13]. Value-based health care aims at achieving higher value for patients while ensuring sustainability of the healthcare system by an efficient and effective delivery of care [14]. This goal is assumed to be achieved by measuring and using outcomes per medical condition for the identification of improvement potential across the full cycle of care [12]. Higher value for patients by measuring outcomes is one of the potential methods for improving quality of healthcare relative to the costs spent. For the purposes of this review, we only focused on outcome measures and not on the respective costs.

Quality of healthcare is generally assessed by using structure, process or outcome measures [15]. The latter provide insights into outcomes of a certain disease or several diseases, for instance on survival, functional status, and quality of life [16]. The aim of measuring outcomes is diverse; guiding clinical decision-making, initiating improvement interventions, benchmarking, monitoring, scientific research and public accountability. Measuring outcomes structurally and using them to identify possible improvements contributes to the aim of achieving higher value for patients [17].

The goal is to summarize the results of studies which use outcome measures from clinical registries to implement and monitor QI initiatives. For the purpose of this study, QI was defined as the application of a defined improvement process to achieve measurable improvement by implementing an improvement intervention. Registry data itself is not sufficient as they need QI methods in order to achieve actual improvement. The second objective is to identify a) facilitators and/or barriers that contribute to the realization of QI efforts, and b) how outcomes are being used as a catalyst to change outcomes over time.

Methods

A systematic review was conducted of studies published between January 1995 and March 2017. The search strategy was designed for PubMed, EMBASE and Cochrane databases. To identify evidence for the use of clinical registries to improve or contribute to patient health outcomes, the following PubMed Mesh terms were used to identify studies: mortality, patient outcome assessment and treatment outcome. These terms were combined with a variety of search terms related to QI and diverse disease specific registry studies. No specific patient group or study design was defined. Details of the complete search strategy are provided in the online supplementary content (Additional file 1: Appendix 1). Additional hand-searching has been conducted for systematic reviews on the subject during the review process. The hand-search was conducted in Google Scholar.

Inclusion and exclusion criteria

Studies were included when they met each of the following criteria: 1) published in peer-reviewed journals, 2) published in English, French or German, 3) the study actively implemented a strategy using outcome data to realize QI, 4) the study relied on structural data collection, and 5) the study evaluated the QI interventions realized. Whether a study made use of a QI effort, falling under criteria 3 and 5, was evaluated after reviewing the full text papers and was therefore not part of the search string. After title screening, included studies were evaluated on criteria 3 and 5. Studies were excluded when they analyzed the effect of new intervention(s) on outcomes (testing drugs, new techniques or the effect of an intervention) or when the data had solely been collected to evaluate an intervention in a clinical trial.

Data extraction and quality assessment

For the initial selection each reviewer reviewed a random set on first title, second abstract, and finally full text to determine eligibility. The full text articles were critically reviewed and judged by all reviewers. Any disagreement between reviewers was discussed by the full review team until consensus was achieved. The selected articles were evaluated using a standardized predesigned form listing whether the inclusion criteria were met.

A thorough review process was carried out for the data quality assessment, which consisted of the following three steps:

Step 1: Data abstraction

The Cochrane data abstraction form for intervention reviews (RCTs and non-RCTs) was used as a tool to extract data on study design and methodological quality (Additional file 1: Appendix 2) [18]. Furthermore, data on the target group, main results, main outcome measures, data source, geographical setting and funding sources was abstracted.

Step 2: Rigor of QI intervention

The included studies were evaluated using the Quality Improvement Minimum Quality Criteria Set (QI-MQCS) as a critical appraisal instrument, developed by the RAND Corporation (Additional file 1: Appendix 3) [19]. The QI-MQCS contains 16 domains to evaluate the QI intervention, resulting in a scoring system to evaluate whether this domain was met or not. The QI-MQSC did not introduce a threshold concerning acceptability of the quality of the papers. Therefore, we agreed on the following criteria in order to adequately interpret the QI-MQSC score. The study was considered to be of perfect quality (> 15 items ranked yes), good quality (> 12 items ranked yes), moderate quality (> 9 items ranked yes) and insufficient quality (≤9 items ranked yes).

Step 3: Rigor of data collection and analysis

In addition to the QI-MQCS, 13 items were added for further evaluation. Two questions (item 2 and 18) from the Downs & Black (1998) criteria were used to reflect on whether the main outcomes to be measured had been clearly described in the introduction or methods section and whether the statistical tests used to assess the main outcomes were appropriate [20]. In addition, three questions (item 10c, 11a and 11b) from the SQUIRE guidelines were used: 1) whether a method was employed for assessing completeness and accuracy of data, 2) whether quantitative methods were used to draw inferences from the data and 3) whether methods were applied for understanding variation within the data, including the effects of time as a variable [21]. Furthermore, it was evaluated how the included studies dealt with missing values, whether they performed audits, reported on secular trends, performed case-mix adjustments, whether clear inclusion and exclusion criteria had been defined for the patient population and when possible whether a power analysis was conducted.

In conclusion, the Cochrane data abstraction form was used to abstract data from the selected articles in order to identify changes in outcomes and facilitators. Data synthesis was guided by 1) the QI-MQCS results, 2) the merged and modified version of the Downs & Black (1998), SQUIRE guidelines, and self-developed questions. Due to the diversity of outcomes, a pooled effect of the results was not conducted.

Results

Search results and included studies

The final systematic search resulted in 11 524 records for initial screening; 117 articles were included to review the full text version of which 96 studies were excluded because they did not meet the inclusion criteria (Fig. 1) [22]. One additional article was included from a relevant systematic review, which emerged from hand-searching [23, 24]. Table 1 presents the characteristics of the 21 included studies. The studies focused on registries for the following patient groups; patients with diabetes [2431], children with chronic conditions [32], patients with lung cancer [33, 34], patients with cystic fibrosis [3537], patients with cardiac anomalies [38], patients undergoing cardiac surgery [3941], patients with acute myocardial infarction [42], and patients referred for home health services [43]. The majority of the registries presented voluntary participation [2527, 2931, 35, 36, 38, 4043]. Three registries required mandatory participation [28, 33, 34]. Most of the presented registries had the purpose of achieving QI [24, 25, 2834, 37, 39, 4143]. The remaining studies have introduced their clinical registry for research and educational purposes [26, 27, 35, 36, 38, 40, 44].
Fig. 1
Fig. 1

Flow diagram. Source: Authors’ analysis, format source from PRISMA [22]. a Exclusion criteria: 1. Studies published in peer-reviewed journals; 2. Studies published in English; 3. Did not actively implement a strategy making use of outcome data to realize quality improvement; 3. Did not relay on structural data collection; 5. Did not evaluate quality improvement interventions using data from outcome registries

Table 1

Characteristics of Included Studies (n = 21)

Characteristics

No. (%)

Geographical setting

 United States [24, 25, 2729, 31, 32, 35, 3741, 43, 44]

15 (71%)

 Sweden [26, 42]

2 (9.5%)

 Denmark [33, 34]

2 (9.5%)

 Germany [36]

1 (4.8%)

 Singapore [30]

1 (4.8%)

Target group

 Diabetes [2431]

8 (38.1%)

 Depression [44]

1 (4.8%)

 Children with chronic conditions [32]

1 (4.8%)

 Lung Cancer [33, 34]

2 (9.5%)

 Cystic fibrosis [3537]

3 (14.3%)

 Congenital heart disease [38]

1 (4.8%)

 Myocardial infarction [42]

1 (4.8%)

 Patients undergoing cardiac or cardiothoracic surgery [3941]

3 (14.3%)

 Patients referred for home health services [43]

1 (4.8%)

Study design

 Observational study [2931, 3337, 39, 41]

10 (47.6%)

 Randomized-Controlled Trial [24, 25, 27]

3 (14.3%)

 Case study [26, 28]

2 (9.5%)

 Cohort study [38]

1 (4.8%)

 Before and after study [32, 40, 42]

3 (14.3%)

 Quasi-experimental study [44]

1 (4.8%)

 Prospective evaluation study [43]

1 (4.8%)

Funding sources

 National funding [25, 27, 28, 31, 32, 36, 42, 43]

8 (38.1%)

 Private funding [24, 29, 35, 37, 44]

5 (23.8%)

 Unknown [26, 30, 33, 34, 3841]

8 (38.1%)

 Registry participation type

 

 Voluntary [2527, 2931, 35, 36, 38, 4043]

13 (62%)

 Mandatory [28, 33, 34]

3 (14.3%)

 Unknown [24, 32, 37, 39, 44]

5 (23.8%)

Registry purpose

 Quality improvement [24, 25, 2834, 37, 39, 4143]

14 (66.7%)

 Research and education [26, 27, 35, 36, 38, 40, 44]

7 (33.3%)

Quality improvement efforts

 Benchmarking [33, 34, 3841]

6 (28.6%)

 Plan-do-check-act (PDCA) [26, 36]

2 (9.5%)

 Collaborative Care Model [26, 28, 42, 44]

4 (19%)

 The Chronic Care Model [25, 32]

2 (9.5%)

 Learning and Leadership Collaborative [35]

1 (4.8%)

 Plan-do-check-act (PDCA) and the Chronic Care Model [37]

1 (4.8%)

 IT application as feedback tool [24, 27, 30, 41]

4 (19%)

 No clear QI method [29, 31, 43]

3 (14.3%)

Impact of quality improvement

Eight studies showed statistically significant improvement in outcomes resulting from the implementation of QI initiatives [25, 27, 29, 31, 33, 34, 42, 44]. Statistically significant improvements were achieved in long-term survival [33, 34], mortality [42], readmission rate [42], bleeding complications [42], systolic blood pressure [27], HbA1C [27, 29], LDL [27, 29], exercise habits [25], depression improved in the acute phase (PHQ-9 score) [44], and hospitalization with ambulatory care-sensitive conditions [31]. The remaining studies did not show statistically significant improvements. All included studies presented outcome measures for their respective improvement work, five of which also measured additional process measures [2733, 35, 41, 42, 44]. Table 2 presents outcomes measures used, QI methods applied and whether statistically significant improvement of outcome measures was achieved. A detailed overview of the significance of outcome measures can be found in the online supplementary content (Additional file 1: Appendix 4). None of the studies identified an impact on patient value or evaluated the impact on costs of care.
Table 2

Improvement in outcomes and/or processes

Author/year

Outcome measures

Significant improvement + 1/02/0a 3

RAND

QI-MQCS score

QI methods

QI focus

Dziuban et al., (1994) [39]

Risk adjusted mortality

0a

12

Benchmarking

Hospital-specific and physician-specific results published annually in cardiac surgery.

Adams et al., (1998) [43]

Ambulation/locomotion

0

10

No clear QI method

Implementation of and outcome-based quality improvement concept including two outcome reports.

Bathing

0

Management of oral medications

0

Pain

0

Dyspnoea

0

Halpin et al., (2004) [40]

Postoperative Atrial fibrillation

0a

11

Benchmarking

Implementation of a new guideline based on insights into outcomes, literature and roundtable discussion. An Outcome Center was formed and a multidisciplinary Performance Improvement Committee.

Operative mortality

0

Cardiac arrest

0

Reoperation for bleeding

0

Pneumonia

0

Deep sternal infection

0

Permanent stroke

0

Transient stroke

0

Prolonged ventilation

0

Length of stay

0

Moller et al., (2005) [38]

Overall operative

Mortality

0a

7

Benchmarking

Developed a centralized data acquisition and analysis method (through the creation of the network paediatric Cardiac Care Consortium). A uniform diagnostic and procedure classification system was created. Differences in patient populations cared for at the cardiac centres were compared.

Thomas et al., (2007) [24]

HgbA1c

0

13

IT application as feedback tool

Registry-generated audit, feedback and patient reminder targeted at residents.

LDL cholesterol

0

Blood pressure

0

Peterson et al., (2008) [27]

Mean systolic blood pressure

+

10

IT application as feedback tool

Multicomponent intervention: implementation of an electronic diabetes registry, visit reminders, and patient-specific physician alerts.

HbA1c

+

Mean LDL

+

Carlhed et al., (2009) [42]

Mortality

+

12

Collaborative Care Model

Multidisciplinary teams consisted of critical care unit nurses and cardiologists were assigned at each of the 19 volunteering hospitals. 19 teams of 4 to 5 persons met at 4 (group A) or 2 (group B) training sessions during which education by QI experts was provided, using the Breakthrough Series curricula.

Readmission rate

+

Bleeding complication

+

Jakobsen et al., (2009) [33]

1-year survival

+

5

Benchmarking

Indicators (staging, surgical procedures, complications and survival) have been registered in 5007 patients who underwent surgery. Each year the results have been audited locally, regionally and nationally and improvements have been proposed, implemented, monitored and evaluated by the audit-plenary.

2-year survival

+

5-year survival

0

30-day mortality

0a

Kraynack et al., (2009) [35]

FEV1

0a

12

Learning and Leadership Collaborative

A QI process is described from the initial team-building phase, through the assessment of care processes, standardization of care, and developing a culture of continuous improvement aiming to improve pulmonary function of the paediatric patients.

MacLean et al., (2009) [25]

Blood pressure

0

10

The Chronic Care Model

Providing decision support and patient decision support in diabetes care delivery.

BMI

0

SF-12

Physical

0a

SF-12 Mental

Quality of life

0

Exercise habit

+

 

Toh et al., (2009) [30]

Poor HbA1c (9% and above)

0a

11

IT application as feedback tool

Chronic disease management system with patient reminders based on registry data.

Good LDL-control below 2.6 mmol/L

0a

 

Baty et al., (2010) [29]

% with HbA1c < 7%

+

10

No clear QI method

Implementing a comprehensive system-based disease management process including a diabetes registry and quality reports.

% with HbA1c < 9%

+

%with LDL < 100

+

Beaulieau et al., (2010) [41]

Mortality

0a

7

Benchmarking

IT application as feedback tool

Implementing a method for linking administrative and registry data to track quality improvement initiatives through dashboards.

Infusion rate

0a

Bricker et al., (2010) [28]

A1C

0a

9

Collaborative Care Model

Implementing the Chronic Care Model through regional care learning collaborative with focus on team-based care, patient-centred care coordination, delivery of evidence-based care, patient self-management, use of a patient registry system and culturally and linguistically competent care.

Blood pressure

0a

LDL Cholesterol levels

0a

Bauer et al. (2011) [44]

Depression improved in acute phase (PHQ-9 score)

+

12

Collaborative Care Model

Implementing a collaborative care model including a web-based disease registry, care management to support treatment and organized psychiatric consultation.

A1C testing

0

10

Plan-do-check-act (PDCA)

Realizing continuous quality improvement through benchmarking in cystic fibrosis care.

Stern et al., (2011) [36]

FEV1 > 80 < 18

0a

FEV1 > 80 > 18

0a

BMI > 19

0a

WH > 90

0a

Jakobsen et al., (2013) [34]

1-year survival

+

5

Benchmarking

Indicators were established, validated, and monitored. 40,000 patients have been included in the database. Results were reported periodically and submitted to realize auditing on an annual basis.

2-year survival

+

5-year survival

+

Siracusaet al., (2014) [37]

Median FEV1

0a

13

Plan-do-check-act (PDCA)

The Chronic Care Model

Several improvement interventions implemented between 2001 and 2007 with focus on patient and family engagement in CF care, improve access and use of data, individualized scheduling, improving vaccination rates, infection control aiway clearance, standardization of care processes, and forming and QI team.

Median body mass index (BMI)

0a

Peterson et al., (2015) [26]

Systolic blood pressure

0a

13

Plan-do-check-act (PDCA)

Collaborative Care Model

The effect of 23 diabetes teams joining a quality collaborative on patient outcomes.

HbA1c

0a

LDL

0a

Han et al. (2016) [31]

Hospitalization with ambulatory care-sensitive conditions

+

7

No clear QI method

Using clinical registry data to identify patients who should receive reminders for preventive/follow-up care and send reminders to those patients. Generate a list of patients by condition to use for quality improvement.

ED visits

+

Lail et al., (2017) [32]

Disease remission

0a

13

The Chronic Care Model

Eighteen condition teams implemented interventions varying from: establishing pre-visit planning (PVP), identifying the target populations, selecting and measuring outcomes and supporting processes, building and implementing care coordination, and assessing and addressing self-management support. The teams were free to choose the interventions that they thought would work best.

Disease control

0a

Quality of life

0a

Symptom management

0a

1+ means that the result was statistically significant at a p-value of 0.05

20 means that there was no significant improvement in outcomes

30a means that there was improvement, but significance was not tested or reported

Quality of the studies

Rigor of quality improvement interventions

The overall quality of included articles was moderate (see Tables 3 and 4). On the 16 domains of the QI-MQCS four articles achieved a score of 13, which is the highest score among included studies [24, 26, 32, 37]. These articles are therefore considered to be of good quality. Four articles were ranked as moderate quality with a score of 12 [35, 39, 42, 44]. Five articles scored poorly on the QI-MQCS with a score ≤ 7, which is ranked as low quality [31, 33, 34, 38, 41].
Table 3

Scoring of the RAND QI-MQCS

 

Dziuban et al., (1994) [39]

Adams et al. (1998) [43]

Halpin et al. (2004) [40]

Moller et al., (2005) [38]

Thomas et al., (2007) [24]

Carlhed et al., (2009) [42]

Peterson et al., (2008) [27]

Jakobsen et al., (2009) [33]

MacLean et al., (2009) [25]

Kraynack & MacBride (2009) [35]

1. Organizational motivation

Y

Y

Y

N

Y

Y

N

N

N

N

2. Intervention rationale

Y

Y

Y

Y

Y

Y

Y

N

Y

Y

3 Intervention description

Y

N

Y

Y

Y

Y

Y

N

Y

Y

4. Organizational characteristics

Y

Y

Y

N

N

Y

N

N

Y

Y

5. Implementation

Y

N

Y

Y

Y

Y

Y

N

Y

Y

6. Study design

Y

Y

N

N

Y

Y

Y

Y

Y

Y

7. Comparator

Y

N

Y

N

Y

Y

Y

N

Y

Y

8. Data source

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

9. Timing

Y

Y

Y

N

Y

Y

Y

N

Y

Y

10. Adherence and fidelity

N

N

Y

N

N

N

N

N

N

Y

11. Health outcomes

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

12. Organizational readiness

Y

N

Y

N

Y

N

N

N

N

Y

13. Penetration and reach

N

N

N

N

N

Y

Y

Y

Y

N

14. Sustainability

N

Y

N

Y

Y

N

N

N

N

N

15. Spread

N

Y

N

Y

Y

N

N

N

N

Y

16. Limitations

Y

Y

N

N

Y

Y

Y

Y

N

N

Total score (Y)

12

10

11

7

13

12

10

5

10

12

Table 4

Scoring of the RAND QI-MQCS

 

Bricker et al. (2010) [28]

Beaulieau et al. (2010) [41]

Bauer et al. (2011) [44]

Stern et al., (2011) [36]

Jakobsen et al., (2013) [34]

Siracusa et al., (2014) [37]

Peterson et al., (2015) [26]

Han et al., (2016) [31]

Lail et al., (2017) [32]

Toh et al., (2009) [30]

Baty et al., (2010) [29]

1. Organizational motivation

N

Y

Y

N

N

Y

N

N

Y

Y

Y

2. Intervention rationale

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

3 Intervention description

N

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

4. Organizational characteristics

N

N

Y

Y

N

Y

Y

N

N

N

Y

5. Implementation

Y

Y

Y

Y

N

Y

Y

N

Y

Y

Y

6. Study design

Y

N

Y

N

N

N

Y

N

N

N

N

7. Comparator

Y

N

Y

Y

N

Y

Y

Y

N

Y

Y

8. Data source

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

9. Timing

Y

N

Y

Y

N

Y

Y

Y

Y

Y

N

10. Adherence and fidelity

N

N

N

N

N

Y

Y

N

Y

N

N

11. Health outcomes

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

12. Organizational readiness

N

Y

Y

N

N

Y

N

N

Y

Y

Y

13. Penetration and reach

Y

Y

N

N

Y

Y

N

N

Y

N

N

14. Sustainability

Y

N

N

Y

N

Y

Y

N

Y

N

N

15. Spread

N

N

N

Y

N

N

Y

N

Y

Y

N

16. Limitations

N

N

Y

N

N

N

Y

Y

Y

Y

Y

Total score (Y)

9

7

12

10

5

13

13

7

13

11

10

Rigor of data collection and analysis

The overall results of the quality assessment on data collection and analyses are displayed in the online supplementary content (Additional file 1: Appendix 5). Four studies have applied generalized linear mixed models for the analysis of change in outcomes [25, 27, 36, 42]. One study used a generalized estimating equation model with repeated measurements [24]. Inferential statistics have also been used in the form of survival analyses, logistic regression and chi-square analyses [29, 31, 33, 39, 44]. The remaining studies made use of descriptive statistical analyses only [26, 30, 32, 38, 43]. In order to monitor change, run charts have been applied in five studies [28, 35, 37, 40, 41].

On the additional item criteria, two studies have applied methods to account for missing values in their data, while also conducting a power analysis [25, 27].

Methods used to achieve improvements

We identified six methods to achieve QI: benchmarking [33, 34, 3841], a collaborative care model [26, 28, 42, 44], Plan-Do-Check-Act [36, 37], the Chronic Care Model [25, 32, 37], Learning and Leadership Collaborative [35] and IT driven interventions [24, 27, 29, 30, 41]. There were some studies where no clear QI method was used [29, 31, 43]. We will discuss these methods in the following paragraphs.

Benchmarking

Benchmarking has been applied in several of the included studies [33, 34, 38, 39, 41]. Data was mostly compared among different hospitals [33, 34, 38]. Annual publication of data in the form of reports has most commonly been applied to report on results [33, 34, 41].

One study complemented their national report with an additional disease specific report with supplementary measures [33]. Another method of benchmarking was a discussion of the results at a (monthly or annual) meeting. During the annual meeting, results from reports were discussed and further evaluated [38]. Also, short-term feedback cycles with monthly publication of reports were applied [39]. The use of a strong data-driven system in combination with audits was characteristic of initiatives that applied benchmarking in order to improve outcomes as well as a model to change practice [33, 34, 39, 40].

Collaborative care model

Three studies applied the Breakthrough Collaborative Model (BCM) to structure the goal of improving outcomes [26, 28, 42]. One study applied a Web-based disease registry to track patients with symptoms of depression to support treatment management in primary care [44]. In addition, evidence-based depression management training was provided to primary care providers. Moreover, in all sites, most patients experienced meaningful improvement in depression.

The BCM was used to design a cycle of structured discussion sessions during which outcomes were analyzed, presented and variation in work processes were discussed [26, 28]. The model was furthermore used as a guide to facilitate improvement efforts and insights into data [26, 42].

Plan-do-check-act

In two studies Plan-Do-Check-Act (PDCA) cycles were used to improve outcomes and/or processes [26, 36, 37]. Yet, the cycle was presented as a supporting tool to other methods, either for the application of the BCM [26] or for benchmarking [36]. For the latter it was applied as a method to prepare for national benchmarking by organizing three PDCA cycles before data was shared [36]. The method was applied by organizing multidisciplinary meetings, where outcomes were discussed and improvement initiatives were identified [36]. Three cycles were organized in order to prepare public benchmarking after the third cycle [36].

The other study, which primarily used the methods outlined for the BCM, used the PDCA to structure and evaluate the learning sessions [26]. However, it was not the primary method for improving outcomes. In another study PDCA was used to continually evaluate local cystic fibrosis care practices, and they were able to improve pulmonary function and nutritional outcomes [37].

The chronic care model

Three studies applied the Chronic Care Model (CCM) [25, 32, 37]. One study that applied the CCM used supporting techniques such as: audit and feedback, electronic registry, clinician reminders, patient reminders, and abbreviated patient education. It is, thus, rather a framework offering practical tools [25]. They did not find expected improvements in outcomes. Here, authors suggested that another, more collaborative approach would be needed to improve outcomes of chronic diseases [25]. The second study applied the CCM in children with various chronic conditions, in combination with PDCA cycles, failure mode and effect analysis and Pareto charts of failures [32]. This study resulted in improvement of respective outcomes [32]. The third study applied the CCM to ensure that all aspects of cystic fibrosis management were covered, and combined this with the PDCA to continually evaluate the processes of best practices in cystic fibrosis care. They did not evaluate the effectiveness of applying the CCM.

Learning and leadership collaborative

The Learning and Leadership Collaborative (LLC) was applied in one study [35]. Commitment of a team to participate in a QI program, developing a sense of common responsibility as an organization for the improvement, measuring outcomes and processes and patient involvement were defined as key ingredients for QI. LLC has been used for training staff towards structured discussions on outcomes and/or processes and the introduction of a patient registry [35]. Data was registered and analyzed at one particular hospital, but presented to all participating hospitals. Participation in the LLC has led to the initiation of an improvement initiative at the hospital where the data were registered and analyzed.

IT application as feedback tool

Five studies made used of (self-developed) IT applications, to empower patients and/or physicians to manage patients with greater care. The studies aimed at linking administrative and key clinical data and made use of reminder functions [24, 27, 30]. One study concluded their patients received better overall coordination of care [30]. Another two studies reported significant improvements in the percentage of type 2 diabetic patients and at-risk populations utilizing diabetes registries achieving recommended values for SBP, LDL, and HbA1C [27]. In one study, data were in addition displayed in operating room theatre, surgical office suites and nursing units [41]. Another study reported improved adherence to diabetes care processes in a continuity clinic due to the registry-generated audit, feedback, and patient reminders [24].

Facilitators for quality improvements

A noticeable facilitator leading to QI was frequent reporting and feedback either annually or even monthly [28, 33, 34, 3841]. The use of a database with high quality data, audits and reports as well as a strong stakeholder involvement were also found to be important factors contributing to successful QI [33, 34]. Structured registry data and an improvement intervention that can be linked to outcomes led to improvement in respective outcome measures [42]. In addition, other factors mentioned that would be needed for successful QI in one or more of the included studies are (1) patient involvement, communication, and standardization; (2) attitude and enthusiastic commitment from physicians, clinical managers and central administration and (3) appreciation concerning the importance of measurements [28, 35, 40, 41]. Moreover, improvement in outcomes appeared to be successful if supported by a proven QI approach [42]. Inconsistencies were found regarding the importance of involving an expert in the field of QI. On the one hand, involvement of a QI expert was considered positive for the start of an improvement agenda as it contributed to a more rapid implementation of improvement initiatives [42]. On the other hand, involving no additional expert or formal team was not experienced as a contributing factor to the success of outcome improvement [26]. This was only possible because a structured data registry was already present [26].

Catalyst to improve outcomes over time

Outcomes can be improved over time through systematic use of outcome registries and facilitators. Outcome data and its interpretation help to achieve improvements in outcomes over time even faster compared to studies that did not use outcome data [34]. It was stated that outcomes were not only used to identify possible improvement interventions but also to monitor and secure improvements in the long run [34].

A computerized system was presented as a success factor to accelerate data from clinical registries to change outcomes and/or processes [2629, 3136, 42, 45]. Such a computerized system ensured valid and timely results [33]. Moreover, it allows for real-time feedback, which, in turn, leads to faster identification of improvement areas [28, 29, 31, 42].

Further use of outcome data for outcome improvement included the development of checklists, improved use of diagnostic standards, creation of data transparency, guidelines, improved patient recall, patient empowerment and leadership towards improvement [28, 29, 31, 36].

Discussion

Eight out of the 21 included studies reported statistically significant improvements in outcomes including long-term survival, mortality, readmission rate, bleeding complications, systolic blood pressure, HbA1C, LDL, exercise habits (FEV1), depression improved in the acute phase (PHQ-9 score) and hospitalization with ambulatory care-sensitive conditions resulting from the implementation of QI initiatives. Out of these eight studies, the Chronic Care Model, IT application as feedback, benchmarking and the Collaborative Care Model were used as QI methods. A diverse set of clinical outcomes were collected and no patient-reported outcome measures (PROMs) were applied in any of the studies. Yet, only one study that reported statistically significant improvements in outcomes was of good quality. The improvement interventions were diverse, ranging from the implementation of guidelines, development of physician/patient alerts, improved teamwork, patient engagement methods through IT applications and the development of a supportive decision system. Many improvement interventions were combined in order to build a multifaceted approach to QI [24, 27, 28, 32, 37, 42, 44]. Facilitators for realizing QI include a high quality database, the use of pre-defined outcome measures, audits, frequent reporting and feedback, patient involvement, improved communication and standardization. Systematic approaches were used for structuring the improvement cycle. In order to use data from clinical registries as a catalyst to change outcomes, this review suggests that having a strong computerized system is supportive in aiding frontline clinical process management and improvement work.

A facilitator identified in this review was the organization of discussions for mapping and selecting best practices. It was further shown that a sound data management has a catalyzing effect. This data can be aggregated in annual reports, while it can also be used to compare with peers and/or perform nationwide comparisons. Also, a registry can facilitate access to real-time outcome and process data which can engage the team in realizing active improvements. Other registry programs such as the Get With The Guidelines-Stroke study, a large registry and performance improvement program for hospitalized patients with stroke and transient ischemic attack, also use annual reports for benchmark and feedback purposes [46].

Other systematic reviews concluded that audit and feedback can lead to small but important improvements in professional practice and healthcare outcomes [47]. They furthermore concluded that the effectiveness of audit and feedback depends on how the feedback was provided as well as on baseline performance. In addition, comparing this review to ours, there was one paper we have both included [24]. However, the objectives are very different, which can explain that there was not more overlap in included studies.

In addition, barriers and success factors to the effectiveness of feedback have been identified [48]. However, the authors were not able to draw sound conclusions on the effect of feedback on the quality of care and its potential to improve outcomes. Another review concerning renal registry data reflected on the potential of registry data and help advancing the nephrology care delivery [49].

None of the reviews studied the effect of QI efforts, besides from audit and feedback, on the quality of care and outcomes. This is the first study for which the literature was searched in detail in order to identify barriers and facilitators supporting QI interventions based on information from clinical registries.

The use of clinical registries can be seen as an important tool in order to systematically measure clinical outcomes and to achieve the goals of value-based health care. This is not only in line with our conclusions, but also acknowledged by others [1, 50, 51]. Other data sources can also be valuable for QI efforts, such as data from randomized controlled trials. However, this review aimed at including studies where structural data was collected through the use of a clinical registry.

In order to improve value, measuring both one or more outcomes and costs is essential [17]. Working with international registries makes it possible to make global comparisons, for example identifying practice variations and therefore improving quality of care for the whole patient group [52].

Implications

We did not observe many efforts to incorporate PROMs. It is, however, generally considered important to measure the impact on health related quality of life (HRQoL) in the evaluation of the effect of QI initiatives [53]. The studies included for this review did not reflect on why they did not use PROMs and what would be the added value if they did. Even so, one study does report however the start of measuring quality of life in patients with cystic fibrosis [36]. The authors report this will lead to more insights into the complexity of QI efforts and personal patient gains in the experienced quality of life. It will also enable reporting on to what extent value was created from the patient’s perspective. Future QI efforts very likely combine QI with benchmarking incorporating quality of life outcomes.

None of the included studies reported costs, causing our study to be unable to evaluate the true impact on value. Incorporating costs will enable to identify cost drivers and comparing improvement interventions as proposed by the value-based healthcare principles [50]. A recent study showed that surgery for the oldest patients with colorectal cancer did not lead to increased hospital costs [51]. However, this study did identify variation in cost driver distribution. Patients under 85 years old had lower costs looking at the ward, operation and intensive care unit. Therefore, identifying costs and its main drivers will enable to develop improvement programs for specific sub-groups. This might be a powerful tool to reduce e.g. complications and thus hospital costs. Value-based health care could be the overarching concept guiding improvement initiatives, combined with well-defined methods. However, the field lacks a clear guide on implementation examples. Studies reflecting on impact, outcomes and costs are needed. Finally, the standardization of outcome measures is key, although they should be defined for a specific patient population. Transparent measurement of outcomes and costs has the potential to improving the value of care for all patients. Both providers, patients and payers can benefit from this collective common goal of transparency.

Limitations

This review has some inherent limitations. Firstly, due to the very heterogeneous types of QI programs and their respective patient groups, it is difficult to generalize the results achieved in the included studies. Moreover, our inclusion criteria for QI programs may be to some extent arbitrary, which could possible lead to a bias in inclusion or exclusion of studies.

Also, the context in which the clinical registry is organized can impact outcomes. Moreover, important differences were observed in e.g. whether the registry was linked to reimbursement or public reporting versus primarily initiated for scientific or QI purposes or whether it was a voluntary or mandatory registry.

Secondly, the studies included in this review mainly focused on experiences in non-communicable diseases and thus often chronic patient groups. However, our aim was not to exclude communicable diseases from the study but we did not identify any studies in our literature search. This could indicate that chronic patient groups benefitted most from the realization of registries and respective QI interventions. As a result, improvement projects concerning other (non-chronic) patient groups have not been included in this review. Thirdly, due to publication bias, studies reporting no effect will be very likely not published and therefore missed out. Finally, two studies randomized practices [25, 27]. One study randomly allocated 19 volunteering hospitals to 1 of 2 intervention groups, where the intervention differed both in design and intensity [42]. In the other studies it should be noted that complete randomization was not possible, since the intervention hospitals involved were e.g. volunteering. Therefore, these hospitals might differ in their willingness to improve, causing potential selection bias.

Conclusion

The results from this evaluation of studies which use outcome measures from clinical registry data to implement and monitor QI initiatives may help policy makers, managers and clinicians to understand the effectiveness, practicality and challenges of implementing QI interventions. An active and systematic approach is needed to improve outcomes. Continuous feedback from the data linked to clinical practice is crucial. Our review indicates that successful QI and consequently improved outcomes, is dependent on an active approach and organizational readiness.

There are many QI methods, and the majority of improvement interventions contain a combination of several methods. Clinical registries can be seen as supportive instruments in the process of improving quality of care. However, a clinical registry can only be successful in realizing QI efforts when there is commitment and leadership at both the physician and manager level, as well as a benchmarking facility, a well-integrated computerized system, and a collective aim to identify best practices.

Notes

Abbreviations

BCM: 

the Breakthrough Collaborative Model

CCM: 

Chronic Care Model

FEV1: 

Forced expiratory volume in one second

HbA1C: 

Glycated Hemoglobin

HRQoL: 

Health related quality of life

LDL: 

Low-density lipoprotein

LLC: 

Learning and Leadership Collaborative

PDCA: 

Plan-Do-Check-Act

PROMs: 

Patient-reported outcome measures

QI: 

Quality Improvement

QI-MQSC: 

Quality Improvement Minimum Quality Criteria Set

SQUIRE: 

Standards for Quality Improvement Reporting Excellence

Declarations

Acknowledgements

We thank Carla Sloof for her support in formulating the search strategy, the English language professional Valesca Hulsman for editing, and Jozé Braspenning who performed a critical review of the manuscript.

Funding

This study was supported by The Netherlands Organisation for Health Research and Development (ZonMw) under project number 842001005. The funder had no influence on the study design, collection, analysis and interpretation of the data, the writing of the report, and the decision in where to submit the article for publication.

Availability of data and materials

Not applicable.

Authors’ contributions

NK, NZ, PvdN, PvdW, SG and GW (hereinafter referred to as ‘all authors’) contributed to the conception and design of the study. The first two authors, NK and NZ, shared equally in the development and execution of the work. NK and NZ were the major contributors in writing the manuscript. NK, NZ, PvdN, PvdW, SG analyzed and interpreted the data. NK, NZ, PvdN, PvdW, SG had full access to all data in the study and take responsibility for the integrity of the data and accuracy of the data analysis. All authors contributed to the analysis and interpretation, and provided drafting of the article. All authors contributed to the revision of the article for important intellectual content. All authors read and approved the final manuscript.

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Department of Value-Based Healthcare, St. Antonius Hospital, Nieuwegein, the Netherlands
(2)
Radboud Institute for Health Sciences, Scientific Center for Quality of Healthcare (IQ healthcare), Radboud university medical center, Nijmegen, the Netherlands

References

  1. Larsson S, Lawyer P. Improving health care value - the case for disease registries; 2011.Google Scholar
  2. Gliklich RE, Dreyer NA, Leavy MB. Registries for evaluating patient outcomes: a user’s guide. Rockville (MD): Government Printing Office; 2014.Google Scholar
  3. Gitt AK, Bueno H, Danchin N, Fox K, Hochadel M, Kearney P, Maggioni AP, Opolski G, Seabra-Gomes R, Weidinger F. The role of cardiac registries in evidence-based medicine. Eur Heart J. 2010;31(5):525–9.View ArticleGoogle Scholar
  4. Hickey GL, Grant SW, Cosgriff R, Dimarakis I, Pagano D, Kappetein AP, Bridgewater B. Clinical registries: governance, management, analysis and applications. Eur J Cardiothorac Surg. 2013;44(4):605–14.View ArticleGoogle Scholar
  5. Black N. High-quality clinical databases: breaking down barriers. Lancet. 1999;353(9160):1205–6.View ArticleGoogle Scholar
  6. Pryor DB, Califf RM, Harrell FE, Hlatky MA, Lee KL, Mark DB, Rosati RA. Clinical data bases: accomplishments and unrealized potential. Med Care. 1985;23(5):623–47.View ArticleGoogle Scholar
  7. Aljurf M, Rizzo J, Mohty M, Hussain F, Madrigal A, Pasquini M, Passweg J, Chaudhri N, Ghavamzadeh A, Solh H. Challenges and opportunities for HSCT outcome registries: perspective from international HSCT registries experts. Bone Marrow Transplant. 2014;49(8)1016–21.Google Scholar
  8. Bhatt DL, Drozda JP, Shahian DM, Chan PS, Fonarow GC, Heidenreich PA, Jacobs JP, Masoudi FA, Peterson ED, Welke KF. ACC/AHA/STS statement on the future of registries and the performance measurement Enterprise: a report of the American College of Cardiology/American Heart Association task force on performance measures and the Society of Thoracic Surgeons. J Am Coll Cardiol. 2015;66(20):2230–45.View ArticleGoogle Scholar
  9. Evans SM, Scott IA, Johnson NP, Cameron PA, McNeil JJ. Development of clinical-quality registries in Australia: the way forward. Med J Aust. 2011;194(7):360–3.PubMedGoogle Scholar
  10. McNeil JJ, Evans SM, Johnson NP, Cameron PA. Clinical-quality registries: their role in quality improvement. Med J Aust. 2010;192(5):244–5.PubMedGoogle Scholar
  11. Stey AM, Russell MM, Ko CY, Sacks GD, Dawes AJ, Gibbons MM. Clinical registries and quality measurement in surgery: a systematic review. Surgery. 2015;157(2):381–95.View ArticleGoogle Scholar
  12. Porter ME. Value-based health care delivery. Ann Surg. 2008;248(4):503–9.PubMedGoogle Scholar
  13. Porter ME. What is value in health care? N Engl J Med. 2010;363(26):2477–248.View ArticleGoogle Scholar
  14. Erekson EA, Iglesia CB. Improving patient outcomes in gynecology: the role of large data registries and big data analytics. J Minim Invasive Gynecol. 2015;22(7):1124–9.View ArticleGoogle Scholar
  15. Donabedian A. Evaluating the quality of medical care. Milbank Q. 2005;83(4):691–729.View ArticleGoogle Scholar
  16. Hedges LV, Vevea JL. Fixed-and random-effects models in meta-analysis. Psychol Methods. 1998;3(4):486.View ArticleGoogle Scholar
  17. Porter ME, Pabo EA, Lee TH. Redesigning primary care: a strategic vision to improve value by organizing around patients' needs. Health Aff (Millwood). 2013;32(3):516–25.View ArticleGoogle Scholar
  18. Cochrane Training. Data collection forms for intervention reviews. https://dplp.cochrane.org/data-extraction-forms. Accessed 2 Mar 2017.
  19. Hempel S, Shekelle PG, Liu JL, Sherwood Danz M, Foy R, Lim YW, Motala A, Rubenstein LV. Development of the quality improvement minimum quality criteria set (QI-MQCS): a tool for critical appraisal of quality improvement intervention publications. BMJ Qual Saf. 2015;24(12):796–804.View ArticleGoogle Scholar
  20. Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health. 1998;52(6):377–84.View ArticleGoogle Scholar
  21. Ogrinc G, Mooney SE, Estrada C, Foster T, Goldmann D, Hall LW, Huizinga MM, Liu SK, Mills P, Neily J, Nelson W, Pronovost PJ, Provost L, Rubenstein LV, Speroff T, Splaine M, Thomson R, Tomolo AM, Watts B. The SQUIRE (standards for QUality improvement reporting excellence) guidelines for quality improvement reporting: explanation and elaboration. Qual Saf Health Care. 2008;17(Suppl 1):i13–32.View ArticleGoogle Scholar
  22. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–9.View ArticleGoogle Scholar
  23. Hoque DME, Kumari V, Hoque M, Ruseckaite R, Romero L, Evans SM. Impact of clinical registries on quality of patient care and clinical outcomes: a systematic review. PLoS One. 2017;12(9):e0183667.View ArticleGoogle Scholar
  24. Thomas KG, Thomas MR, Stroebel RJ, McDonald FS, Hanson GJ, Naessens JM, Huschka TR, Kolars JC. Use of a registry-generated audit, feedback, and patient reminder intervention in an internal medicine resident clinic—a randomized trial. J Gen Intern Med. 2007;22(12):1740–4.View ArticleGoogle Scholar
  25. MacLean CD, Gagnon M, Callas P, Littenberg B. The Vermont diabetes information system: a cluster randomized trial of a population based decision support system. J General Intern Med. 2009;24(12):1303–10.View ArticleGoogle Scholar
  26. Peterson A, Gudbjornsdottir S, Lofgren UB, Schioler L, Bojestig M, Thor J, Andersson Gare B. Collaboratively improving diabetes Care in Sweden Using a National Quality Register: successes and challenges-a case study. Qual Manag Health Care. 2015;24(4):212–21.View ArticleGoogle Scholar
  27. Peterson KA, Radosevich DM, O'Connor PJ, Nyman JA, Prineas RJ, Smith SA, Arneson TJ, Corbett VA, Weinhandl JC, Lange CJ, Hannan PJ. Improving diabetes Care in Practice: findings from the TRANSLATE trial. Diabetes Care. 2008;31(12):2238–43.View ArticleGoogle Scholar
  28. Bricker PL, Baron RJ, Scheirer JJ, DeWalt DA, Derrickson J, Yunghans S, Gabbay RA. Collaboration in Pennsylvania: rapidly spreading improved chronic care for patients to practices. J Contin Educ Heal Prof. 2010;30(2):114–25.View ArticleGoogle Scholar
  29. Baty PJ, Viviano SK, Schiller MR, Wendling AL. A systematic approach to diabetes mellitus care in underserved populations: improving care of minority and homeless persons. Fam Med. 2010;42(9):623–7.PubMedGoogle Scholar
  30. Toh MP, Leong HS, Lim BK. Development of a diabetes registry to improve quality of care in the National Healthcare Group in Singapore. Ann Acad Med Singap. 2009;38(6):546.PubMedGoogle Scholar
  31. Han W, Sharman R, Heider A, Maloney N, Yang M, Singh R. Impact of electronic diabetes registry 'meaningful use' on quality of care and hospital utilization. J Am Med Informatics Assoc. 2016;23(2):242–7.View ArticleGoogle Scholar
  32. Lail J, Schoettker PJ, White DL, Mehta B, Kotagal UR. Applying the chronic care model to improve care and outcomes at a pediatric medical center. Jt Comm J Qual Patient Saf. 2017;43(3):101–12.View ArticleGoogle Scholar
  33. Jakobsen E, Palshof T, Osterlind K, Pilegaard H. Data from a national lung cancer registry contributes to improve outcome and quality of surgery: Danish results. Eur J Cardiothorac Surg. 2009;35(2):348–52 discussion 352.View ArticleGoogle Scholar
  34. Jakobsen E, Green A, Oesterlind K, Rasmussen TR, Iachina M, Palshof T. Nationwide quality improvement in lung cancer care: the role of the Danish lung Cancer group and registry. J Thorac Oncol. 2013;8(10):1238–47.View ArticleGoogle Scholar
  35. Kraynack NC, McBride JT. Improving care at cystic fibrosis centers through quality improvement. Semin Respir Crit Care Med. 2009;30:547–58.View ArticleGoogle Scholar
  36. Stern M, Niemann N, Wiedemann B, Wenzlaff P, German CFQA Group. Benchmarking improves quality in cystic fibrosis care: a pilot project involving 12 centres. Int J Qual Health Care. 2011;23(3):349–56.View ArticleGoogle Scholar
  37. Siracusa CM, Weiland JL, Acton JD, Chima AK, Chini BA, Hoberman AJ, Wetzel JD, Amin RS, McPhail GL. The impact of transforming healthcare delivery on cystic fibrosis outcomes: a decade of quality improvement at Cincinnati Children's Hospital. BMJ Qual Saf. 2014;23(Suppl 1):i56–63.View ArticleGoogle Scholar
  38. Moller JH, Hills CB, Pyles LA. A multi-center cardiac registry. A method to assess outcome of catheterization intervention or surgery. Prog Pediatr Cardiol. 2005;20(1):7–12.View ArticleGoogle Scholar
  39. Dziuban SW, McIlduff JB, Miller SJ, Dal Col RH. How a New York cardiac surgery program uses outcomes data. Ann Thorac Surg. 1994;58(6):1871–6.View ArticleGoogle Scholar
  40. Halpin LS, Barnett SD, Burton NA. National databases and clinical practice specialist: decreasing postoperative atrial fibrillation following cardiac surgery. Outcomes Manag. 2004;8(1):33–8.PubMedGoogle Scholar
  41. Beaulieu PA, Higgins JH, Dacey LJ, Nugent WC, DeFoe GR, Likosky DS. Transforming administrative data into real-time information in the Department of Surgery. Qual Saf Health Care. 2010;19(5):399–404.PubMedGoogle Scholar
  42. Carlhed R, Bojestig M, Peterson A, Aberg C, Garmo H, Lindahl B, Quality Improvement in Coronary Care Study Group. Improved clinical outcome after acute myocardial infarction in hospitals participating in a Swedish quality improvement initiative. Circ Cardiovasc Qual Outcomes. 2009;2(5):458–64.View ArticleGoogle Scholar
  43. Adams CE, Wilson M, Haney M, Short R. Using the outcome-based quality improvement model and OASIS to improve HMO patients' outcomes. Outcome assessment and information set. Home Healthc Nurse. 1998;16(6):395–401.View ArticleGoogle Scholar
  44. Bauer AM, Azzone V, Goldman HH, Alexander L, Unützer J, Coleman-Beattie B, Frank RG. Implementation of collaborative depression management at community-based primary care clinics: an evaluation. Psychiatr Serv. 2011;62(9):1047–53.View ArticleGoogle Scholar
  45. Lee TH, Pearson SD, Johnson PA, Garcia TB, Weisberg MC, Guadagnoli E, Cook EF, Goldman L. Failure of information as an intervention to modify clinical management: a time-series trial in patients with acute chest pain. Ann Intern Med. 1995;122(6):434–7.View ArticleGoogle Scholar
  46. Fonarow GC, Reeves MJ, Smith EE, Saver JL, Zhao X, Olson DW, Hernandez AF, Peterson ED, Schwamm LH, GWTG-Stroke Steering Committee and Investigators. Characteristics, performance measures, and in-hospital outcomes of the first one million stroke and transient ischemic attack admissions in get with the guidelines-stroke. Circ Cardiovasc Qual Outcomes. 2010;3(3):291–302.View ArticleGoogle Scholar
  47. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, O’Brien MA, Johansen M, Grimshaw J, Oxman AD. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev Reviews 2012, Issue 6. Art. No.: CD000259. p 1–229.Google Scholar
  48. van der Veer SN, de Keizer NF, Ravelli ACJ, Tenkink S, Jager KJ. Improving quality of care. A systematic review on how medical registries provide information feedback to health care providers. Int J Med Inform. 2010;79(5):305–23.View ArticleGoogle Scholar
  49. Lim T, Goh A, Lim Y, Morad Z. Use of renal registry data for research, health-care planning and quality improvement: what can we learn from registry data in the Asia–Pacific region? Nephrology. 2008;13(8):745–52.View ArticleGoogle Scholar
  50. Porter ME, Lee TH. The strategy that will fix health care. Harv Bus Rev. 2013;91(12):24.Google Scholar
  51. Govaert JA, Govaert MJ, Fiocco M, van Dijk WA, Tollenaar RA, Wouters MW. Hospital costs of colorectal cancer surgery for the oldest old: a Dutch population-based study. J Surg Oncol. 2016;114(8):1009–15.View ArticleGoogle Scholar
  52. McNamara RL, Spatz ES, Kelley TA, Stowell CJ, Beltrame J, Heidenreich P, Tresserras R, Jernberg T, Chua T, Morgan L, Panigrahi B, Rosas Ruiz A, Rumsfeld JS, Sadwin L, Schoeberl M, Shahian D, Weston C, Yeh R, Lewin J. Standardized Outcome Measurement for Patients With Coronary Artery Disease: Consensus From the International Consortium for Health Outcomes Measurement (ICHOM). J Am Heart Assoc. 2015;4(5). https://doi.org/10.1161/JAHA.115.001767.
  53. Porter ME, Larsson S, Lee TH. Standardizing patient outcomes measurement. N Engl J Med. 2016;374(6):504–6.View ArticleGoogle Scholar

Copyright

© The Author(s). 2018

Advertisement