Table 1 Characteristics of the included studies and effects of PR on clinical outcomes

Dziuban et al., 1994 [21]

To evaluate the impact of NYS CSRS program in CABG related mortality in one hospital identified as poor performing

Cohort study for the same facility before and after the introduction of a PR mechanism

One poorly performing, high risk hospital in New York (1992–1993)

NYS CSRS

Cardiac mortality

The NYS CSRS program has been associated with a reduction in the actual CABG-related mortality from 3.52 % in 1989 to 2.78 % in 1992. The risk-adjusted mortality, using pooled data from 1989 to 1992, decreased from 4.17 % in 1989 to 2.45 % in 1992

Positive

Hannan et al., 1994 [14]

To examine changes in the risk-adjusted mortality and operation volume associated with CABG procedures performed during the first 4 years of NYS CSRS in three groups of hospitals

Cohort study for the same facility before and after the introduction of a PR mechanism

New York Hospital cardiac surgeons performing CABG (1989–1992)

NYS CSRS

Cardiac mortality and CABG operation volume

During 4 years of NYS CSRS program, the risk-adjusted mortality decreased from 2.72 to 2.19 % for group1, from 4.24 to 2.51 % for group 2 and from 7.12 to 2.77 % for group 3. The groups of providers that showed the highest initial mortalities manifested the most improvement. The volume of operations performed by the various provider groups did not change substantially in the 4-year period

Positive

Rosenthal et al., 1997 [22]

To measure changes in hospital mortality that occurred after implementation of the CHQC, which publicly released in-hospital mortality rates

Cohort study for the same facility before and after the introduction of a PR mechanism

Discharges with 8 diagnosis from Northeastern Ohio hospitals (1992–1993)

CHQC

Cardiac, Respiratory, Neurologic, Gastro-Intestinal mortality

Risk-adjusted mortality for most conditions declined for 3 subsequent periods after publication of mortality data (July-December 1992/January-June 1993/ July-December 1993). Decreases in mortality rates were statistically significant in weighted linear regression analyses for heart failure (0.50 % per period) and pneumonia (0.38 % per period).

Positive

Peterson et al., 1998 [23]

To examine the impact of the NYS CSRS on in-hospital mortality rates by comparing mortality rates in New York to those in other states

Cohort study among different facilities with and without PR mechanisms, over the same period of time

New York Hospital Medicare beneficiaries aged 65 years and older who underwent bypass surgery between (1987–1992)

NYS CSRS

Cardiac mortality

After NYS CSRS program initiation, unadjusted 30-day mortality rates following bypass declined by 33 % in NY Medicare patients compared with a 19 % decline nationwide. Risk-adjusted 30-day mortality of bypass surgery in NY patients declined an average of 10.30 % per year (1987–1992) compared with 5.80 % for patients in the rest of the nation.

Positive

Baker et al., 2002 [24]

To examine mortality trends during a period (1991–1997) when the CHQC program was operational in Cleveland Hospitals

Cohort study for the same facility before and after the introduction of a PR mechanism

Cleveland Hospital Medicare patients for cardiac, respiratory, neurologic or gastro-intestinal diseases (1991–1997)

CHQC

Cardiac, Respiratory, Neurologic and Gastro-intestinal mortality (GIH)

During CHQC program risk-adjusted in-hospital mortality declined for all conditions except stroke and GIH. The 30-day mortality declined significantly only for CHF to 1.40 %, and COPD to 1.60 %. For stroke, risk-adjusted 30-day mortality actually increased by 4.30 %.

Mixed

Chassin, 2002 [25]

To examine the impact of NYS CSRS implementation on mortality rate outlier status and CABG mortality

Cohort study for the same facility before and after the introduction of a PR mechanism

Lowest And Highest CABG Mortality Hospitals In New York, (1989–1995)

NYS CSRS

Cardiac mortality

After NYS CSRS program was implemented risk-adjusted mortality fell 41 % statewide in New York. Mortality statewide continued to fall in the next period; the crude mortality reached 2.15 % in 1998 from 3.52 % in 1989.

Positive

Clough et al., 2002 [26]

To verify the decline in in-patient mortality in Cleveland Hospitals and to better understand relationship with CHQC project

Cohort study among different facilities with and without PR mechanisms, over the same period of time

Hospitals included in the Ohio Hospital Association’s in-patient discharge data (1992–1995)

CHQC

Cardiac, Respiratory, Neurologic and Gastro-intestinal mortality

No significant beneficial effect of the CHQC project on hospital mortality in Cleveland was demonstrated. The rate of decline in mortality in Cleveland (−0.218 % per six months) was statistically indistinguishable from that in the rest of the state.

None

Baker et al., 2003 [27]

To describe trends in risk-adjusted mortality for six acute conditions for hospitals that were identified as outliers by CHQC compared with other hospitals.

Cohort study for the same facility before and after the introduction of a PR mechanism

Medicare patients with AMI, heart failure, gastrointestinal hemorrhage, obstructive pulmonary disease, pneumonia, or stroke receiving care at Cleveland-area hospitals (1991–1997)

CHQC

Cardiac, Respiratory, Neurologic and Gastro-intestinal mortality

Hospital outlier status was not significantly related to changes in risk-adjusted 30-day mortality. During CHQC reporting period, the absolute decline in risk-adjusted 30-day mortality at “average” hospitals was 0.50 %.

None

Dranove et al., 2003 [28]

To study the effects of PR in New York and Pennsylvania on health care providers and patient outcomes

Cohort study for the same facility before and after the introduction of a PR mechanism

Medicare beneficiaries and hospitals found in a Medicare claims data set and hospitals participating in the American Hospital Association annual survey (1987–1994)

New York and Pennsylvania CABG report card

Cardiac, Respiratory, Neurologic and Gastro-intestinal mortality and readmission

Report card provided statistically marginal evidence that the average mortality rate in NY and PA increased by 0,45 % point on a base of 33 %. Report cards increased significantly the average rate of readmission with heart failure by approximately 0,50 % point.

Negative

Moscucci et al., 2005 [29]

To compare in-hospital mortality from large multicenter PCI databases in Michigan, where PR is not mandated, and in New York where PR of PCI data is mandatory

Cohort study among different facilities with and without PR mechanisms, over the same period of time

Patients included in a multicenter PCI database in Michigan Hospitals and statewide PCI database in New York Hospitals (1998–2000)

PCI database in Michigan and New York

Cardiac mortality

The unadjusted in-hospital mortality rate was significantly lower in New York than in Michigan (0.83 % vs. 1.54 %, OR = 0.54). However, after adjustment for comorbidities, there was no significant difference in mortality between the two groups (adjusted OR = 1.05).

None

Carey et al., 2006 [30]

To examine the relationship between CCSIP and cardiac surgery mortality in California Hospital

Cohort study for the same facility before and after the introduction of a PR mechanism

Cardiac surgery patients (CABG, PCI, Valve) in California Hospital (1998–2004)

CCSIP

Cardiac mortality

The risk-adjusted in-hospital mortality for CABG decreased and PCI mortality remained unchanged. Combining the two procedural groups, the average annual mortality was 1.88 % (1998–2002) compared with 1.67 % (2003–2004)

Positive

Guru et al., 2006 [31]

To evaluate the differences in clinical outcomes observed during the transition from no reporting to confidential, and ultimately PR cards for CABG surgery in a public health system in Ontario

Cohort study for the same facility before and after the introduction of a PR mechanism

CABG surgery patients in Ontario Hospitals (1 September 1991–31 March 2002)

Ontario institution-level performance report cards on outcomes of CABG surgery

Cardiac mortality

The risk-adjusted 30-day mortality rate decreased 29 % from the era of no reporting (1991–1993) to confidential reporting (1994–1998). There was no further decrease with PR (1999–2001). In-hospital mortality fell significantly faster in Ontario during the period of confidential reporting than in other parts of Canada

Positive

Jha et al., 2006

To examine the impact of NYS CSRS fifteen years after its launch on cardiac surgery mortality

Cohort study for the same facility before and after the introduction of a PR mechanism

All New York Hospital cardiac surgeons performing CABG (1989–2002)

NYS CSRS

Cardiac mortality

Users who picked a top-performing hospital or surgeon from the latest available report had approximately half the chance of dying (risk-adjusted mortality rate = 1.59) as did those who picked a hospital or surgeon from the bottom quartile (risk-adjusted mortality rate = 2.78).

Positive

Hollenbeak et al., 2008 [33]

To assess effect of intensive PHC4 on hospital mortality for 6 high-frequency, high-mortality medical conditions

Cohort study among different facilities with and without PR mechanisms, over the same period of time

Cardiac surgery patients in Pennsylvania Hospitals (1997–2003)

PHC4

Cardiac, Respiratory, Neurologic and Sepsis mortality

Patients treated at hospitals subjected to intensive PR had significantly lower odds of in-hospital mortality when compared with similar patients treated at hospitals in environments with no PR or only limited reporting. The 2000–2003 in-hospital mortality OR for Pennsylvania patients versus non-Pennsylvania patients ranged from 0.59 to 0.79 across 6 clinical conditions. For the same comparison using the 1997–1999 period, OR ranged from 0.72 to 0.90.

Positive

Friedberg et al., 2009 [34]

To determine association of PR with over diagnosis of pneumonia, excessive antibiotic use, or inappropriate prioritization of patients with respiratory symptoms

Cohort study for the same facility before and after the introduction of a PR mechanism

Patients with respiratory symptoms in the National Hospital Ambulatory Medical Care Survey (2001–2005)

Hospital Quality Alliance data on antibiotic timing in pneumonia

Rates of pneumonia diagnosis, antibiotic use, and waiting times to see a physician

Public reporting of hospital antibiotic timing scores has not led to increased pneumonia diagnosis, antibiotic use, or change in patient prioritization. Comparing outcomes before and after antibiotic timing score reporting, there were no differences in rates of pneumonia diagnosis (10 % vs. 11 %) or antibiotic administration (34 % vs. 35 %).

None

Ryan, 2009 [35]

To evaluate the effects of the PHQID, a public quality reporting and P4P program, on Medicare patient mortality

Cohort study among different facilities with and without PR mechanisms, over the same period of time

Medicare patients with AMI, heart failure, pneumonia, or a CABG procedure from acute care hospitals (2000–2006).

PHQID program

Cardiac and Respiratory mortality

No evidence that the PHQID had a significant effect on risk-adjusted 30-day mortality for AMI, heart failure, pneumonia, or CABG.

None

Li et al., 2010 [36]

To evaluate the impact of PR by comparing CABG volume and mortality for hospitals and surgeons in the first year of state-mandated PR (2003), and with the most recent data available (2006)

Cohort study for the same facility before and after the introduction of a PR mechanism

Cardiac surgery patients from the California Hospital CABG Outcomes Reporting Program database for 2003 and 2006

California CABG Outcomes Reporting Program

Cardiac mortality

The statewide observed mortality declined from 2.90 % in 2003 to 2.22 % in 2006. Overall, the empiric odds ratio of operative death for 2006 patients was 24 % lower than for 2003 patients. Total CABG volume decreased from 2003 to 2006 by almost 27 %.

Positive

Werner et al., 2010 [37]

To estimate changes in cardiac and respiratory mortality, length of stay and readmission rate after Hospital Compare was initiated

Cohort study for the same facility before and after the introduction of a PR mechanism

Patients with AMI, heart failure, pneumonia from 3,476 acute care, nonfederal U.S. hospitals that publicly reported quality information on the CMS Hospital Compare Web site (2004–2006)

The Centers for Medicare and Medicaid Services and other health care organizations participate in the Hospital Quality Alliance

Cardiac and Respiratory mortality, length of stay, readmission rate

There was a decline in mortality rates (0.6 % points), lengths-of-stay (0.19 days), and readmission rates (0.5 % points) for acute myocardial infarction from 2004 to 2006. Changes in outcomes for heart failure and pneumonia were less consistent and smaller, when present at all.

Positive

Jha et al., 2012 [38]

To assess the long-term effect of the Medicare PHQID on cardiac and respiratory mortality at Premier versus Non Premier hospitals

Cohort study among different facilities with and without PR mechanisms, over the same period of time

Patients with AMI, CHF, CABG, pneumonia in New England Hospital (2003–2009)

Premier Healthcare Informatics program

Cardiac and Respiratory 30-day mortality

No evidence that the largest hospital-based pay-for-performance program led to a decrease in 30-day mortality. The rates of decline in mortality per quarter at Premier and Non Premier hospitals were also similar (0.04 and 0.04 %, respectively; and mortality remained similar after 6 years under the pay-for-performance system (11.82 % for Premier hospitals and 11.74 % for non-Premier hospitals)

None

Joynt et al., 2012 [39]

To evaluate PCI mortality in PR states versus non-reporting states in USA

Cohort study among different facilities with and without PR mechanisms, over the same period of time

Medicare patients admitted with acute MI to US acute care hospitals (2002–2010)

Mandatory state PR programs (NY, MA and PA) for PCI

Cardiac mortality

There were no differences in overall mortality among patients with acute MI in reporting vs non reporting states. In Massachusetts, odds of PCI for acute MI were comparable with odds in non reporting states prior to PR (40.6 % vs 41.8 %; OR, 1.00). Among Medicare beneficiaries with acute MI, the use of PCI was lower for patients treated in states with PR compared with patients treated in states without PR

Mixed

Renzi et al., 2012 [40]

To evaluate association between public reporting of hospital performance and PCI rates, hip fractures, cesarean deliveries in Lazio versus other regions of Italy

Cohort study for the same facility before and after the introduction of a PR mechanism and among different facilities with and without PR mechanisms over the same period of time

Patients with acute MI, hip fractures, and maternity patients discharged from any hospital within the Italian National Health Service (2006–2009)

Regional Outcome Evaluation Program P.Re. Val.E.

PCI rates, hip fractures, cesarean deliveries

In Lazio PCI within 48 h, changed from 22.49 to 29.43 % following reporting of the P.Re.Val.E results. In the other regions this proportion increased from 22.48 to 27.09 % during the same time period. Hip fractures operated on within 48 h increased from 11.73 to 15.78 % in Lazio, and not in other regions (from 29.36 to 28.57 %). Cesarean deliveries did not decrease in Lazio (from 34.57 to 35.30 %, and only slightly decreased in the other regions (from 30.49 to 28.11 %).

Positive

Ryan et al., 2012 [41]

To estimate the effect of Hospital Compare, Medicare’s PR initiative on 30-day mortality for heart attack, heart failure, and pneumonia

Cohort study for the same facility before and after the introduction of a PR mechanism

Medicare patients in USA Hospitals with heart attack, heart failure, pneumonia, stroke, gastrointestinal hemorrhage, and hip fracture (2000–2008)

Hospital Compare, Medicare’s PR initiative

Cardiac and Respiratory 30-day mortality

Hospitals that reported quality data under Hospital Compare had no reductions in mortality beyond existing trends for heart attack and pneumonia and led to a modest reduction in mortality for heart failure (RR = 0.92)

None

Linkin et al., 2013 [42]

To evaluate the association between state-legislated PR of hospital-acquired infection with infection control process

Cohort study among different facilities with and without PR mechanisms, over the same period of time

Patients from 137 eligible US hospitals in 35 states (2008–2011)

Medicare’s Hospital Compare website reports

Improvements in infection prevention

There is not estimated improvement in infection prevention program or hospital-acquired infection rates in hospitals in states legislating mandatory PR

None

McCabe et al., 2013 [43]

To evaluate the impact of PR of hospitals as negative outliers, on PCI risk adjusted mortality and case mix selection

Cohort study among different facilities with and without PR mechanisms, over the same period of time

Cardiac patients at all non-federally funded Massachusetts hospitals performing PCI (2003–2010)

National Cardiovascular Data Registry and Massachusetts Data Analysis Center model

In-hospital cardiac mortality

After public identification as a negative outlier institution, there was an 18 % relative reduction in predicted mortality among PCI patients at outlier institutions compared with non-outlier institutions

None

Marsteller et al., 2014 [44]

To evaluate mandatory reporting in participation and performance in reducing CLABSI in a national patient safety collaborative

Cohort study among different facilities with and without PR mechanisms, over the same period of time

Patients of intensive care units participating in the US national Comprehensive Unit-based Safety Program: Stop Bloodstream Infections (2009–2011)

Comprehensive Unit-based Safety Program: Stop Bloodstream Infections

CLABSI mortality

There was a reductions in CLABSI rates in the first 6 months compared with the units in states with no reporting requirement. During months 13–18, both state groups with mandatory PR of CLABSI showed a trend toward greater reduction in CLABSI compared with states with no requirement.

Positive

Wang et al., 2014 [45]

To evaluate the effect of publicly reporting performance data of medicine use on the injection prescribing rate

Cohort study for the same facility before and after the introduction of a PR mechanism

Effective electronic injection prescriptions in Primary healthcare institutions of Hubei province (China 2013–2014)

Database of electronic prescriptions of the local health bureau

Injection prescribing rates

PR led to a reduction of approximately 4 % in the injection prescribing rate four months after intervention (OR = 0.96). The intervention effect was inconsistent in each month after intervention, and it was most positive in the second month after intervention (OR = 0.90)

Positive

Yang et al., 2014 [46]

To evaluate the impact of PR on antibiotic prescribing for URTI in a sample of primary care institutions

Cohort study among different facilities with and without PR mechanisms, over the same period of time

URTI patients in Primary healthcare institutions of Hubei province (China 2013–2014)

Electronic health information system

Antibiotic prescribing

PR interventions reduced the incidence of oral antibiotic prescription (9 % point reduction adjusted RR = 39 %) and slowed down the increase of combined use of antibiotics for URTIs (7 % point reduction (adjusted RR = 36 %), while the use of injectable antibiotics remained unchanged. The intervention had little impact on the use of IV injections or infusions, or the total prescription expenditure

Mixed