Risk adjustment for inter-hospital comparison of primary cesarean section rates: need, validity and parsimony

The evaluation of medical quality and performance has become an integral part of health care systems. Hundreds of indicators have been developed to assess the quality and performances of health facilities and health systems and to make inter-hospital comparisons. Cesarean section (c-section) rate is one of the most frequently used quality indicators to evaluate or compare obstetric departments. The utilization of c-section for reasons other than medical necessity [1–3] and the associated costs [4] make this procedure particularly subject to observation by policy makers and public health experts [5].

Hospitals and health systems are often evaluated based on their cesarean delivery rates, with the implicit assumption that lower rates reflect more appropriate, as well as more efficient, clinical practice.

Cesarean section rate has high face validity and is easy to measure because its predictors are reported in administrative data. However, the apparent simplicity of calculating the cesarean section rate can be deceptive. In fact, there is little consistency across performance measurement systems in the specifications of how it is calculated. There are differences in how the population is defined (i.e. who is included and excluded) and how the risk adjustment methodologies are applied [5].

Cesarean delivery is indicated for many clinical situations such as placenta previa, HIV and other risk factors [4, 6, 7] and the failure to account for such patient-specific risk factors may lead to biased comparisons. This may be particularly problematic for making inter-hospital comparisons, given the wide variations in socio-demographic and clinical characteristics of patients at different hospitals, and the referral bias that can result from differences in the availability of clinical services for women with high-risk pregnancies [8].

Risk adjustment is one technique to identify and control for potential confounders [9]. It is increasingly used in observational studies, and has already been used to compare c-section rates between hospitals that are homogenous regarding their "a priori" risk of cesarean delivery. Various authors have demonstrated modest to poor agreement between hospital rankings based on unadjusted and adjusted c-section rates [5, 8, 10–12]. Only one author did not observe important differences in hospital rankings, suggesting that additional resources for complex data adjustment may not be warranted [6].

In most cesarean c-section studies, rates were adjusted by controlling for a large number of potential confounders, including socio-demographic, maternal and fetal clinical conditions, even if they were not actual confounders [13]. However, collecting many variables is onerous and is associated with problems of data completeness, accuracy, and reliability, and tends to reduce the precision of the adjusted measures [14]. Moreover, no consensus has been reached regarding which clinical, demographic, and/or hospital organizational factors should be considered actual confounders.

There are only a few studies [1, 10, 15] regarding risk-adjustment methods for inter-hospital comparison of c-section rates that have been conducted in Europe. We present one carried out in Emilia Romagna (Italy), to determine whether risk adjustment is necessary for inter-hospital comparison and to identify a risk-adjustment model based on a limited number of variables to increase the precision of estimates without compromising validity.

Since 1995 in the Emilia Romagna Region of Northern Italy (RER), all hospital discharge abstracts have been electronically recorded, using a Hospital Information System (HIS). The data stored in the system includes demographics [ID number, gender, date and place of birth, place of residence], discharge ID, admission and discharge dates, up to 9 discharge diagnoses and 9 procedures (International Classification of Diseases, 9 ^th Revision, Clinical Modification ICD-IX-CM), ward(s) of hospitalization, date(s) of in-hospital transfer, and the regional code of the admitting facility.

We selected all hospital discharge abstracts for women in labor and of newborns from 36 maternity units in the region from January 2003 to December 2004.

This study takes as its sample live births for whom the discharge records for the mothers and infants were linked by hospital code, mother's discharge ID and date of delivery.

To identify the delivery, we used Diagnosis-Related Groups (DRGs) 370–375 from the discharge data. DRG 370 and 371 (cesarean section with and without complication, respectively) were used to identify cesarean deliveries. ICD-IX-CM diagnosis code 654.2x was used to identify any previous cesarean deliveries [16]. The number of births from primary cesareans was calculated as the difference between the number of births from c-sections deliveries and number of births from c-sections deliveries in women with previous cesareans.

Therefore, primary cesarean rates were calculated with the formula:

$\frac{\text{Number of births from primary cesarean deliveries}\cdot 100}{\text{Number of births from deliveries with no previous c-section}}$

There were 62,836 births from deliveries to women with no previous c-section and they were included in our study population, excluding the following:

Hospitals with fewer than 100 deliveries per year were excluded to warrant sufficient power of comparison.

The following socio-demographic variables, considered as potential risk factors for cesarean sections, were collected: maternal age (<17, 18–20, 21–24, 25–28, 29–33, 34–38 ≥ 39), citizenship, (Italian, from developing countries, undeveloping countries other than Italy), residency (RER or other), and marital status (married, divorced-separated, single, widow). Maternal and neonatal clinical factors were also retrieved. These factors were defined using the primary and secondary discharge diagnoses of the delivery and newborn admission ' [see Additional file 1]'.

We did not consider dystocia and fetal distress as potential risk factors because of the poor reliability of their definition and because this diagnosis may reflect post ad hoc justifications of cesarean use, rather than objectively assessed conditions [17, 18].

The study was conducted in collaboration with the Azienda Ospedaliera Sant'Orsola-Malpighi, the teaching hospital of the University of Bologna, Italy

Of a total of 62,836 births from deliveries with no previous c-sections in the RER during 2003–2004, 15,197 (24.2 %) were births from primary caesarean deliveries.

The change-in estimate procedure identified seven variables which act as confounders in at least one comparison: marital status, age of mother, infant birth weight, fetopelvic disproportion/excessive development of the infant, eclampsia or pre-eclampsia, antepartum hemorrhage/abruptio placentae/placenta previa, malposition and malpresentation of fetus. These factors were included in the parsimonious model; antepartum hemorrhage/abruptio placentae/placenta previa, malposition and malpresentation of fetus accounted for the greatest adjusted ORs.

The discrimination capacities were high for both models (0.78 full model; 0.73 parsimonious model) and the Hosmer-Lemeshow statistic showed a lower calibration in the parsimonius than in the full model (H-L = 24.76 p = 0.002, H-L = 7.32 p = 0.503 respectively). The AIC statistic was similar in the two models (AIC= 52.195,9 full model; AIC= 54.803,4 parsimonious model).

Hospital W had the highest C-section rate when analyzing crude ORs (OR = 5.79; 95% CI: 4.83–6.94) or adjusted ORs, estimated by both models (OR = 8.77; 95% CI: 7.11–10.80 and OR = 8.55; 95% CI: 6.97–10.49 respectively). Adjusted ORs by the full model were greater than crude ORs in 24 out of 25 hospitals, adjusted ORs by the parsimonious model were greater than the crude value in 24 out of 25 hospitals

Eighteen units had their rank change after adjustment by the full model: seven facilities moved 1 position, four moved 2 or 3 positions, seven moved 4–10 positions. The parsimonious model changed the ranking of 19 facilities: ten moved 1 position, four moved 2 or 3 positions, five moved 4–10 positions.

The correlation coefficient between hospital rankings ordering crude and adjusted ORs by the full model was 0.87, and 0,80 by the parsimonious model. The correlation coefficient between hospital rankings ordering adjusted ORs by the two models was 0.92.

The ratios between upper and lower 95% confidence intervals for the hospital specific ORs estimated by the "parsimonious" model were lower than those obtained by the "full" model; the former improved the precision of the estimates.

Our results indicate that risk adjustment, by removing the inherent bias associated to non random allocation of deliveries, substantially changes inter-hospital comparisons.

After adjusting for heterogeneity of distribution of risk factors for c-section, the ranking of maternity units was substantially modified, with most hospitals registering higher adjusted than crude c-section rates.

In addition to adjusting for a large number of potential confounders, the specific goal of this study was to identify an efficient model that included only actual confounders of the comparison between hospitals. Including factors in a risk adjustment model that do not induce a relevant bias on the measure of association may reduce precision of estimates. The change-in-estimate is one method that improves the parsimony of the model and still results in precise estimates, by eliminating variables that are not actual confounders. To act as a confounder a variable must be associated with the outcome of interest (i.e. c-section) and heterogeneously distributed between categories of exposure (i.e. hospitals). Among the 24 factors identified as c-section predictors by the full model, only seven were the actual confounders used in the parsimonious model. Six out of seven actual confounders were clinical conditions of the mother (antepartum hemorrhage/abruptio placentae/placenta previa, malposition and malpresentation of fetus, eclampsia or pre-eclampsia), the fetus (birth weight), or both (fetopelvic disproportion/excessive development of the infant); these factors and maternal age have already been recognized by previous studies as risk factors for c-section [7, 8, 23–28]. Marital status, the remaining actual confounder, could be a surrogate of unmeasured risk. It is beyond the scope of our study to discuss the possible reasons of the heterogeneous distribution of these risk factors across hospitals, likely to be related to selection factors at work in the health care system.

Two general categories of factors might explain the variation in primary cesarean section rates between hospitals: case mix and hospital performance. In Emilia Romagna, there was an increase in inter-hospital variability after adjusting for clinical case mix, confirming that differences in c-section rates are mainly due to non clinical factors.

Although many Authors [5, 11, 12, 18] advocate considering case mix when comparing c-section risk, the impact risk adjustment has on hospital comparisons and rankings differs between studies [5, 8, 28].

In addition to the mixed findings regarding the role of case mix in explaining inter-hospital variations, there are important methodological differences between studies. They are related to the source of data used (birth certificates, medical records and insurance claims), to criteria used to define c-section, to inclusion and exclusion criteria, to the final summary indicators (rates or ORs) produced, to the methods used for risk adjustment and to the variables controlled [17].

In this study we defined c-section based on DRGs. A previous study [5] showed high reliability among different methods used to calculate c-section rates.

The highest adjusted ORs obtained for the clinical variables identified as actual confounders in the parsimonious model address an important issue regarding appropriate adjustment factors. Although one would think that risk factors for cesarean section would be consistent across studies, there is inconsistency in the risk factors included in the adjustment models [8, 11, 29–32]. For example, factors like presentation other than vertical malposition or malpresentation, fetopelvic disproportion/excessive development, and placenta previa tend to indicate complicated pregnancies where c-section is often the only choice.

It is therefore possible that instead of adjusting for complicated pregnancies, they should be evaluated independently from others without such complications [33].

This study, as many other studies, evaluated c-section performance on administrative discharge data. Problems in accuracy, completeness, and quality might differ from hospital to hospital. The potential for inconsistencies in the coding of discharge records challenges the accuracy of the assessment of the outcome and of the risk factors in both the study population and in other populations [12]. Errors in coding could have occurred, which would have resulted in subsequent errors in adjustment. Omissions of ICD codes identifying risk factors, were more likely in the group without c-section leading to an excess of risk adjustment. Nevertheless, discharge databases are widely available at the state and regional levels, and are already routinely used. Administrative data have proved to be an accurate source to monitor c-section rates and a reliable data source to adjust for risk factors [33–36]. Moreover, administrative data from Emilia Romagna are considered of good quality, especially when compared with those from other regions in Italy. Methods used to develop models based on administrative information have the potential to be generalized to other populations. However, any risk adjustment model should be considered time and population specific.

Another limit of the study is the impossibility of including all possible clinical factors in the model; maternal parity, primipary, fetal distress and dystocia, for example are known risk factors for cesarean section [7, 37], but were not included in the model because the information was either unavailable, incomplete, or considered unreliable. The result being that c-section ORs in some hospitals, especially teaching and referral hospitals might have been underestimated.

Risk adjustment is necessary to compare hospital c-section rates, it shows differences in rankings and highlights inappropriateness of some hospitals. By adjusting for only actual confounders valid and more precise estimates could be obtained.

Anyway, additional studies, including qualitative studies, are recommended to identify which clinical and non clinical factors can explain inter-hospital variability. These factors should be explored in order to address the inappropriate use of this procedure.