Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Risk adjustment for cesarean delivery rates: how many variables do we need? An observational study using administrative databases

  • Elisa Stivanello1Email author,
  • Paola Rucci1,
  • Elisa Carretta1,
  • Giulia Pieri1 and
  • Maria P Fantini1
BMC Health Services Research201313:13

DOI: 10.1186/1472-6963-13-13

Received: 13 August 2011

Accepted: 26 December 2012

Published: 10 January 2013

Abstract

Background

Various studies indicate that inter-hospital comparisons have to take case mix into account and that risk adjustment procedures are necessary to control for potential predictors of cesarean delivery (CD). Different data sources have been used to retrieve information on potential predictors of CD. The aim of this study was to compare the discrimination capacity and fit of predictive models of CD created using different sources and to assess whether more complex models improve inter-hospital comparisons.

Methods

We created 4 predictive models of CD. One model included only variables from Hospital Discharge Records of the index hospitalization, one included also information from previous hospitalizations, one also clinical variables from birth certificates (BC) and one also socio-demographic variables. We compared the four models using the Receiver Operator Curve and the Akaike and Bayesian Information Criteria.

Results

Information from Birth Certificates improved the discrimination and model fit. Adding socio-demographic variables or past comorbidities did not improve the discrimination capacity or the model fit. Hospital-specific CD resulting from the models were highly correlated.

Conclusions

Record linkage improves the performance of the models but does not affect inter-hospital comparisons.

Background

Since the eighties, the World Health Organization (WHO) has recommended not to exceed a cesarean rate of 10–15% [1]. However, more recently, the WHO stated that, ‘both very low and very high rates of caesarean section can be dangerous, but the optimum rate is unknown’ and that ‘there is no empirical evidence for an optimum percentage or range of percentages, despite a growing body of research that shows a negative effect of high rates.’ In the absence of a optimum rate, it is worthwhile to compare cesarean delivery (CD) rates across hospitals in order to identify the birth units that mostly deviate from average values and decide where most efforts should be made to improve the quality and safety of birth [2].

Hospitals can be compared in terms of overall or primary CD rates. Primary CD rates are calculated after excluding women with a previous CD, who are at a very high risk for another cesarean section [35]. Moreover, various studies indicated that inter-hospital comparisons have to take case mix into account [611]. Risk adjustment procedures are necessary to control for potential confounders (socio-demographic and clinical risk factors of the mother and the fetus) that are not homogeneously distributed among different hospital populations.

Various data sources have been used to determine the overall or primary CD rate and retrieve information on potential predictors of CD. Some authors used only one data source, i.e. medical records [6] or hospital discharge records (HDR) [1114] that, in some countries, such as the UK, include a maternity tail containing additional information on the pregnancy [11]. Other authors used multiple data sources linked to each other: maternal HDR linked to infant HDR [9, 15], maternal HDR linked to clinical records [16] or to birth certificates (BC) [10] or maternal HDR linked to both infant HDR and BC [3, 5, 17].

Socio-demographic factors and clinical variables such as fetal weight and parity are usually retrieved from BC because here they are more complete, whereas co-morbidities of the mother are more frequently retrieved from HDR of the index hospitalization or of previous hospitalizations [3, 9, 10]. Database linkage entails the availability of more information but linkage procedures may be cumbersome, unfeasible or the proportion of records linked over the total records may be very low [18]. Empirical data from other clinical populations, such as general and cardiac surgery, reveal that most risk prediction comes from a relatively small number of variables [19, 20]. To our knowledge no study explored this issue in relation to cesarean delivery. This study aimed to determine whether using one instead of two data sources and reducing the number of covariates impact CD risk prediction and affect inter-hospital comparisons of primary CD rates.

Methods

The study was carried out in Emilia Romagna, a region located in North-Eastern Italy with about 4.4 million inhabitants and approximately 40,000 births per year. The national health service is statutorily required to guarantee the uniform provision of comprehensive care throughout the country. Essential health services are provided free of charge, or at a minimal charge. Local Health Authorities, that roughly coincide with administrative units (provinces) are responsible for the overall health of, and for the services offered to the population.

The HDR of the women who delivered in the Emilia Romagna from 1st January 2007 to 30th June 2009 were linked to the BC using the HDR identification code, the hospital code and the year of delivery.

The HDR include demographics (ID number, sex, date and place of birth, place of residence), discharge ID, admission and discharge dates, discharge diagnoses and procedures (International Classification of Diseases, 9th Revision, Clinical Modification ICD-IX-CM), ward(s) of hospitalization, date(s) of in-hospital transfer, and the regional code of the admitting facility.

BC include demographic data of the mother, information on presentation and multiple pregnancy (singleton cephalic, singleton breech, transverse or oblique lie, etc.), parity (nulliparous, multiparous), the course of labour and delivery (spontaneous labour, induced labour or CS before labour) and gestational age (defined as the number of completed weeks at the time of birth) and other information on the newborn.

HDR were identified by using DRG codes (370–375), diagnosis (ICD-9-CM 640.xy - 676.xy (y = 1,2), V27 ) and procedure codes (ICD-9-CM 72.xy - 74.xy).

Records of women with a previous cesarean (ICD-9 CM code 654.2), discharged from hospitals without an operating room or small hospitals (<150 deliveries per year) or with a diagnosis of intrauterine death or still births (ICD-9 CM codes 656.4 V27.1, V27.4, V27.7) were excluded.

Primary CD were defined using procedure codes (ICD 9CM codes: 74.0, 74.1, 74.2, 74.4, 74.99 ) or diagnosis codes (ICD-9-CM 669.7) or DRG codes (370–371) or one BC variable (type of delivery).

We created 4 predictive models for primary CD (the Additional file 1 shows all variables of the four models and sources of information) using logistic regression:
  • Model 1 included age and maternal comorbidities and information about delivery recorded in the HDR of the index hospitalization

  • Model 2 included, in addition, past comorbidities recorded in HDR of hospitalizations occurred two years before delivery

  • Model 3 included additional clinical variables retrieved from the BC

  • Model 4 included also socio-demographic variables retrieved from the BC.

We applied a backward stepwise procedure to model 4 to identify the subset of variables significantly associated with caesarean section. Adjusted Odds ratios (OR) with 95% Confidence intervals (CI) were calculated for all models.

We evaluated the models at both patient and hospital level by comparing the full model (model 4) including variables derived from the two data sources with more parsimonious models, following the procedure of Dimick et al. [20]. In particular, to evaluate patient level risk prediction we used:
  • the Receiver Operating Characteristic (ROC) to assess how well the model discriminates between women with and without a CD. The area under the curve ranges from 0.50 (no ability to discriminate) to 1 (perfect discrimination).

  • the AIC (Akaike Information Criterion) and BIC (Bayesian Information Criterion), to obtain measures that combine fit and complexity of the model. Lower values indicate a better fit of the model taking complexity into account.

To evaluate hospital level risk adjustment, we compared CD measures obtained using the 4 models. To this purpose, we calculated the ratio of observed to expected CD (“O/E ratio”) at each hospital. The O/E ratio was calculated using logistic regression to predict a probability of CD (i.e., the expected outcome) for each woman. These probabilities were then summed for every hospital. The observed number of events was divided by the expected number, to obtain a risk-adjusted estimate of the outcome of interest; an O/E ratio of 1.0 is as expected given that hospital’s women characteristics, less than 1.0 is better than expected and greater than 1.0 is worse than expected. We calculated O/E ratios using the 4 models to estimate the expected CD.

We then analysed the correlations between the hospital O/E ratios obtained with different models. A correlation coefficient of 1.0 implies perfect agreement in O/E ratios [20].

Results

We identified 102695 HDR and 100946 BC. 97% of the HDR were linked with BC (99626). After applying inclusion and exclusion criteria we obtained 87849 deliveries and a study population of 87574 women with complete data for the variables of interest in 29 hospitals. The primary cesarean section rate was 22.36%. The number of variables that were left in the models was: 21 in model 1 and 2, 26 in model 3 and 29 in model 4. Additional file 2 shows the characteristics of the study sample and the adjusted OR of CD for all selected variables in the four models. Across all models, a maternal age lower than 30 years was associated with a lower likelihood of CD, with aOR ranging from 0.34 to 0.88, while an age higher than 34 years was associated with an increased likelihood of CD, with aOR ranging from 1.18 to 2.36. Other demographic characteristics, included in model 4, contributed to predicting CD. In particular, a citizenship in high-income countries, compared to Italian citizenship, was associated with a 27% reduced likelihood of CD (aOR=0.73, 95% CI 0.59-0.91). Middle school education and University-level education were associated with a 14% and 18% reduced likelihood of CD compared to high-school education and a non-declared marital status was also associated with a 11% reduction in the likelihood of CD compared with being married.

Cord prolapse, malpresentation, placenta praevia, placental abruption or uterine hemorrhage, HIV and multiple births, were the risk factors associated with the highest risk of CD in all models.

Table 1 shows the patient level discrimination capacity and the model fits. The areas under the ROC curves ranged from 75.89% in model 1 to 83.11% in model 4, with significant differences between models 2 and 3 and almost no differences between models 1 and 2 and between models 3 and 4 (Figure 1). In terms of goodness of fit and taking complexity into account, model 3 and 4 performed better than model 1 and 2, with only small differences between models 1 and 2 and between models 3 and 4. When considering the hospital level risk prediction, the hospital O/E ratios obtained with different models (Table 2) were highly correlated with each other. The correlation between model 1 and 2 was 1, between model 3 and 4 was 0.99 and between model 2 and 3 was 0.95.
Table 1

Discrimination and calibration capacity and goodness of fit of the four predictive models

 

Model 1

Model 2

Model 3

Model 4

ROC

75.89

75.90

82.98

83.11

(95% CI)

(75.62–76.49)

(75.63–76.50)

(82.63–83.32)

(82.76–83.45)

AIC

69321.11

69325.12

64048.5

63975.75

BIC

69565

69569.01

64367.43

64379.1

https://static-content.springer.com/image/art%3A10.1186%2F1472-6963-13-13/MediaObjects/12913_2011_Article_2406_Fig1_HTML.jpg
Figure 1

Receiver operating characteristic curves of the four models.

Table 2

Hospital volumes and O/E ratios

  

O/E ratios

hospital

n. deliveries

Model 1

Model 2

Model 3

Model 4

1

3000

1.23

1.22

1.25

1.25

2

1710

1.22

1.22

1.25

1.24

3

1513

0.98

0.98

1.01

1.00

4

473

1.30

1.30

1.40

1.40

5

1869

0.98

0.98

1.09

1.07

6

1914

0.97

0.97

1.02

1.02

7

1558

1.18

1.18

1.25

1.24

8

510

1.39

1.38

1.44

1.43

9

3735

0.68

0.68

0.71

0.71

10

1364

0.83

0.83

0.84

0.82

11

1085

0.66

0.66

0.74

0.73

12

2501

1.09

1.08

1.09

1.08

13

6408

0.92

0.92

0.89

0.89

14

2315

1.16

1.16

1.15

1.15

15

1289

0.98

0.99

0.99

1.05

16

1687

0.69

0.69

0.74

0.79

17

3271

1.05

1.05

1.02

1.02

18

2337

0.76

0.76

0.78

0.78

19

2033

0.77

0.77

0.79

0.79

20

3456

1.04

1.04

1.05

1.04

21

4837

0.75

0.75

0.76

0.76

22

6195

1.04

1.04

0.98

0.98

23

2370

0.98

0.98

1.05

1.05

24

781

1.41

1.40

1.35

1.34

25

5520

1.17

1.17

1.17

1.17

26

5402

0.89

0.89

0.89

0.88

27

7240

1.03

1.03

1.06

1.06

28

7916

1.18

1.18

1.12

1.13

29

3285

0.91

0.92

0.87

0.87

We carried out secondary analyses excluding from the models preterm deliveries, women with herpes or antepartum hemorrhage/ placental abruption/placenta previa or malpresentations. The areas under the ROC curves for the 4 models ranged from 64.23% to 75.02% denoting a poorer discrimination capacity of the remaining variables to discriminate cesarean section. Results confirmed that models 3 and 4 were almost equivalent and performed better than models 1 and 2. The correlations of the hospital O/E ratios obtained with the four models ranged from 0.99 to 1.

Discussion

Our study shows that all the four models have a high discrimination capacity but models including variables retrieved from both BC and HDR show a higher discrimination capacity than the ones including only variables from the HDR. In addition, the models including variables from both BC and HDR show a better fit taking complexity into account. Socio-demographic variables, such as maternal education, citizenship and maternal status, or variables derived from previous hospitalizations did not modify the discrimination capacity or the fit of the models substantially.

The discrimination capacity of our predictive models is in line with previously reported findings [6, 7, 11, 16] even if other studies including only clinical risk factors report a higher discrimination capacity [3, 10, 21]. This suggests that in our population other not considered factors may play an important role in predicting CD. In terms of predicting the probability of a CD, the fit of all models was imperfect as reported by other authors [10].

In our study malpresentation, cord prolapse, placenta praevia and placental abruption proved to be the most important risk factors for cesarean delivery. This finding is consistent with prior epidemiological studies and clinical knowledge [6] and supports the predictive validity of the models.

A significant difference in terms of discrimination and fit was observed between model 2 and 3. The latter includes clinical variables retrieved from BC: experience of previous abortion or stillbirth, fetal weight, fetal malformation, gestational age and parity. None of these variables are absolute indications for caesarean delivery but they are often included in risk adjustment models by other authors because they are important and frequent predictors of CD [9, 11]. In our sample, 10% of the women did not deliver at term, 13% delivered a low, very low or high weight baby, 16% had a previous experience of previous abortion or stillbirth and 40% were multipara. Only fetal malformations occurred rarely (less than 1%). In particular, two of these variables, gestational age and parity are considered discriminatory when classifying deliveries according to the 10 Robson classification groups [22].

We found that adding variables retrieved from previous hospitalizations did not modify the performance of the models substantially. Cesarean section does not seem to be predicted by comorbidities registered in previous hospitalizations. These co-morbidities may have resolved and do not represent predictive factors any more or are sufficiently explained by comorbidities reported in the index hospitalization. Women who deliver are generally healthy and comorbidities present during delivery are probably sufficient to describe health problems.

Maternal and paternal education, citizenship and marital status proved to contribute very little to the model fit. The only study that examined the impact of socio-demographic variables [5] found that race and ethnicity do not affect risk adjustment models for cesarean delivery.

Our study also shows that even if the models with more variables explain more variation in the proportion of cesarean deliveries, at hospital level simple adjustment work as well as the complex one: hospital O/E ratios based on the most parsimonious model (model 1) correlated very well with the O/E ratios obtained using the most comprehensive set of variables (model 4).

Similarly Dimick et al. [20] studied risk adjustment models for surgical procedures and found that predictive models with a different number of variables had a very similar discrimination, good calibration and that hospital specific indicators resulting from parsimonious models were highly correlated [20].

This is a large population study, with very recent data. It relies on administrative data, which despite issues regarding their validity, are the most feasible option for searching variation in clinical practices, health outcomes and quality of care [14, 16]. Previous work [14] suggests that administrative data may be equally discriminatory compared with data abstracted from clinical charts with respect to key outcomes. A limitation of this study is the potential lack of generalizability to other settings: the variables needed in adjustment models may not be the same if the quality of the coding is different or if databases include different sets of variables than those included in our HDR and BC. Our additional analyses excluding preterm gestation and other conditions such as herpes, antepartum hemorrhage, placental abruption, placenta previa or malpresentations showed, for instance, that homogenized populations yield a poorer predictive ability of the model (because the most important risk factors and indications for CD are omitted) but very similar O/E ratios across models.

The identification of the best predictive models is only the first step when applying risk adjustment procedures; variables in predictive models might be homogeneously distributed across hospitals and therefore are not actual confounders even if strongly associated with the outcome.

Conclusions

This study highlights the need to consider the potential tradeoff between the accuracy of risk adjustment, the efficiency of data collection and the precision of the estimate. Undoubtedly, adding information from BC (parity, gestational age, previous stillbirth, fetal characteristics) to that available from HDR provides the most accurate data for risk adjustment. However, linkage between different datasets might be unfeasible or lead to a high proportion of missing data and potential selection bias. We argue that when the focus is on hospitals, using only demographic and clinical information from HDR for risk adjustment is a very reasonable option, given the high correlation of O/E ratios obtained between the most parsimonious models and the full model.

Declarations

Acknowledgements

The authors thank Dr. Martina Taglioni for providing data for the project.

Authors’ Affiliations

(1)
Department of Medicine and Public Health, University of Bologna

References

  1. WHO – World Health Organization: Appropriate technology for birth. Lancet. 1985, 2: 436-437.Google Scholar
  2. World Health Organization, UNFPA, UNICEF and Mailman School of Public Health. Averting Maternal Death and Disability (AMDD): Monitoring Emergency Obstetric Care: a handbook. 2009, ISBN: 978 92 4 154773 4 http://www.who.int/reproductivehealth/publications/monitoring/9789241547734/en/index.html.Google Scholar
  3. Keeler EB, Park RE, Bell RM, Gifford DS, Keesey J: Adjusting cesarean delivery rates for case mix. Health Serv Res. 1997, 32 (4): 511-528.PubMedPubMed CentralGoogle Scholar
  4. Main EK, Moore D, Farrell B, Schimmel LD, Altman RJ, Abrahams C, Bliss MC, Polivy L, Sterling J: Is there a useful cesarean birth measure? Assessment of the nulliparous term singleton vertex cesarean birth rate as a tool for obstetric quality improvement. Am J Obstet Gynecol. 2006, 194: 1644-1651. 10.1016/j.ajog.2006.03.013.View ArticlePubMedGoogle Scholar
  5. Bailit JL, Lov TE, Bailit JL, Love TE: The role of race in cesarean delivery rate case mix adjustment. Am J Obstet Gynecol. 2008, 198 (1): 69.e1-5.View ArticlePubMedGoogle Scholar
  6. Aron DC, Harper DL, Shepardson LB, Rosenthal GE: Impact of risk-adjusting cesarean delivery rates when reporting hospital performance. JAMA. 1998, 279: 1968-1972. 10.1001/jama.279.24.1968.View ArticlePubMedGoogle Scholar
  7. Bailit JL, Dooley SL, Peaceman AN: Risk adjustment for interhospital comparison of primary cesarean rates. Obstet Gynecol. 1999, 93: 1025-1030. 10.1016/S0029-7844(98)00536-5.PubMedGoogle Scholar
  8. American College of Obstetricians and Gynecologists (ACOG): Evaluation of cesarean delivery. 2000, Washington, DCGoogle Scholar
  9. Fantini MP, Stivanello E, Frammartino B, Barone AP, Fusco D, Dallolio L, Cacciari P, Perucci CA: Risk adjustment for inter-hospital comparison of primary cesarean section rates: need, validity and parsimony. BMC Health Serv Res. 2006, 15: 100.View ArticleGoogle Scholar
  10. Tang CH, Wang HI, Hsu CS, Su HW, Chen MJ, Lin HC: Risk-adjusted cesarean section rates for the assessment of physician performance in Taiwan: a population-based study. BioMed Cent Public Health. 2006, 6: 246.View ArticleGoogle Scholar
  11. Bragg F, Cromwell DA, Edozien LC, Gurol-Urganci I, Mahamood TA, Templeton A, van der Meulen JA: Variation in rates of caesarean section among English NHS trusts after accounting for maternal and clinical risk: cross sectional study. BMJ. 2010, 341: c5065-10.1136/bmj.c5065.View ArticlePubMedPubMed CentralGoogle Scholar
  12. Gregory KD, Korst LM, Platt LD: Variation in elective primary cesarean delivery by patient and hospital factors. Am J Obstet Gynecol. 2001, 184 (7): 1521-1532. 10.1067/mob.2001.115496. discussion 1532–4View ArticlePubMedGoogle Scholar
  13. Linton A, Peterson MR: Effect of preexisting chronic disease on primary cesarean delivery rates by race for births in U.S. military hospitals, 1999–2002. Birth. 2004, 31 (3): 165-175. 10.1111/j.0730-7659.2004.00301.x.View ArticlePubMedGoogle Scholar
  14. Korst LM, Gornbein JA, Gregory KD: Rethinking the cesarean rate: how pregnancy complications may affect interhospital comparisons. Med Care. 2005, 43 (3): 237-245. 10.1097/00005650-200503000-00006.View ArticlePubMedGoogle Scholar
  15. Aelvoet W, Windey F, Molenberghs G, Verstraelen H, Van Reempts P, Foidart JM: Screening for inter-hospital differences in cesarean section rates in low-risk deliveries using administrative data: an initiative to improve the quality of care. BMC Health Serv Res. 2008, 8: 3-10.1186/1472-6963-8-3.View ArticlePubMedPubMed CentralGoogle Scholar
  16. DiGiuseppe DL, Aron DC, Payne SM, Snow RJ, Dierker L, Rosenthal GE: Risk adjusting cesarean delivery rates: a comparison of hospital profiles based on medical record and birth certificate data. Health Serv Res. 2001, 36 (5): 959-977.PubMedPubMed CentralGoogle Scholar
  17. Li T, Rhoads GG, Smulian J, Demissie K, Wartenberg D, Kruse L: Physician cesarean delivery rates and risk-adjusted perinatal outcomes. Obstet Gynecol. 2003, 101 (6): 1204-1212. 10.1016/S0029-7844(03)00175-3.PubMedGoogle Scholar
  18. Cuttini M, Marini C, Bruzzone S, Prati S, Saracci R: Protection of health information in Italy: a step too far?. Int J Epidemiol. 2008, 38 (6): 1739-1740. Epub 2008 Sep 4View ArticlePubMedGoogle Scholar
  19. Dimick JB, Birkmeyer JD: Ranking hospitals on surgical quality: does risk-adjustment always matter?. J Am Coll Surg. 2008, 207 (3): 347-351. 10.1016/j.jamcollsurg.2008.04.014. Epub 2008 Jun 2View ArticlePubMedGoogle Scholar
  20. Dimick JB, Osborne NH, Hall BL, Ko CY, Birkmeyer JD: Risk adjustment for comparing hospital quality with surgery: how many variables are needed?. J Am Coll Surg. 2010, 210 (4): 503-508. 10.1016/j.jamcollsurg.2010.01.018.View ArticlePubMedPubMed CentralGoogle Scholar
  21. Gregory KD, Korst LM, Gornbein JA, Platt LD: Using administrative data to identify indications for elective primary cesarean delivery. Health Serv Res. 2002, 37 (5): 1387-401. 10.1111/1475-6773.10762.View ArticlePubMedPubMed CentralGoogle Scholar
  22. Robson MS: Classification of Cesarean Sections. Fetal and Matern Med Rev. 2001, 12 (1): 23-39.View ArticleGoogle Scholar
  23. Pre-publication history

    1. The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/13/13/prepub

Copyright

© Stivanello et al.; licensee BioMed Central Ltd. 2013

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.