Skip to main content

Validation of a measure of hospital maternal level of care for the United States



Lack of a validated assessment of maternal risk-appropriate care for use in population data has prevented the existing literature from quantifying the benefit of maternal risk-appropriate care. The objective of this study was to develop a measure of hospital maternal levels of care based on the resources available at the hospital, using existing data available to researchers.


This was a secondary data analysis. The sample was abstracted from the American Hospital Association Annual Survey Database for 2018. Eligibility was limited to short-term acute general hospitals that reported providing maternity services as measured by hospital reporting of an obstetric service level, obstetric services, or birthing rooms. We aligned variables in the database with the ACOG criteria for each maternal level of care, then built models that used the variables to measure the maternal level of care. In each iteration, the distribution of hospitals was compared to the distribution in the CDC Levels of Care Assessment Tool Validation Pilot, assessing agreement with the Wilson Score for proportions for each level of care. Results were compared to hospital self-report in the database and measurement reported with another published method.


The sample included 2,351 hospitals. AHA variables were available to measure resources that align with ACOG Levels 1, 2, and 3. Overall, 1219 (51.9%) of hospitals reported resources aligned with Maternal Level One, 816 (34.7%) aligned with maternal level two, and 202 (8.6%) aligned with maternal level Three. This method overestimates the prevalence of hospitals with maternal level one compared to the CDC measurement of 36.1% (Mean 52.9%; 95% CI47.2%—58.7%), and likely includes hospitals that would not qualify as level one if all resources required by the ACOG guidelines could be assessed. This method underestimates the prevalence of hospitals with maternal critical care services (Level 3 or 4) compared to CDC measure of 12.1% (Mean 8.1%; 95%CI 6.2% – 10.0%) but is an improvement over hospital self-report (24.7%) and a prior published method (32.3%).


This method of measuring maternal level of care allows researchers to investigate the value of perinatal regionalization, risk-appropriate care, and hospital differences among the three levels of care. This study identified potential changes to the American Hospital Association Annual Survey that would improve identification of maternal levels of care for research.

Peer Review reports


In the United States, severe maternal morbidity occurs in 1.1 – 1.4% of inpatient deliveries [1, 2]. Severe maternal morbidity increases maternal and infant care costs and is associated with an increased risk for neonatal mortality [3,4,5]. Women from racialized groups experience higher rates of severe maternal morbidity [6,7,8,9,10]. Most strategies for preventing severe maternal morbidity are focused on ensuring quality care once the woman arrives at the hospital for birth [11, 12]. One strategy, risk-appropriate care, prevents severe maternal morbidity by ensuring the woman reaches a facility with the resources necessary to provide care commensurate with the clinical complexity of the pregnancy [13].

The purpose of risk-appropriate care is highlighted in the Three Delays Framework which hypothesizes that maternal death is the cumulative result of a lack of timely care [14]. According to the framework, there are three points where delay threatens a woman’s health to the extent that death may not be preventable, even when appropriate care is eventually provided. These points are seeking care, reaching an appropriate health facility, and receiving adequate care at the facility. Risk-appropriate care prevents the second delay in the process, reaching an appropriate health facility. In the United States, delay in reaching the appropriate health facility may be due to failure to identify the patient’s high-risk status, inequitable distribution of critical care services, or structural systems that discourage or limit transfers of care [15, 16]. Both delays in identifying a patient as high risk and failure to transfer to a higher level of care have been found to contribute to potentially preventable maternal morbidity and mortality in the U.S. and other developed countries [15, 17, 18].

The existing literature has been unable to quantify the reduction in severe maternal morbidity that can be attributed to maternal risk-appropriate care [19, 20]. The main limitation to measuring the benefits of risk-appropriate care is the lack of a validated assessment of maternal level of care in population data. Prior studies measured risk-appropriate care using a hospital self-reported maternal level of care, but were unable to measure reduction in severe maternal morbidity [19, 20]. Subsequent evidence demonstrated that hospitals over-estimate their maternal level of care, likely resulting in misclassification bias when used to evaluate outcomes with risk-appropriate care [21].

Definitions for maternal levels of care in the U.S. were published in 2015 and updated in 2019 [22,23,24,25]. The Centers for Disease Control and Prevention developed an assessment tool that aligns with the definitions of maternal level of care, but the tool has not been implemented in all states and the data is not available for research [26]. To date, two attempts have been made to measure maternal levels of care using population data [27, 28]. Both methods used the types of patients a hospital serves to identify levels of care rather than restricting measurement to the resources available at the hospitals. This means they are measuring where women at high-risk currently give birth instead of measuring the hospitals’ resources for providing complex care. This is a problem when 43% of hospitals overestimate their ability to care for complex patients [26].

The objective of this study was to develop a measure of hospital maternal levels of care based on the resources available at the hospital, using existing data available to researchers. Creation of a measure for hospital level of care for research would advance science by allowing population-level analysis of the barriers to risk-appropriate and conditions that are most effectively treated at facilities with critical care services.


This was a cross-sectional study. The sample used to build the model was the American Hospital Association (AHA) Annual Survey Database for 2018 [29]. This year was chosen to align with the data from the CDC LOCATe Verification Pilot which was used to validate the model. Closure of hospital obstetric units since 2018 may mean more recent versions of the AHA Database are no longer representative of the sample available during the CDC LOCATe Verification Pilot. The unit of analysis was the hospital. Hospitals in the database were included in the study if they reported the hospital had an obstetric service (OBHOS = 1) or reported one or more dedicated obstetric beds (OBBD > 0). Hospitals were excluded from the study if they reported 0 births or did not report the number of births.

Creation of measure

We began with a review of the AHA Annual Survey data dictionary to identify all variables that potentially aligned with the ACOG/SMFM criteria [24]. We created frequency tables for variables and removed from consideration any that could not be used due to low response. We organized the remaining variables according to ACOG/SMFM criteria to identify which variables had the potential to identify each level.

We worked through multiple iterations of the model validating each iteration by measuring the agreement between the distribution of hospital maternal levels of care achieved with the model to the distribution achieved in the CDC LOCATe Verification Pilot [26]. The CDC LOCATe Verification Pilot was selected as the gold standard because the survey was created to measure the resources needed for each level of care as defined by the ACOG/SMFM criteria.

Although agreement at the individual hospital level would be the most rigorous method for validation, individual hospital levels of care are not available from the CDC LOCATe Verification. Therefore, the agreement of the distribution of hospital levels of care was measured. Agreement was measured with the Wilson Score of Proportions to calculate the probability to obtaining the CDC Verification Pilot result if our measurement of the AHA data accurately represented the nation [30]. We also measured the agreement in the proportion of hospitals that self-reported a higher level than was calculated with assessment of resources using the self-reported maternal level of care in the AHA database [26].


The American Hospital Association (AHA) database included information for 6,218 hospitals, of which 2331 (37.5%) were identified as eligible for inclusion in this study. The sampling method excluded 558 hospitals that reported births, but did not report obstetric services or obstetric beds (Mean Births 740 Standard Deviation 841.6 Range 1 – 9,264). None of the excluded hospitals reported the number of operating rooms or if the hospital had ultrasound services, which are used to identify hospitals with level 1 services. Therefore, a level of care could not be measured to compare excluded hospitals to the included hospitals.

The team identified 137 variables with potential to align with resources that measure maternal level of care. After review of the missingness of these variables, the team selected 32 variables with the potential to identify resources at the hospital. The variables considered are included in Table 1, with the variable name, description, frequency and final inclusion or exclusion decision. Of these, seventeen variables had the potential to provide information that identified nine of the resources used to define the ACOG/SMFM maternal levels of care.

Table 1 Description of services as reported by hospitals with maternity services

The model underwent four iterations, which can be viewed in the supplement, prior to finalizing the fifth version of the model. The first model included eight criteria identified by fourteen variables. This model failed to discriminate between specialty and critical care resources with 46.2% of the hospitals identified as having level III services and 34.3% as having level IV services. Results of the Wilson Score for each model are available in Table 2. As only one variable was potentially aligned with level IV criteria, we determined the data was not appropriate for discriminating between level III and level IV services and would instead aim for identification of level III/IV services. In the second iteration, the team reduced the model to 15 variables to identify eight criteria. The second model over-corrected, with only 6.6% of hospitals qualifying for level III/IV services.

Table 2 Distribution of hospitals across maternal levels of care for model iterations

In the third model, the team added a measure of on-site laboratory services to improve discrimination between hospitals with less than level I and level I resources, and expanded the criterion for interventional radiology to better discriminate level II services. The third model included thirteen variables across nine categories. The distribution of hospitals indicated the third model was likely overestimating the number of hospitals with level III/IV services. The fourth model added an additional variable to identify hospitals with surgical capability and removed the expanded criterion for interventional radiology. The fourth model had fourteen variables across nine categories, but was still overestimating the proportion of hospitals with level III/IV services.

The final model is presented in Table 3. The final model included a criterion for the presence of subspecialists to identify level II services. The final model also used trauma center levels, when available, to identify hospitals that did not meet the temporal requirements for availability of providers at levels II and III. Applying trauma level criteria removed 34 hospitals from the level III/IV category. The final model included seventeen variables that identified ten criteria. The distribution of hospitals using this model aligned with the CDC reporting of distribution of hospitals as assessed by non-significant Wilson Scores for each level category.

Table 3 Alignment of Final Model Variables with Maternal Levels of Care Criteria

Using the final model, 1219 (51.9%) of hospitals reported resources aligned with maternal level I, 816 (34.7%) aligned with maternal level II, and 202 (8.6%) aligned with maternal level III/IV. In the current study, 57 (2.4%) hospitals did not provide a self-assessment for level of care. The comparison of hospital self-reported level with the final model level revealed a discrepancy for 1176 (51.7%) hospitals, which was different from the 46.4% discrepancy reported by the CDC when compared by Wilson Score (p < 0.001). Of those with a level discrepancy, 789 (67.1%) self-reported a level higher than assessed, which was lower than the 89.2% reported by the CDC.


This study examined the feasibility of measuring hospital maternal level of care using the variables available in the American Hospital Association Annual Survey. The final model achieved a distribution of hospital levels that was similar to the CDC LOCATe Verification Pilot. The model created in this study can be used by researchers to estimate the hospital maternal level of care when studying risk-appropriate care.

This study builds on prior research into methods to estimate hospital level of maternal care by designing a method that relies only on the presence of hospital resources to define levels of care. Limiting measurement to the available resources is necessary if the levels of care are used to measure maternal risk-appropriate care [31]. Risk-appropriate care means that the patient is treated in a hospital with the resources that match the complexity of the case [31]. The CDC found that 46.4% of hospitals had a discrepancy in their self-identified maternal level of care with most over-estimating their capacity [21]. This means that classifying hospital maternal level of care based on the types of patients served or the types of care provided will cause misclassification bias. Misclassification bias is likely the reason prior examination of the effects of risk-appropriate care using hospital self-identified levels of service did not find improved outcomes at the highest-level hospitals [19, 20].

A major challenge to identifying the level of maternal care using the AHA survey is that the AHA survey has limited information about the qualifications and presence of specific staff required to discriminate between maternal levels of care. The AHA includes obstetricians in the count of primary care physicians, which prevents the survey from identifying hospitals with 24-h in-house obstetrician coverage. The survey does not provide information about maternal–fetal medicine subspecialists which is needed to distinguish level II from level III services. The inability to measure presence of obstetricians and obstetrical subspecialists may be why this study found a lower proportion of hospitals overestimating their level of service than was found in the CDC Levels of Care Validation Pilot [21] Amending the AHA Annual Survey to include a question about the number of laborists employed by the hospital may help provide the needed level of discrimination and would align with the existing questions about the number of hospitalists and intensivists employed.

The AHA Annual Survey Data allowed identification of many of the physical resources included in the maternal levels of care but could not distinguish the timeliness of care provision. For example, the survey clearly identifies which hospitals have CT Scan and MRI machines, but does not distinguish between those that can perform an immediate scan in the middle of the night from those that operate the equipment during a daily schedule. We tested variables such as presence of an emergency department or the number of full-time radiology technicians, but nearly all hospitals reported these resources preventing them from being useful measures of level of care. The one exception to this problem was immediate availability of an echocardiogram. Because the echocardiogram must be read by a cardiologist, variables that indicated the presence of cardiology intensivists, presence of a closed cardiac intensive care unit, and the number of cardiac intensivists, provided an estimate of the availability of this service.

One limitation of using the AHA to measure hospital level of care is that the variables for intensive care units and presence of health care providers do not allow researchers to distinguish between resources for maternal levels III & IV. The final decision of the team was to collapse these categories for measurement. Collapsing the two categories is likely to be sufficient for most research into risk-appropriate care because hospitals in the level III category have critical care services. Researchers whose work depends on measuring hospitals with level IV services should use a more precise measurement, preferably verified using the CDC Levels of Care Assessment Tool (LOCATe).


This study demonstrated that it is possible to use the American Hospital Association Annual Survey Database to estimate the hospital maternal level of care for research. The method of measurement presented in this study will allow researchers to quantify benefits with risk-appropriate care. In addition, this study identified potential changes to the American Hospital Association Annual Survey that would improve identification of maternal levels of care for research.

Availability of data and materials

Data sharing is not applicable to this article as no datasets were generated during the current study. The data analyzed in this study, the American Hospital Association Annual Survey Database, is available for purchase from the American Hospital Association at


  1. Oot A, Huennekens K, Yee L, Feinglass J. Trends and risk markers for severe maternal morbidity and other obstetric complications. J Womens Health. 2021;30(7):964–71.

    Article  Google Scholar 

  2. Black CM, Vesco KK, Mehta V, Ohman-Strickland P, Demissie K, Schneider D. Incidence of severe maternal morbidity during delivery hospitalization in U.S. commercially insured and Medicaid populations. J Womens Health. 2022;31(1):91–9.

    Article  Google Scholar 

  3. Wall-Wieler E, Carmichael SL, Gibbs RS, et al. Severe maternal morbidity among stillbirth and live birth deliveries in California. Obstet Gynecol. 2019;134(2):310–7.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Phibbs CM, Kozhimannil KB, Leonard SA, et al. The effect of severe maternal morbidity on infant costs and lengths of stay. J Perinatol. 2022;42(5):611–6.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Black CM, Vesco KK, Mehta V, Ohman-Strickland P, Demissie K, Schneider D. Costs of severe maternal morbidity in U.S. Commercially insured and Medicaid populations: an updated analysis. Womens Health Rep (New Rochelle). 2021;2(1):443–51.

    Article  PubMed  Google Scholar 

  6. Guglielminotti J, Wong CA, Friedman AM, Li G. Racial and ethnic disparities in death associated with severe maternal morbidity in the United States: failure to rescue. Obstet Gynecol. 2021;137(5):791.

    Article  PubMed  Google Scholar 

  7. Guglielminotti J, Landau R, Wong CA, Li G. Patient-, hospital-, and neighborhood-level factors associated with severe maternal morbidity during childbirth: a cross-sectional study in New York State 2013–2014. Matern Child Health J. 2019;23(1):82–91.

    Article  PubMed  Google Scholar 

  8. Wall-Wieler E, Bane S, Lee HC, Carmichael SL. Severe maternal morbidity among U.S.- and foreign-born Asian and Pacific Islander women in California. Ann Epidemiol. 2020;52:60-63e2.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Luke AA, Huang K, Lindley KJ, Carter EB, Joynt Maddox KE. Severe maternal morbidity, race, and rurality: trends using the National Inpatient Sample, 2012–2017. J Womens Health. 2021;30(6):837–47.

    Article  Google Scholar 

  10. Kozhimannil KB, Interrante JD, Tofte AN, Admon LK. Severe maternal morbidity and mortality among indigenous women in the United States. Obstet Gynecol. 2020;135(2):294–300.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Ahn R, Gonzalez GP, Anderson B, Vladutiu CJ, Fowler ER, Manning L. Initiatives to reduce maternal mortality and severe maternal morbidity in the United States. Ann Intern Med. 2020;173(11_Supplement):S3–10.

    Article  PubMed  Google Scholar 

  12. Main EK, Cape V, Abreo A, et al. Reduction of severe maternal morbidity from hemorrhage using a state perinatal quality collaborative. Am J Obstet Gynecol. 2017;216(3):298.e1-298.e11.

    Article  PubMed  Google Scholar 

  13. Kroelinger CD, Goodman DA. Levels of risk-appropriate care: ensuring women deliver at the right place at the right time. J Womens Health. 2020;29(3):281–2.

    Article  Google Scholar 

  14. Thaddeus S, Maine D. Too far to walk: maternal mortality in context. Soc Sci Med. 1994;38(8):1091–110.

    Article  CAS  PubMed  Google Scholar 

  15. Lawton B, MacDonald EJ, Brown SA, et al. Preventability of severe acute maternal morbidity. Am J Obstet Gynecol. 2014;210(6):557.e1-557.e6.

    Article  PubMed  Google Scholar 

  16. Mehta PK, Kieltyka L, Bachhuber MA, et al. Racial inequities in preventable pregnancy-related deaths in Louisiana, 2011–2016. Obstet Gynecol. 2020;135(2):276–83.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Geller SE, Koch AR, Martin NJ, Rosenberg D, Bigger HR. Assessing preventability of maternal mortality in Illinois: 2002–2012. Am J Obstet Gynecol. Published online 2014.

  18. Geller SE, Koch AR, Garland CE, MacDonald EJ, Storey F, Lawton B. A global view of severe maternal morbidity: moving beyond maternal mortality. Reprod Health. 2018;15(1):31–43.

    Article  Google Scholar 

  19. Vanderlaan J, Rochat R, Williams B, Dunlop A, Shapiro SE. Associations between hospital maternal service level and delivery outcomes. Women’s Health Issues. 2019;29(3):252–8.

    Article  PubMed  Google Scholar 

  20. Srinivas SK, Kozhimannil K, Hung P, Attanasio L, Jou J, Grobman WA. Do hospitals with a higher level of maternal care designation have better maternal outcomes? Am J Perinatol. 2019;36(6):653–8.

    Article  PubMed  Google Scholar 

  21. Madni SA, Ewing AC, Beauregard JL, Brantley MD, Menard MK, Goodman DA. CDC LOCATe: discrepancies between self-reported level of maternal care and LOCATe-assessed level of maternal care among 463 birth facilities. J Perinatol. Published online 2021.

  22. Stark AR, American Academy of Pediatrics Committee on F, Newborn. Levels of neonatal care. Pediatrics. 2004;114(5):1341–7.

    Article  PubMed  Google Scholar 

  23. Barfield WD, Papile LA, Baley JE, et al. Levels of neonatal care. Pediatrics. 2012;130(3):587–97.

    Article  Google Scholar 

  24. Kilpatrick SJ, Menard MK, Zahn CM, Callaghan WM. Obstetric care consensus #9: levels of maternal care. Am J Obstet Gynecol. 2019;221(6):B19–30.

    Article  PubMed  Google Scholar 

  25. Menard MK, Kilpatrick S, Saade G, et al. ACOG/SMFM Obstetric care consensus: levels of maternal care. Am J Obstet Gynecol. Published online 2015.

  26. Zahn CM, Remick A, Catalano A, Goodman D, Kilpatrick SJ, Menard MK. Levels of maternal care verification pilot. Obstetrics, Gynecology. 2018;132(6):1401–6.

    Article  PubMed  Google Scholar 

  27. Handley SC, Passarella M, Srinivas SK, Lorch SA. Identifying individual hospital levels of maternal care using administrative data. BMC Health Serv Res. 2021;21(1):538.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Easter SR, Robinson JN, Menard MK, et al. Potential effects of regionalized maternity care on U.S. hospitals. Obstet Gynecol. 2019;134(3):545–52.

    Article  PubMed  Google Scholar 

  29. American Hospital Association. American Hospital Association Dataset. Published online; 2012.

  30. Wilson EB. Probable inference, the law of succession, and statistical inference. J Am Stat Assoc. 1927;22(158):209.

    Article  Google Scholar 

  31. DeSisto CL, Kroelinger CD, Levecke M, Akbarali S, Pliska E, Barfield WD. Maternal and neonatal risk-appropriate care: gaps, strategies, and areas for further research. J Perinatol. Published online January 11, 2023.

Download references


The authors would like to thank Dr. James M. Alexander for his assistance with selecting variables to consider.


This study was not funded.

Author information

Authors and Affiliations



JV was responsible for conceptualization, formal analysis, and writing the original draft. JV & JS designed the methodology. JV, JS & IM conducted the investigation and completed review and editing of the final paper

Corresponding author

Correspondence to Jennifer Vanderlaan.

Ethics declarations

Ethics approval and consent to participate

This study is not human subjects research and was exempt from review by the University of Nevada Institutional Review Board.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vanderlaan, J., Shen, J.J. & McDonough, I.K. Validation of a measure of hospital maternal level of care for the United States. BMC Health Serv Res 24, 286 (2024).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: