Access to after-hours primary care: a key determinant of children’s medical home status

Abstract

Background

The medical home (MH) model has been promoted by both the federal and state governments in the United States in recent years. To ascertain American children’s MH status, many studies have relied on a large set of survey items, posing a considerable burden on their parents. We aimed to identify individual survey items or domains that best predict MH status for children and use them to develop brief markers of MH status. We also examined whether the identified items differed by status of special health care needs and by racial/ethnic group.

Method

Using the 9-year data from Medical Expenditure Panel Survey, we examined associations between children’s MH status and individual survey items or domains. We randomly split the data into two halves with the first half (training sample, n = 8611) used to identify promising items, and the second half (validation sample, n = 8779) used to calculate all statistical measures. After discovering significant predictors of children’s MH status, we incorporated them into several brief markers of MH status. We also conducted stratified analyses by status of special health care needs and by racial/ethnic group.

Results

Less than half (48.7%) of the 8779 study children had a MH. The accessibility domain has stronger association with children’s MH status (specificity = 0.84, sensitivity = 1, Kappa = 0.83) than other domains. The top two items with the strongest association with MH status asked about after-hours primary care access, including doctors’ office hours at night or on the weekend and children’s difficulty accessing care after hours. Both belong to the accessibility domain and are one of several reliable markers for children’s MH status. While each of the two items did not differ significantly by status of special health care needs, there were considerable disparities across racial/ethnic groups with Latino children lagging behind other children.

Conclusion

Accessibility, especially the ability to access health care after regular office hours, appears to be the major predictor of having a MH among children. The ongoing efforts to promote the MH model need to target improving accessibility of health care after regular hours for children overall and especially for Latino children.

Background

Recent years have witnessed an increased interest in promoting the pediatric medical home (MH) model, which was defined by the American Academy of Pediatrics (AAP) as “a model of delivering primary care that is accessible, continuous, comprehensive, family-centered, coordinated, compassionate, and culturally effective (i.e., the seven defining domains or components of MH) to every child and adolescent [1, 2].” Prior studies found that children with MHs had fewer visits and lower costs to emergency care than children without MHs [3,4,5,6,7,8]. MH has also been linked to increased preventive visits and dental care visits among children [7], as well as higher ratings of health care quality [6]. One recent systematic review identified generally positive associations between having a MH and various outcomes of child well-being [9].

Despite the growing literature, there is no consensus among researchers about how to empirically define children’s MH status. For example, the National Committee for Quality Assurance has promoted the Patient-Centered MH model by offering different levels of accreditation to physician practices for achieving specific model features [10]. A practice’s accreditation can be used to define MH status for the patients served at the practice [11,12,13]. Whereas this approach measures MH status from the practice perspective, other studies have defined MH status based on health care experiences [6, 7, 14, 15]. Arguably, patients’ experiences are as important as practice features in measuring the quality of primary care. That is why many studies have used pediatric care experience to define MH status for children. Those studies typically operationalized the AAP definition of MH by using survey data about pediatric care experience [9, 14, 16, 17]. For example, one study examined children’s MH status using the National Survey of Children’s Health [17]. Other studies have used the Medical Expenditures Panel Survey to define children’s MH status [7].

Theoretically, a MH should be defined by all the above seven domains with equal relevance. In empirical studies, however, domains can have very heterogeneous associations with the MH status. For example, one study reported that the family-centered domain had a much stronger association with a child’s MH status than the comprehensiveness domain [17]. While the study represents the first analysis of the heterogeneous associations between individual domains and the overall MH status for a child, further research on this issue is needed for several reasons. First, a better understanding of the specific domains and survey items that are significantly associated with the overall MH status would greatly inform future efforts and resource allocation to study MHs for children. Conversely, domains or survey items without notable associations with the overall MH status may lack variation (i.e. they may be endorsed by most patients), and subsequently will not be useful for future studies of children’s MH statuses. Second, after identifying the survey items with the strongest associations with the overall MH status, they may be used to develop a shorter list of survey items as a brief marker for MH status. A shorter list of survey items would make screening for MH more efficient, compared with the current survey instrument usually consisting of 20 or more survey questions, which can be a significant burden on patients and their families, raising concerns about the scarce resources available for conducting surveys. Finally, beyond a research application, there may also be utility of a marker in a clinical setting by helping identify patients’ needs either at initial patient intake or in an ongoing fashion to ensure that the new needs are addressed.

To examine these questions, we analyzed the association between the overall MH status for a child and individual domains in a nationally representative survey. We identified specific survey items with strong associations with MH status, and used them to explore the development of a short list, or a marker for children’s MH status. We also examined whether the domains or items that were strongly associated with the MH status differed by status of special health care needs and by racial/ethnic group since prior studies indicated that MH status varied by these factors [18,19,20].

Methods

Data

We used the Medical Expenditure Panel Survey (MEPS) from 2004 to 2012. The MEPS is a nationally representative survey conducted by the Agency for Healthcare Research and Quality. Compared to other data sources, such as the National Survey of Children’s Health, MEPS has a much more comprehensive list of survey items on pediatric care experiences. Each MEPS panel consists of a nationally representative sample that is followed-up for 2 years. We used the first year in each panel to avoid the intra-person correlation in all variables.

MH status

We used the method developed by Romaire and Bell, who used 22 MEPS survey items to define children’s MH status [7]. To have a MH, a child first must have a qualified usual source of care (USC) (See details in Table 1 about the definition of a qualified USC). Then, those children having a qualified USC were scored based on the MEPS survey items (See details in Table 1) in four MH domains (accessible, comprehensive, family-centered, and compassionate). The other three domains of the AAP definition of MH model could not be measured because of inadequate information in the MEPS. Table 1 provides detailed information about the 22 MEPS survey items. The minimum of the four domain scores needed to be no less than the cutoff of 75 points. Missing data in survey items that was not a legitimate skip led to a missing MH status for a child. We identified 17,390 children having sufficient information to apply the MH definition in Table 1.

Table 1 MEPS survey items and scoring by MH domain ^a

Item status

Responses to each survey item in Table 1 was assigned a score, ranging from 0 to 100, by using the method in a prior study [7]. Any item on which the individual child scored 75 points or more was considered a positive item status. Otherwise, a negative item status was considered for an item scored less than 75 points. For example, if a child had a response of “sometimes” to question CPA1 “Provider shows respect for what the parent has to say”, then the CPA1 item score was 25, and consequently the CPA1 item status was negative for the child.

Domain status

The domain score was the average of all the item scores in a domain. Dichotomous domain statuses (positive/negative) was defined using the cutoff of 75 points on domain scores. For example, if a child received item scores of 0, 25, 100, and 100 for the four items under the accessible domain respectively (ACC1-ACC4), then the accessible domain score was 56 and the accessible domain status was negative for the child.

It is worth noting that, for the domain of comprehensive care, it is possible to legitimately skip some of the items if a child did not have any needs in a specific category in the past 12 months. For example, if a child did not need to see a specialist in the last 12 months (answered “No” to CPR1), the child qualified as a legitimate skip for the question CPR2 that asked if there were any problems seeing a specialist.

Statistical analysis

We randomly split the data into two halves. The first half (training sample, n = 8611) was used to identify promising items for comparisons among all possible short lists, and the second half (validation sample, n = 8779) was used to calculate all statistical measures reported in this paper. Ideally, the reported statistical measures should be based on data external to the development of the marker. In practice, split-sample development and validation sets are frequently used to develop instruments for screening diseases when a true external validation set from a unique sample is not available [21, 22], a limitation we have in this study.

For each half, we conducted a classification analysis using two-by-two contingency tables between children’s MH status and an item status, a domain status, or a marker status. Classification tests are a standard technique in clinical studies, where the two binary conditions consist of the true condition (e.g. sick/healthy) and a clinic diagnosis (e.g., at risk/no risk). Since all status variables are dichotomous, the classification test statistics is suitable in our study, where we treated the overall MH status as the true condition, and we used an item status, a domain status, or a marker status as a diagnosis.

The statistical measures in two-way clinic classifications include sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), efficiency, and Cohen’s Kappa [23, 24]. The true positive (TP) cases were those children who were positive on the diagnosis status (e.g., an item status) and had a MH. The false positive (FP) cases included those children who also scored positively on an item or domain but did not have a MH. Among those children who had a negative item or domain status, the true negative (TN) cases and the false negative (FN) cases were defined with the former indicating children who had a negative item or domain status and did not have a MH and the latter indicating children who had a negative item or domain status but had a MH. Using these notations, we defined:

• PPV = TP/(TP + FP),

• NPV = TN/(TN + FN),

• Sensitivity = TP/(TP + FN),

• Specificity = TN/(TN + FP), and.

• Efficiency = (TP + TN)/N,

• where N = TP + TN + FP + FN is the total sample size.

We defined the positive rate of an item, domain, or marker as the proportion of children who had a positive status, i.e., (TP + FP)/N. Specificity, sensitivity, PPV, and NPV are conditional probabilities. Efficiency is the probability that a diagnosis agrees with the true condition, which can be less informative when the prevalence rate of the disease is too high or too low. To assess the probabilities, we adopted the general rule of thumb: a probability above 0.9 is considered as high, between 0.8 and 0.9 as moderately high, and moderate between 0.7 and 0.8.

We also calculated Cohen’s Kappa, which adjusted for the probability of agreement between two measures by chance. Kappa is usually between 0 and 1 with a Kappa above 0.6 considered as substantial agreement, and a Kappa value around 0.5 indicating a good level of agreement [25].

Brief marker

We developed several brief markers using subsets of items that are strongly associated with the overall MH status. Hypothetically, a brief marker can reliably predict the MH status, and can be easier to implement in a survey than the full list of the items currently used in the literature. We considered two types of markers. The first type dropped the concept of domains and used between two and four items from Table 1. The marker score was defined as the average of all items in the marker. The overall MH status was positive if the marker score was greater than 75 points. The second type maintained the four domains—within each domain, one or two items mostly associated with the domain status were selected. For the second type, we first calculated each individual domain score. A domain status was considered positive if the average score of selected items within the domain was greater than 75 points. The overall MH status was positive if all four domains were positive. We enumerated all possible combinations among the 22 MEPS items in Table 1 for each type of markers. To select the best markers, we required specificity, sensitivity, NPV, PPV, and efficiency all greater than 0.7, and Kappa greater than 0.5.

Disparities analyses

After identifying the items and domains with strong association with MH status for all children, we conducted stratified analyses by status of special health care needs and by racial/ethnic group. We identified children with special health care needs by adopting the method by prior research that used the MEPS data to provide detailed information about this group of children [26]. We included three racial/ethnic groups, Latinos, non-Latino Whites, non-Latino Blacks while other groups, such as non-Latino Asians, were not included due to small sample size.

Results

MH status

Among the 8779 children in the validation sample, 4279 (48.7%) had a MH.

Statistical measures of the associations between individual items/domains and MH status

Due to the large sample size, all statistical measures had very small standard errors, and the 95% confidence intervals for all measures had a half-length no greater than 0.01. Thus, we omitted 95% confidence intervals in all tables to save space. Statistical measures between item status and MH status are reported in Table 2. The rate of positive status on each individual item was reported under the “positive rate” column. The positive rate was above 85% for most items, except for ACC1 (“Does the provider have office hours at night or on the weekend?”) and ACC2 (“Any difficulty accessing care after hours?”). The positive rate for ACC1 and ACC2 was 51 and 70%, respectively, indicating that only 51% of children’s providers had office hours at night or on the weekends, and 70% of children did not have difficulty accessing care after regular hours.

Table 2 Reliability of single survey item in predicting MH status

Table 2 also showed that sensitivity and NPV were generally high for all the items. Nevertheless, except for ACC1 and ACC2, all items had very poor specificity (below 0.25) and PPV (below 0.55). This combination of extreme values in statistical measures (high sensitivity and NPV, and low specificity and PPV) reflected the fact that most children endorsed almost all the survey items except for ACC1 and ACC2. Further, PPV for all the items (except for ACC1 and ACC2) was close to the rate of having a MH among the study children (48.7%), suggesting that responding affirmatively to these items (except for ACC1 and ACC2) was only slightly better than a random guess of the overall MH status. In comparison, both ACC1 and ACC2 have relatively high PPV (0.74 and 0.66 respectively).

Overall, ACC1 and ACC2 had moderate to high values on all statistical measures. ACC1 had a sufficiently strong prediction power for the MH status (efficiency = 0.75, Kappa = 0.51), and ACC2 was only slightly weaker (efficiency = 0.73, Kappa = 0.47).

Table 3 reported the statistical measures between domain status and MH status. Sensitivity and NPV were all equal to 1 because the definition of MH requires a positive status in each of the domains. Due to the key role of two accessibility measures (i.e., ACC1 and ACC2) in predicting the MH status, the accessibility domain had the strongest association with the overall MH status (efficiency = 0.92, Kappa = 0.83). The family-centered domain had a weaker but notable association with the overall MH status (efficiency = 0.64, Kappa = 0.29), while the other two domains were not predictive with Kappa below 0.2.

Table 3 Reliability of domain status in predicting MH status

Brief markers

Among those brief markers without domains that met the requirement in all statistical measures and had the highest Kappa, the selected 2-item marker included only ACC1 and ACC2. Adding one or two more items to it did not improve substantially the statistical measures, especially specificity. Of the markers with the four domains, either the 4- or the 7-item marker did not increase classification measures, compared with the above selected 2-item marker (Table 5 in Appendix).

Disparities in the selected items and domains

Table 4 summarized disparities in the selected items and domain that had the strongest associations with MH status in Tables 2 and 3. For ACC1, while there were small differences between children with special health care needs and children without special health care needs, the differences were substantial across racial/ethnic groups. For example, the sensitivity of ACC1 in predicting MH status was 0.80 for Latino children, higher than that for either non-Latino Whites (0.71) or non-Latino Black (0.65). The positive rate of ACC1 was 0.42 for Latino children, markedly lower than that for non-Latino Whites (0.56) or non-Latino Black (0.59).

Table 4 Reliability of the selected items and domain in predicting MH status by special health care needs status and race/ethnicity

For ACC2 or the Accessibility domain, a similar pattern appeared with (1) small differences between children with special health care needs and children without special health are needs, (2) considerable differences across racial/ethnic groups, and (3) lower positive rate for Latino children, compared with non-Latino Whites or non-Latino Blacks.

Discussion

Despite the efforts to promote MH in recent years, we found that only around 50% children nationwide had a MH, a finding that is consistent with the results from previous studies [6, 7]. Similar to prior research [17], this study found the family-centered care domain to be more predictive of MH status than the comprehensiveness domain. However, it is the accessibility domain that has the strongest association with the MH status. Thus, accessibility measures may be the most significant marker for predicting the MH status among pediatric patients. The top two items with the strongest association with the MH status were regarding doctors’ office hours at night or on the weekend and children’s difficulty accessing care after hours, highlighting the importance of after-hours primary care access for a well-functioning health care system, which has been documented by both international studies and prior research in the US [27, 28].

We also found that, while the two items and the accessibility domain did not differ significantly by status of special health care needs, there were considerable disparities in the two items and the accessibility domain across racial/ethnic groups with Latino children lagging behind other children. Similar findings were reported by prior studies. For example, one study also used the MEPS data and found small differences between children with special health care needs and children without special health care needs in terms of either the overall MH status or specific MH components [29]. On the other hand, like our analysis, prior research indicated lower likelihood of Latino children having a MH [30, 31].

This study is subject to a number of limitations. First, the definition of MH based on MEPS data can only cover four domains besides having a qualified USC. This is a limitation of the MEPS data as recognized in the literature [7, 8, 16]. Studies using other survey data, for example, the National Survey of Children’s Health, also included only three or four domains [17, 32]. It will be interesting to examine how the results might change if a comprehensive patient experience survey can cover all seven domains of the AAP definition of MH. Second, a large proportion of children in MEPS did not have complete information across the 22 survey items to define their MH status. Our findings may be subject to selection bias due to potential non-ignorable missing data. However, the sheer amount of missing data also strongly suggests the necessity and utility of a brief marker for children’s MH status, which may lead to less missing data in future studies, allowing for greater accuracy in ascertaining MH status for children. Third, although we used the split samples to develop and examine reliability measures of the markers, the split samples were retrieved from the same data source, and hence subject to the same sampling bias [33]. A truly external validation sample, independent of instrument development, is still needed for validation [21, 34, 35].

Despite these limitations, our analysis showed that accessibility is the most significant marker of children’s MH status. The high positive rates of all other items and domains make them less informative in predicting the MH status although they are equally important in measuring different perspectives of a MH. On the other hand, the relatively low positive rate of accessibility measures led to high predictive power, but also revealed a significant practical issue for children’s health care nationwide. Specifically, our finding that only 51% of children’s providers had office hours at night or on the weekends indicates that accessibility of health care after regular hours appears to be a major hurdle for pediatric patients to have a MH. The finding suggested that improving access to after-hours primary care services should be a priority for promoting the MH model for children.

Another implication of this study is to encourage a reconsideration of the effects of MHs on outcomes such as utilization, expenditures, quality of health care, and well-being outcomes. Given our findings, it is reasonable to suspect that accessibility to health care, particularly after regular office hours, may play an important role in producing these reported effects of a MH. One study found that among seniors, items in the accessibility domains affected various expenditure outcomes: no difficulty contacting providers over the telephone during regular business hours was associated with lower total and inpatient expenditures, and having a provider with office hours at night or on weekends was associated with lower outpatient, emergency department, and other expenditures [28]. While the precise nature of the relationship between accessibility measures and health care outcomes would benefit from further examination, one potential explanation is that not all MH domains are equal in practice, and among the domains of a MH, accessibility is likely the most difficult for a practice to achieve while also lying largely within a practice’s control. In comparison, measures in the comprehensive domain, such as specialist referrals, depend largely on patient-level factors and health care providers outside a primary care physician’s practice, making it difficult for the practice to directly affect changes in that domain. A patient-centered practice that considers the needs of patients and their families and works providers’ schedules around those of working parents and families may be more committed to improving accessibility and more likely to exhibit other features of a MH. Therefore, accessibility may not be a fundamental cause for conferring the benefits of a MH but serve as a critical mediator between practice features and the benefits of a MH. Future studies are needed to disentangle and confirm the effect of accessibility measures and the overall impact of having a MH on health care outcomes in the general pediatric population.

Finally, our finding of racial/ethnic disparities in the accessibility domain and the two accessibility items suggested that the ongoing efforts to promote MHs among children should target Latino children by increasing their access to care in order to improve their MH status.

Conclusions

Less than half of children in the U. S had a MH. Among the survey domains, the accessibility domain had the strongest association with children’s MH status. The top two items with the strongest associations with the status ask about doctors’ office hours at night or on the weekend and children’s difficulty accessing care after hours. The two items can be used as a brief marker of children’s health status while combining them with more items will not necessarily improve the marker’s statistical measures. The significant associations between the two items and children’s MH status and the racial/ethnic disparities in these two items suggest that the ongoing efforts to promote the MH model need to target improving accessibility of health care after regular hours for children overall and especially for Latino children.

Appendix

Table 5 Reliability of top Type 1 (without domains) and Type 2 (with domains) Marker in predicting MH status