- Research article
- Open Access
The Mitigating Toxic Stress study design: approaches to developmental evaluation of pediatric health care innovations addressing social determinants of health and toxic stress
BMC Health Services Research volume 21, Article number: 71 (2021)
Health care administrators must establish and promote effective partnerships with community agencies to address social determinants of health, including reducing exposure of infants and young children to chronic stress. Because infants’ experiences are inextricably tied to their caregivers, an important target for mitigating “toxic” stress exposure in early childhood is through reducing parents’ experiences of chronic stress in addition to protecting children from direct experiences of harm such as physical or sexual abuse. Conducting screening to identify when children are exposed to early life adversity is a first step; connecting families to needed support services is an essential component to addressing identified challenges. This paper presents the methodology of a three-year study of health care systems innovations designed to engage and support parents of infants to prevent and mitigate children’s toxic stress exposures.
Key study features included: 1) multi-component study in five U.S. communities and nine pediatric health care clinics and the families they serve, 2) a developmental evaluation approach to describe how innovations are experienced over time at three levels—community systems, pediatric providers, and families, and 3) rapid cycle feedback conducted with communities, clinics and families to co-interpret data and findings. Data sources included: 1) focus groups and interviews with community stakeholders, clinic staff, and families, 2) electronic health record and Medicaid services data extracted to assess health care quality, utilization, and financial impact, and 3) clinic-recruitment of 908 parents of infants in a longitudinal survey. Results. The sample is briefly characterized based on responses to the enrollment phase of the parent survey.
We discuss the study design elements’ contribution to generating evidence needed by innovators, communities, and clinics to modify and sustain investments in these innovations to prevent or mitigate the effects of exposure to toxic stress on young children.
Health care screening and referring families to resources are critical roles for pediatric health care practices to consider as part of addressing social determinants of health (SDoH). Social determinants of health are the conditions or circumstances in which people are born, grow, live, work, and age, and the wider set of forces and systems shaping the conditions of daily life [1, 2]. This includes social, cultural, economic, and political forces that contribute to health disparities . During the first post-natal year, these social determinants may help or hinder families’ access to services and could play a significant part in parents’ experience of strain [4, 5]. Infants can experience much of the same social stress and its impact as their parents. Poverty, inadequate food or nutrition, or exposure to violence, for example, can prompt a physiological stress response, primarily indicated in the amount of cortisol that is produced and for how long . Infants may also be impacted by chronic stress through the relationship with their parents; developmentally supportive parenting can help buffer children from stress by promoting children’s early development . The term “toxic stress” is used to refer to the strong, frequent, or prolonged activation of the body’s stress response systems in infants while not being buffered by protection from a supportive adult . Protective factors including supportive relationships, and access to basic needs such as high-quality food and transportation, can mitigate chronic stressors and promote resilience. These positive childhood experiences in the face of adversity have been shown to independently predict adult health . Assessing parent experiences of stress and burden can ensure timely access to services and promote child health. The American Academy of Pediatrics and the American Academy of Family Physicians are just two of the professional practice organizations in the U.S. that underscore pediatric and family health care providers’ role in addressing the social needs of families [10, 11].
Because the origins of stress are multi-faceted and multi-determined, reducing the development of toxic stress through enhancing protective factors must also be multi-systemic. One challenge facing pediatric health care is the effective integration of screening for families’ social needs, referring to services (such as food banks or housing) and facilitating family access to services. Child health practitioners and managers frequently endorse recommendations for screening, however, they also report that community services (e.g. early childhood behavioral health) are often limited or unavailable and therefore are reluctant to implement screening guidelines [12, 13]. Health care providers also do not typically have the capacity, nor the financial model including staff roles and time necessary to monitor the available service array in their community. The impact of innovations in health care to prevent or mitigate toxic stress in children is therefore influenced by the available service continuum and effective partnership across health care and community-based services.
Latino families are even more disenfranchised from health care and therefore more likely to have less access to referral and formal support networks. Child health insurance programs have extended well-child and illness care to millions of uninsured families throughout the U.S. , making more apparent the many unaddressed physical health and social needs they face. Latino families in the U.S. have the highest uninsured rates, compared to other racial or ethnic groups, with only 49% reporting insurance coverage in 2017 . As a marker of socioeconomic adversity, lack of health insurance often co-occurs with other social needs including insecure housing and food [16, 17]. There is evidence that accessing health and economic supports decreased from 2015 to 2018 among Latino families . Public charge rules that were adopted in 2019 also created a chilling effect that further reduced Latino families’ enrollment in benefits.
In this paper, we describe a study that triangulates three important perspectives (community systems, health care clinics, and families) on strategies for promoting child health through bolstered partnerships between health care and early childhood systems to support families of infants. Participating health clinics serve predominantly Latino families. The study is designed to examine two approaches to pediatric health innovations that are being implemented in the U.S. to prevent and mitigate conditions related to exposure to early childhood adversity and the lack of protective factors. This study documents systems change, healthcare provider behavior, and family engagement with health and community services in relation to health and stress outcomes over time. Our theory of change asserts that the effects of these innovations are first at the systems level, influencing collaboration, provider behavior and family engagement efforts. Strengthening these systems thus provides families with increased supports for both concrete and health needs. This in turn increases parent sense of agency, prompting behavioral changes that promote child health and well-being.
Developmental Understanding and Legal Collaboration for Everyone (DULCE; 19) and Improving Screening, Connections with Families, and Referral Networks (I-SCRN; 20) are two approaches designed to increase the pediatric care provider’s role in strengthening family supports by identifying family needs in areas of SDoH and referring and linking families to resources. Both innovations involve: screening during health care visits; a multidisciplinary team; enhanced conversations with families about child development, maternal depression, and SDoH; and collaboration with a local system of care. DULCE was developed, implemented and tested in a randomized control trial of 330 parents of infants in Boston, showing that the intervention accelerated family access to concrete resources and that participants were more likely to complete their 6-month immunizations by age 8 months and less likely to have visited the emergency department by age 6 months . DULCE embeds a family specialist to work in tandem with pediatricians and families during the first 6 months of preventive health care visits, and also uses a legal partnership approach. The legal partners provide: training and support for the family specialist around screening for legal issues associated with social determinants of health, legal information, and referrals for families for legal intake if appropriate. Attorneys also participate in a weekly care planning meeting for families and can provide immediate legal advice and represent families in court. DULCE was implemented in seven of the nine clinics in our study.
The American Academy of Pediatrics’ (AAP) I-SCRN is a quality improvement collaborative to support pediatric primary care practices in the implementation of effective processes for screening, discussion, referral, and follow-up of child development, autism spectrum disorder (ASD), maternal depression, and social determinants of health [20, 21]. I-SCRN was implemented in two of the nine study clinics. Practices participated in I-SCRN over 1 year with other collaborative members and the AAP, involving virtual and in-person learning opportunities, in-clinic practice teams centered on early childhood screening, and a continuous quality improvement process based on monthly chart reviews and surveys . Using the monthly chart reviews (10 per month per clinic for well-child visits at 9, 18, and 30 months) among 19 participating practices, significant increases in screenings conducted were observed, as well as the proportion of visits where child development, ASD, maternal depression, and SDoH were discussed, and referrals for developmental and maternal depression services were made .
The two clinic innovations by design promote access to services that are difficult for the most stressed, marginalized populations to access while beneficial for women and children across socio-economic circumstances. Clinics can both refer families directly to services and leverage centralized referral systems like Help Me Grow (HMG) that build on existing community resources to develop a comprehensive network of early childhood services in a community . The HMG model includes a centralized access point (e.g. call center), family and community outreach, child health care provider outreach, and referral tracking. Help Me Grow was available to four of the five study communities.
This paper describes a developmental evaluation study using mixed methods to understand the experiences of five communities, nine health care clinics, and 908 families in the implementation of two healthcare/early childhood systems innovations to promote resilience and reduce stress in the child’s family during the first year of life. The study’s evaluation strategy is rooted in prevention science and incorporates two approaches critical for evaluating innovation. The first approach is an articulation of the multiple levels at which an innovation operates, including documenting the variance between the intended versus actual implementation experienced by stakeholders. Second, because outcomes are rarely driven by single causes, it is important to document the paths of theoretically meaningful mediators of outcomes . This evaluation plan developed for the study highlights multiple levels (community systems, clinics, and families) and pathways at which the innovations operate through the project design, measurement, and analysis strategy. We capture change over the time period of the study using a developmental evaluation approach [24, 25]. Focus groups and interviews (qualitative methodology) with parents, clinic staff, and early childhood community leaders complemented parent longitudinal survey interviews and electronic health records extracted for the study.
Figure 1 illustrates the evaluation’s focus on rapid-cycle feedback and co-interpretation approaches that are part of the developmental evaluation approach. This model, adapted from a synthesized member checking methodology for qualitative research, captures evolution of the health innovations and the communities in which they are embedded .
The five communities in the study were chosen from a larger set of communities engaging in an early childhood systems transformation effort in 2018, The Early Childhood Learning and Innovation Network for Communities (EC-LINC; https://cssp.org/our-work/project/early-childhood-learning-and-innovation-network-for-communities/) led by the Center for the Study of Social Policy (CSSP). Participating communities in EC-LINC bring together early childhood brokering or hub organization with specific community partners to strengthen systems and services to better meet the needs of families with young children. In 2018, there were 10 EC-LINC communities. Those in the study were chosen because of their interest in strengthening how they partnered with health care systems to promote family protective factors. During this time, pediatric health care clinics within each community also self-selected and received resources to implement DULCE or I-SCRN as part of program developers’ larger efforts to implement the models. Clinics that opted in to DULCE or I-SCRN in the communities were then approached to participate in the study. DULCE clinics were recruited by the research team and CSSP, and interested I-SCRN clinics responded to a recruitment flyer distributed by the research team and the AAP. The final sample included nine clinics—seven implementing DULCE and two implementing I-SCRN. All study procedures were approved by the University of Chicago Social Services Administration-Chapin Hall Institutional Review Board (IRB).
We describe each data collection method next, beginning with qualitative data collection methods across various stakeholder groups over time and then transitioning to quantitative methods including electronic health record extraction and longitudinal survey methods. For the quantitative data collection with families, community-based Field Interviewers completed the interviews (see Family Longitudinal Interviews section). For qualitative interviews and focus groups, data collection was conducted by a research team led by the PhD-level qualitative study lead. Interviews and focus groups were conducted in English, Spanish and in one community Haitian-Creole, and lasted 45 to 90 min and were audiotaped and transcribed. Table 1 describes the levels of primary data collected, data sources, methods, and number of participants per time point. Data were collected in each of 3 years (2018–2020) of the study. In 2018, interviews and focus groups were conducted with national leadership of the three innovations, EC-LINC leadership in the five communities, and leaders of the clinic innovation teams. We commenced family longitudinal interviews in March 2018 completing the final longitudinal interviews with families in February 2020. In 2019, we focused data collection at the community and clinic levels.
Interviews were completed with 16 of the DULCE, I-SCRN and HMG national teams members (N = 16). The interviews centered on three topics: (1) lessons learned about support clinic innovation; (2) the conceptual approach to addressing toxic stress and family engagement; and (3) emerging challenges and opportunities for the field. The purpose of this investigation was to understand the innovation model and implementation plans at the outset of the study in 2018.
Multiple levels of community-level data were collected to understand each community’s early childhood service system. First, we conducted telephone interviews with 1–2 EC-LINC leadership staff per community (n = 11). EC-LINC leaders were asked about the vision for early childhood-healthcare partnerships and lessons learned from DULCE and I-SCRN implementation and from supporting Help Me Grow as a centralized referral network.
We then conducted focus groups with community service providers (n = 36) recruited by the early childhood organizations such as mental and behavioral health providers, child care and early education providers, home visiting providers, regional centers, and public health departments. The focus groups inquired about approaches to mitigating toxic stress, healthcare-early childhood partnerships, policies and practices related to screening and referral for social determinations of health, and family engagement strategies.
To understand family perspective, we asked the early childhood organizations and providers to nominate and invite parents or caregivers who could speak to experiences accessing early childhood services to a focus group. We conducted 8 focus groups (4 in English and 4 in Spanish) with a total of 58 parents/caregivers. Parents/caregivers had diverse experiences with the early childhood and healthcare systems (participation in home visiting, healthy pregnancy programs, or as community leaders). The 90-min focus groups inquired about experiences with community providers and pediatric primary care in their community.
Finally, we partnered with one participating state to understand the role of Help Me Grow in facilitating the referral network and bridging health care and community-based services for families. The purpose was to learn from HMG staff, community providers, health care providers, and families about their experiences with the roles of HMG as a centralized early childhood referral system. Three HMG affiliate locations responded to a request for applications and were selected for geographic representation and diversity in the number of years of HMG implementation. Within the three communities (two urban, one rural/suburban), we conducted 45-min telephone interviews with HMG staff and health care providers (n = 11). We conducted 3 English and 2 Spanish focus groups with families (n = 17), and 3 focus groups with community providers (n = 24). Focus groups were 60 min and asked about perceptions of facilitators and barriers to participating in the centralized referral system and critical opportunities to strengthen referral pathways in each community. Healthcare provider interviews also centered on these topics.
Pediatric Health care clinics
We collected data from three groups at each clinic: (1) innovation team members (DULCE or I-SCRN), (2) clinicians, and (3) families. Innovation team members (n = 54) were interviewed in 2018 to gain perspective about early experiences with implementation. The interviews asked about core innovation components, opportunities, and barriers to integrating SDoH screening into the practice’s standard of care. Participants included the physician or nurse practitioner “champion,” trained family specialists, social workers, clinic administrators, and other team members supporting implementation. In 2019, we surveyed a broader group of clinic staff. We invited staff at each clinic (physicians, pediatricians, nurse staff, medical assistants and physician assistants) to complete an online survey, resulting in 63 completed surveys. The survey was designed to measure clinician attitudes and practices related to early childhood healthcare services, screening, and perceptions of occupational burnout (see Table 1 for the measures used).
Finally, we asked clinic innovation team members and early childhood organizations to nominate parents/caregivers to be invited to a 90-min focus group. Reflecting the languages preferred by families served by the clinics, we conducted 7 focus groups in English and 8 focus groups in Spanish with 110 total participants and at least one focus group conducted at each clinic. Focus groups asked about experiences with screening and referral for social needs in the clinic setting, and interactions with community providers following referrals.
Family Longitudinal Interviews
Aiming to enroll 110 parents/caregivers of infants per clinic, we recruited 908 parents/caregivers to participate in three in-person survey interviews before their infant’s 2-year birthday (92% of recruitment goal). The first interview was conducted when infants were 2 weeks to 6 months of age, followed by midpoint and final interviews at infant age 7–11 months and 12–15 months, respectively. After the initial interview, each follow-up interview was scheduled to allow a period of no fewer than 30 days between interview waves.
Recruitment was facilitated through close collaboration with clinics, and conducted by 12 community-based field interviewers. Field interviewers were bicultural and fluent in English and Spanish and in one community: Haitian-Creole. Recruitment strategies were discussed, reviewed, and refined collaboratively with clinics throughout the study time period to accommodate clinic flow and patient protection. Early challenges in recruiting families led to highly effective collaborative decisions. Recruitment techniques included waiting room recruitment (field interviewers approached families in the waiting room), warm hand-offs from clinic staff to the field interviewer, clinic staff introducing the study to parents by phone followed by field interviewer phone contact, and texting parents. Field interviewers used study-issued surface notebooks for all communication and data collection with families.
Data sharing agreements enabled collaboration between researchers and clinic staff and between clinics and field interviewers. The study included extensive field interviewer supervision by a bilingual and bicultural researcher using real-time information and data about contact attempts and interview completion rates with parents to support field interviewers in reaching their recruitment and retention goals. Supervision also addressed the challenges experienced by Field Interviewers when engaging with caregivers and their infants to maintain boundaries. Field interviewers also were provided monthly summaries of their progress-towards-goals based on the targeted recruitment number, also discussed during monthly individual supervision. Study retention was 84% at midpoint, and 78% at the final interview (Table 1).
Family longitudinal interview survey measures
Table 2 presents the measures set administered with parents/caregivers. Interviews included a group of established and newly developed measures of five areas of family experiences: (1) the home and neighborhood environment, (2) parent adverse childhood experiences, (3) resilience and personal mastery, (4) parenting, and (5) access to social support and resources. Measures were selected to provide a snapshot of comprehensive household and environmental risk and protective factors in infancy among families receiving pediatric care in participating clinics. The study will use this snapshot to examine how clinic engagement and support for impacted families entering pediatric care with diverse constellations of strengths and needs. Among the measures assessed at enrollment, we collected parents’ report of Adverse Childhood Experiences (ACEs), using the expanded scale developed by the Center for Youth Wellness . Though primarily interested in current household and environmental circumstances, caregiver ACEs are associated with factors directly impacting the child’s environment, including caregiver mental health, substance use, and relational well-being and were therefore important to capture as part of the overall assessment of the household. Licenses were obtained as required for use of six study instruments (see Table 2).
Electronic Health records
The study used electronic health record (EHR) and Medicaid administrative data to examine trends in quality of care, health care utilization and financial impact of the innovations. Outcomes assessed include: health care quality (adherence to immunization and well-child visit schedules; developmental, maternal depression, and lead screening; and continuity of care); ambulatory care-sensitive hospitalizations; avoidable Emergency Department (ED) visits; and urgent care use among children served in the participating clinics. These data were used to analyze Medicaid spending on service use (recommended preventive versus urgent care/hospitalization) to inform the cost value of the innovations. We extracted the EHR and Medicaid data for all children age birth to two served in the participating study clinics from January 1, 2014 (corresponding with implementation of the Affordable Care Act) to February 29, 2020 (corresponding to study end). For each outcome using these data, analyses compare children’s health care and ED use before and after implementation of the innovations, as well as outcomes for children who did and did not receive the innovations.
The final sample recruited into the study included 908 parents, primarily mothers (97.6%) whose average age was 29.5 years. Nearly two-thirds reported being in two-parent households (married = 42.2%, domestic partner = 26.8%). Additionally, the sample was predominately comprised of parents who reported Hispanic ethnicity (69.2%). Children in the sample were 46.9% male. Finally, although the mean income across the sample of $37,788 depicted a moderate income-level, income varied substantially across the clinic sites. Of note, families at six of the clinics reported average income near or below the Federal Poverty Line for a family of 5 (FPL = $30,170). Range of annual family income was $0 to $550,000. The enrollment interview was completed by 98% of the recruited sample (n = 888); 764 parents completed the midpoint interview (84% of recruited sample), and 707 parents completed the final interview (78% of recruited sample). Table 3 presents sample demographics.
This paper describes the developmental approach used to execute a comprehensive, multi-level investigation of five communities and nine pediatric health care clinics implementing approaches to bolster screening and support for families of infants to address toxic stress and social determinants of health. We used this developmental evaluation approach to describe the early stages of adoption of the innovations and to tell the story of the intersecting systems essential to addressing the multi-factorial nature of toxic stress on families with young children. The layered community, pediatric clinic, and family perspectives form a comprehensive view of families with infants and their experiences with healthcare and community services. We used rapid-cycle feedback and co-interpretation approaches to maintain study engagement, and to elicit insight on preliminary findings from innovation developers, community providers, clinics, and families. This was a critical opportunity to gather early reactions from data sources to enhance our understanding of local context and add credibility of findings. This also ensured that clinics could respond to early findings to refine their service approaches.
The study answers questions about the value of social need screening and referral systems utilizing two approaches; that of having an embedded family specialist and legal consultation in medical practices (such as DULCE) and the other using existing health care practitioners and a multi-practice learning collaborative approach (such as I-SCRN). We do not compare the approaches’ outcomes given the developmental nature of their implementation, but rather we draw themes and implications that indicate the strengths and challenges to the implementation of each from multiple perspectives. We focus on barriers and facilitators to families connecting to and accepting services (such as early developmental interventions) as one of the significant challenges that must be addressed in the near term if the innovations are to be successful. Our integrated study of Help Me Grow in three communities aims to increase understanding of the role of a centralized referral system, such as HMG, in facilitating referrals that originate from the increased screening and family engagement approaches occurring in pediatric health care clinics. Primary care practices are stretched thin in their efforts to connect families to needed social services [52, 53].
The study advances policy and practices in the field of health care and community partnerships that will help to reduce the health implications of toxic stress exposure and lack of protective factors in infancy and early childhood. There is a business case that is central to integrating innovations such as those described in this study into pediatric health care: the potential for improved population health and reduced system cost over the life course, first among children—and later adolescents and adults—a preventive service model that is intended to shore up family access to resources early in children’s lives so that children do not experience chronic stress that can have health impact .
A strength of the study is that it was able to find common ground across five communities and nine clinics about these approaches to evaluation. An invaluable asset was being able to join an already established network, Early Childhood Learning and Innovation Network for Communities, where a commitment to data-driven decision-making and evidence creation was valued. While developmental evaluations of innovation adoption, implementation, and sustainability are often focused on single innovations , we believe there is value added in bringing the same questions and approaches to multiple approaches addressing social determinants of health. This paper is limited to describing the methodological approaches; outcomes are not addressed here but are being prepared for publication.
Health care practices considering screening and referral innovations for social needs in their medical setting face three primary challenges. First, because there is yet insufficient evidence that investing in innovations to address families’ social needs and reduce toxic stress will result in intended outcomes of improved health, increased health care quality, and cost savings, implementing these approaches currently requires practices to take a significant “leap of faith.” Health care systems and payors need the backing of rigorous research and evaluation to make a business case for the models and this is not yet possible. There is preliminary evidence of similar models in the area of increasing families’ access to resources and improving socio-economic circumstances. For example, using the WE CARE model to systematically screen and refer families to services, results of a randomized control trial including 336 mothers in 8 community health centers showed that mothers who received the intervention were more likely to be employed, less likely to be homeless, and to have accessed at least one resource compared with mothers who did not receive the intervention . Studies such as this are promising, but more is needed to particularly understand health impact. More evidence is needed about what works best for whom and in what context, the key ingredients to successful approaches, and the approaches that show the most cost efficiency.
Second, health care in the U.S. does not yet have a universal administrative framework to pay providers to implement screening and the service coordination and referral roles that accompany identifying families’ social needs. A number of U.S. federal funding initiatives were launched in 2016–2017 to spark innovation through the Center for Medicare and Medicaid Innovation (CMMI). The initial phase evaluation results show challenges in areas such as having an adequate work force to address social needs, lack of standardization, adequate data systems, and closing referral loops between health care and community services [57, 58]. Medicaid demonstration programs are testing different payment and delivery models that include broadening services to health-related needs such as providing community health workers to assist families with housing, food and income [59, 60]. The recently introduced Social Determinants Accelerator Act of 2019 would convene a federal inter-agency technical advisory council to identify opportunities for state and local governments to coordinate funding and administration of federal programs that may be underutilized or unknown. These emerging approaches will provide many lessons and eventually provide direction for more widespread adoption if positive financial and quality of care impacts are realized.
Finally, health care practices need to consider the strengths and gaps in their partnerships with community service providers. Linkages between health care and community services are absent in most communities and building these systems takes time. Our study identifies social needs reported by families of infants receiving pediatric primary care. From family interview data, we will create profiles of families formed by common characteristics in the areas of risk and resilience and assess differences in health care engagement and longitudinal child and family outcomes. This will help health care stakeholders prioritize and tailor services to meet common social needs.
There are important limitations of this study. First, generalizability to under-served populations is limited. Black/African Americans, Asian, and indigenous people were less represented across the clinic sites and may have different experiences to share. Second, recruiting families to participate in focus groups that are held at a specific time/location was challenging. The intention was to have larger numbers participate but interest, feasibility with infants and young children, timing and transportation obstacles limited the size of some groups. Child care was provided during focus groups to facilitate participation. Third, clinic electronic health records (EHR) were maintained in highly disparate formats and accessibility, making it challenging to address questions across multiple states, communities, and clinics. While all of the clinics had EHR, the accessibility to data, functionality of the systems, documentation, and the ease of use created significant challenges. Finally, the study concluded when the children of the families were under the age of 2. Studying families beyond infancy will permit greater insights into the role that parent engagement with clinics and early childhood community services play in empowering families and whether their early experience with screening and referral services in their child’s infancy has an enduring impact on health and wellness.
This study is one of the first comprehensive investigations of health innovations in primary care designed to prevent and mitigate toxic stress. Our articulated theory of change asserts that the effects of systems change, provider behavior, and family engagement efforts will continue to evolve and as a consequence of this change, families will experience increased and enhanced health and concrete support, in turn, affecting parent sense of agency and behavioral change that promotes child health and well-being. As the number of families living in stressful environmental conditions exacerbated by poverty continues to increase in the U.S., it is critical to develop effective methods to mitigate the impacts of toxic stress. Crucially needed are cost-effective supports for families that occur at critical transition points, are driven by families, and sustainable within the existing funding structures. The products of the evaluation will serve as benchmark evidence in the field and provide not only a vehicle for clarifying approaches to reducing stress but seed a basis for beginning to discern critical ingredients that inform replication. Investigating how innovations arise and impact families in pediatric primary care-community service partnerships that provide services to low-income as well as middle-income families allows the documentation of under-told stories of family resilience and strain to systems serving families.
Availability of data and materials
The datasets generated and/or analyzed during the current study are not publicly available but are available from the corresponding author on reasonable request.
Adverse Childhood Experiences
American Academy of Pediatrics
Brief Infant-Toddler Social and Emotional Assessment
Connor-Davidson Resilience Scale
Clinic Inclusivity and Regard for Parents Scale
Center for the Study of Social Policy
Center for Medicare and Medicaid Innovation
Developmental Understanding and Legal Collaboration for Everyone
Early Childhood Learning and Innovation Network for Communities
Electronic Health Records
Functional Impact of Life Events for Parents
Federal Poverty Line
Help Me Grow
Infant Development Questionnaire
Institutional Review Board
Improving Screening, Connections with Families, and Referral Networks
National Survey of Children’s Health
Parent Patient Activation Measure
Parenting Interactions with Children: Checklist of Observations Linked to Outcomes
Parental Stress Scale
Safe Environment for Every Kid
Social determinants of health
- U.S. DHHS:
U.S. Department of Health and Human Services
World Health Organization, Commission on Social Determinants of Health. Closing the Gap in a Generation: Health Equity through Action on the Social Determinants of Health. In: Final Report of the Commission on Social Determinants of Health. Geneva: World Health Organization; 2008.
World Health Organization. Social Determinants of Health; 2020. www.who.int/social_determinants. Accessed 29 June 2020.
Dankwa-Mullan I, Rhee KB, Stoff DM, Pohlhaus JR, Sy FS, Stinson N, Ruffin J. Moving toward paradigm-shifting research in health disparities through translational, transformational, and transdisciplinary approaches. Am J Public Health. 2010;100:S19–24.
Rosen Valverde JN, Backstrand J, Hills L, Tanuos H. Medical-legal partnership impact on parents’ perceived stress: a pilot study. Behav Med. 2018;45:70–7. https://doi.org/10.1080/08964289.2018.1481011.
Sandel M, Sheward R, Ettinger de Cuba S, Coleman S, Heeren T, Black MM, Casey PH, Chilton M, Cook J, Cutts DB, Rose-Jacobs R, Frank DA. Timing and duration of pre- and postnatal homelessness and health of young children. Pediatrics. 2018;142(4):e20174254. https://doi.org/10.1542/peds.2017-4254.
Brisson D, McCune S, Wilson JH, Speer SR, McCrae JS, Calhoun KH. A systematic review of the association between toxic stress and poverty. J Evid Base Soc Work. 2020;12:1–18. https://doi.org/10.1080/26408066.2020.1769786.
Roggman LA. Home visiting to promote developmental parenting: measurement to ensure quality. In: Roggman L, Cardia N, editors. Home visitation programs: preventing violence and promoting healthy early childhood. New York: Springer Press; 2016. p. 35–62.
Shonkoff JP, Garner AS, The Committee on Psychosocial Aspects of Child and Family Health, Committee on Early Childhood, Adoption, and Dependent Care, Section on Developmental and Behavioral Pediatrics. The lifelong effects of early childhood adversity and toxic stress. Pediatrics. 2012;129(1):e232–46. https://doi.org/10.1542/peds.2011-2663.
Bethell C, Jones J, Gombojav N, Linkenback J, Sege R. Positive childhood experiences and adult mental and relational health in a statewide sample. JAMA Pediatr. 2019;173(11):e193007. https://doi.org/10.1001/jamapediatrics.2019.3007.
Daniel H, Bornstein SS, Kane GC. Addressing social determinants to improve patient care and promote health equity: an American College of Physicians position paper. Ann Intern Med. 2018;168(8):577–8. https://doi.org/10.7326/M17-2441.
Garner AS, Shonkoff JP, Siegel BS, Dobbins MI, Earls MF, Garner AS, Wood DL. Early childhood adversity, toxic stress, and the role of the pediatrician: translating developmental science into lifelong health. Pediatrics. 2012;129(1):e224–31. https://doi.org/10.1542/peds.2011-2662.
King TM, Tandon D, Macias MM, Healy JA, Duncan PM, Swigonski NL, Skipper SM, Lipkin PH. Implementing developmental screening and referrals: lessons learned from a national project. Pediatrics. 2010;125(2):350–60. https://doi.org/10.1542/peds.2009-0388.
Radecki L, Sand-Loud N, O’Connor KG, Sharp S, Olson LM. Trends in the use of standardized tools for developmental screening in early childhood: 2002–2009. Pediatrics. 2011;128(1):14–9. https://doi.org/10.1542/peds.2010-2180.
Adams EK, Johnston EM, Guy G, Joski P, Ketsche P. Children’s health insurance program expansions: what works for families? Glob Pediatric Health. 2019;6:1–12. https://doi.org/10.1177/2333794X19840361.
U.S. Dept. of Health and Human Services, Office of Minority Health. Minority Population Profiles; 2017. https://minorityhealth.hhs.gov/omh/browse.aspx?lvl=3&lvlid=64. Accessed 29 June 2020.
Berkowitz SA, Basu S, Gundersen C, Seligman HK. State-level and county-level estimates of health care costs associated with food insecurity. Prev Chronic Dis. 2019;16:180549. https://doi.org/10.5888/pcd16.180549.
Carroll A, Corman H, Curtis MA, Noonan K, Reichman NE. Housing instability and children’s health insurance gaps. Acad Pediatr. 2017;17(7):732–8. https://doi.org/10.1542/peds/2017-0297.
Urban Institute. With public charge rule looming, one in seven adults in immigrant families reported avoiding public benefit programs in 2018: Urban Institute; 2018. https://www.urban.org/urban-wire/public-charge-rule-looming-one-seven-adults-immigrant-families-reported-avoiding-public-benefit-programs-2018. Accessed 12 Jan 2020.
Sege R, Preer G, Morton SJ, Cabral H, Morakinyo G, Lee V, Abreu C, De Vos E, Kaplan-Sanoff M. Medical-legal strategies to improve infant health care: A randomized trial. Pediatrics. 2015;136(1). https://doi.org/10.1542/peds.2014-2955.
Flower K, Earls M, Nagy B, Janies K, Massie S, Bassewitz J. I-SCRN: A quality improvement collaborative to increase early childhood screening, referral, and follow-up in pediatric primary care practices. Pediatrics. 2019;144(2). https://doi.org/10.1542/peds.144.2_MeetingAbstract.72.
Flower KB, Massie S, Janies K, Bassewitz JB, Coker TR, Gillespie RJ, Macias MM, Whitaker TM, Zubler J, Steinberg D, DeStigter L, Earls MF. Increasing early childhood screening in primary care through a quality improvement collaborative. Pediatrics. 2020;146(3). https://doi.org/10.1542/peds.2019-2328.
Help Me Grow. The Help Me Grow System Model: Help Me Grow National Center; 2020. https://helpmegrownational.org/hmg-system-model/. Accessed 29 June 2020.
Gottfriedson DC, Cook TD, Gardner FEM, Gorman-Smith D, Howe GW, Sandler IN, Zafft KM. Standards of evidence for efficacy, effectiveness, and scale-up research in prevention science: next generation. Prev Sci. 2015;6:151–75. https://doi.org/10.1007/s11121-015-0555-x.
Beckett K, Farr M, Kothari A, Wye L, le May A. Embracing complexity and uncertainty to create impact: exploring the processes and transformative potential of co-produced research through development of a social impact model. Health Res Policy Sys. 2018;16:118. https://doi.org/10.1186/s12961-018-0375-0.
Patton MQ. Developmental evaluation: applying complexity concepts to enhance innovation and use. New York: Guilford Press; 2010.
Birt L, Scott S, Cavers D, Campbell C, Walter F. Member checking: a tool to enhance trustworthiness or merely a nod to validation? Qual Health Res. 2016;26(13):1802–11. https://doi.org/10.1177/1049732316654870.
Dubowitz H, Feigelman S, Lane W, Kim J. Pediatric primary care to help prevent child maltreatment: the safe environment for every kid (SEEK) model. Pediatrics. 2009;123(3):858–64. https://doi.org/10.1542/peds.2008-1376.
Dubowitz H, Feigelman S, Lane W, Prescott L, Blackman K, Grube L, Meyer W, Tracy JK. Screening for depression in an urban pediatric primary care clinic. Pediatrics. 2007;119(3):435–43. https://doi.org/10.1542/peds.2006-2010.
Dubowitz H, Prescott L, Feigelman S, Lane W, Kim J. Screening for intimate partner violence in a pediatric primary care clinic. Pediatrics. 2008;121(1):e85–91. https://doi.org/10.1542/peds.2007-0904.
Lane W, Dubowitz H, Feigelman S, Prescott L, Meyer W, Tracy JK. Screening for parental substance abuse in pediatric primary care. Ambul Pediatr. 2007;7(6):458–62. https://doi.org/10.1016/j.ambp.2007.07.007.
Fauth RC, Leventhal T, Brooks-Gunn J. Short-term effects of moving from public housing in poor to middle-class neighborhoods on low-income, minority adults’ outcomes. Soc Sci Med. 2004;59(11):2271–84. https://doi.org/10.1016/j.socscimed.2004.03.020.
Rollins C, Glass NE, Perrin NA, Billhardt KA, Clough A, Barnes J, Hanson GC, Bloom TL. Housing instability is a strong a predictor of poor health outcomes as level of danger in an abusive relationship: findings from the SHARE study. J Interpers Violence. 2012;27(4):623–43. https://doi.org/10.1177/0886260511423241.
Burke Harris N, Renschler T. Center for Youth Wellness ACE-questionnaire (CYW ACE-Q child, teen, teen SR). San Francisco: Center for Youth Wellness; 2015. https://centerforyouthwellness.org/wp-content/uploads/2018/06/CYW-ACE-Q-USer-Guide-copy.pdf. Accessed 29 June 2020.
Moreno A. Functional impact of life events for parents (FITS-P). Unpublished measure. Chicago: Erikson Institute; 2016.
Connor KM, Davidson JR. Development of a new resilience scale: the Connor Davidson resilience scale (CD-RISC). Depress Anxiety. 2003;18:76–82. https://doi.org/10.1002/da.1011339.
Pearlin LI, Schooler C. The structure of coping. J Health Soc Behav. 1978;19(1):2–21. https://doi.org/10.2307/2136319.
Pearlin LI, Lieberman M, Menaghan E, Mullan J. The stress process. J Health Soc Behav. 1981;22(4):337–56. https://doi.org/10.2307/2136676.
Turner RJ, Noh S. Physical disability and depression: a longitudinal analysis. J Health Soc Behav. 1988;29(1):23–37. https://doi.org/10.2307/2137178.
Rahman A, Iqbal Z, Robert C, Husain N. Cluster randomized trial of a parent-based intervention to support early development of children in a low-income country. Child Care Health Dev. 2009;35(1):56–62. https://doi.org/10.1111/j.1365-2214.2008.00897.x.
Berry JO, Jones WH. The parental stress scale: initial psychometric evidence. J Soc Pers Relat. 1995;12(3):463–72. https://doi.org/10.1177/0265407595123009.
Norman V, Christiansen K. Validity of the PICCOLO tool in child care settings: can it assess caregiver interaction behaviors? Infant Ment Health J. 2013;34(4):319–29. https://doi.org/10.1002/imhj.21391.
Briggs-Gowan MJ, Carter AS, Irwin JR, Wachtel K, Cicchetti DV. The Brief Infant-Toddler Social and Emotional Assessment: Screening for social-emotional problems and delays in competence. J Pediatr Psychol. 2004;29(2):143–55. https://doi.org/10.1093/jpepsy/jsh017.
Briggs-Gowan MJ, Carter AS, McCarthy K, Augustyn M, Caronna E, Clark R. Clinical validity of a brief measure of early childhood social-emotional/behavioral problems. J Pediatr Psychol. 2013;38(5):577–87.
Mathematica Policy Research, Inc. & the Administration on Children, Youth, and Families, U.S. Department of Health and Human Services. In: Early Head Start Parent Interview: For Parents of 14-Month-Old Infants; 1998.
LeCroy C, Milligan Associates, Inc. Healthy Families Parenting Inventory. In: Unpublished users Manual; 2004.
Krysik J, LeCroy CW. Development and initial validation of an outcome measure for home visitation: the Healthy Families Parenting Inventory. Infant Ment Health J. 2012;33(5):496–505. https://doi.org/10.1002/imhj.21343.
Help Me Grow. The Help Me Grow System Model. In: Help Me Grow National Center. https://helpmegrownational.org/hmg-system-model/; 2020.
Insignia Health. Parent Patient Activation Measure. http://www.insigniahealth.com. 2011.
Hibbard JH, Stockard J, Mahoney ER, Tusler M. Development of the patient activation measure (PAM): conceptualizing and measuring activation in patients and consumers. Health Serv Res. 2004;39:1005–26. https://doi.org/10.1111/j.1475-6773.2004.00269.x.
Maslach C, Jackson S, Leiter M. Maslach Burnout Inventory: manual and non-reproducible instrument and scoring guides; forms included: general (MBI – GS), human services (MBI – HSS), et educators (MBI – ES). Menlo Park: Mind Garden; 2013.
American Academy of Pediatrics Periodic Survey of Fellows. Menlo Park: 2017. https://www.aap.org/en-us/professional-resources/Research/pediatrician-surveys/Pages/survey-findings.aspx. Accessed 22 Mar 2017.
LaForge K, Gold R, Cottrell E, Bunce AE, Proser M, Hollombe C, Clark KD. How 6 organizations developed tools and processes for social determinants of health screening in primary care. J Ambul Care Manage. 2018;41:2–14. https://doi.org/10.1097/JAC.0000000000000221.
Berry C, Paul M, Massar R, Marcello RK, Krauskopf M. Social needs screening and referral program at a large US public hospital system, 2017. Am J Public Health. 2020;110:S211–4. https://doi.org/10.2105/AJPH.2020.305642.
Murray G, Rodriguez H, Lewis V. Upstream with a small paddle: how ACOs are working against the current to address patients’ social needs. Health Aff. 2020;39(2):199–206.
Miller TR, Hendrie D. Nurse family partnership: comparing costs per family in randomized trials versus scale-up. J Prim Prev. 2015;36(6):419–25. https://doi.org/10.1007/s10935-015-0406-3.
Garg A, Toy S, Tripodis Y, Silverstein M, Freeman E. Addressing social determinants of health at well child care visits: a cluster RCT. Pediatrics. 2015;135(2):e296–304. https://doi.org/10.1542/peds.2014-2888.
Beil H, Feinberg RK, Patel SV, Romaire MA. Behavioral health integration with primary care: implementation experience and impacts from the state innovation model round 1 states. Milbank Q. 2019;97(2):543–82. https://doi.org/10.1111/1468-0009.12379.
Tumber MB, Bunzli L, Rosenberg M. Addressing the social determinants of health: the Rhode Island state innovation model (RI SIM) experience. Rhode Island Med J. 2019;102(5):22–5.
Alley DE, Asomugha CN, Conway PH, Sanghavi DM. Accountable health communities – addressing social needs through Medicare and Medicaid. N Engl J Med. 2016;374:8–11. https://doi.org/10.1177/2333794X19840361.
Hinton E, Artiga S, Musumeci M, Rudowitz R. A first look at North Carolina’s section 1115 Medicaid waiver’s healthy opportunities pilots. San Francisco: Kaiser Family Foundation; 2019. https://www.kff.org/medicaid/issue-brief/a-first-look-at-north-carolinas-section-1115-medicaid-waivers-healthy-opportunities-pilots/. Accessed 29 June 2020.
The authors thank Mickie Anderson, Julia Bachman, Cheryl Austein-Casnoff, Megan Finno-Velasquez, Kamia Rathore, and Sameer Vohra for their contributions to this manuscript.
This project was funded by a grant from The JPB Foundation. The Chapin Hall team would like to thank our national partners—The Center for the Study of Social Policy and the American Academy of Pediatrics—for their support of this work, for example, facilitating initial relationships with DULCE and I-SCRN clinics and sharing programmatic information. Data was obtained with the support and guidance of our clinic partners. We are grateful to them for their willingness to support this project and for their commitment to sharing the authentic voice of the families they serve. The opinions, findings, and conclusions or recommendations expressed in this publication are solely those of the authors and do not necessarily reflect those of The JPB Foundation, The Center for the Study of Social Policy, the American Academy of Pediatrics, or our clinic partners.
Ethics approval and consent to participate
All study procedures were approved by the University of Chicago Social Services Administration-Chapin Hall Institutional Review Board (SSA IRB). Participant consent procedures were approved by the SSA IRB. Written consent was obtained for all family and stakeholder interview participants. All focus group participants provided verbal consent. We obtained a waiver for written informed consent for the focus groups because the research presented no more than minimal risk of harm and involved no procedures for which written consent is normally required outside of a research context, and written consent would be the only record linking the subjects to the research. At each focus group, we asked subjects to affirm their agreement to participate in the study and to be recorded at the start of the audio recording.
Families (adult parents/caregivers) provided written consent for researchers to access electronic health record and Medicaid claims data. Data sharing agreements and business associate agreements for sharing protected health information were executed with each clinic, health system, or state Medicaid agency and approved by the SSA IRB and in the case of one clinic as required, their hospital system IRB.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
McCrae, J.S., Robinson, J.A.L., Spain, A.K. et al. The Mitigating Toxic Stress study design: approaches to developmental evaluation of pediatric health care innovations addressing social determinants of health and toxic stress. BMC Health Serv Res 21, 71 (2021). https://doi.org/10.1186/s12913-021-06057-4
- Toxic stress
- Social determinants of health
- Pediatric primary care
- Early childhood