Identifying and developing strategies for implementation of a guided internet- and mobile-based infant sleep intervention in well-baby and community mental health clinics using group concept mapping

Abstract

Background

This study aimed to identify strategies for the implementation of a guided internet- and mobile-based intervention (IMI) for infant sleep problems (“Sleep Well, Little Sweetheart”) in well-baby and community mental health clinics.

Study design

We used group concept mapping, a two-phased mixed methods approach, conducted as a two-day workshop in each clinic. We recruited 20 participants from four clinics and collected sorting and rating data for implementation strategies based on the Expert Recommendations for Implementing Change taxonomy and brainstorming sessions. Data were analyzed using descriptive statistics, multidimensional scaling, and hierarchical cluster analysis to create cluster maps, laddergrams, and Go-Zone graphs. Participants were presented with the results and discussed and interpreted the findings at each of the clinics in spring 2022.

Results

Participants identified 10 clusters of strategies, of which Training, Embedding and Coherence, User Involvement and Participation, and Clinician Support and Implementation Counseling were rated as most important and feasible. Economy and Funding and Interactive and Interdisciplinary Collaboration were rated significantly lower on importance and feasibility compared to many of the clusters (all ps < 0.05). There was a correlation between the importance and feasibility ratings (r =.62, p =.004).

Conclusions

The use of group concept mapping made it possible to efficiently examine well-baby and community clinics’ perspectives on complex issues, and to acquire specific knowledge to allow for the planning and prioritization of strategies for implementation. These results suggest areas of priority for the implementation of IMIs related to infant sleep problems.

Trial registration

The study was pre-registered at Open Science Framework (www.osf.io/emct8).

Background

About 20% of infants and toddlers experience sleep problems related to sleep onset, night waking, and sleep scheduling (i.e., pediatric insomnia) [1, 2]. Few parents, however, raise their concerns about their child’s sleep problems with health professionals (HPs) and many HPs have little or no formal training in pediatric sleep [1]. There are effective psychosocial and behavioral interventions [3, 4], but their descriptions in research often lack in detail to help HPs in their implementation [5]. This may result in underidentification and undertreatment of pediatric sleep problems [1], and the endorsement and delivery of sleep assessment and treatments that are not evidence-based [6]. Thus, HPs and parents would benefit from easy access to standardized and evidence-based screening and intervention programs.

Digital interventions can provide standardized, evidence-based and accessible care ‘where parents are’, that is, in their local communities, at home, and online. Parents actively search for information about infant sleep and health online [7, 8], and most parents and HPs are interested in internet-based infant sleep programs [8, 9]. Despite this, research on internet- and mobile-based interventions (IMIs) for pediatric insomnia in typically developed young children is still in its infancy. There is currently only one randomized trial of an internet intervention, that showed reductions in problematic sleep, sleep onset latency, and the number and duration of night wakings [10]; improvements that were maintained one year later [11]. In terms of mobile interventions, only one app was empirically supported [12], according to a recent review [13]. Thus, the few initial results seem promising. A few studies have also examined the dissemination of online sleep information and the usability of online tools or interventions [14,15,16,17]. Such studies are useful for identifying barriers and facilitators to implementation such as the parental needs for professional guidance and cultural adaptations (e.g., many practice co-sleeping) and the time restrictions and training needs of HPs [15].

Internet interventions have existed for 20 years. However, with a few exceptions [18], efforts to integrate these into routine practice have had mixed success. There are many factors that may promote or impede the implementation of IMIs. Provision of information, training, and infrastructure to those involved is key to success [19], but is insufficient by itself. Many HPs may still be skeptical about using IMIs, and experience excessive workloads and disruptions to their workflow [19, 20]. Compatibility (or lack thereof) with existing systems, ambiguous policies, and costs, are other known challenges that must be addressed at an organizational or policy level [19,20,21]. Practical guidelines may provide important insights and key points to consider (e.g., privacy, funding schemes, policy, and regulatory context) [22, 23]; however, the development of IMIs still requires careful consideration of the context in which they are to be used [24].

To integrate IMIs into practice, it is necessary to identify and select contextually appropriate implementation strategies. Implementation strategies can be defined as “methods or techniques used to enhance the adoption, implementation, and sustainability of a clinical program or practice” [25, p. 2]. However, selecting implementation strategies is challenging for several reasons; it requires careful consideration of contextual variations [24, 26]; despite there being around 170 implementation theories and models [27], most have a limited evidence-base due to being underutilized [28]; and, there is an abundance of strategies that can be combined in numerous ways [29,30,31]. The literature thus provides limited guidance on the selection of implementation strategies. Therefore, the objective of this study was to advance our practical understanding of barriers and facilitators that can influence the implementation of IMIs. More specifically, our aim was to identify implementation strategies that are important and feasible to integrate a guided IMI for infant sleep problems (“Sleep Well, Little Sweetheart”) into well-baby and community mental health clinics by using group concept mapping [32, 33].

Methods

Group concept mapping (GCM) is a two-phased, participatory sequential mixed methods approach to guide planning and program evaluation [34]. In the first qualitative phase (i.e., the first workshop), participants received a demonstration of Sleep Well, Little Sweetheart and explained that “the goal of the workshop is to arrive at a framework for the implementation of the program”. They were specifically instructed to “consider the strategies related to Sleep Well, Little Sweetheart”, although the focus prompt was formulated as “what conditions must be present for successful implementation at your workplace?”, to help with the sorting and rating tasks, and to broaden their mind during brainstorming. Finally, participants individually sorted and rated a set of pre-defined implementation strategies and engaged in a brainstorming session (see Supplementary Materials). In the second phase, researchers performed quantitative analyzes to represent the sorting and rating data, which were presented at the second workshop where participants discussed and interpreted the findings.

Intervention

Sleep Well, Little Sweetheart is a guided IMI for pediatric insomnia in infants from six months to three years. It consists of eight program modules: (1) infant sleep assessment [35], (2) psychoeducation about infant sleep [35, 36], (3) individual bedtime routine [37], (4) infant crying [38], (5) parent emotion regulation [39], (6) individual sleep plan (i.e., extinction-based plans, bedtime fading or scheduled awakenings) [40], (7) relapse prevention, and (8) a sleep diary [41]. The program is administered and delivered by HPs, using the Youwell platform (www.youwell.no). HPs task is to establish and maintain a high-quality working alliance with the parents, motivate them to use the program, and adapt the program contents to the individual family (e.g., individual bedtime routine and sleep plan) [42], either during routine face-to-face consultations or via technology (i.e., text messages or mobile phone).

Participants

We recruited 20 participants from two well-baby clinics (n = 13, 65%) and two community mental health clinics (n = 7, 35%), which is sufficient for GCM and above the typically recommended sample size of 15 [43]. All participants were women, with an average age of 47.1 years, and a university or college degree. Eleven (55%) of the participants worked as public health nurses (PHNs), four (20%) as psychologists, and four (20%) in other positions (e.g., family therapists). Three (15%) of the participants were clinic leaders. The majority had some clinical training and experience with infant sleep (n = 18, 90%), but few had any personal or professional experience with IMIs (n = 4, 20%).

Data collection

Prior to the first workshop, participants gave their consent and provided background information (e.g., age, employment, and experience with infant sleep). GCM was conducted in each clinic from March to May/June 2022. The first step of data collection consisted of individual sorting of 73 implementation strategies from the Expert Recommendations for Implementing Change (ERIC)–taxonomy [29], including suggested labels for each group of piles. Participants were instructed to group strategies in a meaningful way, based on their similarities. The participants then rated each strategy in terms of its relative importance and feasibility in their workplace, as two separate ratings, on a scale from 1 (“not at all important/feasible”) to 5 (“extremely important/feasible”). To ensure that the piles were not mixed, participants were instructed to use rubber bands and zipped bags (see Supplementary Materials for participant instructions).

The ERIC–taxonomy is a widely applicable, standardized, and manageable set of implementation strategies that minimizes participant burden and maximizes breadth. However, it may not cover context-specific strategies outside U.S. or North–American settings and ensure data saturation. Thus, participants brainstormed individually and created 56 additional strategies that were written on post-its and placed in separate zipped bags. These were reduced to 24 novel strategies by plenary discussions and a final review by the second author, which can be considered indicative of saturation. The list of statements and translations is provided in Table 1. At the second workshop, participants were presented with the results from the first workshop, they discussed and interpreted the findings (e.g., if the results were surprising or reflected their opinion), and sorted and rated strategies from the brainstorming. Analyses were then updated to include sorting and rating data from the second workshop. The participants’ interpretations of the findings were audio-recorded, summarized, and integrated in the Discussion below. The first and second authors moderated the workshops.

Table 1 Overview and translations of implementation strategies.^a

Data analyses

Demographics, importance, and feasibility ratings were analyzed using descriptive statistics. Analyses were performed using the open-source software R [44] and R-CMap package [45]. Ward’s algorithm was used for multidimensional scaling and hierarchical cluster analysis to characterize how participants grouped strategies and how they were rated in terms of their importance and feasibility (i.e., a cluster rating map). The stress value of the multidimensional scaling was 0.336 and is an indicator of the relationship between the strategies, their similarities, and distances on the map, in which lower values reflect a better fit. Values ≤ 0.39 are acceptable and unlikely to have either no structure or a random two-dimensional configuration [43, 46]. Furthermore, we calculated the split-half reliability as a measure of the overall consistency of the card sort, using 20 random splits of participants, to 0.39.

There is no true number of clusters in a final map. The goal is to produce a set of clusters that are intuitive and meaningful. The within-cluster sum of squares, a measure of the variability of observations within each cluster, indicated an 11-cluster solution as a point of departure (see Fig. 1). A backward process with a stepwise reduction in clusters ended when further merging disrupted the meaning of the strategies in each separate cluster, as sorted by participants. It should be noted that the structure of hierarchical trees is determined by analysis, and not by the researchers [47]. In this study, the first and fifth authors examined clusters emerging from each step and agreed on the final number, after reviewing the content and meaning within each cluster. Each cluster was labeled based on the names proposed by the participants. After determining the number of clusters, each strategy’s importance and feasibility score were plotted on a scatterplot and divided into four quadrants using the mean of each dimension to identify actionable strategies for an implementation plan (i.e., ‘Go-Zone’ analyses). A laddergram was created to show cluster-level differences in importance and feasibility ratings and an analysis of variance and Tukey’s post-hoc test of multiple comparisons was used for mean comparisons between clusters.

Fig. 1

Number of clusters– within cluster sums of squares

Results

All participants sorted and rated all strategies, except one that only sorted and rated strategies in the ERIC–taxonomy. On average, participants created 8.25 piles (SD = 2.92; Range = 5–16). Two participants put more than one-third of the cards in one pile. Strategies 8, 10, 11, 12, 68, and 87 formed a separate cluster in the initial 11-cluster solution but were merged to form a single cluster labeled Preparation and Facilitation, as the original clusters were not judged as sufficiently distinct or intuitive. The final cluster map consists of 10 clusters with 4 to 13 strategies per cluster. Table 2 presents a summary of the clusters, their corresponding strategies, and mean importance and feasibility ratings at the cluster level. Table 3 summarizes the implementation strategies and their mean importance and feasibility ratings, organized by cluster and Go-Zone quadrant.

Table 2 Summary of clusters of implementation strategies and their importance and feasibility ratings

Table 3 Summary of implementation strategies organized by Go-Zone quadrants globally and per cluster

Figure 2 presents a point and cluster rating map that visually represents the relationship between the 97 strategies, accompanied by a number for cross-referencing to the strategies in Tables 2 and 3. In general, the closer two strategies are together, the more often they were sorted together (e.g., strategies 1 (access new funding) and 49 (place innovation on fee for service lists) in the Economy and Funding cluster; see Fig. 2). Strategies farther apart from each other were less often, if at all, sorted together (e.g., strategies 11 (change physical structure and equipment) and 16 (conduct educational outreach visits) in the Preparation and Facilitation and Training clusters, respectively). Similarly, clusters near one another are more closely connected than those farther away. Clusters in the middle of the map (i.e., Quality Assurance and Embedding and Coherence) can be considered to function as a bridge for interaction between other clusters. For example, establishing a coherent individual and collective understanding of a new practice, can make any preparations and training, both more meaningful and thus easier to embed into routine practice.

Fig. 2

A point and cluster rating map for each cluster by importance (n = 20)

Figure 3 shows the global Go-Zone graph for each of the 97 strategies. The graph was divided into four quadrants by the average importance (M = 3.61, SD = 0.70) and feasibility (M = 3.21, SD = 0.59) ratings. There was a significant correlation between ratings (r =.62, p =.004), indicating that most strategies fell within quadrants I (n = 44, 45.4%) or IV (n = 34, 35.1%). The upper right quadrant (I), referred to as the Go-Zone, shows strategies that were rated above average on both importance and feasibility. These strategies were mostly from clusters 2, 5, 6, 7, 8, and 10 (i.e., Preparation and Facilitation, Embedding and Coherence, Leadership and Organization, Training, Clinician Support and Implementation Counseling, and User Involvement and Participation; see also Tables 2 and 3), and should be prioritized and addressed first in any ensuing implementation plan. Conversely, strategies rated lowest on both importance and feasibility, fell within the lower left quadrant (IV; i.e., the No-Go zone). These were predominantly from clusters 1 to 4 (i.e., Economy and Funding, Preparation and Facilitation, Implementation, and Interactive and Interdisciplinary Collaboration; see also Tables 2 and 3). Only a few strategies were rated relatively important (upper left quadrant III; n = 11, 11.3%) or feasible (lower right quadrant II; n = 8, 8.2%). Table 3 also shows the Go-Zone quadrants for each cluster independently. Although most of the strategies (n = 66, 68.0%) remained in the same quadrant as in the global Go-Zone analysis, 31 (32.0%) strategies were classified into another quadrant in the per cluster analysis. Changes in quadrants among strategies between the global and per cluster analyses occurred across all clusters, but mainly in Economy and Funding (n = 7; 58.3%) and Interactive and Interdisciplinary Collaboration (n = 5, 50.0%).

Fig. 3

Global go-zone graph for all 97 strategies (n = 20)

Figure 4 compares the average importance and feasibility ratings of strategies at the cluster level. It shows that all clusters were judged relatively important, but also consistently more difficult to accomplish. Training was considered the most important and feasible, while Economy and funding had the greatest mean difference between importance and feasibility ratings and was perceived as least feasible. An analysis of variance revealed statistically significant differences in importance (F (9, 1 906) = 8.91, p <.001) and feasibility (F (9, 1 906) = 21.69, p <.001) between two or more clusters. Table 4 includes significant differences from Tukey’s test for multiple comparisons of all possible pairs. Most notably, differences show that Economy and Funding and Interactive and Interdisciplinary Collaboration were rated significantly less important than most other clusters (all ps < 0.05). In terms of feasibility, Training was perceived as more applicable than all clusters, except Embedding and Coherence, while Economy and Funding was considered harder to accomplish than all other clusters (all ps < 0.05).

Fig. 4

A laddergram comparing the average cluster ratings (n = 20)

Table 4 Multiple comparisons of mean differences in importance and feasibility with 95% family-wise confidence level (CI)

Discussion

We used GCM to identify strategies for the implementation of a guided IMI for infant sleep problems in well-baby and community mental health clinics. We identified 10 clusters of strategies, of which Training, Embedding and Coherence, User Involvement and Participation, and Clinician Support and Implementation Counseling were rated as most important and feasible. In contrast, Economy and Funding and Interactive and Interdisciplinary Collaboration were rated as least important and feasible. There was a positive linear correlation between the importance and feasibility ratings. Therefore, more strategies from the most important and feasible clusters fell into the Go-Zone quadrant, while more strategies from the least important and feasible clusters fell into the No-Go quadrant. Reflecting on data saturation, the study team found that the discussions added no major changes to the interpretation of results across groups. This likely reflected the narrow study aim, relevance/adequacy of the sample, and the applied use of a specific taxonomy and methodology providing a clear and structured dialogue between researchers and participants. In what follows, we discuss the most prominent results and interpretation of findings from discussions with participants. The references to strategies are numbered in parentheses for cross-referencing to the cluster map (Fig. 1) and Tables 2 and 3.

Overall, the results resonated with the participants. The stress value was acceptable, indicating that there is a structure to the data. Further validation of these findings can be found in Waltz and colleagues who conducted a GCM-study using the ERIC-taxonomy with implementation experts [48]. Although our study included clinical staff and additional strategies, participants conceptualized strategies in similar ways. Apart from the slightly different labels, Waltz and colleagues also identified clusters related to financial strategies, training, engaging users, collaboration with stakeholders, and supporting clinicians [48]. However, how participants sorted strategies within clusters, varied greatly. For example, we found that participants conceptualized ongoing training (19) and consultation (55) as Clinical Support and Implementation Counseling, rather than training and educating stakeholders. This may be due to the different groups of participants in the studies but may also show how the conceptualization of strategies can vary across cultures and contexts.

In discussing the findings, the participants recognized that IMIs do have setup, operation, and maintenance costs [21], but that they, as clinical staff, rarely have opportunities to influence the funding of their clinic. Therefore, Economy and Funding should not be interpreted as unimportant but must be taken care of at higher system levels (e.g., municipal or government funding), as successful examples of IMIs in routine practice have taught us [22]. Thus, it makes sense that Leadership and Organization were close to Economy and Funding on the cluster map. Participants were more concerned about learning to administer the program and any counseling methods [42], but did not consider that Training in IMIs needed to be extensive. IMIs do not require the same level of competency or skills training as face-to-face methods, as parents carry out much of the intervention themselves. However, the participants were clear about the need for active involvement in Training by participating in group work, testing, and administration of the program.

Interactive and Interdisciplinary Collaboration was rated less important and feasible compared to many clusters. During the discussions, it became clear that some strategies in this cluster were considered not currently relevant (e.g., 62), useful but not necessary (e.g., 72), or the responsibilities of other stakeholders (e.g., 24 and 73). It was more important to create structures for learning collaboratives (20) where clinicians could meet regularly to learn and share experiences. Further building of coalitions (6) and network weaving (52) could be considered important, but mainly to embed the IMI into routine care. Embedding new practices is made possible by an understanding of their meaning, uses, and utility. It requires a coherent set of beliefs and behaviors that define and organize the work, and that are seen as meaningful and different from other practices (78) [49]. In this sense, participants were surprised that informing local opinion leaders (38) was not in the Go-Zone, as they considered it essential, and should include key managers, clinicians, and administrative staff.

Regardless of clustering, certain strategies fell in the No-Go zone because they are rarely used in public healthcare in Norway (e.g., 2 (alter incentive/allowance structures) and 59 (revise professional roles)) or may be experienced as unpleasant and stressful (e.g., 26 and 27 (tools for quality monitoring), and 81 (certification schemes)). Participants agreed that purveyors must set certain quality requirements for the delivery of IMIs, but this can be achieved in other ways than through licensing standards (22) or certification schemes (81). Quality Assurance and evaluative strategies in general require a high level of psychological safety [50]. Therefore, many were reluctant to such strategies and are not used to being monitored or measured in performance. Quality Assurance and evaluative strategies were also compared to established initiatives such as specialist breastfeeding centers, which use formal quality requirements [51]. According to the participants, it must be acknowledged that evaluative strategies only provide a snapshot of the current situation and take time away from families in an already busy work schedule.

Finally, participants acknowledged the distances and spatial relationships between strategies in the cluster map (Fig. 2). Strategies within clusters spread in different spatial directions and some were farther from the center of the cluster than others, approaching neighboring clusters such as conducting local consensus discussion (17) in Embedding and Coherence and needs assessment (18) in Preparation and Facilitation. Participants explained that some strategies, such as those related to Training and Economy and Funding, were easier to sort than others, and not all the strategies were equally clear or understandable. Several mentioned that strategies could have been sorted in multiple ways or placed in several piles. They perceived that there were subtle differences between several strategies and even considered them interchangeable (e.g., reexamining the implementation (56) and audit and feedback (5)). Furthermore, the participants pointed out that the role they had in the clinic (leader vs. clinician) and professional background (PHN vs. psychologist) could also have impacted how they sorted and rated the strategies.

Strengths and limitations

GCM is an efficient and engaging method to obtain insights on a topic. It is less resource intensive than interviews, but there are also limited possibilities to probe and explore new concepts and may not provide sufficient in-depth data [52]. However, one of its strengths is the mixed methods approach. This became evident in the discussions where participants attributed less value to Economy and Funding but recognized that the ratings reflected their opportunities to influence the funding of a clinic, more than its actual importance.

For practical reasons, it was not possible to discuss the updated maps that included the strategies from the brainstorming. However, the discussions of maps based on the ERIC-taxonomy were audio-recorded, summarized, and reviewed against the final maps. Most clusters and sorting of strategies remained the same, but discussions may have given participants more time to reflect and fine-tune their sorting to truly represent their views.

It is important to note that although each cluster is unique, there are overlapping ideas between them. Participants expressed that certain strategies were more difficult to sort and that they could have sorted the strategies in multiple ways or placed certain strategies in several piles. However, in GCM, a strategy or statement can only be placed in one pile. Thus, overlaps are inevitable and common; also, because participants are instructed to sort strategies in a way that makes sense to them, without being guided by any theory or logic. Although this may be a limitation, it can also be considered a strength as it highlights challenging, ambivalent, or even contradictory concepts or ideas that may have important implications for, in our case, the implementation of the IMI.

Although the study included more than the recommended number of participants for GCM [43], subgroups of interest became unbalanced and small due to the modest sample size. For such reasons, we did not compare clustering or ratings between primary care services (i.e., well-baby vs. community clinics), professions (i.e., PHNs vs. psychologists), or roles (e.g., leaders and clinicians). This could have provided a more nuanced understanding of the implementation strategies. It could also be argued that the heterogeneity among participants more closely resembles the real-world setting in which the IMI will be implemented and thus has captured important variations in their responses. Taken together, the overlapping strategies and heterogeneity among our group of participants may reflect the modest overall consistency (i.e., split-half reliability) of the sorting task. Yet, the stress value indicated that the relationship between the data, similarity matrix, and distances on the map, was acceptable and that there is an underlying structure in the data.

Conclusions

GCM made it possible to efficiently examine the perspectives of the well-baby clinics and community clinics on complex issues, and to acquire specific knowledge to allow for the planning and prioritization of implementation strategies. Training, Embedding and Coherence, User Involvement and Participation, and Clinician Support and Implementation Counseling were identified as the most important and applicable areas for implementation. In contrast, Economy and Funding and Interactive and Interdisciplinary Collaboration were rated as least important and feasible, although they should not be ignored but taken care of for sustainable implementation. Cluster-level Go-Zone analyzes and the discussions of the findings with participants may help identify which strategies within clusters to target. These results suggest areas of priority for the implementation of IMIs related to infant sleep problems such as Sleep Well, Little Sweetheart, and potentially other practices in primary care for parents with young children.