Skip to main content

Identifying and developing strategies for implementation of a guided internet- and mobile-based infant sleep intervention in well-baby and community mental health clinics using group concept mapping



This study aimed to identify strategies for the implementation of a guided internet- and mobile-based intervention (IMI) for infant sleep problems (“Sleep Well, Little Sweetheart”) in well-baby and community mental health clinics.

Study design

We used group concept mapping, a two-phased mixed methods approach, conducted as a two-day workshop in each clinic. We recruited 20 participants from four clinics and collected sorting and rating data for implementation strategies based on the Expert Recommendations for Implementing Change taxonomy and brainstorming sessions. Data were analyzed using descriptive statistics, multidimensional scaling, and hierarchical cluster analysis to create cluster maps, laddergrams, and Go-Zone graphs. Participants were presented with the results and discussed and interpreted the findings at each of the clinics in spring 2022.


Participants identified 10 clusters of strategies, of which Training, Embedding and Coherence, User Involvement and Participation, and Clinician Support and Implementation Counseling were rated as most important and feasible. Economy and Funding and Interactive and Interdisciplinary Collaboration were rated significantly lower on importance and feasibility compared to many of the clusters (all ps < 0.05). There was a correlation between the importance and feasibility ratings (r =.62, p =.004).


The use of group concept mapping made it possible to efficiently examine well-baby and community clinics’ perspectives on complex issues, and to acquire specific knowledge to allow for the planning and prioritization of strategies for implementation. These results suggest areas of priority for the implementation of IMIs related to infant sleep problems.

Trial registration

The study was pre-registered at Open Science Framework (

Peer Review reports


About 20% of infants and toddlers experience sleep problems related to sleep onset, night waking, and sleep scheduling (i.e., pediatric insomnia) [1, 2]. Few parents, however, raise their concerns about their child’s sleep problems with health professionals (HPs) and many HPs have little or no formal training in pediatric sleep [1]. There are effective psychosocial and behavioral interventions [3, 4], but their descriptions in research often lack in detail to help HPs in their implementation [5]. This may result in underidentification and undertreatment of pediatric sleep problems [1], and the endorsement and delivery of sleep assessment and treatments that are not evidence-based [6]. Thus, HPs and parents would benefit from easy access to standardized and evidence-based screening and intervention programs.

Digital interventions can provide standardized, evidence-based and accessible care ‘where parents are’, that is, in their local communities, at home, and online. Parents actively search for information about infant sleep and health online [7, 8], and most parents and HPs are interested in internet-based infant sleep programs [8, 9]. Despite this, research on internet- and mobile-based interventions (IMIs) for pediatric insomnia in typically developed young children is still in its infancy. There is currently only one randomized trial of an internet intervention, that showed reductions in problematic sleep, sleep onset latency, and the number and duration of night wakings [10]; improvements that were maintained one year later [11]. In terms of mobile interventions, only one app was empirically supported [12], according to a recent review [13]. Thus, the few initial results seem promising. A few studies have also examined the dissemination of online sleep information and the usability of online tools or interventions [14,15,16,17]. Such studies are useful for identifying barriers and facilitators to implementation such as the parental needs for professional guidance and cultural adaptations (e.g., many practice co-sleeping) and the time restrictions and training needs of HPs [15].

Internet interventions have existed for 20 years. However, with a few exceptions [18], efforts to integrate these into routine practice have had mixed success. There are many factors that may promote or impede the implementation of IMIs. Provision of information, training, and infrastructure to those involved is key to success [19], but is insufficient by itself. Many HPs may still be skeptical about using IMIs, and experience excessive workloads and disruptions to their workflow [19, 20]. Compatibility (or lack thereof) with existing systems, ambiguous policies, and costs, are other known challenges that must be addressed at an organizational or policy level [19,20,21]. Practical guidelines may provide important insights and key points to consider (e.g., privacy, funding schemes, policy, and regulatory context) [22, 23]; however, the development of IMIs still requires careful consideration of the context in which they are to be used [24].

To integrate IMIs into practice, it is necessary to identify and select contextually appropriate implementation strategies. Implementation strategies can be defined as “methods or techniques used to enhance the adoption, implementation, and sustainability of a clinical program or practice” [25, p. 2]. However, selecting implementation strategies is challenging for several reasons; it requires careful consideration of contextual variations [24, 26]; despite there being around 170 implementation theories and models [27], most have a limited evidence-base due to being underutilized [28]; and, there is an abundance of strategies that can be combined in numerous ways [29,30,31]. The literature thus provides limited guidance on the selection of implementation strategies. Therefore, the objective of this study was to advance our practical understanding of barriers and facilitators that can influence the implementation of IMIs. More specifically, our aim was to identify implementation strategies that are important and feasible to integrate a guided IMI for infant sleep problems (“Sleep Well, Little Sweetheart”) into well-baby and community mental health clinics by using group concept mapping [32, 33].


Group concept mapping (GCM) is a two-phased, participatory sequential mixed methods approach to guide planning and program evaluation [34]. In the first qualitative phase (i.e., the first workshop), participants received a demonstration of Sleep Well, Little Sweetheart and explained that “the goal of the workshop is to arrive at a framework for the implementation of the program”. They were specifically instructed to “consider the strategies related to Sleep Well, Little Sweetheart”, although the focus prompt was formulated as “what conditions must be present for successful implementation at your workplace?”, to help with the sorting and rating tasks, and to broaden their mind during brainstorming. Finally, participants individually sorted and rated a set of pre-defined implementation strategies and engaged in a brainstorming session (see Supplementary Materials). In the second phase, researchers performed quantitative analyzes to represent the sorting and rating data, which were presented at the second workshop where participants discussed and interpreted the findings.


Sleep Well, Little Sweetheart is a guided IMI for pediatric insomnia in infants from six months to three years. It consists of eight program modules: (1) infant sleep assessment [35], (2) psychoeducation about infant sleep [35, 36], (3) individual bedtime routine [37], (4) infant crying [38], (5) parent emotion regulation [39], (6) individual sleep plan (i.e., extinction-based plans, bedtime fading or scheduled awakenings) [40], (7) relapse prevention, and (8) a sleep diary [41]. The program is administered and delivered by HPs, using the Youwell platform ( HPs task is to establish and maintain a high-quality working alliance with the parents, motivate them to use the program, and adapt the program contents to the individual family (e.g., individual bedtime routine and sleep plan) [42], either during routine face-to-face consultations or via technology (i.e., text messages or mobile phone).


We recruited 20 participants from two well-baby clinics (n = 13, 65%) and two community mental health clinics (n = 7, 35%), which is sufficient for GCM and above the typically recommended sample size of 15 [43]. All participants were women, with an average age of 47.1 years, and a university or college degree. Eleven (55%) of the participants worked as public health nurses (PHNs), four (20%) as psychologists, and four (20%) in other positions (e.g., family therapists). Three (15%) of the participants were clinic leaders. The majority had some clinical training and experience with infant sleep (n = 18, 90%), but few had any personal or professional experience with IMIs (n = 4, 20%).

Data collection

Prior to the first workshop, participants gave their consent and provided background information (e.g., age, employment, and experience with infant sleep). GCM was conducted in each clinic from March to May/June 2022. The first step of data collection consisted of individual sorting of 73 implementation strategies from the Expert Recommendations for Implementing Change (ERIC)–taxonomy [29], including suggested labels for each group of piles. Participants were instructed to group strategies in a meaningful way, based on their similarities. The participants then rated each strategy in terms of its relative importance and feasibility in their workplace, as two separate ratings, on a scale from 1 (“not at all important/feasible”) to 5 (“extremely important/feasible”). To ensure that the piles were not mixed, participants were instructed to use rubber bands and zipped bags (see Supplementary Materials for participant instructions).

The ERIC–taxonomy is a widely applicable, standardized, and manageable set of implementation strategies that minimizes participant burden and maximizes breadth. However, it may not cover context-specific strategies outside U.S. or North–American settings and ensure data saturation. Thus, participants brainstormed individually and created 56 additional strategies that were written on post-its and placed in separate zipped bags. These were reduced to 24 novel strategies by plenary discussions and a final review by the second author, which can be considered indicative of saturation. The list of statements and translations is provided in Table 1. At the second workshop, participants were presented with the results from the first workshop, they discussed and interpreted the findings (e.g., if the results were surprising or reflected their opinion), and sorted and rated strategies from the brainstorming. Analyses were then updated to include sorting and rating data from the second workshop. The participants’ interpretations of the findings were audio-recorded, summarized, and integrated in the Discussion below. The first and second authors moderated the workshops.

Table 1 Overview and translations of implementation strategies.a

Data analyses

Demographics, importance, and feasibility ratings were analyzed using descriptive statistics. Analyses were performed using the open-source software R [44] and R-CMap package [45]. Ward’s algorithm was used for multidimensional scaling and hierarchical cluster analysis to characterize how participants grouped strategies and how they were rated in terms of their importance and feasibility (i.e., a cluster rating map). The stress value of the multidimensional scaling was 0.336 and is an indicator of the relationship between the strategies, their similarities, and distances on the map, in which lower values reflect a better fit. Values ≤ 0.39 are acceptable and unlikely to have either no structure or a random two-dimensional configuration [43, 46]. Furthermore, we calculated the split-half reliability as a measure of the overall consistency of the card sort, using 20 random splits of participants, to 0.39.

There is no true number of clusters in a final map. The goal is to produce a set of clusters that are intuitive and meaningful. The within-cluster sum of squares, a measure of the variability of observations within each cluster, indicated an 11-cluster solution as a point of departure (see Fig. 1). A backward process with a stepwise reduction in clusters ended when further merging disrupted the meaning of the strategies in each separate cluster, as sorted by participants. It should be noted that the structure of hierarchical trees is determined by analysis, and not by the researchers [47]. In this study, the first and fifth authors examined clusters emerging from each step and agreed on the final number, after reviewing the content and meaning within each cluster. Each cluster was labeled based on the names proposed by the participants. After determining the number of clusters, each strategy’s importance and feasibility score were plotted on a scatterplot and divided into four quadrants using the mean of each dimension to identify actionable strategies for an implementation plan (i.e., ‘Go-Zone’ analyses). A laddergram was created to show cluster-level differences in importance and feasibility ratings and an analysis of variance and Tukey’s post-hoc test of multiple comparisons was used for mean comparisons between clusters.

Fig. 1
figure 1

Number of clusters– within cluster sums of squares


All participants sorted and rated all strategies, except one that only sorted and rated strategies in the ERIC–taxonomy. On average, participants created 8.25 piles (SD = 2.92; Range = 5–16). Two participants put more than one-third of the cards in one pile. Strategies 8, 10, 11, 12, 68, and 87 formed a separate cluster in the initial 11-cluster solution but were merged to form a single cluster labeled Preparation and Facilitation, as the original clusters were not judged as sufficiently distinct or intuitive. The final cluster map consists of 10 clusters with 4 to 13 strategies per cluster. Table 2 presents a summary of the clusters, their corresponding strategies, and mean importance and feasibility ratings at the cluster level. Table 3 summarizes the implementation strategies and their mean importance and feasibility ratings, organized by cluster and Go-Zone quadrant.

Table 2 Summary of clusters of implementation strategies and their importance and feasibility ratings
Table 3 Summary of implementation strategies organized by Go-Zone quadrants globally and per cluster

Figure 2 presents a point and cluster rating map that visually represents the relationship between the 97 strategies, accompanied by a number for cross-referencing to the strategies in Tables 2 and 3. In general, the closer two strategies are together, the more often they were sorted together (e.g., strategies 1 (access new funding) and 49 (place innovation on fee for service lists) in the Economy and Funding cluster; see Fig. 2). Strategies farther apart from each other were less often, if at all, sorted together (e.g., strategies 11 (change physical structure and equipment) and 16 (conduct educational outreach visits) in the Preparation and Facilitation and Training clusters, respectively). Similarly, clusters near one another are more closely connected than those farther away. Clusters in the middle of the map (i.e., Quality Assurance and Embedding and Coherence) can be considered to function as a bridge for interaction between other clusters. For example, establishing a coherent individual and collective understanding of a new practice, can make any preparations and training, both more meaningful and thus easier to embed into routine practice.

Fig. 2
figure 2

A point and cluster rating map for each cluster by importance (n = 20)

Figure 3 shows the global Go-Zone graph for each of the 97 strategies. The graph was divided into four quadrants by the average importance (M = 3.61, SD = 0.70) and feasibility (M = 3.21, SD = 0.59) ratings. There was a significant correlation between ratings (r =.62, p =.004), indicating that most strategies fell within quadrants I (n = 44, 45.4%) or IV (n = 34, 35.1%). The upper right quadrant (I), referred to as the Go-Zone, shows strategies that were rated above average on both importance and feasibility. These strategies were mostly from clusters 2, 5, 6, 7, 8, and 10 (i.e., Preparation and Facilitation, Embedding and Coherence, Leadership and Organization, Training, Clinician Support and Implementation Counseling, and User Involvement and Participation; see also Tables 2 and 3), and should be prioritized and addressed first in any ensuing implementation plan. Conversely, strategies rated lowest on both importance and feasibility, fell within the lower left quadrant (IV; i.e., the No-Go zone). These were predominantly from clusters 1 to 4 (i.e., Economy and Funding, Preparation and Facilitation, Implementation, and Interactive and Interdisciplinary Collaboration; see also Tables 2 and 3). Only a few strategies were rated relatively important (upper left quadrant III; n = 11, 11.3%) or feasible (lower right quadrant II; n = 8, 8.2%). Table 3 also shows the Go-Zone quadrants for each cluster independently. Although most of the strategies (n = 66, 68.0%) remained in the same quadrant as in the global Go-Zone analysis, 31 (32.0%) strategies were classified into another quadrant in the per cluster analysis. Changes in quadrants among strategies between the global and per cluster analyses occurred across all clusters, but mainly in Economy and Funding (n = 7; 58.3%) and Interactive and Interdisciplinary Collaboration (n = 5, 50.0%).

Fig. 3
figure 3

Global go-zone graph for all 97 strategies (n = 20)

Figure 4 compares the average importance and feasibility ratings of strategies at the cluster level. It shows that all clusters were judged relatively important, but also consistently more difficult to accomplish. Training was considered the most important and feasible, while Economy and funding had the greatest mean difference between importance and feasibility ratings and was perceived as least feasible. An analysis of variance revealed statistically significant differences in importance (F (9, 1 906) = 8.91, p <.001) and feasibility (F (9, 1 906) = 21.69, p <.001) between two or more clusters. Table 4 includes significant differences from Tukey’s test for multiple comparisons of all possible pairs. Most notably, differences show that Economy and Funding and Interactive and Interdisciplinary Collaboration were rated significantly less important than most other clusters (all ps < 0.05). In terms of feasibility, Training was perceived as more applicable than all clusters, except Embedding and Coherence, while Economy and Funding was considered harder to accomplish than all other clusters (all ps < 0.05).

Fig. 4
figure 4

A laddergram comparing the average cluster ratings (n = 20)

Table 4 Multiple comparisons of mean differences in importance and feasibility with 95% family-wise confidence level (CI)


We used GCM to identify strategies for the implementation of a guided IMI for infant sleep problems in well-baby and community mental health clinics. We identified 10 clusters of strategies, of which Training, Embedding and Coherence, User Involvement and Participation, and Clinician Support and Implementation Counseling were rated as most important and feasible. In contrast, Economy and Funding and Interactive and Interdisciplinary Collaboration were rated as least important and feasible. There was a positive linear correlation between the importance and feasibility ratings. Therefore, more strategies from the most important and feasible clusters fell into the Go-Zone quadrant, while more strategies from the least important and feasible clusters fell into the No-Go quadrant. Reflecting on data saturation, the study team found that the discussions added no major changes to the interpretation of results across groups. This likely reflected the narrow study aim, relevance/adequacy of the sample, and the applied use of a specific taxonomy and methodology providing a clear and structured dialogue between researchers and participants. In what follows, we discuss the most prominent results and interpretation of findings from discussions with participants. The references to strategies are numbered in parentheses for cross-referencing to the cluster map (Fig. 1) and Tables 2 and 3.

Overall, the results resonated with the participants. The stress value was acceptable, indicating that there is a structure to the data. Further validation of these findings can be found in Waltz and colleagues who conducted a GCM-study using the ERIC-taxonomy with implementation experts [48]. Although our study included clinical staff and additional strategies, participants conceptualized strategies in similar ways. Apart from the slightly different labels, Waltz and colleagues also identified clusters related to financial strategies, training, engaging users, collaboration with stakeholders, and supporting clinicians [48]. However, how participants sorted strategies within clusters, varied greatly. For example, we found that participants conceptualized ongoing training (19) and consultation (55) as Clinical Support and Implementation Counseling, rather than training and educating stakeholders. This may be due to the different groups of participants in the studies but may also show how the conceptualization of strategies can vary across cultures and contexts.

In discussing the findings, the participants recognized that IMIs do have setup, operation, and maintenance costs [21], but that they, as clinical staff, rarely have opportunities to influence the funding of their clinic. Therefore, Economy and Funding should not be interpreted as unimportant but must be taken care of at higher system levels (e.g., municipal or government funding), as successful examples of IMIs in routine practice have taught us [22]. Thus, it makes sense that Leadership and Organization were close to Economy and Funding on the cluster map. Participants were more concerned about learning to administer the program and any counseling methods [42], but did not consider that Training in IMIs needed to be extensive. IMIs do not require the same level of competency or skills training as face-to-face methods, as parents carry out much of the intervention themselves. However, the participants were clear about the need for active involvement in Training by participating in group work, testing, and administration of the program.

Interactive and Interdisciplinary Collaboration was rated less important and feasible compared to many clusters. During the discussions, it became clear that some strategies in this cluster were considered not currently relevant (e.g., 62), useful but not necessary (e.g., 72), or the responsibilities of other stakeholders (e.g., 24 and 73). It was more important to create structures for learning collaboratives (20) where clinicians could meet regularly to learn and share experiences. Further building of coalitions (6) and network weaving (52) could be considered important, but mainly to embed the IMI into routine care. Embedding new practices is made possible by an understanding of their meaning, uses, and utility. It requires a coherent set of beliefs and behaviors that define and organize the work, and that are seen as meaningful and different from other practices (78) [49]. In this sense, participants were surprised that informing local opinion leaders (38) was not in the Go-Zone, as they considered it essential, and should include key managers, clinicians, and administrative staff.

Regardless of clustering, certain strategies fell in the No-Go zone because they are rarely used in public healthcare in Norway (e.g., 2 (alter incentive/allowance structures) and 59 (revise professional roles)) or may be experienced as unpleasant and stressful (e.g., 26 and 27 (tools for quality monitoring), and 81 (certification schemes)). Participants agreed that purveyors must set certain quality requirements for the delivery of IMIs, but this can be achieved in other ways than through licensing standards (22) or certification schemes (81). Quality Assurance and evaluative strategies in general require a high level of psychological safety [50]. Therefore, many were reluctant to such strategies and are not used to being monitored or measured in performance. Quality Assurance and evaluative strategies were also compared to established initiatives such as specialist breastfeeding centers, which use formal quality requirements [51]. According to the participants, it must be acknowledged that evaluative strategies only provide a snapshot of the current situation and take time away from families in an already busy work schedule.

Finally, participants acknowledged the distances and spatial relationships between strategies in the cluster map (Fig. 2). Strategies within clusters spread in different spatial directions and some were farther from the center of the cluster than others, approaching neighboring clusters such as conducting local consensus discussion (17) in Embedding and Coherence and needs assessment (18) in Preparation and Facilitation. Participants explained that some strategies, such as those related to Training and Economy and Funding, were easier to sort than others, and not all the strategies were equally clear or understandable. Several mentioned that strategies could have been sorted in multiple ways or placed in several piles. They perceived that there were subtle differences between several strategies and even considered them interchangeable (e.g., reexamining the implementation (56) and audit and feedback (5)). Furthermore, the participants pointed out that the role they had in the clinic (leader vs. clinician) and professional background (PHN vs. psychologist) could also have impacted how they sorted and rated the strategies.

Strengths and limitations

GCM is an efficient and engaging method to obtain insights on a topic. It is less resource intensive than interviews, but there are also limited possibilities to probe and explore new concepts and may not provide sufficient in-depth data [52]. However, one of its strengths is the mixed methods approach. This became evident in the discussions where participants attributed less value to Economy and Funding but recognized that the ratings reflected their opportunities to influence the funding of a clinic, more than its actual importance.

For practical reasons, it was not possible to discuss the updated maps that included the strategies from the brainstorming. However, the discussions of maps based on the ERIC-taxonomy were audio-recorded, summarized, and reviewed against the final maps. Most clusters and sorting of strategies remained the same, but discussions may have given participants more time to reflect and fine-tune their sorting to truly represent their views.

It is important to note that although each cluster is unique, there are overlapping ideas between them. Participants expressed that certain strategies were more difficult to sort and that they could have sorted the strategies in multiple ways or placed certain strategies in several piles. However, in GCM, a strategy or statement can only be placed in one pile. Thus, overlaps are inevitable and common; also, because participants are instructed to sort strategies in a way that makes sense to them, without being guided by any theory or logic. Although this may be a limitation, it can also be considered a strength as it highlights challenging, ambivalent, or even contradictory concepts or ideas that may have important implications for, in our case, the implementation of the IMI.

Although the study included more than the recommended number of participants for GCM [43], subgroups of interest became unbalanced and small due to the modest sample size. For such reasons, we did not compare clustering or ratings between primary care services (i.e., well-baby vs. community clinics), professions (i.e., PHNs vs. psychologists), or roles (e.g., leaders and clinicians). This could have provided a more nuanced understanding of the implementation strategies. It could also be argued that the heterogeneity among participants more closely resembles the real-world setting in which the IMI will be implemented and thus has captured important variations in their responses. Taken together, the overlapping strategies and heterogeneity among our group of participants may reflect the modest overall consistency (i.e., split-half reliability) of the sorting task. Yet, the stress value indicated that the relationship between the data, similarity matrix, and distances on the map, was acceptable and that there is an underlying structure in the data.


GCM made it possible to efficiently examine the perspectives of the well-baby clinics and community clinics on complex issues, and to acquire specific knowledge to allow for the planning and prioritization of implementation strategies. Training, Embedding and Coherence, User Involvement and Participation, and Clinician Support and Implementation Counseling were identified as the most important and applicable areas for implementation. In contrast, Economy and Funding and Interactive and Interdisciplinary Collaboration were rated as least important and feasible, although they should not be ignored but taken care of for sustainable implementation. Cluster-level Go-Zone analyzes and the discussions of the findings with participants may help identify which strategies within clusters to target. These results suggest areas of priority for the implementation of IMIs related to infant sleep problems such as Sleep Well, Little Sweetheart, and potentially other practices in primary care for parents with young children.

Data availability

Data are stored at the Regional Center for Child and Adolescent Mental Health, Eastern and Southern Norway, but cannot be shared publicly as consent for publication of the dataset was not obtained. Requests to access the datasets should be directed to Filip Drozd:



Expert Recommendations for Implementing Change-taxonomy


Health professionals


Internet- and mobile-based interventions


Group concept mapping


Public health nurses


  1. Honaker SM, Meltzer LJ. Sleep in pediatric primary care: a review of the literature. Sleep Med Rev. 2016;25:31–9.

    Article  PubMed  Google Scholar 

  2. Kang EK, Kim SS. Behavioral insomnia in infants and young children. Clin Exp Pediatr. 2021;64:111–6.

    Article  CAS  PubMed  Google Scholar 

  3. Magee L, Goldsmith LP, Chaudhry UAR, Donin AS, Wahlich C, Stovold E, Nightingale CM, Rudnicka AR, Owen CG. (2022) Nonpharmacological interventions to lengthen sleep duration in healthy children: a systematic review and meta-analysis. JAMA Pediatr e1–e14.

  4. Drozd F, Leksbø TS, Størksen HT, Weyde CE, Slinning K. An overview of reviews for preventing and treating sleep problems in infants. Acta Paediatr. 2022;111:2071–6.

    Article  PubMed  Google Scholar 

  5. Meltzer LJ, Wainer A, Engstrom E, Pepa L, Mindell JA. Seeing the whole elephant: a scoping review of behavioral treatments for pediatric insomnia. Sleep Med Rev. 2021;56:101410.

    Article  PubMed  Google Scholar 

  6. Zhou ES, Mazzenga M, Gordillo ML, Meltzer LJ, Long KA. Sleep education and training among practicing clinical psychologists in the United States and Canada. Behav Sleep Med. 2021;19:744–53.

    Article  PubMed  Google Scholar 

  7. Jaks R, Baumann I, Juvalta S, Dratva J. Parental digital health information seeking behavior in Switzerland: a cross-sectional study. BMC Public Health. 2019;19:225.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Thorndike FP. Commentary: interest in internet interventions - an infant sleep program as illustration. J Pediatr Psychol. 2009;34:470–3.

    Article  PubMed  Google Scholar 

  9. Størksen HT, Haga SM, Slinning K, Drozd F. Health personnel’s perceived usefulness of internet-based interventions for parents of children younger than 5 years: cross-sectional web-based survey study. JMIR Ment Health. 2020;7:e15149.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Mindell JA, du Mond CE, Sadeh A, Telofski LS, Kulkarni N, Gunn E. Efficacy of an internet-based intervention for infant and toddler sleep disturbances. Sleep. 2011;34:451–8.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Mindell JA, du Mond CE, Sadeh A, Telofski LS, Kulkarni N, Gunn E. Long-term efficacy of an internet-based intervention for infant and toddler sleep disturbances: one year follow-up. J Clin Sleep Med. 2011;7:507–11.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Leichman ES, Gould RA, Williamson AA, Walters RM, Mindell JA. Effectiveness of an mHealth intervention for infant sleep disturbances. Behav Ther. 2020;51:548–58.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Simon SL, Kaar JL, Talker I, Reich J. Evidence-based behavioral strategies in smartphone apps for children’s sleep: content analysis. JMIR Pediatr Parent. 2022;5:e32129.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Yoshizaki A, Mohri I, Yamamoto T, et al. An interactive smartphone app, Nenne Navi, to improve children’s sleep: a pilot study. JMIR Pediatr Parent. 2020;3:e22102.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Speth TA, Coulombe JA, Markovich AN, et al. Barriers, facilitators, and usability of an internet intervention for children aged 1 to 10 years with insomnia. Transl Issues Psychol Sci. 2015;1:16–31.

    Article  Google Scholar 

  16. Howlett MD, Jemcov A, Adams A, Corkum Pv. ABCs of SLEEPING tool: improving access to care for pediatric insomnia. Clin Pract Pediatr Psychol. 2020;8:1–12.

    Google Scholar 

  17. Mindell JA, Leichman ES, Walters RM, Bhullar B. Development and dissemination of a consumer health information website on infant and toddler sleep. Transl Behav Med. 2021;11:1699–707.

    Article  PubMed  Google Scholar 

  18. Titov N, Dear BF, Nielssen OB, et al. ICBT in routine care: a descriptive analysis of successful clinics in five countries. Internet Interv. 2018;13:108–15.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Ganapathy A, Clough BA, Casey LM. Organizational and policy barriers to the use of digital mental health by mental health professionals. Telemedicine and e-Health. 2021;27:1332–43.

    Article  PubMed  Google Scholar 

  20. Granja C, Janssen W, Johansen MA. Factors determining the success and failure of ehealth interventions: systematic review of the literature. J Med Internet Res. 2018;20:e10235.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Ross J, Stevenson F, Lau R, Murray E. Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update). Implement Sci. 2016;11:146.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Newby J, Mason E, Kladnistki N, Murphy M, Millard M, Haskelberg H, Allen A, Mahoney A. Integrating internet CBT into clinical practice: a practical guide for clinicians. Clin Psychol. 2021;25:164–78.

    Article  Google Scholar 

  23. Titov N, Hadjistavropoulos HD, Nielssen OB, Mohr DC, Andersson G, Dear BF. From research to practice: ten lessons in delivering digital mental health services. J Clin Med. 2019;8:1239.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Lyon AR, Koerner K. (2016) User-centered design for psychosocial intervention development and implementation. Clinical Psychology: Science and Practice 23:180–200.

  25. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013;8:139.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Nilsen P, Bernhardsson S. Context matters in implementation science: a scoping review of determinant frameworks that describe contextual determinants for implementation outcomes. BMC Health Serv Res. 2019;19:189.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Amsterdam University Medical Centers Implementation theories., models & frameworks. Accessed 3 Nov 2022.

  28. Davies P, Walker AE, Grimshaw JM. A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Implement Sci. 2010;5:14.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, Proctor EK, Kirchner JE. A refined compilation of implementation strategies: results from the Expert recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Michie S, Richardson M, Johnston M, Abraham C, Francis JJ, Hardeman W, Eccles MP, Cane J, Wood CE. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med. 2013;46:81–95.

    Article  PubMed  Google Scholar 

  31. Flottorp SA, Oxman AD, Krause J, Musila NR, Wensing M, Godycki-Cwirko M, Baker R, Eccles MP. A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci. 2013;8:35.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK, Mandell DS. Methods to improve the selection and tailoring of implementation strategies. J Behav Health Serv Res. 2017;44:177–94.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Kane M, Trochim WM. Concept mapping for planning and evaluation. Thousand Oaks, CA: Sage; 2007.

    Book  Google Scholar 

  34. Trochim WMK, Kane M. Concept mapping: an introduction to structured conceptualization in health care. Int J Qual Health Care. 2005;17:187–91.

    Article  PubMed  Google Scholar 

  35. Mindell JA, Owens JA. A clinical guide to pediatric sleep: diagnosis and management of sleep problems. 3rd ed. Philadelphia, PA: Wolters Kluwer; 2015.

    Google Scholar 

  36. Meltzer LJ, Crabtree VM. Pediatric sleep problems: a clinician’s guide to behavioral interventions. Washington, DC: American Psychological Association; 2015.

    Book  Google Scholar 

  37. Mindell JA, Leichman ES, Lee CI, Williamson AA, Walters RM. Implementation of a nightly bedtime routine: how quickly do things improve? Infant Behav Dev. 2017;49:220–7.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Chóliz M, Fernández-Abascal EG, Martínez-Sánchez F. Infant crying: pattern of weeping, recognition of emotion and affective reactions in observers. Span J Psychol. 2012;15:978–88.

    Article  PubMed  Google Scholar 

  39. Fjorback LO, Arendt M, Ornbøl E, Fink P, Walach H. Mindfulness-based stress reduction and mindfulness-based cognitive therapy: a systematic review of randomized controlled trials. Acta Psychiatr Scand. 2011;124:102–19.

    Article  CAS  PubMed  Google Scholar 

  40. Morgenthaler TI, Owens JA, Alessi C, et al. Practice parameters for behavioral treatment of bedtime problems and night wakings in infants and young children. Sleep. 2006;29:1277–81.

    PubMed  Google Scholar 

  41. Honaker SM, Meltzer LJ. Bedtime problems and night wakings in young children: an update of the evidence. Paediatr Respir Rev. 2014;15:333–9.

    PubMed  Google Scholar 

  42. Størksen HT, Leksbø TS, Drozd F, Sandtrø HP, Slinning K. Søvnvansker hos barn: Ny Metode Kan hjelpe [Sleep problems in children: a new method can help]. Sykepl Fag. 2022;110:e–88549.

    Article  Google Scholar 

  43. Rosas SR, Kane M. Quality and rigor of the concept mapping methodology: a pooled study analysis. Eval Program Plann. 2012;35:236–45.

    Article  PubMed  Google Scholar 

  44. R Core Team. (2022) R: A language and environment for statistical computing.

  45. Bar H, Mentch L. R-CMap—An open-source software for concept mapping. Eval Program Plann. 2017;60:284–92.

    Article  PubMed  Google Scholar 

  46. Sturrock K, Rocha J. A multidimensional scaling stress evaluation table. Field Methods. 2000;12:49–60.

    Article  Google Scholar 

  47. Jackson KM, Trochim WMK. Concept mapping as an alternative approach for the analysis of open-ended survey responses. Organ Res Methods. 2002;5:307–36.

    Article  Google Scholar 

  48. Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, Chinman MJ, Smith JL, Proctor EK, Kirchner JE. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert recommendations for Implementing Change (ERIC) study. Implement Sci. 2015;10:109.

    Article  PubMed  PubMed Central  Google Scholar 

  49. May CR, Finch TL. Implementing, embedding, and integrating practices: an outline of normalization process theory. Sociology. 2009;43:535–54.

    Article  Google Scholar 

  50. O’Donovan R, McAuliffe E. A systematic review of factors that enable psychological safety in healthcare teams. Int J Qual Health Care. 2020;32:240–50.

    Article  PubMed  Google Scholar 

  51. Norwegian Institute of Public Health. (2021) Ammekyndig helsestasjon [Specialized breastfeeding centre]. Accessed 10 Nov 2022.

  52. Humphrey L, Willgoss T, Trigg A, Meysner S, Kane M, Dickinson S, Kitchen H. A comparison of three methods to generate a conceptual understanding of a disease based on the patients’ perspective. J Patient Rep Outcomes. 2017;1:9.

    Article  PubMed  PubMed Central  Google Scholar 

Download references


We thank the participants who contributed to the study during their busy working schedules and clinic managers for making their staff and offices available to us. We also thank Thomas Engell, RBUP Øst og Sør, who shared some of his translations of the ERIC-taxonomy.


Supported by Stiftelsen Dam (Dam Foundation; reference number: 353518 [to FD]) through Rådet for psykisk helse (The Norwegian Council for Mental Health). Study sponsors did not have any role in the study design; collection, analysis, and interpretation of data; the writing of the manuscript; and the decision to submit the manuscript for publication.

Author information

Authors and Affiliations



FD, SMH, and HTS conceptualized and designed the study. FD acquired funding for the study, had the overall responsibility for the study and investigation, and contributed to data curation, formal analysis, oversight, and drafting of the initial manuscript. HPS contributed to carrying out the study, investigation, data curation, formal analysis, and oversight. TSL had the responsibility for recruitment and contributed to data curation. HJ contributed to the formal analysis. All authors were involved in reviewing, editing, providing feedback, and approving the final manuscript.

Corresponding author

Correspondence to Filip Drozd.

Ethics declarations

Ethics approval and consent to participate

The study was performed in accordance with the Declaration of Helsinki and national norms and standards for conducting research in Norway with approval from the Sikt– kunnskapssektorens tjenesteleverandør (Sikt– Norwegian Agency for Shared Services in Education and Research,; project number: 684718). All participants gave informed, written consent to participate in the study.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Drozd, F., Pettersen Sandtrø, H., Leksbø, T.S. et al. Identifying and developing strategies for implementation of a guided internet- and mobile-based infant sleep intervention in well-baby and community mental health clinics using group concept mapping. BMC Health Serv Res 24, 175 (2024).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: