Skip to main content
Download PDF

Implementation of a program to support direct support professionals to promote a healthy lifestyle for people with moderate to profound intellectual disabilities

BMC Health Services Research volume 22, Article number: 15 (2022) Cite this article

Abstract

Background

There is a lack of theory-based interventions for direct support professionals (DSPs) to support a healthy lifestyle for people with moderate to profound intellectual disabilities (ID) despite their major role in this. This study aims to evaluate the preparation, implementation, and preliminary outcomes of a theory-based training and education program for DSPs to learn how to support these individuals.

Methods

The program consisting of e-learning, three in-person sessions, and three assignments was implemented. The implementation process was evaluated with a mixed method design with the following components: preparation phase, implementation phase, and the outcomes. These components were measured with project notes, questionnaires, interviews, reflections, assignments, food diaries, Actigraph/Actiwatch, and an inventory of daily activities.

Results

Regarding the preparation phase, enough potential participants met the inclusion criteria and the time to recruit the participants was 9 months. The program was implemented in four (residential) facilities and involved individuals with moderate to profound ID (n = 24) and DSPs (n = 32). The e-learning was completed by 81% of the DSPs, 72–88% attended the in-person sessions, and 34–47% completed the assignments. Overall, the fidelity of the program was good. DSPs would recommend the program, although they were either negative or positive about the time investment. Mutual agreement on expectations were important for the acceptability and suitability of the program. For the outcomes, the goals of the program were achieved, and the attitudes of DSPs towards a healthy lifestyle were improved after 3 months of the program (nutrition: p = < 0.01; physical activity: p = 0.04). A statistically significant improvement was found for food intake of people with ID (p = 0.047); for physical activity, no statistically significant differences were determined.

Conclusions

The theory-based program consisting of a training and education section for DSPs to support a healthy lifestyle for people with moderate to profound ID was feasible to implement and, despite some barriers regarding time capacity and mutual expectations, it delivered positive changes in both persons with moderate to profound ID and DSPs. Thus, the program is a promising intervention to support DSPs.

Peer Review reports

Background

People with moderate to profound intellectual disabilities (ID) often do not have a healthy lifestyle [1,2,3,4] with regard to physical activity and healthy nutrition [5, 6]. These individuals often have low levels of physical activities and an unhealthy diet [2, 6]. Consequently, people with moderate to profound ID have a higher prevalence for health problems like constipation and being under- or overweight [7]. Severe motor disabilities as an example of prevalent health problems are also barriers for physical activity participation of people with ID. Other barriers are related to individual motivation and preferences; support on financial level, transportation and staffing levels. Facilitators for physical activity are for example social interaction and engagement, rewarding’s, and having fun [8]. People with moderate to profound ID require support from their social environment in performing their daily activities and thus to live healthily. In the Netherlands, this support is often provided by direct support professionals (DSPs) in residential facilities. Therefore, DSPs play a major role in the support of a healthy lifestyle for people with moderate to profound ID [9, 10]. These DSPs usually have an educational background in social work or nursing for which a healthy lifestyle is not an element of the program. However, training and education can help DSPs to support a healthy lifestyle [11].

To support a healthy lifestyle, there are theory-based motor activity programs for people with profound intellectual and multiple disabilities [12, 13]. However, previous research shows a lack of theory-based interventions and education for DSPs to support a healthy lifestyle [14]. Theory-based interventions are shown to have more potential for effective outcomes than interventions without a theoretical basis [15]. In a previous study, a theory-based training and education program was developed in co-creation with daily practice [16]. The content of the training and education program is based on the Theoretical Domains Framework (TDF), related to the COM-B System [17, 18], and Behaviour Change Techniques (BCTs) [19]. The TDF gives insights about the conditions to support a healthy lifestyle. The COM-B system is complementary to the TDF whereas this system explains the nature of behaviour to change it. In addition, one of the skills DSPs need is motivating people with ID towards healthy behaviour, therefore BCTs are added to the program. The mode of delivery of the program was based on Kolb’s theory of Learning Styles [20]. The developed program consists of a training and education section with online and in-person components to support DSPs in promoting healthy living for people with moderate to profound ID. In the online component, DSPs gather knowledge and increase awareness of physical activity and healthy nutrition for this population. For the in-person component, DSPs participate in three sessions to discuss health promotion with the following themes: Social/Professional Role and Identity, Skills (Behaviour Change Techniques), and Social Influences, Environmental Context and Resources.

A next step is to examine the potential of this intervention by implementing the program and evaluating this process [21]. However, the implementation of newly developed programs has been shown to be a major challenge in organizations in general [22] and for ID care providers [11, 12, 23]. Reporting of implementation processes and identifying key facilitators and barriers is important for compiling a knowledge base for successful implementation [22]. ID care providers and researchers can learn from previously implemented interventions and apply that knowledge to advance implementation processes [12, 23]. In order to evaluate its potential for widespread implementation, the aim of this study is to evaluate the preparation, implementation, and preliminary outcomes of a theory-based training and education program for DSPs to learn how to support people with moderate to profound ID in a healthy lifestyle.

Methods

Design

A theory-based program consisting of a training and education section to facilitate DSPs in promoting a healthy lifestyle for people with moderate to profound ID was implemented in four (residential) facilities of ID care providers in the Netherlands. A mixed method design was used to evaluate the implementation phase. This implementation process was operationalised and evaluated with the following components: recruitment, reach, context, dose delivered, dose received, fidelity [24], recruitment capacity, acceptability/suitability of the program, factors during implementation, data collection process, and changes in both DSPs and persons with ID after the program [25]. Figure 1 illustrates the design of the study.

Fig. 1
figure 1

Design of the study

Full size image

Sample and context

The participants of four residential facilities and/or day activity centres from four different ID care providers were recruited within a consortium of eight ID care providers across the Netherlands. The inclusion criteria for the (residential) facilities were as follows:

  • A (residential) facility where people with moderate to profound ID (≥18 years) receive support.

    In this study people with moderate to profound ID are defined as individuals who need support in several domains, for example in language, motor skills, sensory, and in activities for daily living [26, 27]. These people experience limitations in both intellectual and adaptive functioning, in the following domains: conceptual, social, and practical adaptive skills. In addition, people with more severe intellectual disabilities experience more often mobility problems [28]. Based on the support needs people with ID are categorized as having moderate, severe, or profound ID.

  • Readiness for implementation of the (residential) facility by checking the fit of the goal of the program in consultation with the facility.

  • Availability of one trainer to support the program.

  • Time capacity of the team for participating in the research study by receiving approval from the manager for the indicated time investment. .

The contact persons of the ID care providers assisted in the recruitment of the teams working at the selected (residential) facilities by informing in person and distributing information flyers. If the team was interested in participating, the researchers contacted the team to discuss participation. If the recruited team had the capacity to participate, the DSPs, people with ID, and legal representatives were informed and asked for consent for the study.

The Discovering Health-promoting Assets in Settings for people with Intellectual Disabilities (DIHASID) tool was used to gain insight in the implementation context [29]. The DIHASID tool provides actionable knowledge about the social, physical, financial, and organizational assets for physical activity and healthy nutrition at a residential or day-activity accommodation [30]. From the enquired physical assets, seven out of 13 were available at all (residential) facilities, and an additional three were available at three (residential) facilities. For example, there was enough space at all of the (residential) facilities for physical activities. Some aids and equipment, such as activity-stimulating games, were not available for all settings. At all (residential) facilities, social assets were available, for example, health professionals to help with exercise activities. However, to support healthy living, friends, clients, volunteers, and a massage therapist were accessible at only one or two (residential) facilities. At the policy level, the focus on healthy living in organisations was moderate to satisfactory; financial assets were also moderate to satisfactory [31].

Theory-based program

A theory-based program for DSPs to learn how to support physical activity and healthy nutrition for people with moderate to profound ID was implemented [16]. The program consisted of (see also Fig. 1):

  1. 1.

    E-learning to increase knowledge and awareness of physical activity and healthy nutrition for people with moderate to profound ID.

  2. 2.

    Three in-person group sessions of 2 h with the following themes:

    • Social/Professional Role and Identity.

    • Skills (Behaviour Change Techniques, BCTs).

    • Social Influences and Environmental Context and Resources.

The e-learning could be performed individually or with colleagues; the in-person sessions were performed within the team of DSPs at the (residential) facility where they work. Between and after the last in-person group session, practical assignments (three in total) were carried out within the team of DSPs.

The program was in operation from April 2019 to July 2019. The training was performed by two trainers: one trainer from the Hanze University of Applied Sciences (an experienced teacher from the social work program) and one trainer of each ID care provider (e.g. a behaviour scientist). Prior to the beginning of the program, the trainers received an instruction manual including PowerPoint presentations in order to prepare for the in-person sessions. A common meeting with the trainers was held to coordinate the sessions and to clarify uncertainties. After each session, there was brief contact with the trainers and with the first author about the process of the program. The e-learning was implemented in collaboration with the technical staff of the involved care providers. Before and during the program, there was consultation with the managers regarding the implementation (e.g. manager Knowledge and Innovation as well as manager residential facilities and day activity centers) and contact persons of the involved ID care providers in order to improve the implementation phase.

Data collection

To measure the implementation components, the definitions and underlying questions of Linnan and Steckler [24] and the objectives of Orsmond and Cohn [25] were used. The methods for data collection can be found in Table 1; the design of the study is depicted in Fig. 1. In the preparation phase, for the evaluation of recruitment and reach, project notes were captured. For the changes in the context, questionnaires and interviews were used.

Table 1 Components of implementation, indicators, data collection and analysis
Full size table

In the implementation phase, the dose delivered, dose received, and fidelity were captured with project notes, questionnaires, reflections, and assignments. The acceptability/suitability of the program was measured with reflections, questionnaires, and interviews. The last parts of the implementation phase, the factors during implementation, and the evaluation of the data collection process were measured with project notes. During the implementation phase of the program, the first author was present at one in-person session of the program with each of the four participating teams in order to observe whether the sessions were carried out as intended.

For the preliminary outcomes, the changes after the program for DSPs were measured in terms of goal achievement, actual application in practice, and attitude of DSPs towards nutrition and physical activity. These data were aggregated from questionnaires, interviews, and assignments. Goal achievement was measured with a questionnaire for DSPs 1 week after the last program session. The actual application of healthy lifestyle support in daily practice was measured during the program with practical assignments for DSPs and 3 months after the program with interviews with them. Managers were also surveyed about the actual application of the program in daily practice. The attitude was defined as the thoughts and feelings of DSPs regarding a healthy lifestyle [11]. The attitude of DSPs towards physical activity and healthy nutrition was measured before the program (T0), 1 week after (T1), and 3 months after (T2) the last program session [11]. The attitude was measured on a 5-point Likert scale for which 1 indicates a negative and 5 a positive attitude. The psychometric properties of the questionnaires were favourable [11] (Overwijk A, Krijnen WP, Hilgenkamp TIM, Van der Schans CP, Van der Putten AAJ, Waninge A: A questionnaire to measure direct support professionals’ attitude towards healthy nutrition of people with intellectual disabilities, submitted).

To evaluate the program, individual telephone interviews were held with half of the participating DSPs (n = 17) as a sufficient representation on the following topics: recommendation of the program, connection to support needs, actual implementation in daily practice, and other interventions during the program. In addition, changes in the level of physical activity and the amount of food intake of the participating people with ID were measured. Physical activity was measured with the Actigraph wGT3X BT for walking respondents [36, 37] and with the Actiwatch for non-walking respondents [38]. The data for physical activity of the walking people with moderate to profound ID was collected with a frequency of 30 Hz [37], over 1 min epoch periods [39]. Additionally, DSPs recorded the planned movement activities during the measurements of physical activity. Food intake was measured with food diaries covering 3 days [40]. Food intake and physical activity were measured before (T0) and 3 months after (T2) the last program session.

Data analyses

In the last column of Table 1, details regarding the analyses are described. In the preparation phase, the project notes for the evaluation of recruitment and reach were analyzed with documentary analyses [32], and the DSPs per team were calculated. For the changes in the context, questionnaires and interviews were checked in other interventions (yes/no) and influential changes in the environment were examined.

In the implementation phase, the dose that was delivered was calculated based on the completion of the e-learning, in-person sessions, and the assignments. For the dose received, descriptive statistics were used, and the quality of assignments was checked, and they were scored for correctness [23]. For the fidelity of the program, the descriptive statistics for the reflections of the trainers and the researcher (AO) were analyzed. The acceptability and suitability of the program was analyzed with a document analysis in order to determine the reflection of the trainers regarding the open questions about the learning questions of the teams and additional comments. Descriptive statistics were used to analyze the questionnaires of DSPs and managers. In addition, a conventional content analysis [33] using ATLAS.ti was employed to analyze the suitability and acceptability of the program for DSPs. The factors during implementation and the evaluation of the data collection process was analyzed with a document analysis of the project notes made during the implementation.

For the preliminary outcomes, the changes after the program were analyzed with descriptive statistics for goal achievement. The actual application in practice was analyzed with a conventional content analysis [33], and the application was checked in the assignments. The changes in attitudes of DSPs towards a healthy lifestyle were analyzed with linear mixed models [34] in the statistical programming language R [35].

To analyze the changes after the program for people with ID, data for the food intake of people with ID were compared with the Dutch nutrition guidelines via a food meter. This is a Dutch tool to support appropriate composition of food intake according to the recommended nutrition guidelines. Of the food intake, 85% should meet the nutrition guidelines [41]. For physical activity, the cutoff points of Freedson et al. [42] were used for the Actigraph data according to Chow et al. [39]. Non-wear time was defined as 60 min, and a minimum wear time per day was set for 600 min with a minimum of 4 days of valid wear time. Only the non-wear time was excluded from analysis. The low frequency filter (LFE) was checked for low sensitive movements. The cutoff points of Van Alphen et al. [38] were used for the respondents wearing the Actiwatch; 0–15 counts inactive, > 15 counts active. The data regarding both physical activity and nutrition before (T0) and 3 months after (T2) the program were compared using the Wilcoxon signed rank test. For the Actiwatch data, the percentages of inactive and active time were calculated.

To obtain insight into the differences between the participating (residential) facilities and the outcome variables for DSPs (goal achievement, changes after program, relevance program, satisfaction program, and attitude) and people with ID (food intake, and physical activity), a one-way ANOVA was performed.

Two trained master students of Inclusive and Special Needs Education supported analysing the interview data with a conventional content analysis [33] and the food diaries, and one bachelor student of human movement sciences supported with the entry of the food diaries.

Results

The results are described in the order of the implementation process (see Table 1). The preparation phase will be reported first, and then the implementation phase will be discussed and, lastly, the outcomes with corresponding variables will be examined.

Preparation phase

Evaluation of recruitment and reach

Four residential facilities and/or day activity centres supporting adults with moderate to profound ID were included. All of the approached teams were enthusiastic about the program and its relevance. The time to recruit the appropriate number of participants was 9 months (April 2018 until January 2019). DSPs also spent time to inform and obtain informed consent of the people with ID. The inclusion criteria could be met for this study, although difficulties were faced, for example, if people with mild ID lived or worked together with people with moderate to profound ID, and thus the target group did not completely match. In addition, the required time investment of the DSPs for the program and for the data collection was a challenge for the (residential) facilities.

Participants in this study were DSPs (n = 32), people with moderate to profound ID (n = 24), managers/coordinator of the participating (residential) facilities (n = 4), and the trainers of the program (n = 6). In total, 32 DSPs out of 41 DSPs from the four (residential) facilities participated in the study. The distribution of DSPs over the four (residential) facilities was as follows: 4, 16, 7, and 5 DSPs. Reasons for not doing so were: lack of presence at the (residential) facility at the time of the implementation due to attendance in another department of the ID care provider, not willing to commit to the time investment, or too recently hired. The characteristics of the DSPs and the people with moderate to profound ID supported by the participating teams are described in Table 2.

Table 2 Characteristics of the DSPs and people with ID
Full size table

Context

During the program, there were changes in the context that were not provoked by the implementation of the program. These changes were with regard to staff: decrease in staff at one (residential) facility (number 1) and additional education for two DSPs at one (residential) facility in order to become physical activity consultants (number 2). With regard to a healthy lifestyle, the following changes were made: decrease in budget for food at one (residential) facility (number 2), increasing offer of physical activity at one (residential) facility (number 4), and health related activities at two (residential) facilities (numbers 1 and 4), for example, receiving recommendations from a movement expert regarding physical activity, more indoor activities, a short class for people with ID about health, and a cooking activity 1 day per week for people with ID.

Implementation phase

Dose delivered, dose received and fidelity

All program components (e-learning, three in-person sessions, and three assignments) were available, although not all DSPs completed all of the components of the program. The e-learning was completed by 26 out of the 32 DSPs. All of the three in-person sessions were attended by 18 DSPs. The remaining 14 DSPs attended one or two in-person sessions. Reasons for not participating in the in-person sessions were illness, vacation, time, or private circumstances. An overview of dose delivered and dose received is described in Table 3. With regards to completing the assignments, six DSPs completed all three assignments, 13 completed either one or two assignments, and 13 did not complete any assignments.

Table 3 Dose delivered and dose received
Full size table

The quality of the submitted assignments (38 out of 96) was mostly positive (23 assignments); see Table 4. The quality of the assignments was considered not sufficient if not all parts of the assignment were completed (e.g. there was no consultation with colleagues), the link to the theme of the in-person session was missing, a question was not well understood, or the BCTs were not clear enough.

Table 4 Results of implementation: dose received, fidelity scores, acceptability and suitability of the program, and changes after the program (goal achievement, actual application in practice)
Full size table

According to the DSPs, 13 out of 32 prepared all of the in-person sessions. Some DSPs prepared only partly for the sessions. Reasons for not preparing were illness, absence, time, and not remembered. The trainers indicated that DSPs from two (residential) facilities in particular prepared inadequately for the in-person sessions. However, the participation during the sessions at these two (residential) facilities was neutral to good. One (residential) facility had different expectations (working with more clear action points) for the in-person sessions. Additionally, the group size for one (residential) facility was large (n = 16), making the sessions less effective according to the reflection of the trainers. Overall, the execution of the in-person sessions in relation to theoretical methods and determinants was good (see Table 4).

Acceptability and suitability of the program

Table 4 shows the mean ratings of DSPs and the managers for the acceptability and fit of the program. It also indicates that, for two of the four (residential) facilities, the acceptability of the program was sufficient, and the fit of the program in daily practice was sufficient to good according to DSPs. For the other two (residential) facilities, the acceptability and suitability were insufficient to good; for one (residential) facility, the acceptability and relevance of the e-learning was considered insufficient whereas the satisfaction and relevance of the in-person sessions were good. The overall satisfaction about the preparation before the in-person sessions was insufficient according to DSPs. The satisfaction and the relevance of the in-person sessions was, for (residential) facility 2, statistically significantly lower in comparison to the other three (residential) facilities. The DSPs of (residential) facility 2 were overall less satisfied about the program and its relevance. All managers granted the program an eight as a rating (one manager could not answer this question), and three out of four were very positive about the content of the program (missing information for one manager).

Almost all of the interviewed DSPs (14 out of 17) indicated that they would recommend the program to colleagues. The DSPs mentioned that their recommendation depends on the status of the team in the domain of healthy lifestyle. Comments that were more critical by some of DSPs pertained to the lack of connection with the target group and the significant time investment of the program. In addition, all of the interviewed DSPs indicated that they were positive (n = 17) about the connection of the program to their support needs. Approximately half of them (n = 8) considered the awareness of a healthy lifestyle and communicating with each other in this context as positive. One DSP illustrates: ‘I thought it was good. We do a lot of things, but it is the awareness of just simple things, like housekeeping, that is also physical activity. Often, we take over activities of our clients which they can do by themselves’ (respondent 5:9). Beside the positive statements about the connection of the program to support needs, there were also DSPs who made neutral (n = 8) or negative (n = 7) comments. They indicated that the program was not necessary or that they expected more tips for daily practice.

Time and capacity to complete the program

Per DSP, the time investment of the program, including preparation and assignments, was an average of 11 h. The feasibility of the time investment of the e-learning was, according to the DSPs, slightly negative to neutral. The feasibility of the time investment of the in-person sessions was neutral to positive. They indicated that the number of sessions, the available time, and the time between the in-person sessions was good. The time investment of the program and the proceeds were rated neutral to positive. The experienced support from their manager was slightly negative to positive according to the DSPs (see Table 4 for their ratings). Overall, the managers indicated the time investment of the program was feasible with the exception of a single manager indicating that the time investment was too much, especially for the research element. The costs associated with the time investment were feasible according to the managers. They were positive about the time spent on the program and the outcomes (see Table 4 for the ratings of the managers). The capacity to complete the program as operationalized by DSPs’ participation in the activities during the meetings was sufficient (score 4.1 on a scale from 1 to 5).

Factors during implementation

Factors for implementation were addressed when developing the program and designing the study, for example, continuous communication about the program [16]. The primary researcher was able to manage the conditions to facilitate the implementation of the study and the program. The willingness to participate was good, however, despite careful discussions of expectations, the time for the DSPs to participate in the study and the program was limited. In addition, according to two of the four managers, the implementation of the program requires improvement, for example. With respect to the alignment of mutual expectations before the beginning of the program.

Evaluation of the data collection process

Based on the goals of the program, the overall sensitivity of the outcome measures to the changes after the program was good. The DSPs understood the questions and guidelines to complete the data collection. They were requested to fill in questionnaires, participate in interviews, and track food intake and physical activities of people with ID. This data collection process appeared, as noted before, time consuming for the participating teams which resulted in missing data. Regarding the missing data for the food diaries, at T2, 6 days of six people with ID were missing. Additionally, part of the diaries or details were missing which could not be included. For physical activity at T0, six people with ID did not wear the Actigraph for at least 4 days for 10 h, and one of them lost the Actigraph. Additionally, at T2, three people with ID did not wear the Actigraph.

Preliminary outcomes

Changes after the program

Figure 1 and Table 1 provide an overview of the design and the indicators for the changes after the program. The outcomes will be described based on the indicators: goal achievement of the program, actual application in practice, attitude of the DSPs, food intake of people with ID, and physical activity of people with ID.

Improving knowledge and skills

Overall, the DSPs’ goals of the program have been achieved for both the e-learning and the in-person sessions; see Table 4. They rated their knowledge about physical activity and nutrition for people with ID; these ratings were a 6.6 and a 7.2, respectively, at T0 and T1. This shows an improvement in knowledge directly after the program (Z = -2.923, p = 0.003).

Actual application in practice

The change in daily practice following the program was sufficient to good for two of the four (residential) facilities according to DSPs. For one (residential) facility, the change was insufficient and, for one (residential) facility, just the change in daily practice from the e-learning was insufficient. For (residential) facility 2, the in-person sessions made statistically significantly less change on daily practice. See Table 4 for the ratings.

The actual applications in practice mentioned by the DSPs were: ‘awareness about a healthy lifestyle’ (n = 16); a DSP illustrated: ‘It is in the little things. I am more aware that I let people with ID clean up themselves after dinner instead of doing that for them’ (respondent 3.33); ‘introducing more physical activities’ (n = 14); using the BCTs (n = 13); prepare or offer healthy nutrition (n = 12); and let people with ID choose themselves (n = 12). Almost all of the DSPs (n = 15) indicated nothing has changed in involving family and others in a healthy lifestyle of people with ID. Besides these comments, 11 DSPs had some neutral comments. They indicated the use of the BCTs depends on the people with ID (n = 6), and the actual application in practice is dependent on the attitude of the DSP (n = 5).

Overall, the managers indicated that the teams spent more attention on a healthy lifestyle in daily practice (see Table 4 for the ratings of the managers). Despite the improvement, the DSPs still struggle with the integration of healthy lifestyle behaviour into working processes.

Attitude of DSPs on supporting healthy lifestyle

There is a significant increase over time on the attitudes of nutrition and physical activity of the DSPs between T0 and T2 (3 months after the program), resulting from a mixed model analysis with random effects for DSPs; see Table 5 which is illustrated in Fig. 2. At T1, the change in attitude after the program is not statistically significant for physical activity and, for nutrition, the change is borderline significant.

Table 5 Change on attitude of DSPs over time; fixed effects (Estimate) with T0 as reference from mixed modeling using random DSP effects
Full size table
Fig. 2
figure 2

Means of attitude of DSPs on nutrition (NU) and physical activity (PA) at T0, T1, and T2

Full size image

Food intake of people with ID

The food intake of people with ID who were supported by the participating DSPs was measured before (T0) and three months after the program (T2). A statistical difference was ascertained between T0 and T2 for food intake from the recommended nutrition guidelines (Z = -1.979, p = 0.047). Table 6 shows the descriptives from T0 and T2. For the mean percentages of food intake from the recommended nutrition guidelines, this guideline is not reached.

Table 6 Descriptive statistics for percentage food intake from the recommended nutrition guidelines at T0 and T2
Full size table

Physical activity of people with ID

The physical activity of people with ID who were supported by the participating DSPs was measured before (T0) and three months after the program (T2). Table 7 shows the mean levels of activity of the people with ID who are able to walk. No significant differences between T0 and T2 were found (% Sedentary: Z = -0.459, p = 0.695; % Light: Z = -0.357, p = 0.770; % Moderate: Z = -0.357, p = 0.750; % Vigorous: Z = -1.604, p = 0.250; % Very Vigorous: Z = -1.414, p = 0.500). The percentages demonstrate that people with ID spend most of their time in a sedentary state. Table 8 shows the percentages of time spent in activities for people with ID who use a wheelchair; the percentage of time in activities before and after the program is almost the same. During the measurements, two Actigraphs were lost, and one participant did not accept wearing the Actigraph. In addition, the daily activity programs before and after the program were compared. No statistical differences were determined between T0 (Mean: 22.75, SD: 23.77) and T2 (Mean: 24.79, SD: 34.01): Z = -0.280, p = 0.844.

Table 7 Mean (SD) level of physical activity of walking people with ID (measured by Actigraph)
Full size table
Table 8 Mean (SD) level of physical activity of people with ID using a wheelchair (measured by Actiwatch)
Full size table

Differences in outcomes for the four participating (residential) facilities

For the differences in outcomes between the four participating (residential) facilities, statistically significant variances were ascertained for the changes after the in-person sessions (actual application in practice)(F = 5.848; p = 0.004) and for the relevance (F = 11.606; p = 0.000) and satisfaction (F = 14.004; p = 0.000) of the in-person sessions (acceptability/suitability). No statistical differences between the four participating (residential) facilities were found for goal achievement, the attitude of DSPs, food intake, and physical activity of people with ID.

Discussion

Main findings

The aim of this study was to evaluate the preparation, implementation, and preliminary outcomes of a theory-based training and education program for DSPs to learn how to support people with moderate to profound ID in a healthy lifestyle. The results provide preliminary empirical evidence for this program. Results show that the preparation phase was feasible. Regarding the implementation phase, the overall fidelity of the program was good, although completing the program, the presence, and preparation for the in-person sessions were points of attention. All of the DSPs were positive about the connection of the program to their support needs whereas, for half of the (residential) facilities, the fit of the program was sufficient. The acceptability and suitability of the program were sufficient, however, the face-to-face sessions were rated more positively than the e-learning. The DSPs ranged from slightly negative to positive about the time investment, and the alignment of the mutual expectations of all stakeholders about the program was considered as important and need to be improved. Almost all of the DSPs would recommend the program. The managers were also positive about the content of the program, and the awareness of the importance of a healthy lifestyle in the teams, even though the teams still struggle with the integration of healthy lifestyle behaviour into working processes. Despite the implementation barriers, the DSPs considered the goals of the program to be achieved. The changes after the program were sufficient.

During implementation, barriers were encountered: the DSPs were not always satisfied, the program was not always relevant to them, and they had different experiences about the changes in daily practice after the program. Nevertheless, there were no statistically significant differences in changes after the program between the different (residential) facilities. This may suggest that, despite the mixed opinions of the DSPs regarding the program, it indeed has impact on their attitudes for supporting a healthy lifestyle, which was recognized in a previous implementation study [23]. In this study, the DSPs made positive changes although, in their own perception, they believed their participation in the training program was obligatory.

The DSPs’ attitudes towards supporting a healthy lifestyle on nutrition increased directly after the program. For both nutrition and physical activity, their attitudes significantly increased after 3 months. This is in accordance with previous research in which education was mentioned as a factor that positively influences the attitude of DSPs for supporting physical activity [14, 43, 44]. This improvement of DSPs’ attitudes is important because it contributes to successful implementation of a healthy lifestyle [45, 46].

In addition, for the changes for people with ID after the program, statistically significant improvements were found for food intake; however, for physical activity, no statistical differences were determined. Additionally, the involvement of family and others in a healthy lifestyle for people with ID needs improvement. This social network can be supportive in adapting to the needs of people with ID by focusing on their strengths and, for example, including them in decision-making [47].

Strengths and limitations

This study was executed in daily practice at four ID care providers; this is where the actual implementation of the program should occur. Although this strength of executing the study in daily practice also brought some limitations for the study, it was the reality of the workplace for the DSPs and therefore provided a realistic representation of the implementation of the program. This study shows important factors that must be taken into account when implementing programs in daily practice.

The changes after the program should be interpreted with caution. The study had a relative small sample size of 32 DSPs and 24 people with ID. In addition, there were missing data in the food diaries of people with ID, and not all persons with ID wore the Actigraphs for the recommended amount of time. Therefore, these outcomes are based on a small sample. However, this is the reality of data collection in daily practice and, nevertheless, the changes after the program were promising.

Although the data collection was suitable for measuring the changes after the program, the feasibility of the data collection process needs improvement regarding the time investment for DSPs. For example, filling in food diaries and wearing physical activity trackers can be components of the daily routine to facilitate the data collection process. The lack of time for implementation corresponds with findings from other implementation studies [12, 23].

Furthermore, physical activity and food intake was measured 3 months after the last program session. This post-test was after the summer holidays, taking into account the absence of regular staff. This may be a limitation because it is questionable if there is a causality between the program and its outcomes, for example, context factors such as changes in the organization, or the different seasons can be a mediating variable. In addition, for future research, a follow-up measurement can indicate healthy lifestyle changes in the long term.

Implications for implementation

For further implementation of the program, communication and alignment with all stakeholders is an important factor. By discussing mutual expectations in advance, disappointments are avoided, and it improves the relevance of the intervention in daily practice [11]. A contact person who is responsible for managing the mutual expectations and the sustainable execution of the plans would support the actual implementation of the program. The program in this study was flexible to the needs at a specific (residential) facility and adaptable to the target group that DSPs work with; this is an important factor for implementing a health promotion program [22]. Therefore, it is important that at least one of the trainers is familiar with the (residential) facility so that the program can be aligned to the needs of the DSPs. The staff change in teams also affects the knowledge and skills of the team about a healthy lifestyle and impedes the continuation of an intervention [48]. Therefore, it may be considered to perform the program again, maybe in a short form, when hiring new DSPs. In addition, a maximum group size for an intervention should be considered in order to be able to properly conduct the intervention. For larger teams, it can be considered to split them up for optimal utilization.

In the evaluation of the program, for the measurements of people with ID, the residential facility where they live should be responsible for the physical activity and food intake measurements. They are with the people with ID at the beginning and end of the day so are able to check the measurements. This data collection should be incorporated into the daily routine to avoid missing data. The importance of the commitment of staff in the implementation is also addressed in a previous evaluation study [49].

Conclusions

The theory-based program consisting of a training and education section for DSPs to learn how to support a healthy lifestyle for people with moderate to profound ID was feasible to implement and, despite some barriers regarding time capacity and mutual expectations, it delivered positive changes in both DSPs and persons with moderate to profound ID. Thus, the program is a promising intervention to support DSPs.

Availability of data and materials

The datasets used and analysed during the current study are available from the corresponding author upon reasonable request.

Abbreviations

BCTs:

Behavior Change Techniques

DIHASID:

Discovering Health-promoting Assets in Settings for people with Intellectual Disabilities

DSPs:

Direct support professionals

ID:

Intellectual disability

IM:

Intervention Mapping Protocol

LFE:

Low Frequency Filter

NU:

Nutrition

PA:

Physical Activity

TDF:

Theoretical Domains Framework

References

  1. Gawlik K, Zwierzchowska A, Celebańska D. Impact of physical activity on obesity and lipid profile of adults with intellectual disability. J Appl Res Intellect Disabil. 2018;31(2):308–11.

    PubMed  Google Scholar 

  2. Hsieh K, Rimmer JH, Heller T. Obesity and associated factors in adults with intellectual disability. J Intellect Disabil Res. 2014;58(9):851–63.

    CAS  PubMed  Google Scholar 

  3. Humphries K, Traci MA, Seekins T. Nutrition and adults with intellectual or developmental disabilities: systematic literature review results. Intellect Dev Disabil. 2009;47(3):163–85.

    PubMed  Google Scholar 

  4. van der Putten AAJ, Bossink LWM, Frans N, Houwen S, Vlaskamp C. Motor activation in people with profound intellectual and multiple disabilities in daily practice. J Intellect Develop Disabil. 2016;42(1):1–11.

    Google Scholar 

  5. Hsieh K, Hilgenkamp TIM, Murthy S, Heller T, Rimmer JH. Low levels of physical activity and sedentary behavior in adults with intellectual disabilities. Int J Environ Res Public Health. 2017;14(12):1503.

    PubMed Central  Google Scholar 

  6. Koritsas S, Iacono T. Weight, nutrition, food choice, and physical activity in adults with intellectual disability. J Intellect Disabil Res. 2016;60(4):355–64.

    CAS  PubMed  Google Scholar 

  7. van Timmeren EA, van der Schans CP, van der Putten AAJ, Krijnen WP, Steenbergen HA, van Schrojenstein Lantman-de Valk HM, et al. Physical health issues in adults with severe or profound intellectual and motor disabilities: a systematic review of cross-sectional studies. J Intellect Disabil Res. 2017;61(1):30–49.

    PubMed  Google Scholar 

  8. Bossink LWM, van der Putten AAJ, Vlaskamp C. Understanding low levels of physical activity in people with intellectual disabilities: a systematic review to identify barriers and facilitators. Res Dev Disabil. 2017;68:95–110. https://doi.org/10.1016/j.ridd.2017.06.008.

    Article  CAS  PubMed  Google Scholar 

  9. Leser KA, Pirie PL, Ferketich AK, Havercamp SM, Wewers ME. The perceived role of direct support professionals in the health promotion efforts of adults with developmental disabilities receiving support services. Intellect Dev Disabil. 2018;56(1):40–55.

    PubMed  Google Scholar 

  10. Kuijken NMJ, Vlot-van Anrooij K, HMJ VS L-d V, Leusink G, Naaldenberg J, Nijhuis-van der Sanden MW. Stakeholder expectations, roles and responsibilities in Dutch health promotion for people with intellectual disabilities. Health Promot Int. 2019;34(5):59–70.

    Google Scholar 

  11. Steenbergen HA. Healthy lifestyle of people with intellectual disabilities. University of Groningen; 2020. p. 81–105.

    Google Scholar 

  12. van Alphen HJM, Waninge A, Minnaert AEMG, van der Putten AAJ. Development and process evaluation of a motor activity program for people with profound intellectual and multiple disabilities. BMC Health Serv Res. 2021;21(1):1–13.

    Google Scholar 

  13. Dijkhuizen A, Waninge A, Hermans S, van der Schans CP, Krijnen WP. Progressive resistance training for persons with intellectual disabilities and visual impairment. J Appl Res Intellect Disabil. 2019;32(5):1194–202.

    PubMed  Google Scholar 

  14. Steenbergen HA, van der Schans CP, van Wijck R, de Jong J, Waninge A. Lifestyle Approaches for People With Intellectual Disabilities: A Systematic Multiple Case Analysis. J Am Med Dir Assoc. 2017;18(11):980–987.e3. https://doi.org/10.1016/j.jamda.2017.06.009.

    Article  PubMed  Google Scholar 

  15. Willems M, Hilgenkamp TIM, Havik E, Waninge A, Melville CA. Use of behaviour change techniques in lifestyle change interventions for people with intellectual disabilities: a systematic review. Res Dev Disabil. 2017;60:256–68. https://doi.org/10.1016/j.ridd.2016.10.008.

    Article  PubMed  Google Scholar 

  16. Overwijk A, Hilgenkamp ThessaIM, Van der Schans CP, Van der Putten AAJ, Waninge A. Development of a Dutch training/education program for a healthy lifestyle of people with intellectual disabilities. Intellect Dev Disabil. 2021.

  17. Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005;14(1):26–33.

    CAS  PubMed  PubMed Central  Google Scholar 

  18. Cane J, O’Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012;7(1):1–17.

    Google Scholar 

  19. Michie S, Ashford S, Sniehotta FF, Dombrowski SU, Bishop A, French DP. A refined taxonomy of behaviour change techniques to help people change their physical activity and healthy eating behaviours: the CALO-RE taxonomy. Psychol Health. 2011;26(11):1479–98.

    PubMed  Google Scholar 

  20. Kolb AY, Kolb DA. The learning way. Simul Gaming. 2009;40(3):297–327 Available from: http://journals.sagepub.com/doi/10.1177/1046878108325713.

    Google Scholar 

  21. Bartholomew Eldredge LK, Markham CM, Ruiter RAC, Fernández ME, Kok G, Parcel GS. Planning health promotion programs: an intervention mapping approach. San Francisco: Jossey-Bass; 2016.

    Google Scholar 

  22. Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41(3–4):327–50.

    PubMed  Google Scholar 

  23. Hanzen G, Van Nispen RMA, Vlaskamp C, Korevaar EL, Waninge A, Van der Putten AAJ. Improving the participation of adults with visual and severe or profound intellectual disabilities: a process evaluation of a new intervention. BMC Health Serv Res. 2020;20(1):1–15.

    Google Scholar 

  24. Linnan L, Steckler A. Process evaluation for public health interventions and research. In: Process evaluation for public health interventions and research. San Francisco: Jossey-Bass; 2002. p. 1–23.

    Google Scholar 

  25. Orsmond GI, Cohn ES. The distinctive features of a feasibility study: objectives and guiding questions. OTJR: Occupation Participation Health. 2015;35(3):169–77.

    Google Scholar 

  26. Schalock RL, Luckasson R, Tassé MJ. Intellectual disability: definition, diagnosis, classification, and Systems of Support. 12th ed. AAIDD: Washington, D.C; 2021.

    Google Scholar 

  27. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 5th ed. Washington, DC: (DSM-5); 2013.

    Google Scholar 

  28. Nakken H, Vlaskamp C. A need for a taxonomy for profound intellectual and multiple disabilities. J Policy Pract Intellect Disabil. 2007;4(2):83–7 Available from: http://onlinelibrary.wiley.com/doi/10.1111/j.1741-1130.2007.00104.x/abstract.

    Google Scholar 

  29. Vlot-van Anrooij K, Hilgenkamp TIM, Leusink GL, van der Cruijsen A, Jansen H, Naaldenberg J, et al. Improving environmental capacities for health promotion in support settings for people with intellectual disabilities: inclusive design of the DIHASID tool. Int J Environ Res Public Health. 2020;17(3):1–13.

    Google Scholar 

  30. Vlot-van AK. Embedding health promotion in support settings for people with intellectual disabilities. An innovative adoption of the settings approach. Nijmegen: Radboudumc; 2021.

    Google Scholar 

  31. Vlot-van Anrooij K, Naaldenberg J, Hilgenkamp TIM, Overwijk A, van der Velden K, Leusink GL. Gaining actionable knowledge to improve local health-promoting capacities in long-term care support settings for people with intellectual disabilities. Patient Educ Couns. 2021;xxxx:0–1.

    Google Scholar 

  32. Bowen GA. Document analysis as a qualitative research method. Qual Res J. 2009;9(2):27–40.

    Google Scholar 

  33. Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88.

    PubMed  Google Scholar 

  34. Pinheiro JC, Bates DM. Mixed-effects models in S and S-plus. New York: Springer; 2000.

    Google Scholar 

  35. Core Team RC. R: a language and environment for statistical computing. Vienna: R Foundation for statistical Computing; 2017.

    Google Scholar 

  36. Chow BC, Huang WYJ, Choi PHN, Yu PC. Design and methods of a multi-component physical activity program for adults with intellectual disabilities living in group homes. J Exerc Sci Fit. 2016;14(1):35–40.

    PubMed  PubMed Central  Google Scholar 

  37. Nordstrøm M, Hansen BH, Paus B, Kolset SO. Accelerometer-determined physical activity and walking capacity in persons with Down syndrome, Williams syndrome and Prader-Willi syndrome. Res Dev Disabil. 2013;34(12):4395–403.

    PubMed  Google Scholar 

  38. Van Alphen HJM, Waninge A, Minnaert AEMG, Post WJ, Van der Putten AAJ. Construct validity of the Actiwatch-2 for assessing movement in people with profound intellectual and multiple disabilities. J Appl Res Intellect Disabil. 2020:1–12.

  39. Chow BC, Choi PHN, Huang WYJ. Physical activity and physical fitness of adults with intellectual disabilities in group homes in Hong Kong. Int J Environ Res Public Health. 2018;15(7):1370.

    PubMed Central  Google Scholar 

  40. Bastiaanse LP, Hilgenkamp TIM, Echteld MA, Evenhuis HM. Prevalence and associated factors of sarcopenia in older adults with intellectual disabilities. Res Dev Disabil. 2012;33(6):2004–12.

    PubMed  Google Scholar 

  41. Nutrition Centre of the Netherlands. Richtlijnen Schijf van Vijf. Richtlijnen Schijf van Vijf. 2020.

    Google Scholar 

  42. Freedson PS, Melanson E, Sirard J. Calibration of the computer science and applications, Inc. accelerometer. Med Sci Sports Exerc. 1998;30(5):777–81.

    CAS  PubMed  Google Scholar 

  43. Steenbergen HA, Van Wijck R, De Jong J, Van der Schans CP, Krijnen WP, Waninge A. Exploring the attitude of direct support persons towards promoting healthy physical activity of persons with intellectual disabilities: using the ADSP-HEPA questionnaire. Submitted

  44. Bossink LWM, Van der Putten AAJ, Paap MCS, Vlaskamp C. Factors associated with direct support professionals’ behaviour in the physical activity support provided to people with intellectual disabilities. J Intellect Disabil Res. 2019;63(8):981–91 Available from: https://onlinelibrary.wiley.com/doi/abs/10.1111/jir.12616.

    CAS  PubMed  PubMed Central  Google Scholar 

  45. Martin E, McKenzie K, Newman E, Bowden K, Morris PG. Care staff intentions to support adults with an intellectual disability to engage in physical activity: an application of the theory of planned behaviour. Res Dev Disabil. 2011;32(6):2535–41. https://doi.org/10.1016/j.ridd.2011.07.006.

    Article  PubMed  Google Scholar 

  46. Wensing M, Grol R, Grimshaw J. Improving Patient Care: The Implementation of Change in Health care; 2020.

    Google Scholar 

  47. Vlot-Van Anrooij K, Naaldenberg J, Hilgenkamp TIM, Vaandrager L, Van Der Velden K, Leusink GL. Towards healthy settings for people with intellectual disabilities. Health Promot Int. 2020;35(4):661–70.

    PubMed  Google Scholar 

  48. Elinder LS, Sundblom E, Zeebari Z, Bergström H. Effect and process evaluation of a structural health intervention in community residences for adults with intellectual disabilities. J Policy Pract Intellect Disabil. 2018;15(4):319–28.

    Google Scholar 

  49. Dixon-Ibarra A, Driver S, VanVolkenburg H, Humphries K. Formative evaluation on a physical activity health promotion program for the group home setting. Eval Program Plann. 2017;60:81–90.

    PubMed  Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge the care providers, their staff, and the people with moderate to profound ID for participating in this study. We also thank the trainers for executing the program. The authors would also like to acknowledge Sanne Nijhoff and Iris Hoedemaker for their assistance with interviewing and analysing the interviews and food diaries. In addition, thanks to Mirjam Kalisvaart for entering the food diary data.

Funding

This work was supported by the Dutch organization for health research and development ZonMw in the program ‘Gewoon Bijzonder, Nationaal Programma Gehandicapten’ [grant number: 80–84500–98-118] and the ‘Innovatie Werkplaats Active Ageing van mensen met VB’, a collaboration between the Hanze University of Applied Sciences, Alfa-college, and care provider organizations for people with ID.

Author information

Authors and Affiliations

  1. Research Group Healthy Ageing, Allied Health Care and Nursing, Hanze University of Applied Sciences Groningen, Groningen, the Netherlands

    A. Overwijk, C. P. van der Schans, W. P. Krijnen & A. Waninge

  2. Department of Health Psychology, University of Groningen, University Medical Center Groningen, Groningen, the Netherlands

    A. Overwijk, C. P. van der Schans & A. Waninge

  3. Department of General Practice, Intellectual Disability Medicine, Erasmus MC, University Medical Center Rotterdam, Rotterdam, the Netherlands

    T. I. M. Hilgenkamp

  4. Department of Physical Therapy, University of Nevada, Las Vegas, USA

    T. I. M. Hilgenkamp

  5. Department of Rehabilitation Medicine, University of Groningen, University Medical Center Groningen, Groningen, the Netherlands

    C. P. van der Schans

  6. Department of Primary and Community Care, Research group Intellectual Disabilities and Health, Radboud Institute of Health Sciences, Radboud University Medical Center, Nijmegen, the Netherlands

    K. Vlot-van Anrooij

  7. Department of Inclusive and Special Needs Education, University of Groningen, Groningen, the Netherlands

    A. A. J. van der Putten

Authors
  1. A. Overwijk
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. T. I. M. Hilgenkamp
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. C. P. van der Schans
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. W. P. Krijnen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. K. Vlot-van Anrooij
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. A. A. J. van der Putten
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. A. Waninge
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

AO, TH, CvdS, KVvA, AvdP, and AW contributed substantially to the overall conceptualization and design of the study. AO coordinated the implementation process, gathered the data, and drafted the manuscript. CvdS, AvdP, TH, and AW supervised AO during the process. WK supported and performed part of the analysis. The content of the article was discussed with all of the authors who were also involved in drafting or revising the manuscript. All authors have read and approved the final manuscript.

Corresponding author

Correspondence to A. Overwijk.

Ethics declarations

Ethics approval and consent to participate

This study was conducted according to the principles of the Declaration of Helsinki and has been independently reviewed by the Medical Ethical Committee of the University Medical Centre Groningen (study number: 201700164). Exemption of this Committee was received because in this type of research no action or code of conduct is imposed on persons (WMO: Medical Research involving Human Subjects Act). In addition, the research is conducted according to the ethical guidelines of the Medical Treatment Contracts Act (WGBO), the General Data Protection Regulation (GDPR/ENG or AVG/NL), and FEDERA’s codes of conduct. Participation of all respondents was voluntary and of no consequences to their work evaluations or rewards. Participants had the option to withdraw from participation without any consequences. Resistance among people with ID has been taken into account in the context of the Medical Research involving Human Subjects Act. DSPs, people with ID, managers/coordinator of the participating (residential) facilities, and the trainers of the program provided written informed consent for the study. Legal representatives provided written informed consent for the people with moderate to profound ID.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Overwijk, A., Hilgenkamp, T.I.M., van der Schans, C.P. et al. Implementation of a program to support direct support professionals to promote a healthy lifestyle for people with moderate to profound intellectual disabilities. BMC Health Serv Res 22, 15 (2022). https://doi.org/10.1186/s12913-021-07389-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-021-07389-x

Keywords

BMC Health Services Research

ISSN: 1472-6963

Contact us