Skip to main content

TIME to reduce agitation in persons with dementia in nursing homes. A process evaluation of a complex intervention



The Targeted Intervention Interdisciplinary Model for Evaluation and Treatment of Neuropsychiatric Symptoms (TIME) has recently in a three-month cluster randomised controlled trial demonstrated reduction in agitation in nursing home residents with dementia. To ease replication and future implementation, and to clarify possible causal mechanisms, we performed a process evaluation of the intervention based on the RE-AIM framework (Reach, Effectiveness, Adoption, Implementation, Maintenance).


An exploratory and a quasi-experimental design with mixed methods were used. The RE-AIM dimensions were explored by questionnaires to 807 staff members and 46 leading ward nurses in both the intervention nursing homes (INH) and the control nursing homes (CNH), before the start of the trial (baseline), and six and 12 months later. These questionnaires assessed data regarding the reach, effectiveness (staff level) and adoption dimensions. To assess implementation, we used a checklist for performance of the main components in TIME and analysed the minutes from 84 case conferences in the INH. To explore adoption and maintenance, five focus group interviews with 32 participants from the staff in the INH were conducted three to 6 months after the end of the trial.


Reach: On average 61% (SD 22) of the staff in each ward in the INH attended the training sessions. Effectiveness at staff level: There were no between-group differences throughout the study period for attitudes towards dementia, perceived competence or perception of mastery and social interaction. Adoption: 16 of the 17 INH completed the intervention. Implementation: 75% or more of the components of TIME were performed for 91% of the included residents. Maintenance: Most of the nursing homes used TIME three to 6 months after the end of the trial. An easy to grasp model and an engaged leadership facilitated the intervention and maintenance.


A high degree of reach, adoption, implementation and maintenance contributed to the effectiveness of TIME at resident level. One other causal assumption of the effectiveness of TIME is the development in the staff of a new, shared and situated knowledge about each individual resident, not reflected by measurements in general knowledge and attitudes.

Trial registration

The trial was registered January 6, 2016 with (NCT02655003).

Peer Review reports


Nearly all persons living with dementia in nursing homes will develop neuropsychiatric symptoms (NPS) during the course of the condition [1, 2]. Of these symptoms, agitation including verbal and physical aggression and excessive motor activity, is one of the most frequent and persistent symptoms [1, 3, 4]. Agitation results in distress for the patients and their caregivers and is associated with reduced quality of life, referral to specialist health care and hospitalization, and a more rapid progression toward severe dementia and death [5,6,7]. The effect of psychotropic drugs used for agitation is modest, and their use is associated with major side-effects [8]. Treatment guidelines recommend non-pharmacological interventions for agitation as a first-line approach, even though the evidence of their effectiveness is contradictory [9,10,11]. Though some training interventions for nursing home staff aiming to reduce agitation in dementia have shown promising results [12, 13], most of these interventions have failed to demonstrate effectiveness at residential level [10, 14]. Nursing homes represent complex social systems which differ in their characteristics, sometimes changing during a trial. The settings will affect both the content of an intervention (i.e. standardisation) and the implementation process [15, 16]. These characteristics of the care settings pose important challenges for both the intervention and the implementation process to be efficacious [16, 17]. Findings from a systematic review to identify features of effective dementia educational programs by Surr et al. (2017), emphasised that for learning to take place, theory must underpin practice-based learning, involve active face-to face participation and provide a structured tool to guide practice [17]. To address the challenges in complex systems and improve the adoption of interventions, Hawe et al. (2004) recommended allowing for a certain degree of flexibility and the adaption of the intervention to the local context (i.e. a more pragmatic design in trials) [16].

In a recent three-month cluster-randomized controlled trial conducted in Norway, The Targeted Interdisciplinary Model for Evaluation and Treatment of Neuropsychiatric Symptoms (TIME), was compared to usual care supplemented by a brief educational intervention [18]. In this trial clinically significant reduction in agitation (primary outcome) and in symptoms of delusions, depression, and disinhibition, as well as improved quality of life (secondary outcomes) was found in favour of the intervention group [19]. An important question regarding evidence-based interventions like TIME, is how to obtain a simple, effective, and sustainable implementation [10, 20].

Description and development of TIME

The published protocol paper for the trial and the manual at the TIME website (English version) give a complete description of TIME, the model’s development, and its theoretical underpinnings [18, 21]. TIME is an evidence based, multicomponent, biopsychosocial intervention to be used by the nursing home staff and physicians. Principles from cognitive behavioural therapy (CBT) and person-centred care form the theoretical base for TIME [22, 23]. TIME is divided in three overlapping phases: a registration and assessment phase (Table 1); a guided reflection phase, including case conferences (Table 2); and an action and evaluation phase. The actions to take in this last phase are to be described as SMART. SMART stands for Specific, Measurable, Actual, Realistic and Time framed [30]. The evaluation of these actions is performed using the same clinical scales and observation forms as described in the first phase. Because the actions and treatment measures are supposed to be tailored to each resident, they will display great variations between patients. In this way, TIME serves as a guide for the staff to create actions and treatment measures that are customised to the resident. These three phases are in line with reviews describing the “state of the art” for management of NPS [9, 31]. A qualitative study exploring the staffs’ experiences with TIME showed that the model shifted the way of learning for the staff from a traditional to a shared, innovative and reflection-based learning at work using the case conferences. The staff experienced increased coping when approaching complex problems [32].

Table 1 Registration and assessment phase
Table 2 Agenda and timeframe for the case conference

Implementation strategies used in the trial

The published study protocol gives a detailed description of the control and intervention phases in the trial [18]. The implementation process started with an information meeting (1.5 h) with leaders (i.e. leading ward nurses, managers, and physicians) from the nursing home and the health and care authorities in the municipality to introduce TIME and the necessary steps for its implementation. Before randomisation, three nurses from each ward in both the intervention nursing homes (INH) and the control nursing homes (CNH) were offered a three-hour lecture about the trial and the assessments instruments to be used for the residents. After randomisation, a two-hour lecture covering dementia and NPS was given to the staff in both the INH and the CNH. For the staff in the CNH, this lecture represented the education-only intervention. For the remainder of the trial, the staff of the CNH continued with practice as usual. An additional training program with a three-hour lecture and role play about TIME was offered to the staff in the INH. The responsibility for the implementation of TIME was given to three nurses in each ward in the INH together with the leading ward nurse. These three nurses were given the title TIME administrators. The leading ward nurse selected the TIME-administrators for each ward, based on their interest in professional development, their legitimacy with the rest of the staff, and not having a part-time job. The TIME-administrators received three additional hours of training. A specialist nurse from the education and training team attended the TIME administrators’ first case conference on their first resident in their nursing home in a supervisory capacity. The education and training team in the TIME trial consisted of five specialist registered nurses in geriatrics or old age psychiatry and one physician with special competence in nursing home medicine. All members of the team were familiar with TIME and had used the model for some years in real-world clinical settings. In addition to the training sessions, all the staff members in the INH were given the TIME manual and they had access to a website with assessment sheets, an educational film and other educational material to support the intervention [21, 33].

The intervention with TIME can be regarded as a pragmatic trial due to relevant clinical inclusion criteria and the primary outcome measure, the low involvement of the researchers in the implementation process, the low use of other external resources, the rather modest time allocated for educational purposes and the moderate degree of flexibility for adaption of the intervention to the context of the nursing homes [34].

The need for process evaluations of complex interventions

TIME can be regarded as a complex intervention because it is composed of multiple components with the potential for interactions between them and with a number of possible outcomes [35]. Few studies of complex interventions in nursing homes report from the implementation process and to what degree the interventions were adopted and sustained over time [36,37,38,39,40]. A common conclusion in these reports is that management support and organisational factors are the main issues in the implementation process. The comprehensive review of the effect of staff training on staff outcomes by Spector et al. in 2016 concluded that many staff factors are difficult to change [40]. One important question for health leaders and policy makers is to what extent the intervention is flexible, easy to implement and can be adapted to the local context. Furthermore, a process evaluation is important to guide projects which aim to replicate the trial. And finally, it is necessary to clarify possible causal mechanisms to better inform and prepare further dissemination and implementation of the intervention on a larger scale [20, 41].

The RE-AIM framework for evaluation of complex interventions

The Medical Research Council (MRC) defines an overarching framework for the development and evaluation of complex interventions emphasizing the value of combining both quantitative and qualitative data in the process [42]. In the MRC guidance the RE-AIM framework is described as a useful framework for assessing the overall implementation impact of interventions. RE-AIM is an acronym for five evaluation dimensions: Reach (the number, proportion and characteristics of the target population that participated), Effectiveness (the ability to change desired outcomes), Adoption (the number or proportion of wards who adopted the intervention), Implementation (how well the intervention was delivered as designed), and Maintenance (how well program effects are maintained and the continued use of the program) [43,44,45]. In this report, we have followed the recommendations from the MRC guidance and performed a process evaluation of the intervention with TIME using the RE-AIM evaluation framework.


The primary aim of the study was to perform a process evaluation of the intervention of TIME to ease replication and future implementation. As a part of this evaluation, we wanted to explore what factors inhibited or promoted the implementation. Secondly, we aimed to clarify possible causal mechanisms for the effectiveness of TIME at resident level.



We used a quasi-experimental (a pre-test post-test, control group) and a qualitative exploratory design. Mixed methods of the convergent parallel type according to Creswell and Clark were used [46]. In this type of mixed-method study, qualitative data and quantitative data are collected in the same study and with equal priority. The data are analysed separately but integrated in the overall interpretation of the results.

Sites and participants

A total of 89% of nursing homes in Norway are run by the municipalities and 11% are run by non-profit organisations or private enterprises. According to national specific regulations, each nursing home must have an administrative manager, a registered nurse responsible for the nursing care, adequate staffing to ensure necessary care for the residents, and a nursing home physician [47]. Most physicians working in nursing homes are general practitioners in part-time positions. Sixty-three municipalities with a total of 130 nursing homes located in the north, middle, and south-eastern parts of Norway were invited to take part in the cluster-randomised controlled trial (TIME). The choice of this convenience sample was made because of resource reasons to minimise travel time and costs for the research and educational group and, at the same time, strive for a representative sample of NH. The approach was effectuated by an e-mail invitation to the manager of the elderly care department of each municipality. In the e-mail intervention, we clarified that nursing homes already using TIME, nursing homes engaged in other research projects and nursing homes that primarily offered short-term care could not participate in the trial. The research team arranged information meetings for managers and physicians from nursing homes in 32 municipalities who responded positively to the invitation. In these municipalities there were 63 nursing homes. Participation in the study for the municipalities and their NH, as is the rule for most other RCTs, was based on interest and was voluntary [48]. After the information meetings 12 of the 32 municipalities declined to participate in the trial. The reasons they gave for their refusal were, ‘being engaged in other research projects’ (five), ‘lack of resources and being engaged in internal educational projects ‘(five) and ‘already using TIME’ (one). One municipality did not give any reason for the refusal to participate. Finally, a total of 33 nursing homes from 20 municipalities accepted our invitation to take part in the trial. After baseline assessments, 17 nursing homes with 104 included residents from 22 wards were randomly assigned to the intervention group (INH), and 16 nursing homes with 125 included residents from 24 wards were randomly assigned to the control group (CNH) [49].

In this process evaluation of the trial, 797 staff members and 22 leading ward nurses from the INH and 889 staff members and 24 leading ward nurses from the CNH were invited to participate in the survey by receiving questionnaires at baseline. The questionnaires were distributed by e-mail, using a work address list of all employees including both regular and temporary staff members, in each ward who had residents included in the trial. The five focus groups consisted of 32 staff members, leaders, and physicians from 11 of the 17 INH. The nursing homes that were represented in the groups were selected randomly from the pool of INH to minimize selection bias [50]. We did not conduct focus groups in all the 17 INH to minimise travel time and costs for the research group. To achieve information-rich cases and a purposeful sample, the leading ward registered nurse in each nursing home selected who of the staff would attend the focus groups [51]. The criteria for the selection of participants were that they should be familiar with TIME and be able to promote views in a group discussion. A detailed description of the composition of the focus groups can be found in the Additional file 2 (Methods used for the collection of qualitative data and their analyses).

Data collection


The dimensions from the RE-AIM framework were assessed using questionnaires to the staff and leaders. An overview of the questionnaires, their timeframe, the target population (i.e. staff members or leadership in INH or CNH) and their relation to the RE-AIM dimensions is presented in Table 3. The complete set of questionnaires is presented in the Additional file 1 (Questionnaires).

Table 3 Overview of questionnaires distributed to the staff and the leading ward registered nurses, timeframe, and their relation to the RE-AIM-frameworka

Using a questionnaire, we asked the staff to provide their demographic information, such as age, employment relationship, percentage of full time engagement, number of years of experience in health-related jobs, and number of years of health-related continuing education. Organizational and structural factors of the NH wards were obtained from the leading ward nurses: type of ward (special care ward or regular ward), the ward size (number of residents and the number of full-time employment equivalent), the number of staff present per resident during daytime, and the number of hours the nursing home physician is available per resident per week.

The following three questionnaires were administered to the staff before the start of the intervention (i.e. before randomisation of the nursing homes), and six and 12 months later, in both INH and CNH (name or acronym of questionnaires, RE-AIM dimensions): the Approaches to Dementia Questionnaire (ADQ, Effectiveness) [52, 54], the General Nordic Questionnaire for Psychological and Social Factors at Work regarding mastery and social interaction (QPS-Nordic, Effectiveness) [53], and a brief self-developed questionnaire assessing perceived competence for individual staff members regarding NPS (Competence Questionnaire, Effectiveness). Only those of the staff who answered a questionnaire at baseline received a follow-up questionnaire at 6 months, and only those who answered at 6 months received a follow-up questionnaire at twelve months.

The QPS-Nordic has been validated in a Norwegian version and has acceptable construct and content validity as well as internal consistency and test-retest reliability [53, 55]. The QPS-Nordic consists of 13 subscales covering essential social and psychological factors at work. The two subscales labelled perception of mastery and perception of social interaction were used. To respond, staff members chose a value on a five-point Likert scale ranging from 1 (very seldom or never) to 5 (very often or always) to indicate how appropriate each statement was for them. Each subscale score for six items ranged from 6 to 30.

The ADQ assesses general attitudes to dementia. The questionnaire has acceptable content and construct validity as well as internal consistency, inter-rater and test-retest reliability [54]. ADQ has been translated to a Norwegian version [52]. The questionnaire consists of 19 statements, where respondents indicate on a five-point Likert scale from 1 (strongly agree) to 5 (strongly disagree) the extent to which each statement was in accordance with their attitudes. The total sum score ranges from 19 to 95, with higher scores signifying more positive attitudes. A factor analysis resulted in two domains: the “hope” attitude (8 items, range 8–40) and the “person-centred” attitude (11 items, range 11–55) [54].

The Competence Questionnaire consisted of five statements. The respondent indicated how appropriate each statement was for their situation on a seven-point Likert scale from 1 (very low competence) to 7 (very high competence) giving a total sum score between 7 and 35. Competence was defined in the questionnaire as a composition of the concepts “knowledge” and “skills” [56].

A questionnaire assessing the performance of the main 10 components of the TIME (Fidelity Questionnaire, Implementation) were addressed by a brief telephone interview to one of the TIME administrators in the INH for each included resident, up to three times during the trial with 3 to 4 weeks interval. Each component was given a score of 1 if it was performed, except for the performance of the case conference which was given a score of 8, and the evaluation procedure which was given a score of 4. The weighting of the score was based on the presumed time to perform each component. The sum score of the questionnaire thus had a score ranging from 0 to 20, where a higher score indicates greater fidelity to the model. A score of 20 for a resident, means that 100% of the components for the resident was performed.

A self-developed questionnaire for assessment and treatment routines of NPS at ward level (Current Practice Questionnaire, Adoption and Maintenance) were distributed to the leading ward nurses before the start of the intervention, and six and 12 months later, in both the INH and CNH. This questionnaire was developed by the research team based on the TIME manual and on evidence-informed best practice [9, 21, 31]. Respondents indicated how appropriate each statement was for their situation on a five-point Likert scale from 1 (very seldom or never) to 5 (very often or always). The questionnaire consists of 13 items, giving a sum score from 13 to 65 where a higher score indicates a better practice for assessment and treatment of NPS.

A registration form to assess participation of staff, leaders and physicians in education and training sessions were used (Reach). The Reach fraction for the attendance to the training sessions was calculated by the number of participants per ward who attended the training sessions divided by the number of potential participants (i.e. the total number of regular and temporary staff in the ward) [44].

Focus groups and minutes from case conferences

To explore adoption and maintenance, five focus groups interviews with 32 of the caregivers, leaders, and physicians from 11 of the 17 INH were conducted 3 to 6 months after the end of the 3 month intervention [57]. To further assess implementation, we collected the minutes from 84 of the 85 performed case conferences in the INH. A detailed description of the selection and the composition of the focus groups and the collection of the minutes can be found in the Additional file 2 (Methods used for the collection of qualitative data and their analyses).

Data analysis

Statistical analysis

Data were presented as means and standard deviations (SD) or frequencies and percentages, as appropriate. Differences between the two groups in the sum scores for items from the questionnaires were assessed by a linear mixed model with random effects for wards, controlling for possible intra-ward correlations. Fixed effects for time, group allocation and the interaction between these two were included into the model. A significant interaction implies differences between the groups throughout the study period. The results were presented as means and 95% confidence intervals estimated at each time point within the groups with p-values quantifying the between- and within-groups differences. Statistical analyses were performed using SPSS v24 and SAS v 9.4. The results with p-values below 0.05 were considered statistically significant.

Qualitative analysis

For the analysis of the interviews of the focus groups, thematic content analysis was used. Thematic content analysis is a method with the purpose to identify, analyse, and report patterns and themes in qualitative data [58, 59]. The aim is to provide a systematic description of both the manifest and latent content of the data, and in the end to evolve new concepts and understanding of phenomena [58]. Our approach to the coding process was mainly inductive and data-driven [58]. Preliminary analyses were conducted after each group, and when no obvious new meanings units or new sub-themes occurred in the last interview, we considered having met saturation of the data [60]. Qualitative analyses were performed using the word processor Word creating new documents based on coding of meaningful units, subthemes, and themes in the text to be analysed. Detailed description of this process can be found in the Additional file 2.

The documentary analysis of the minutes from the case conferences were performed by using structuring content analysis [61, 62]. This analysis looks for types or formal structures in the documents and uses preformed categories as codes to analyse frequencies and different degrees of quality in a category. The analysis is mainly deductive. Our categories were the main structuring components of the case conferences used in TIME, i.e. use of a problem list, selection of a prioritized problem, the degree of details in the description of the prioritized problem, the overall understanding and use of the five columns, actions described as SMART, and the degree of details in the description of the evaluation procedures of treatment actions. A detailed description of both the qualitative analysis methods, can be found in the Additional file 2.


Eight-hundred and seven (48%) of the staff, 366 from the INH (46%) and 441 (50%) from the CNH, accepted to participate in the survey by answering the baseline questionnaires. At follow-up 6 months from baseline, the response rates among those who answered at baseline were 188 (51%) and 222 (50%) in the INH and the CNH respectively. The response rate at 12 months follow-up, among those who answered at baseline, was 141 (39%) and 156 (35%) in INH and CNH, respectively.

On behalf of their wards, 21 (95%) of the leading ward nurses from INH and 24 (100%) from the CNH answered at baseline the Current Practice Questionnaire. At 6 months from baseline the response rate was 21 (95%) and 23 (96%) in INH and CNH, respectively. The response rate at 12 months was 20 (91%) and 22 (92%) in INH and CNH, respectively.

The Fidelity Questionnaire for the INH and the registration form to assess participation in education and training sessions was completed for 100% of the wards in the INH and the CNH.

Staff and ward characteristics are presented in Table 4. Staff and ward characteristics for those who participated in the focus groups are presented in Additional file 2.

Table 4 Baseline characteristics of the nursing staff (N = 804a) and the wards (N = 46). Values are numbers (%), unless otherwise specified


The mean reach fraction for the attendance to the training sessions in the INH were 61% (SD 22) and 51% (SD 24) for the brief educational session in the CNH. In the INH, seven out of 17 nursing home physicians (41%) and 14 out 22 (64%) of the leading ward nurses attended the training sessions. Of the 78 TIME administrators, 43 (55%) were registered nurses or had an education equivalent to at least 3 years of college or university education in the field of elderly or health care and 35 (45%) were auxiliary nurses. On average there were 8 (SD 2.9) staff members participating in the case conferences ranging from 3 to 19. Nursing home physicians and leading ward nurses attended 27 (32%) and 42 (49%) of the case conferences, respectively.


The results from the questionnaires on the effectiveness at staff level are presented in Table 5. There were no statistically significant group differences throughout the study period for the sum scores for attitudes towards dementia, perceived competence or perception of mastery and social interaction for attitudes towards dementia, perceived competence or perception of mastery and social interaction. The sum score for attitudes towards dementia increased from baseline to twelve months in the INH (p = 0.029), and in the CNH (p = 0.006). The sub-score for hope attitudes increased from baseline to twelve months in the intervention group (p = 0.012) and from baseline to 6 months (p = 0.003) and to twelve months (p = 0.001) in the control group. The sum score for perceived competence for individual staff members regarding NPS increased from baseline to twelve months in the INH (p = 0.002). There was a reduction in the social interaction score for the CNH from baseline to 6 months (p = 0.009).

Table 5 Outcome measures on effectiveness at staff level and change in current practice at ward level (adoption). Mean scores and 95% confidence intervals (CI) before start of the intervention (Month 0), 6 and 12 months, and results of linear mixed models


Sixteen of the 17 INH completed the intervention. One INH, with one ward and seven residents included, withdrew from the trial approximately 2 weeks after the start of the intervention (i.e. after both the randomisation process and the training sessions). The reason given for this by the leading ward nurse, was an extremely high and prolonged level of sickness leave in the staff. It was not possible for her to assign someone from the staff to be interviewed for the assessments of the residents at eight and 12 weeks. Of the 22 wards in the INH that completed the study, all but two wards performed more than 75% of the components in TIME for their included residents as measured by the Fidelity Questionnaire. These two wards completed all or part of the first phase of TIME but performed no case conferences or evaluations of treatment actions. For both these wards the reasons given from the leaders for this reduced adoption of TIME, were lack of time and resources.

The INH and the CNH were not statistically different throughout the study period regarding adherence at ward level to recommended clinical practice for the assessment and treatment of NPS, as measured by the Current Practice Questionnaire (See Table 5). Both groups did however increase their scores from baseline to six and twelve weeks indicating better adherence to - and maintenance of - recommended clinical practice.


Overall, the staff performed 75% or more of the components in TIME for 91% of the included residents as measured by the Fidelity Questionnaire (i.e. a score ranging from 15 to 20). The INH performed a case conference in 91% of the included residents. The number of included residents with completed main components in TIME, is presented in Table 6. No nursing home performed more than one case conference for each resident during the 3 month trial. The average number of weeks that elapsed from intervention initiation (i.e. the randomisation date) to the performance of the case conferences was 5.0 (SD 1.8).

Table 6 The number of included residents with completed main components in TIME. Values are numbers (%)

The results from the documentary content analyses of the 84 min of the case conferences are illustrated in Fig. 1. The description of the problems analysed during the case conferences (i.e. description in detail of symptoms or behaviour) was performed adequately in 22 (26%) of the case conferences. For the description of the evaluation procedures this was done adequately in 49 (48%) of the case conferences. For the other structuring components of the case conferences, adequately understanding and use of them varied among the components from 69 (82%) to 84 (100%) of the minutes.

Fig. 1
figure 1

Documentary analysis of minutes from 84 case conferences. Notes: SMART, Specific, Measurable, Actual, Realistic, Time framed

Maintenance - facilitators and inhibitors to the implementation process

All but one of the 11 nursing homes represented in the five focus groups were still using the model 3 to 6 months after the end of the intervention. The frequency of the case conferences varied from once a week to once a month. The nursing home that had stopped to use the model, did so because they had moved all their residents to a new building. They considered re-implementing TIME as soon as they settled in. Several factors were considered when the focus groups discussed what promoted and what inhibited the maintenance of the model. These findings are summarised in Table 7.

Table 7 Findings from analyses of focus groups: Excerpt from the development of the theme: facilitators and barriers for maintenance

One important condition for using the model was that it was considered simple to learn, to understand, and to start using. Another condition was that they could see an immediate result of their efforts for the residents.

All the groups believed that the intervention had to be properly supported by the leading registered nurse. Time allocated and active support by the leader for assessment tasks was important. In addition, the staff thought that the leaders ought to actively give support in the case conferences and support for follow-up and evaluation procedures. It was deemed vital that as many staff members as possible could participate in the initial educational program. This was important for creating acceptance among all the staff for this way of working. The TIME administrator’s role was highlighted by both the staff and the leaders, and by themselves as essential for the implementation process. At the same time as they were an integrated part of the staff, they also felt a commitment to engage each other as a group, and the rest of the staff in the work to implement the different components of the model.

A theme that was recurring was the difficulty encountered in arranging and finding time for the case conferences. These difficulties were regarded as the main barrier to a proper implementation of the model. Here too, wards with an engaged leader who saw the benefits of the intervention, did better. The leaders in these wards were able to incorporate the case conferences as a part of the routines in the wards. The participants also reported better attendance at the case conferences in these wards. In other nursing homes, attending the case conference was voluntary for the staff who were not on duty. Some nursing homes did not provide compensation for the attendance on free time. Several participants attributed varying attendance at meetings and varying frequency of the case conferences to these factors and to the leader’s engagement.


Our results revealed a high degree of reach, adoption, and implementation of the intervention with TIME. Effectiveness at staff level measured by competence (i.e. general knowledge on NPS and skills), general attitudes towards dementia, mastery and social interaction at the working place, did not differ between the INH and the CNH. The analyses of the qualitative data showed that an easy to grasp model and a present and an engaged leadership are important facilitators for maintenance. We will discuss these results in the light of the RE-AIM dimensions and analyse to which extent these results can contribute to assumptions of causal mechanisms of the effectiveness of TIME at resident level.

Reach, adoption and implementation

The importance of aiming to train most staff (i.e. high degree of reach) was highlighted by Rapaport et al. (2017) in a systematic review on the effective components of psychosocial interventions [64]. In this review, staff perceived difficulties sharing new approaches with those who had not attended training sessions.

A high degree of adoption and implementation in our study can to some extent be explained by the finding that TIME was perceived by the staff as an easy to grasp model. To achieve acceptable adoption and implementation, several studies support the importance of interventions to be simple, appreciated by the staff, and not too overly complex [37,38,39].

Though implementation and adoption were high in our study, TIME was used with some variation between the study sites. The variation in the time the staff needed to perform the assessment of each resident, how long it took them to organise a case conference, and how many of the staff from each ward that attended case conferences, may in part be explained by organizational conditions. Allowing for some adaption of complex interventions to the organisational context of the settings is in line with viewing organisations (e.g. nursing homes) as complex systems. This flexibility of a complex intervention has been emphasized as an important factor to improve the effectiveness of the interventions by increasing their applicability [16, 35, 42].

The INH and the CNH did not differ throughout the study period regarding adherence to recommended clinical practice towards NPS as measured by the Current Practice Questionnaire (see Table 5). This can be due to several reasons. Firstly, participants in a research project could be affected by the so-called Hawthorne effect meaning that the participants change their behaviours only because they are monitored in a study [65]. Secondly, the questionnaire is based on self-reporting of practices by the leading ward nurses. Self-reporting of behaviour change in these settings is prone to bias [17]. It is possible that leaders from both groups gave more favourable answers than what are reflected in their practices. This could have offset any differences between the two groups. Thirdly, both groups had similar high scores at baseline, indicating that they already followed most of these recommendations providing less possibilities for improvement. Finally, both groups received lectures on NPS and recommended clinical practice for the approach towards NPS. Since both groups significantly increased their scores on this questionnaire throughout the study, this could indicate that both groups followed, to some degree, the recommendations from these lectures. As will be further discussed in the next paragraph, the effectiveness of TIME at resident level might to a lesser degree depend on the clinical practice routines.


Though there was an increased level of positive attitudes towards dementia in both groups, and an increased level of knowledge in the intervention group, there were no between-group differences in any measurements of effectiveness at individual staff level throughout the study period. These results are in line with the results in two recent systematic reviews on the impact of staff training on staff outcomes [40, 43]. However, in the same reviews some studies found increased staff knowledge and self-efficacy, but these findings were not consistently maintained over time. On the other hand, our recently published report exploring in depth the staff’s learning and coping experiences with TIME, showed that the staff perceived an increase in specific knowledge and coping related to each individual resident [32] . These changes were interpreted as a consequence of a new innovative learning process that took place during the interdisciplinary case conferences. This learning process created a new and shared situated knowledge about the resident and made it easier to customise the approach towards the individual resident. Most questionnaires assess mainly general attitudes and general knowledge about dementia [66]. These general attitudes and knowledge do not necessarily reflect the staff’s capability to translate this general competence into their real-world approach to an individual resident with, for example, severe agitation by which they are confronted on an everyday basis. The creation of a new, shared and situated knowledge about each resident might be one of the most important contributions to the effectiveness of TIME at staff level.

Maintenance - facilitators and inhibitors to the implementation process

Our data suggest that most of the nursing homes were still using TIME 3 to 6 months after the end of the intervention. Adherence to recommended clinical practice was sustained after 12 months. In accordance with our results from the focus groups, process evaluations of complex interventions in nursing homes highlights the importance of management support to achieve both adoption and sustainability [37,38,39,40, 43, 64, 67]. In the systematic review from Rapaport et al. (2017), a leadership attending training sessions and meetings, and allocating space and time for the staff to engage in the intervention, was conceived as one of the most important facilitators for a sustainable implementation [64]. Most of the leading registered nurses in the TIME trial attended the same educational training sessions as the rest of the staff and they also attended nearly half of the case conferences. The finding that the TIME administrators were important for the implementation process and acted as implementation leaders, is in line with other studies [68,69,70,71]. Scalzi et al. (2006) used the label “change champions” and regarded them one of the most important enablers for culture change [69].

As discussed for adoption and implementation, the staff expressed that TIME was an easy to grasp model. They emphasised that this feature had an impact on the possibility for continuing using the model. In the trial, after a short education and training programme, the TIME administrators in cooperation with the leading ward nurse carried out the intervention independently of the research team. This will increase the probability for maintenance of the intervention, reduce the cost for implementation and ease the proliferation of the model.

Turnover of care staff is often reported as a main barrier to sustainable interventions in nursing homes [64, 72]. However, turnover did not emerge as a subtheme in the focus group discussions when raising questions about barriers to the implementation process. The high degree of attendance to training sessions from all staff including those working part-time and with temporary positions, participation of the leading ward nurses in the training sessions, selecting three TIME-administrators among the nursing home staff as implementation champions and an easy-to-grasp model may have outweighed some of the effect of staff turnover.

Limitations and strengths

The main strength of this process evaluation is the rigorous use of a well-established and recommended framework for evaluation of complex interventions [42, 44]. The RE-AIM framework mainly emphasises quantification of the five dimensions reach, effectiveness, adoption, implementation, and maintenance to explore what is called the public health impact of the process of implementation [42, 44]. Less emphasis is put on the process of implementation. We have therefore in our study included issues regarding facilitators and barriers of the implementation process and considerations on causal assumptions of the effectiveness of the intervention [42].

The only interference during the randomised controlled trial was the use of the checklist for assessing performance of the main components of the intervention to explore implementation (fidelity). This checklist could have possibly served as a reminder for the TIME administrators.

During this process evaluation we have used a quasi-experimental design giving us the same quantitative data from the control condition as from the intervention group. Another strength is that we used a mixed-method design which allows us to compare and interpret quantitative results from the survey with findings from the perspectives of the staff, leaders, and physicians derived from the focus groups. This comparison adds to the validity of our results and to the value of our assumptions on possible causal mechanisms of the effect of TIME at resident level [41, 46].

One important limitation is that we do not know to which extent the model became a stable enduring part of the clinical routines of the nursing homes beyond 6 months after the end of the intervention.

One other limitation is that this process evaluation was performed by the same research team as the team responsible for the trial [42]. It was also performed when results from the trial were known, even though all the data collection for the process evaluation were planned for in a published protocol, and collected independently of the trial [18]. Both these phenomena could have created some bias in the analysis of the data. On the other hand, integrating process and outcomes’ evaluation by the same team, may limit the risk of one collection of data disturbing the other.

The relatively low response to some of the questionnaires is also an important limitation. One of the reasons for this is probably that the nursing home staff did not regularly use the e-mail account at work. We do not have data on the part of the staff who did not participate in the survey. This limits the external validity of the results. However, a recently published study comparing staff characteristics in special care units and regular units in Norway did find similar staff characteristics as in our sample [73].

The questionnaires to the staff members at 6 months and at 12 months were only administered to those of the staff wo had answered the questionnaires at baseline and at 6 months, respectively. Given the staff turnover within nursing homes, this procedure of data collection limits our possibility to draw conclusions on the impact of the intervention on the whole group of staff members throughout the trial and on an eventually pass on effect of the intervention among the staff members. Future process evaluations of interventions in care settings, should explore also this aspect of an intervention since this might be crucial for the intervention’s sustainability.

The successive reduction of responders at 6 months and at 12 months may also have influenced the results on the measured outcomes. It is possible that those who continued to respond are more positive to the intervention and in general more interested in professional development. However, the response rates were similar in both INH and CNH throughout the study period. Assuming that the characteristics of the staff that did not participate in the survey and their reasons for not participating are equally distributed in the two groups, the comparison between the two groups should, however, be judged as valid.

Two of the of questionnaires in the study were developed by the research team to be brief and to evaluate the implementation process inherent to TIME. With these questionnaires we wanted to evaluate the self-reported competence of the staff members and the clinical routines in the NH regarding the approach especially towards NPS, not towards dementia in general. Because of these requirements we did not find other equivalent questionnaires. They were not validated, and their psychometric properties are not known. This is an important limitation in this process evaluation study. However, since the questionnaires are carefully developed to reflect the content of the lectures and training sessions in both the INH and the CNH, we do believe they measure, at least to some extent, what the staff members eventually have learned from these lectures and training sessions in both the INH and the CNH. This is supported by the finding that perceived competence of the staff members increased significantly from baseline to 12 months in the INH, but not in the CNH. The Fidelity Questionnaire is a check-list for the performance of the main components in TIME, conducted by interviewing the TIME administrators by telephone, and is, therefore, less prone to self-reporting bias. There is still a concern about the accuracy of self-reporting of behaviours without checking with objective fidelity measures [17]. However, the documentary analysis of the meetings from the case conferences in our study contributed to a more valid judgement of the implementation (fidelity) dimension of the RE-AIM framework.


A high degree of reach, adoption, implementation, and maintenance have probably contributed to the effectiveness of TIME at resident level. Effectiveness measured by changes in general attitudes and competence at an individual staff level, and in clinical routines at ward level, did not reveal differences between the intervention nursing homes and the control nursing homes. An easy to grasp model and an engaged and a present leadership were the most important facilitators for maintenance. Lack of support from the leading ward nurse, and not integrating the case conferences as a part of the routines in the wards, were perceived as the main barriers to implementation and maintenance. Another causal assumption of the effect observed at resident level, is that TIME contributed to the development of a new, shared and situated knowledge in the staff about each individual resident and the actual NPS. Future process evaluations on complex interventions should explore methods for assessing how the staff’s general knowledge and attitudes are translated into their every-day approach to each individual resident.

Availability of data and materials

The quantitative data supporting the conclusions of this article are available on reasonable request to the first author. The transcribed interviews are due to confidentiality, not available.



Approaches to Dementia Questionnaire


behavioural and psychological symptoms in dementia


cognitive behavioural therapy


Clinical Dementia Rating Scale


Confidence Interval


Cohen-Mansfield Agitation Inventory


Control nursing homes


Cornell Scale of Depression in Dementia


General Medical Health Rating Scale


Intervention nursing homes


Mini-Mental State Examination


Mobilisation-Observation-Behaviour-Intensity-Dementia Scale


Medical Research Council


Neuropsychiatric Inventory-Nursing Home Version


Neuropsychiatric symptoms


Physical Self-Maintenance Scale


General Nordic Questionnaire for Psychological and Social Factors at Work




Standard deviation


Standard deviation




Statistical Product and Service Solution


Targeted Interdisciplinary Model for Evaluation and Treatment of Neuropsychiatric Symptoms


  1. Selbaek G, Engedal K, Benth JS, Bergh S. The course of neuropsychiatric symptoms in nursing-home patients with dementia over a 53-month follow-up period. Int Psychogeriatr. 2014;26(1):81–91.

    Article  Google Scholar 

  2. Selbaek G, Engedal K, Bergh S. The prevalence and course of neuropsychiatric symptoms in nursing home patients with dementia: a systematic review. J Am Med Dir Assoc. 2013;14(3):161–9.

    Article  Google Scholar 

  3. Wetzels RB, Zuidema SU, de Jonghe JF, Verhey FR, Koopmans RT. Course of neuropsychiatric symptoms in residents with dementia in nursing homes over 2-year period. Am J Geriatr Psychiatry. 2010;18(12):1054–65.

    Article  Google Scholar 

  4. Cummings J, Mintzer J, Brodaty H, Sano M, Banerjee S, Devanand DP, et al. Agitation in cognitive disorders: international psychogeriatric association provisional consensus clinical and research definition. Int Psychogeriatr. 2015;27(1):7–17.

    Article  Google Scholar 

  5. Brodaty H, Connors MH, Xu J, Woodward M, Ames D. Predictors of institutionalization in dementia: a three year longitudinal study. J Alzheimers Dis. 2014;40(1):221–6.

    Article  Google Scholar 

  6. Peters ME, Schwartz S, Han D, Rabins PV, Steinberg M, Tschanz JT, et al. Neuropsychiatric symptoms as predictors of progression to severe Alzheimer's dementia and death: the Cache County dementia progression study. Am J Psychiatry. 2015;172(5):460–5.

    Article  Google Scholar 

  7. Zeller A, Hahn S, Needham I, Kok G, Dassen T, Halfens RJ. Aggressive behavior of nursing home residents toward caregivers: a systematic literature review. Geriatr Nurs. 2009;30(3):174–87.

    Article  Google Scholar 

  8. Wang J, Yu JT, Wang HF, Meng XF, Wang C, Tan CC, et al. Pharmacological treatment of neuropsychiatric symptoms in Alzheimer's disease: a systematic review and meta-analysis. J Neurol Neurosurg Psychiatry. 2015;86(1):101–9.

    Article  Google Scholar 

  9. Kales HC, Gitlin LN, Lyketsos CG. Detroit expert panel on a, Management of Neuropsychiatric Symptoms of D. Management of neuropsychiatric symptoms of dementia in clinical settings: recommendations from a multidisciplinary expert panel. J Am Geriatr Soc. 2014;62(4):762–9.

    Article  Google Scholar 

  10. Brasure M, Jutkowitz E, Fuchs E, Nelson VA, Kane RA, Shippee T, et al. AHRQ comparative effectiveness reviews. Rockville (MD): Agency for Healthcare Research and Quality: US; 2016.

  11. Helsedirektoratet. Nasjonal faglig retningslinje om demens (Norwegian National Guideline on Dementia) Oslo: Helsedirektoratet (The Norwegian Directorate of Health); 2017 [updated 2017.10.07. Available from: Accessed 8 Mar 2019.

  12. Livingston G, Kelly L, Lewis-Holmes E, Baio G, Morris S, Patel N, et al. Non-pharmacological interventions for agitation in dementia: systematic review of randomised controlled trials. Br J Psychiatry. 2014;205(6):436–42.

    Article  Google Scholar 

  13. Zwijsen SA, Gerritsen DL, Eefsting JA, Smalbrugge M, Hertogh CM, Pot AM. Coming to grips with challenging behaviour: a cluster randomised controlled trial on the effects of a new care programme for challenging behaviour on burnout, job satisfaction and job demands of care staff on dementia special care units. Int J Nurs Stud. 2015;52(1):68–74.

    Article  CAS  Google Scholar 

  14. Livingston G, Barber J, Marston L, Stringer A, Panca M, Hunter R, et al. Clinical and cost-effectiveness of the managing agitation and raising quality of life (MARQUE) intervention for agitation in people with dementia in care homes: a single-blind, cluster-randomised controlled trial. Lancet Psychiatry. 2019.

  15. Medical Research Council. Developing and evaluating complex interventions - new guidance London. 2016 [Available from: Accessed 10 Mar 2017.

  16. Hawe P, Shiell A, Riley T. Complex interventions: how "out of control" can a randomised controlled trial be? BMJ. 2004;328(7455):1561–3.

    Article  Google Scholar 

  17. Surr CA, Gates C, Irving D, Oyebode J, Smith SJ, Parveen S, et al. Effective dementia education and training for the health and social care workforce: a systematic review of the literature. Rev Educ Res. 2017;87(5):966–1002.

    Article  Google Scholar 

  18. Lichtwarck B, Selbaek G, Kirkevold O, Rokstad AM, Benth JS, Myhre J, et al. TIME - targeted interdisciplinary model for evaluation and treatment of neuropsychiatric symptoms: protocol for an effectiveness-implementation cluster randomized hybrid trial. BMC Psychiatry. 2016;16(233):1–12.

    Google Scholar 

  19. Lichtwarck B, Selbaek G, Kirkevold O, Rokstad AMM, Benth JS, Lindstrom JC, et al. Targeted interdisciplinary model for evaluation and treatment of neuropsychiatric symptoms: a cluster randomized controlled trial. Am J Geriatr Psychiatry. In Press 2017.

  20. Glasgow RE, Lichtenstein E, Marcus AC. Why Don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. Am J Public Health. 2003;93(8):1261–7.

    Article  Google Scholar 

  21. Lichtwarck B, Tverå AM, Røen I. TIME - Targeted Interdiciplinary Model for Evaluation and treatment of neuropscychiatric symptoms - Manual 2nd Edition. Ottestad (Norway): The Research Centre for Age-related Functional Decline and Disease - Innlandet Hospital Trust; 2015. Available from: Accessed 30 Apr 2017.

  22. Beck JS. Cognitive behavior therapy: basics and beyond. New York: Guilford Press; 2011.

    Google Scholar 

  23. Kitwood T. Dementia reconsidered - the person comes first. Birmingham: Open University Press; 1997.

    Google Scholar 

  24. Alexopoulos GS, Abrams RC, Young RC, Shamoian CA. Cornell scale for depression in dementia. Biol Psychiatry. 1988;23(3):271–84.

    Article  CAS  Google Scholar 

  25. Barca ML, Engedal K, Selbaek G. A reliability and validity study of the Cornell scale among elderly inpatients, using various clinical criteria. Dement Geriatr Cogn Disord. 2010;29(5):438–47.

    Article  Google Scholar 

  26. Husebo BS, Ostelo R, Strand LI. The MOBID-2 pain scale: reliability and responsiveness to pain in patients with dementia. Eur J Pain. 2014;18(10):1419–30.

    Article  CAS  Google Scholar 

  27. Hughes CP, Berg L, Danziger WL, Coben LA, Martin RL. A new clinical scale for the staging of dementia. Br J Psychiatry. 1982;140:566–72.

    Article  CAS  Google Scholar 

  28. Folstein MF, Robins LN, Helzer JE. The mini-mental state examination. Arch Gen Psychiatry. 1983;40(7):812.

    Article  CAS  Google Scholar 

  29. Lawton MP, Brody EM. Assessment of older people: self-maintaining and instrumental activities of daily living. Gerontologist. 1969;9(3):179–86.

    Article  CAS  Google Scholar 

  30. David L. Consultation skills for the CBM. In: David L, editor. Using CBT in general practice: the 10 minute consultation. Bloxham: Royal College of general practitioners; 2006. p. 61–81.

    Google Scholar 

  31. Kales HC, Gitlin LN, Lyketsos CG. Assessment and management of behavioral and psychological symptoms of dementia. BMJ. 2015;350:h369.

    Article  Google Scholar 

  32. Lichtwarck B, Myhre J, Goyal AR, Rokstad AMM, Selbaek G, Kirkevold O, et al. Experiences of nursing home staff using the targeted interdisciplinary model for evaluation and treatment of neuropsychiatric symptoms (TIME) - a qualitative study. Aging Ment Health. 2018:1–10.

  33. Lichtwarck B, Tverå AM, Nystad HL. TID Tverrfaglig Intervensjonsmodell ved utfordrende atferd ved Demens Opplæringsfilm (Targeted Interdisciplinary Model for Evaluation and Treatment of Neuropsychiatric Symptoms - an educational film) Mo i Rana: Rana Kommune; 2013 [Available from: Accessed 30 Apr 2017.

  34. Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ: British Medical Journal. 2015;350:h2147.

    Article  Google Scholar 

  35. Richards DA, Hallberg IR. Complex interventions in health: an overview of research methods. New York: Routledge; 2015.

    Book  Google Scholar 

  36. Holle D, Kruger C, Halek M, Sirsch E, Bartholomeyczik S. Experiences of nursing staff using dementia-specific case conferences in nursing homes. Am J Alzheimers Dis Other Dement. 2015;30(3):228–37.

    Article  Google Scholar 

  37. Boersma P, van Weert JC, van Meijel B, Droes RM. Implementation of the Veder contact method in daily nursing home care for people with dementia: a process analysis according to the RE-AIM framework. J Clin Nurs. 2017;26(3–4):436–55.

    Article  Google Scholar 

  38. Chenoweth L, Jeon YH, Stein-Parbury J, Forbes I, Fleming R, Cook J, et al. PerCEN trial participant perspectives on the implementation and outcomes of person-centered dementia care and environments. Int Psychogeriatr. 2015;27(12):2045–57.

    Article  Google Scholar 

  39. Zwijsen SA, Smalbrugge M, Eefsting JA, Gerritsen DL, Hertogh CM, Pot AM. Grip on challenging behavior: process evaluation of the implementation of a care program. Trials. 2014;15:302.

    Article  Google Scholar 

  40. Spector A, Revolta C, Orrell M. The impact of staff training on staff outcomes in dementia care: a systematic review. Int J Geriatr Psychiatry. 2016;31(11):1172–87.

    Article  Google Scholar 

  41. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258.

    Article  Google Scholar 

  42. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions UK Medical Research Council (MRC) guidance: MRC Population Health Science Research Network; 2013 [Available from: Accessed 12 Dec 2017.

  43. Boersma P, van Weert JC, Lakerveld J, Droes RM. The art of successful implementation of psychosocial interventions in residential dementia care: a systematic review of the literature based on the RE-AIM framework. Int Psychogeriatr. 2015;27(1):19–35.

    Article  Google Scholar 

  44. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–7.

    Article  CAS  Google Scholar 

  45. Gaglio B, Shoup JA, Glasgow RE. The RE-AIM framework: a systematic review of use over time. Am J Public Health. 2013;103(6):e38–46.

    Article  Google Scholar 

  46. Creswell JW, Clark P. Choosing a mixed methods design. In: Creswell JW, Clark P, editors. Designing and conducting mixed methods research. Los Angeles: SAGE Publications,Inc.; 2011. p. 53–106.

    Google Scholar 

  47. Helse og omsorgsdepartementet (Ministry of Health and Care Services). Forskrift for sykehjem og boform for heldøgns omsorg og pleie (National regulations for nursing homes and residential care) 1998 [updated 21.06.2013. Available from: Accessed 24 June 2018.

  48. Murad MH, Katabi A, Benkhadra R, Montori VM. External validity, generalisability, applicability and directness: a brief primer. BMJ evidence-based medicine. 2018;23(1):17–9.

    Article  Google Scholar 

  49. Lichtwarck B, Selbaek G, Kirkevold Ø, Rokstad AM, Benth JS, Lindstrøm JC, et al. Targeted interdisciplinary model for evaluation and treatment of neuropsychiatric symptoms: a cluster randomized controlled trial. Am J Geriatr Psychiatry. 2018;26(1):25–38.

    Article  Google Scholar 

  50. Krueger RA, Casey MA. Answering questions about the quality of focus goups research. In: Krueger RA, Casey MA, editors. Focus groups: A practical guide for applied reserach. Los Angeles: Sage publications; 2015. p. 237–45.

    Google Scholar 

  51. Patton M. Designing qualitative studies. In: Patton M, editor. Qualitative research and evaluation methods. United States of America: Sage Publications, Inc; 2015.

    Google Scholar 

  52. Kada S, Nygaard HA, Mukesh BN, Geitung JT. Staff attitudes towards institutionalised dementia residents. J Clin Nurs. 2009;18(16):2383–92.

    Article  Google Scholar 

  53. Dallner M, Elo A-L, Gamberale F. Validation of the general Nordic questionnaire (QPSNordic) for psychological and social factors at work. Copenhagen: Nordic Council of Ministers; 2000.

    Google Scholar 

  54. Lintern T. Quality in dementia care: evaluating staff attitudes and behaviour. Bangor: University of Wales Bangor; 2001.

    Google Scholar 

  55. Wannstrom I, Peterson U, Asberg M, Nygren A, Gustavsson JP. Psychometric properties of scales in the general Nordic questionnaire for psychological and social factors at work (QPS): confirmatory factor analysis and prediction of certified long-term sickness absence. Scand J Psychol. 2009;50(3):231–44.

    Article  Google Scholar 

  56. Bing-Jonsson PC, Bjork IT, Hofoss D, Kirkevold M, Foss C. Competence in advanced older people nursing: development of 'nursing older people--competence evaluation tool. Int J Older People Nursing. 2015;10(1):59–72.

    Article  Google Scholar 

  57. Krueger RA, Casey MA. Focus groups: a practical guide for applied research. Los Angeles: Sage publications; 2015.

    Google Scholar 

  58. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.

    Article  Google Scholar 

  59. Vaismoradi M, Turunen H, Bondas T. Content analysis and thematic analysis: implications for conducting a qualitative descriptive study. Nurs Health Sci. 2013;15(3):398–405.

    Article  Google Scholar 

  60. Malterud K. Qualitative research: standards, challenges, and guidelines. Lancet. 2001;358(9280):483–8.

    Article  CAS  Google Scholar 

  61. Bowen GA. Document analysis as a qualitative research method. Qual Res J. 2009;9(2):27–40.

    Article  Google Scholar 

  62. Flick U. Thematic coding and content analysis. In: Flick U, editor. An introduction to qualitative research. London: SAGE Publications Inc.; 2014. p. 420–38.

    Google Scholar 

  63. Cummings JL. The neuropsychiatric inventory: assessing psychopathology in dementia patients. Neurology. 1997;48(5 Suppl 6):S10–6.

    Article  CAS  Google Scholar 

  64. Rapaport P, Livingston G, Murray J, Mulla A, Cooper C. Systematic review of the effective components of psychosocial interventions delivered by care home staff to people with dementia. BMJ Open. 2017;7(2):e014177.

    Article  Google Scholar 

  65. Nguyen VN, Miller C, Sunderland J, McGuiness W. Understanding the Hawthorne effect in wound research-a scoping review. Int Wound J. 2018;15(6):1010–24.

    Article  Google Scholar 

  66. Spector A, Hebditch M, Stoner CR, Gibbor L. A biopsychosocial vignette for case conceptualization in dementia (VIG-Dem): development and pilot study. Int Psychogeriatr. 2016;28(9):1471–80.

    Article  Google Scholar 

  67. Rokstad AM, Vatne S, Engedal K, Selbaek G. The role of leadership in the implementation of person-centred care using dementia care mapping: a study in three nursing homes. J Nurs Manag. 2015;23(1):15–26.

    Article  Google Scholar 

  68. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation science: IS. 2009;4:50.

    Article  Google Scholar 

  69. Scalzi CC, Evans LK, Barstow A, Hostvedt K. Barriers and enablers to changing organizational culture in nursing homes. Nurs Adm Q. 2006;30(4):368–72.

    Article  Google Scholar 

  70. Fossey J, Garrod L, Guzman A, Testad I. A qualitative analysis of trainer/coach experiences of changing care home practice in the well-being and health in dementia randomised control trial. Dementia (London, England) 2018:1471301218772178.

  71. Fossey J, Garrod L, Lawrence V, Testad I, Stafford J, Murray J. "we should see her like part of the team": an investigation into care home staff's experiences of being part of an RCT of a complex psychosocial intervention. Aging Ment Health. 2018:1–8.

  72. Beeber AS, Zimmerman S, Fletcher S, Mitchell CM, Gould E. Challenges and strategies for implementing and evaluating dementia care staff training in long-term care settings. Alzheimer’s Care Today. 2010;11(1):17–39.

    Google Scholar 

  73. Roen I, Kirkevold O, Testad I, Selbaek G, Engedal K, Bergh S. Person-centered care in Norwegian nursing homes and its relation to organizational factors and staff characteristics: a cross-sectional survey. Int Psychogeriatr. 2018;30(9):1279–90.

    Article  Google Scholar 

Download references


We would like to express our gratitude to the participants for sharing their experience and thoughts from their work in the nursing homes, and the nursing homes for their participation in the study.


The study is funded by a grant from the Innlandet Hospital Trust (Grant Number 150 333). The funding body did not take part in the design of the study, data collection, analysis, interpretation of data or in writing of the manuscript.

Author information

Authors and Affiliations



BL is the chief investigator of the TIME study and wrote the manuscript. BL, AMMR, SB, ØK, JSB and GS developed the study design. JSB is responsible for the statistical data management. JM and BL performed and transcribed the interviews, and JM, BL, and AMMR, performed the analysis of the interviews. JM coordinated the study. SB supervised the project. All authors read and approved the manuscript.

Corresponding author

Correspondence to Bjørn Lichtwarck.

Ethics declarations

Ethics approval and consent to participate

The Norwegian Centre for Research Data (NSD) and the Regional Committee for Medical and Health Research Ethics in eastern Norway (REC South East) approved the trial on 19 October 2015 (Project No.: 2015/1549). No residents were directly involved in the part of the study concerning results in this report. All the participants from the staff were provided with written information about the study. They gave written consent to conduct interviews and online consent to participate in the surveys.

Consent for publication

The participants consented to the publication of de-identified material from the interviews and the questionnaires.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

Questionnaires to the staff. (DOCX 30 kb)

Additional file 2:

Methods used for qualitative data collection and analysis. (DOCX 35 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lichtwarck, B., Myhre, J., Selbaek, G. et al. TIME to reduce agitation in persons with dementia in nursing homes. A process evaluation of a complex intervention. BMC Health Serv Res 19, 349 (2019).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: