Skip to main content

Development and evaluation of the content validity, practicability and feasibility of the Innovative dementia-oriented Assessment system for challenging behaviour in residents with dementia

Abstract

Background

One of the most difficult issues for care staff is the manifestation of challenging behaviour among residents with dementia. The first step in managing this type of behaviour is analysing its triggers. A structured assessment instrument can facilitate this process and may improve carers’ management of the situation. This paper describes the development of an instrument designed for this purpose and an evaluation of its content validity and its feasibility and practicability in nursing homes.

Methods

The development process and evaluation of the content validity were based on Lynn’s methodology (1998). A literature review (steps 1 + 2) provided the theoretical framework for the instrument and for item formation. Ten experts (step 3) evaluated the first version of the instrument (the Innovative dementia-oriented Assessment (IdA®)) regarding its relevance, clarity, meaningfulness and completeness; content validity indices at the scale-level (S-CVI) and item-level (I-CVI) were calculated. Health care workers (step 4) evaluated the second version in a workshop. Finally, the instrument was introduced to 17 units in 11 nursing homes in a field study (step 5), and 60 care staff members assessed its practicability and feasibility.

Results

The IdA® used the need-driven dementia-compromised behaviour (NDB) model as a theoretical framework. The literature review and expert-based panel supported the content validity of the IdA®. At the item level, 77% of the ratings had a CVI greater than or equal to 0.78. The majority of the question-ratings (84%, n = 154) and answer-ratings (69%, n = 122) showed valid results, with none below 0.50. The health care workers confirmed the understandability, completeness and plausibility of the IdA®. Steps 3 and 4 led to further item clarification. The carers in the study considered the instrument helpful for reflecting challenging behaviour and beneficial for the care of residents with dementia. Negative ratings referred to the time required and the lack of effect on residents´ behaviour.

Conclusions

There was strong evidence supporting the content validity of the IdA®. Despite the substantial length and time requirement, the instrument was considered helpful for analysing challenging behaviour. Thus, further research on the psychometric qualities, implementation aspects and effectiveness of the IdA® in understanding challenging behaviour is needed.

Peer Review reports

Background

The process of caring for people with dementia in nursing homes is complicated by their manifestation of behavioural and psychological symptoms, also referred to as challenging behaviours [1]. Challenging behaviours occur throughout all phases of the disease [2] and are highly prevalent in nursing homes (82%) [3]. In contrast with loss of memory, cognitive decline, and functional disability, challenging behaviours are more difficult to manage and are often perceived to be a burden by care staff [4, 5].

Studies that investigate patients’ perspectives on the disease commonly consider challenging behaviours to be a way of coping with dementia rather than a behavioural problem [6]. Research has indicated that challenging behaviours often emerge from an incongruence between a person’s needs and the degree to which the environment fulfils those needs [7]. Thus, challenging behaviours can be explained as the best response a person can provide given the limitations imposed by dementia, his or her abilities and elements of the environment [8]. It is highly important that challenging behaviours are not taken for granted; rather, they should be explored by others to discover the possible triggers and causes of these behaviours [6].

Understanding the underlying causes of challenging behaviours is both demanding and complex. Assessment tools designed to systematically guide nursing staff through this process can facilitate these complex behavioural analyses [9]. These instruments enable an objective description and evaluation of challenging behaviours and can provide insight into their potential triggers and causes [10].

At the time of this study, existing assessment instruments (e.g., Behavioural Pathology in Alzheimer’s Disease (Behave-AD) [11], Neuropsychiatric Inventory-Nursing home version (NPI-NH) [12], Cohen-Mansfield-Agitation-Inventory (CMAI) [13], Challenging Behaviour Scale (CBS) [14]) include topographical information about the behaviour (nature, intensity, and frequency) and a measure of its consequences (stress, safety), but they do not comprise information about its possible triggers. These instruments had objectives other than understanding a person’s challenging behaviour. No instruments for use in nursing homes have been developed that capture all the information needed to systematically guide nursing and care staff through the complex process of understanding residents’ challenging behaviours using a biomedical and psychosocial definition of this behaviour. However, several structured approaches for the analysis of challenging behaviour within nursing homes have now been developed [15,16,17,18,19,20] and these approaches include assessment instruments, charts or tools for the analysis of challenging behaviour [21]. Few approaches also address a behavioural analysis in the community setting [22,23,24,25]. Due to a lack of detailed information on these documents, a thorough comparison of these instruments is not possible. These instruments appear to be broadly targeted and address different topics, although the determination of medical and physical causes of challenging behaviour is prominent in most assessment instruments [19, 21]. The assessment instruments often require the expertise of professionals other than nurses (e.g., nursing home physicians or psychologists) [16,17,18,19], or the assessment process is led by a psychologist with a multidisciplinary team [15]. This situation makes it more difficult or even impossible to transfer the assessment instrument to countries, like Germany, where psychologists or nursing home physicians are not employed in nursing homes. In Germany, the nurses are the professionals who are responsible for resident’s care. Additionally, these instruments have objectives other than understanding a person’s challenging behaviour.

To address these shortcomings, an assessment tool called the “Innovative dementia-oriented Assessment system (IdA®)” was developed and validated for use in nursing practice. The objectives of this assessment instrument were as follows: (1) it should comprehensively assess the aspects relevant to evaluating challenging behaviour such as its topography, consequences and need for action; (2) it should collect information that helps identify the triggers and causes and better understand the behaviour; (3) it should include information relevant to providing nursing care for people with dementia; and (4) it should be applicable by care staff.

This paper presents the (1) development process and the (2) content validity of the IdA® as well as the (3) results of its first feasibility and practicability tests.

Methods

For the development and content validity of the new instrument, the approach described by Lynn [26] was used. This approach advocates two stages, in which stage I (development) results in the generation of the instrument’s items and stage II evaluates the performance of the instrument’s items (judgement and quantification). In this study, these two stages were supplemented by a third stage (evaluation), in which the feasibility and practicability of the new instrument were tested within an evaluation study (Fig. 1).

Fig. 1
figure 1

Stages of the development and validity and feasibility testing of the IdA®

Stage I. Development

The development of the IdA® consisted of two comprehensive literature reviews. The first review aimed to identify a conceptual framework for challenging behaviour and to identify the assessment’s components and items [26,27,28]. The second literature review focused on the specific behaviour of wandering and aimed to confirm the validity of the conceptual framework and of the specific components and items found in the first review. Both reviews were conducted in parallel and used an iterative process.

The first literature review was guided by following research questions:

  1. 1.

    Which conceptual framework is most suitable for assessing challenging behaviour and its causes in people with dementia who are living in nursing homes?

  2. 2.

    Which components of the conceptual framework are the most important for establishing the instrument’s dimensions?

  3. 3.

    Which items are the most important for representing/assessing the instrument’s dimensions?

The following research question guided the second literature search:

  1. 4.

    Are additional dimensions or items needed when considering a specific challenging behaviour?

The search process for both reviews has been described elsewhere [29].

Based on the results of both literature reviews, a first draft of the instrument and its items was developed (IdA® version 1.0).

Stage II. Judgement and quantification

The Judgement and Quantification phase included two evaluations. The first assessment consisted of an expert panel that focused on the relevance and clarity of the questions as well as the significance and completeness of responses in the IdA®. The second evaluation comprised a workshop with health care workers that focused on the understandability, completeness, plausibility and management of the IdA®.

1st evaluation: Expert panel

Experts in the expert panel were defined as individuals who worked in practice as a practitioner or in science in the field of dementia care, i.e., the criteria included being a nurse or being very familiar with nursing work and having an understanding of the field of assessment instruments. We focused mainly on nurses since nurses in Germany are responsible for the nursing process and thus for recognition and management of challenging behaviour. Fifteen experts were identified and contacted.

Measurement of content validity

As the IdA® was composed of questions and answers, a content validity index was calculated for both.

Each question was evaluated by rating a) its relevance to the instrument’s aim and b) its understandability/clarity. Each answer was assessed regarding its c) comprehensiveness/completeness and d) meaningfulness/significance for the related question. The four attributes (a-d) of the questions and answers were rated on a 4-point scale (1 = not relevant/not clear/incomplete/meaningless; 4 = highly relevant/clear/complete/meaningful) [26, 30]. In addition, the experts were asked to evaluate whether the items covered all important aspects or if there were missing components. The experts could also comment on every item.

The experts in the first expert panel also completed a questionnaire about their demographics and expertise.

Measurement of the content validity index

A content validity index was calculated both at the item level (I-CVI) and scale level (S-CVI) for all four attributes [26, 31]. The I-CVI was calculated as the number of experts providing a score of 3 or 4 divided by the total number of experts [26]. With more than 5 experts, the I-CVI should not be lower than 0.78 [31].

To calculate the S-CVI, two different indices were calculated: 1) the proportion of the items on one scale that the experts scored as valid (ratings of 3 or 4) (universal agreement by experts = S-CVI/UA) and 2) the average proportion of the items on one scale rated 3 or 4 (average agreement by experts = S-CVI/Ave) [31]. The S-CVI/UA is sensitive to the number of experts: the more experts are included, the greater the possibility of generating a low S-CVI. The S-CVI/Ave is more liberal and is preferred by Polit and Beck [31]; however, they recommend presenting both indices. The acceptable standard for the S-CVI/UA and the S-CVI-Ave is 0.8 [30, 31]; values up to 0.9 indicate an excellent average [21].

Additionally, a modified Kappa index was computed to estimate the I-CVI [32, 33]. The modified Kappa (k*) is an index of agreement among experts that indicates beyond chance that the item is relevant, clear, or another characteristic of interest [32]. We used the formula suggested by Polit et al. [32] (Table 5). The standards recommended by Fleiss [34] and Cicchetti and Sparrow [35] were used to interpret k*.

The results of the expert panel contributed to the second version of the instrument (IdA® 2.0).

2nd evaluation: Workshop with health care workers

According to Lynn [26], the same experts should re-evaluate the modified version of an instrument. Due to the comprehensiveness of the instrument and the limited resources of the first expert panel, a second round of evaluations with the same experts was not possible. Therefore, a second evaluation was organized with other experts, in which key persons from health care practice were invited to participate in a workshop. The nursing homes were free to select the key persons; the only inclusion criteria were that the persons were responsible for implementing the IdA® in this study, e.g., organizing the application of the IdA® and/or using it within the planned study, and that they had experience caring for people with dementia.

During the workshop, which lasted for several hours, the modified version of the IdA® (version 2.0) was introduced, and the objectives were explained. Subsequently, the experts were asked to assess the understandability, plausibility and completeness of the items. The discussion and suggested modifications were recorded, resulting in a further revised version of the IdA® (version 3.0).

Stage III: Evaluation

The third stage aimed to evaluate the feasibility of the IdA®, its usefulness in managing challenging behaviour and its practicability in daily work through an evaluation study. Data from a questionnaire administered in a before-after study were used to evaluate the feasibility and relevance of the IdA® in nursing practice.

Sample

A convenience sample of 11 nursing homes with 17 units participated in the evaluation study. Twelve were traditional care units and 5 were specialized care units for residents with dementia. All units had structures that enabled the use of the IdA® within a team context (e.g., team meetings, case conferences, nursing rounds). All care staff members were invited to the survey.

Application of the IdA®

In this study, the IdA® was expected to be used for 6 weeks with a minimum of 3 residents in one care unit (traditional and/or specialized). The IdA® was introduced to key persons on the units’ care staff team, who received a comprehensive one-day training that included lectures about dementia and challenging behaviour, guidelines about managing challenging behaviour and the IdA® dimensions and items. The remaining care staff team members received an one-hour training on how to use the IdA®.

The IdA® should be used in the study as a team instrument for care staff: the collection of information, the formulation of hypotheses about the causes and triggers of the behaviour and the development of the intervention plan should be conducted within the care staff team (e.g., team meetings, case conferences). Discussions and negotiation are necessary for an appropriate use of the IdA®.

Data collection and analysis

To evaluate the feasibility and relevance of the IdA®, self-developed questionnaires were used. The questionnaires included ratings about its structural aspects (15 items: clarity, meaningfulness, completeness, length, comprehensibility and background information required to use the IdA®); its benefit for daily care (8 items: professionalism and use of the IdA® as a memory aid, communication aid, planning aid, evaluation aid and information tool); carers’ experiences with the IdA® and an overall rating of the IdA®. An open question format was used to ask about missing information, problems with comprehension and other aspects. Finally, the caregivers were asked whether the IdA® supported them in decision making about residents’ challenging behaviour. The data were analysed using descriptive statistics (proportion and mean).

Results

Stage I: Development

The literature review identified a few models or approaches that aimed to explain the development of challenging behaviour in people with dementia (the multidimensional model [36], Progressively Lowered Stress Threshold model [37], Treatment Routes for Exploring Agitation (TREA) [38], Need driven dementia-compromised behaviour model (NDB model) [39], Serial Trial Intervention (STI) [40], Comprehensive model of behaviour [41], Antecedents-Behavior-Consequences (ABC)-approach [31]. In addition to the above mentioned approaches, further dementia-relevant psychosocial models exist [42]. These comprehensive and multidimensional models did not focus solely on behaviour but incorporated assumptions about the development of challenging behaviour in dementia.

The complexity of the behaviour and the influence of multiple determinants call for an explanation model that considers all aspects mentioned in the different approaches [33].

From the nursing perspective, the NDB model fulfilled all of the requirements for this study. It integrated the bio-medical and psychosocial perspective and provided a literature-based, comprehensible and comprehensive framework for assessing the causes of challenging behaviour. The origin of the model was in nursing science [39].

In this model, the causes of challenging behaviour are the unmet needs of people with dementia. They result from the increasing inability of these people to care for themselves, to interpret their needs as such and to communicate them in an understandable manner. The interaction between proximal (physiological and psychological aspects, characteristics of the environment) and background (e.g., neurological factors, cognitive abilities, health status, physical functioning, premorbid personality) factors is hypothesized to lead to the development of challenging behaviour. Background factors are stable characteristics of the person, with low changes over time, and act as risk factors and mediators for the influence of proximal factors on the development of challenging behaviour. Proximal factors are changeable aspects of the person with dementia or the direct environment and can directly produce challenging behaviour [39, 43]. For example, a person has a feeling of fear or experiences pain (proximal factors). The impairments in verbal communication (background factor) and a sensitive and fearful personality (background factor) determine how (through repetitive questioning “Hallo, Help, Hallo, Help”), in which situation, and how rapidly the feeling of anxiety evolves. The interplay between background and proximal factors determines how the person’s needs are communicated, expressed or satisfied. The literature on wandering did not produce new dimensions or aspects that were not covered by the NDB model. The results of the literature review provided substantial information about the priorities of the model and how to define and assess different triggers.

Additionally, other aspects were identified as relevant for documenting the behaviour itself and for deciding whether the behaviour was challenging or problematic and whether it required further consideration. These factors included a description of the type of behaviour, its quantity, context, burden and risk potential. The review showed a large number of scales and measurements, but none of them included all of the issues mentioned above. Thus, an assessment of all attributes of challenging behaviour was developed. The Cohen-Mansfield agitation model [44, 45] was identified as the one most commonly used in research to describe challenging behaviour. The underlying categorization of behaviour types established the theoretical foundation for the development of items in the IdA®. Apathy was added because the Cohens-Mansfield agitation model did not focus on this type of behaviour and apathy is one of the most prevalent challenging behaviours in nursing homes [3].

The first version (1.0) of the instrument consisted of 6 domains within 2 main parts: assessment of behaviour (part 1) and assessment of the causes and triggers of the behaviour (part 2). The behavioural assessment included 18 items, and part 2 was composed of 5 domains (health status & independence in everyday life, communication, personality & life style, moods & emotions, and environmental influences) with 8 dimensions (cognitive status, physical health status, physical discomfort, independence in everyday life, communication, personality and lifestyle before the onset of dementia, mood & emotions, and environmental influences). In total, 93 items were developed (Table 1). The items had different formats: closed-ended and open-ended. Examples are shown in Table 2.

Table 1 Different versions of the IdA®; results from stages I and II
Table 2 Examples of items in version 1.0

Stage 2: Judgement and quantification

1st evaluation: Expert panel

Sample

Ten out of 15 experts who were contacted completed the evaluation forms. The only reason for non-completion was a lack of time. The data confirmed that these persons had the required expertise to evaluate the instrument (Table 3). The experts were from a broad spectrum of disciplines; most of them had a double qualification, for example, as a nurse and a gerontologist. Almost all the experts had additional qualification in dementia-relevant aspects. All the experts had longstanding experience in the dementia field. Four of the experts were members of the group of experts who developed the German guidelines for challenging behaviours of residents living in nursing homes [46].

Table 3 Sample characteristics of the expert panel

Average S-CVI

The S-CVI was calculated for all 9 dimensions, resulting in 36 CVI indices for each calculation method (average and universal agreement).

The S-CVI/Ave ranged from 0.64 to 1.00. Six (16.7%) of the 36 indices were lower than 0.80, and 10 indices (28%) were lower than 0.90. The majority of the dimensions indicated content validity according to both cut-offs (83% and 72% for 0.80 and 0.90, respectively). The questions showed a better performance than the answers. All S-CVI/UA values except one were below the acceptable minimum of 0.80 (Table 4).

Table 4 S-CVI for the IdA® Version 1.0

Questions in the independence in everyday life dimension received the lowest relevance scores for S-CVI/Ave (S-CVI/Ave 0.81), followed by the environmental influences (S-CVI/Ave 0.85) and mood & emotions (S-CVI/Ave 0.87). The best score for the relevance of questions was for the dimension physical health status at 1.0, followed by cognitive status and personality & lifestyle before the onset of dementia (both S-CVI/Ave 0.95) (Table 4).

The average S-CVI for understandability ranged between 0.64 and 0.92. The questions on mood & emotions were the least understood (S-CVI/Ave 0.64). Personality & lifestyle before the onset of dementia (S-CVI/Ave 0.92) as well as communication (S-CVI/Ave 0.91) appeared to be best understood (Table 4).

The scores for meaningfulness of the answers ranged from 0.72 to 0.88. The cognitive status dimension received the fewest negative values (S-CVI/Ave 0.88) followed by the assessment of behaviour (S-CVI/Ave 0.87). The weakest dimension regarding the meaningfulness of answers was independence in everyday life (S-CVI/Ave 0.72) (Table 4).

The answers for 5 of 9 subject areas appeared to be incomplete (S-CVI/Ave below 0.8). S-CVI/Ave varied between 0.72 and 0.9. The dimension personality & lifestyle before the onset of dementia had the highest scores, and the subject areas independence in everyday life, cognitive status and environmental influences received the lowest scores.

Item -CVI

The experts scored 93 items regarding 4 attributes, two for each set of questions and answers, and thus a total of 360 CVI indices and k* were calculated. On the item level, 77% (276) of the ratings had a CVI greater than or equal to 0.78. The majority of the question-ratings (84%, n = 154) and answer-ratings (69%, n = 122) showed valid results. None of the ratings were below 0.50, which would indicate rejection of the item. Fifty-one complete items (question plus answer options) showed an I-CVI value less than 0.78 and k* less than 0.74 for at least one of the 4 rated attributes. Only 6 had I-CVI values ≤0.5 and k* < 0.4. The results for each item are shown in Tables 5, 6, 7, 8, 9, 10, 11, 12 and 13.

Table 5 Content validity of the dimension “Assessment of behaviour”
Table 6 Content validity of the dimension “Cognition status”
Table 7 Content validity of the dimension “Physical health status”
Table 8 Content validity of the dimension “Independence in everyday life”
Table 9 Content validity of the dimension “Physical discomfort”
Table 10 Content validity of the dimension “Communication”
Table 11 Content validity of the dimension “Personality & life style before the onset of dementia”
Table 12 Content validity of the dimension “Mood & emotions”
Table 13 Content validity of the dimension “Environmental influences”

For the challenging behaviour dimension, all 18 questions showed good validity regarding relevance and 16 regarding clarity. One item was considered fair and one poor in terms of clarity. For the answers, 16 of 18 items showed excellent validity in terms of meaningfulness and 14 in terms of completeness. Two answers were considered good regarding meaningfulness and 3 regarding completeness. One answer item showed fair validity in terms of completeness. Altogether, 5 complete items (question plus answer options, no. 12 and no. 17) needed modification (Table 5).

Six of 7 questions for the dimension cognitive status were considered excellent (relevance and clarity). Two items had good validity (no. 4 for relevance; no. 4a for clarity). One answer (no. 3) showed fair values for meaningfulness, and two items were considered good in terms of completeness (no. 1 and no. 4a). In total, 6 complete items (questions and answers) needed some modification (Table 6).

All questions and all answers for the physical health status dimension were considered relevant or meaningful (excellent validity). All questions but one showed excellent validity in terms of clarity (no. 5a showed good validity). Three of 4 answers were considered excellent in terms of completeness, and one showed good validity (no. 5a). Overall, only one complete item (5a) needed revising (Table 7).

Eleven of 13 questions for the dimension physical discomfort were considered excellent, one good and one fair regarding relevance. Ten questions were rated excellent, 2 good and one poor in terms of clarity. Ten of 12 answers were considered excellent, one good and one fair regarding meaningfulness, and 8 were rated excellent and 2 good for completeness. Altogether, 6 complete items needed modification (Table 8).

For the dimension independence in everyday life, 12 of the 17 questions were rated excellent regarding relevance, 3 good and 2 fair. Fifteen questions showed excellent validity in terms of clarity, one showed good validity, and one had fair validity. Only 7 answers were considered excellent in terms of meaningfulness, whereas 5 were good, 5 fair and one poor. Six answers were rated excellent for completeness, 10 good, and one fair. Overall, 14 of the 17 complete items needed some degree of modification (Table 9).

Five of 6 questions for the dimension communication were considered to have excellent validity in terms of relevance and clarity. One question showed good validity for relevance and one for clarity. Four answers had excellent ratings and two had good ratings for meaningfulness and completeness. In total, 2 complete items needed modification (Table 10).

For the dimension personality & lifestyle before the onset of dementia , all questions were considered relevant and clear (excellent validity). Additionally, all answers showed excellent values in terms of meaningfulness, and all but one in terms of completeness. One answer showed good validity and needed modification (Table 11).

The relevance of 6 of the 7 questions in the mood & emotions dimension were rated excellent and one good. In contrast, only one question showed excellent validity, two indicated good validity, two were fair and one poor in terms of clarity. For meaningfulness and completeness, 6 answers were rated excellent for both. One answer showed good validity for meaningfulness. One answer showed fair validity and one poor validity for completeness. Altogether, all of the items needed modification to different degrees (Table 12).

For the environmental influences dimension, 9 of 10 questions showed excellent, two good and one fair validity regarding relevance. Seven questions had excellent, two good and one fair validity values in terms of clarity. For meaningfulness, 7 answers were considered excellent and 3 good. For completeness, 5 answers were considered excellent, 4 good and 1 poor. Overall, 9 complete items had to be revised (Table 13).

The modifications to the instrument were made based on the CVI scores and the free text comments of the expert panel (Table 3). The changes were mainly related to a reduction in the questions and a specification and clarification of the answers. The items in part 1 that focused on the experiences and reactions of the social environment regarding challenging behaviour were the most criticized. These criticisms resulted in the rejection of two items and a merging of other items. In part 2 of the instrument (capturing the triggers of the behaviour), the subject area independence in everyday life was fundamentally revised. The main criticism was that the items were too rich in detail about care dependency and that the relation of the dimension to challenging behaviour was not clear. The experts recommended replacing the assessment of dependency with an assessment of the consequences or experiences of dependency resulting in the development of challenging behaviour. These consequences included the “stress/burden of the person with dementia due to being limited in self-care ability” and “stress/burden related to the care activities themselves”. Additionally, the subject area mood & emotions was further refined. The experts mentioned several times in the free text that because of the large number of questions, it seemed difficult to maintain an overview of all potentially important factors related to the development of challenging behaviour. Similarly, the objective of the assessment seemed to fade during the assessment process, and the reference to the challenging behaviour decreased. To maintain focus on the aim of the instrument during the assessment process and to illustrate the relationship between the items and challenging behaviour, overarching key questions were developed for all subject areas. These key questions introduced revised assessment dimensions (Table 14). The modified version of the IdA® (version 2.0) included 72 items and was presented for discussion during the subsequent workshop with the health care practitioners.

Table 14 Content of the IdA® and the guiding questions (Version 3)

2nd evaluation: Workshop with health care workers

Sample

Seventeen persons from 10 nursing homes participated in the workshop. Most of them were nurses, and a few had a geriatric background, were leaders of the unit or nursing home, or were quality managers. No further information about the participants was collected.

Results

The discussion with the health care workers was very helpful for determining how the items were understood and interpreted. The language was simplified based on the results of the workshop. The subject areas “communication” and “independence in everyday life” again required further clarification. Some items were perceived to be too broad; the participants expressed a need for more differentiated and detailed information. Difficulty summarizing the large amount of information obtained was the strongest piece of criticism. Two main points were discussed. First, the requirements needed to answer the items required further consideration. Some items could only be answered with more in-depth observations of the person with dementia, e.g., their cognitive abilities, or further information was needed from family members or doctors. For example, the domain “Personality & lifestyle before the onset of dementia” contained questions that could not be answered without knowing the person well and for a long period of time. The nursing home care staff was often dependent on information provided by family members or significant others, if present. In some cases, these informants were not available, and the necessary information to answer the IdA® questions could not be obtained. Other items led to actions such as speaking with the occupational therapist about activities or a change in care routines. Second, the participants wanted to have an overview of all relevant aspects that could be associated with the challenging behaviour in the particular individual. To address the first criticism, each item received a new assessment criterion called “what has to be done?”. This question included three selection options: a) further clarification needed, b) action needed to plan, and c) item remains important. Referring to the second point of criticism, key questions with the corresponding significant content were presented on a separate page through bullet points. These modifications resulted in version 3.0 of the IdA®, which contained 7 domains with 89 items and three parts.

Stage III evaluation

Sample

Sixty of the 229 questionnaires were returned. One ward unit withdrew from the study because of being transferred from one location to another. The response rate varied considerably between nursing homes (7% to 58%). The reasons for non-response included holiday time, illness, and a lack of time. The respondents’ characteristics are shown in Table 15. The characteristics of the sample reflect the population of the care staff in German nursing homes.

Table 15 Characteristics of the sample in the evaluation study

Results

Structure of the IdA®

The majority of the respondents positively rated 9 of the 13 structural aspects of the IdA®. Three aspects were rated positively by 40% to 50% of the respondents. However, 71% of the respondents rated the time aspect negatively. The IdA® did not have to be applied by registered nurses only (67%), but the interpretation of information from the IdA® required broad knowledge of dementia (Table 16).

Table 16 Evaluation of the structural aspects of the IdA®

The detailed ratings of each domain of the IdA® showed that the majority of the respondents provided mostly positive assessments of the completeness (71–80%) of the information, the relevance of the information for daily care (64–76%) and the contribution of the IdA® domains for internal (71–82%) and external communication (59–67%).

Benefits of the IdA® for daily care

Six aspects, asked with 8 items, pertained to the benefit of the IdA® for the daily care of residents with dementia. The majority of the respondents (min. 50%) rated six items positively. However, most of the respondents perceived the IdA® as not capable of indicating quality of care and as less supportive of daily communication with other health care workers (Table 17).

Table 17 Evaluation of the benefit of the IdA® for daily care of residents with dementia and challenging behaviour

The overall rating of the IdA® was positive. Two of three respondents (64%) rated the IdA® as good to very good. When asked “Would you continue using the assessment tool after the testing phase?”, 28 of 47 (56%) reported that they would use the IdA®, whereas 19 said they would not. If the IdA® continued to be used together in a team, 84% of the users said that they would keep using the tool.

The IdA® as a tool for decision making

For decision making regarding the causes of challenging behaviour, it was important for the care staff to have individualized information about the residents. In total, 74% of the respondents confirmed the statement “The IdA® gives me the opportunity to express the individuality of the resident with dementia”. The IdA® seemed to mostly stimulate reflection on the residents’ behaviour. The IdA® included aspects related to behaviour that the majority of the respondents had not considered before; furthermore, it helped describe the residents’ behaviour to others and to provide another perspective on the behaviour. Two-thirds of the respondents thought that the IdA® was relevant to their daily work. Regarding the statements of whether the IdA® had changed their own behaviour, helped plan interventions, provided unknown information or improved understanding of the behaviour, between 44% and 47% of the respondents agreed with the statements. Unfortunately, the respondents did not experience an improvement in the behaviour of the residents (Table 18).

Table 18 Benefits of the IdA® for decision making

Discussion

The purpose of this study was to develop a nursing assessment of the triggers and causes of challenging behaviour among residents with dementia as a diagnostic guideline and to assess its content validity, feasibility and practicability. We used the approach suggested by Lynn [26], which is broadly applied in health care and nursing research [47,48,49]. Although Beckstead [50] questioned the meaning of content validity for establishing the validity of an instrument, the two-stage approach is widely accepted in the methodological literature and is considered a requirement for the development of new instruments [31, 51,52,53]. The main criticism refers to the calculation of the agreement indices and the significance of the number of experts for the risk of error. We used the modified kappa as an index that adjusts for chance agreement. This index provides information about the degree of agreement beyond chance. Questions concerning whether the use of other approaches, such as multirater kappa, would result in more correct agreement cannot be answered here. However, the agreement indices are only one part of the process of assessing content validity and should not be the only reason for the rejection or modification of items. The comments of the experts helped form judgements about the type of problems that existed for specific items. Thus, CVI together with modified kappa and written comments of the experts added rigour to the validation process of the content of IdA®.

Content validity expresses the representation of current available knowledge in the construct of interest [28]. Together with face validity, content validity represents the minimum quality requirement for an instrument. However, despite its limited value within the validity hierarchy [50, 54], content validity is an important quality indicator of an instrument’s validity and provides insight into its feasibility and practicability [32, 51, 54]. In this study, the development process of the instrument and the evaluation of its content validity supported the validity of the IdA® and provided a strong foundation from which to begin further examination of its validity and reliability.

To assess the content validity of the IdA®, it was important to assess different attributes of both the questions and the answer options. The experts perceived the majority of the questions as relevant; however, more weaknesses with the answer options were identified. The experts’ judgements together with the comments provided detailed information about the strengths and weaknesses of each item and led to well-reasoned modifications.

The decision to assess more than only the relevance of an item was shown to be very helpful in the development process. The experts distinguished between the relevance of the content (particular trigger or cause of behaviour) and the clarity of the wording. Most of the questions were considered relevant, but the phrasing was criticized. This differentiation facilitated the modifications and improved their comprehensibility. The responses received more negative criticisms than the questions. For example, the relevance of the item asking about pain was confirmed, and the answer options were meaningful. However, in the experts´ opinion, some important aspects relevant to pain assessment were not considered. Additionally, in some cases, the written comments indicated that there were problems with comprehension – the intended content was somewhat misunderstood. This information led to a more precise phrasing of the item.

The importance of the competency of the expert is crucial. Very different factors can be used to confirm a person as an expert, and there are no rules on how to define an expert. Usually, an expert is defined as a person who represents the content of interest in science or practice. In the context of assessment instruments, knowledge about the methodology of assessment is very helpful, and input from stakeholders from additional application fields can be useful [52]. In the first expert panel of this study, we focused on experiences and knowledge in dementia care, in nursing care in the context of nursing homes and in using or developing assessment instruments. The strong focus on nursing was requested, as we expected that those experts would help to develop an instrument with content that is meaningful, understandable and practicable for nurses and the nursing field. We could not clarify the extent to which the narrow focus on nursing field influenced the rating of relevance of the IdA® items and dimensions in favour of more physical and ADL topic than psycho-social aspects. We assume a small risk of bias since most the experts had additional qualifications and professions beyond nursing.

The disadvantage of the development process was the time and effort needed to assess the substantial number of items, as the process could not be replicated after the modifications suggested by the first expert panel had been made. Because the second review process occurred through a workshop with health care workers, individual ratings of the assessment items were not feasible. Thus, the effect of modifying the IdA® items on the CVI obtained in the first assessment could not be quantified, which is a limitation of the study.

The health care practitioner’s comments and discussion contributed significantly to the item clarity and adaptability to nursing practice. In this second modification, aspects such as problems obtaining the information needed to answer the questions were strongly discussed. For example, the domain “Personality & lifestyle before the onset of dementia” contained questions that could not be answered without knowing the person well and for a long period of time. The nursing home care staff was often dependent on information provided by family members or significant others, if present. In some cases, these informants were not available, and the information needed to answer the IdA® questions could not be obtained. This is not a problem specific to IdA® but is a general problem of assessments of complex and multifaceted information. The creation of a professional environment in which the observation, collection, communication, reflection and appraisal of essential information is an obvious part of the nursing process can overcome this challenge. IdA® should not be the start of this process, but it supports this process for a specific topic (dementia or challenging behaviour) and in a specific phase of this process (mainly complementary information collection, communication, reflection and appraisal of information). Although the CVI could not be calculated in this second expert round, the advantage of this round was the clear practice perspective, the simplification of the wording and the willingness of the carers to collect, provide and share information.

The majority of the IdA® users reported that the tools were well structured, clear, complete and comprehensive. The content appeared to be ambiguous. For almost 60% of the users, the relevance of the IdA® information for dementia care seemed to be unclear. This finding was consistent with the evaluation that the IdA® required comprehensive knowledge about dementia and challenging behaviour. This feedback was important for the future implementation of the IdA®. The findings highlighted the insufficient knowledge of the staff working in nursing homes about the triggers of challenging behaviour. Particular emphasis should be placed on expressing the relationships between the IdA® variables and their relevance to the development of challenging behaviour. As previously found, the qualification of the staff caring for people with dementia is known to influence the effectiveness of dementia-specific approaches [55]. However, how the staff experiences challenging behaviour and how they place this behaviour in context also influence their understanding of causes and triggers. In their study, Dupuis et al. [56] showed a complex process of interpretation through which nurses use several filters. The lens of pathology and the influence of cognition on the construction of meaning of challenging behaviour are very dominant. Thus, it is important that nursing home staff are supported in developing more person-centred attitudes towards persons showing challenging behaviour. The combination of evidence-based knowledge about triggers, supported by IdA®, and the reinforcement of dementia-friendly attitudes are a promising basis for the best care for those people.

The IdA® was time consuming because it covered a wide range of information on potential causes, and these causes are multifactorial. The care staff needed time to answer the questions, gather any missing information, discuss potential hypotheses on the causes of behaviours, and then plan interventions. This process required a substantial amount of time at a point when nursing care is characterized by a lack of time [57]. This conflict could not be resolved, as the broadness of the themes and items was necessary to trace the triggers and causes of behaviour [58]. Changes in routine processes, changes in the prioritization of tasks and facilitation of a professional self-understanding that incorporates analytic thinking processes into nursing tasks could help address this conflict. An important form of preparation for the implementation of IdA® is to analyse current practices of nursing assessment and documentation. Most of the information needed for IdA® already exists. Nurses should ensure that the existing information is valid and can be easily linked with the questions on IdA®. This process has the potential to save time. We also expect that the frequent use of IdA® will result in better management of the instrument and better prioritization of the parts of IdA® relevant to a particular person with dementia. We analysed this aspect in a recently completed intervention study [21, 59].

The IdA® should be viewed as a tool that offers support and assistance for carers in understanding the challenging behaviour exhibited by persons with dementia. This tool provides guidelines for navigating a complex network of widely varying potential causes and triggers of these behaviours. It is important to mention that the IdA® does not provide exact solutions for the problem but rather helps generate a hypothesis (one or more) about the causes of challenging behaviour. This type of support is important in Germany, as nurses act practically independently in nursing homes. In Germany, a general practitioner (GP) spends an average of less than 2 h peer week in nursing homes, caring for 20 residents during this time period [60]. Because of the limited time and the high number of residents with challenging behaviours, contact with the GP is mainly restricted to acute or severe situations and focuses mainly on medical and pharmacological questions [61]. Bartholomeyczik et al. [15] found that nurses are only moderately satisfied with GP consultation times for residents with dementia, and only 13% of GPs wanted to participate in case conferences very often or often. In that study, only one GP participated and contributed to one case conference. Consultations with psychologists are unusual. This situation differs from the situation in the Netherlands, for example, where an interdisciplinary team is responsible for managing challenging behaviour [16], and in the UK, where the psychologist has a key role in this context [17].

The IdA® asks for a substantial amount of information about the person with dementia, and therefore different sources of information are important to obtaining a comprehensive understanding of the person, e.g., reports from different shift workers and carers, family and friends, therapists and persons with dementia themselves. Hypotheses regarding the causes or triggers of a behaviour are not formed as a result of a deduction of “hard facts” but rather a construction of plausible and preliminary presumptions. The effects of the action deduced from the hypotheses verify the accuracy of these estimates. Discussing the pros and cons of a hypothesis is therefore necessary to properly apply the IdA®. Favourable opportunities for these discussions include team meetings or case conferences [22]. The initial experiences in which the IdA® was used during case conferences in nursing homes showed that the care staff benefitted from the exchange of individual views and experiences. They were able to realize that residents showed different behaviours with different staff members, and this realization facilitated the recognition of helpful and unhelpful approaches to the interactions between residents and staff. Self-reflection was supported. Residents´ behaviour was no longer considered a personal attack but could be perceived from a broader context. The IdA® helped to more precisely describe the behaviour of residents with dementia as well as the circumstances of behaviour and supported communication with physicians. Through the IdA®, gaps in knowledge regarding the residents became apparent [55]. Thus, the IdA® has the potential to support carers’ self-efficacy and thus reduce their burden in managing challenging behaviour [18]. The effectiveness of using the IdA® within case conferences is currently being examined in a cluster randomized controlled trial in Germany [21, 59].

The IdA® is not a typical measurement instrument that can be easily tested using all routine psychometric properties (e.g., inter-rater reliability, criterion validity, structure analysis). The outcome of the IdA® is not a risk status or physical status that can be prospectively validated or compared to a gold standard. The results of the IdA® as a whole include the formulation of hypotheses about the relationship between the information obtained from the IdA® and challenging behaviour. The “truth” can be verified only if the derived conclusions (hypothesis) lead to the expected effects. The rejection or confirmation of the hypothesis also depends on the choice of the right intervention (e.g., ways to communicate or changes in medication or enjoyable activities) and on the correct execution of the intervention. This requires knowledge about effective interventions and skills and technique in delivering the interventions.

In addition, the effects in each person and situation could differ: challenging behaviour could be reduced or stabilized, the right interactions could be implemented when the behaviour occurs, the behaviour could be integrated into daily living in the nursing home unit, the burden could be relieved for persons in the patient’s environment, the quality of life of the person showing challenging behaviour could be maintained or the challenging behaviour could be better understood or re-interpreted. When the conclusion does not lead to the desired effects, the reasons can vary from incorrect hypotheses, incorrect interventions, or both; incorrect, missing or unimportant items on the IdA®; or a lack of availability of significant information. The evaluation of reliability also has some challenges. For IdA®, as a team instrument, the evaluation of inter-rater reliability is less meaningful, except for some specific aspects such as pain, cognitive status or some behavioural characteristics. A form of intergroup reliability could be an alternative, but the time needed for conducting repeated team measurements should be taken into account. Evaluation of test-retest for the formulated hypotheses could make more sense and help to overcome the previously discussed problems with the interpretation of validity.

Limitations

The study has several limitations. Due to the many items included in the IdA®, it was not possible to conduct additional expert rounds, as suggested by Lynn [26], or to calculate CVIs after each evaluation round. Thus, whether the modifications led to improvements in the content validity remains unknown. Content validity is the weakest form of validity, and therefore this study is a first step in establishing the validity of the IdA®.

We used the CVI and modified kappa for the assessment of agreement, the merit of which has been criticized. Here, we do not know whether e.g. the multirater kappa statistic would provide more valid agreement indices. Therefore, different indices were used to decide the item modifications.

We included in the development process mainly professionals from the nursing field, since our priority was an instrument that covers information relevant to nursing care and can be managed by nurses. The involvement of other disciplines might influence the content and the content validity evaluation of IdA®.

The size of the care staff sample was small, and thus the interpretation of the results was limited.

Additionally, the delivery time was very short (6 weeks); this brief duration might have influenced the correct application of the IdA® and led to over- or underestimations of the IdA® quality. Conclusions regarding the experiences with the IdA® over time cannot be made.

Conclusion

The IdA® is a nursing assessment tool that focuses on the triggers and causes of challenging behaviour. The comprehensive literature review, strong theoretical foundation, and the two evaluations representing specific content expertise as well as practical perspectives contribute to the content validity of the IdA®. The IdA® is recommended to be used as a team instrument, as the information collected with the IdA® requires observations from different caregivers, situations and time periods to create a comprehensive understanding of the persons exhibiting challenging behaviour.

References

  1. Moniz Cook ED, Swift K, James I, Malouf R, De Vugt M, Verhey F. Functional analysis-based interventions for challenging behaviour in dementia. Cochrane Database Syst Rev. 2012;2012:CD006929.

    Google Scholar 

  2. Wetzels R, Zuidema S, Jansen I, Verhey F, Koopmans R. Course of neuropsychiatric symptoms in residents with dementia in long-term care institutions: a systematic review. Int Psychogeriatr. 2010;22:1040–53.

    Article  PubMed  Google Scholar 

  3. Selbaek G, Engedal K, Bergh S. The prevalence and course of neuropsychiatric symptoms in nursing home patients with dementia: a systematic review. J Am Med Dir Assoc. 2013;14:161–9.

    Article  PubMed  Google Scholar 

  4. Schmidt KH, Diestel S. Job demands and personal resources in their relations to indicators of job strain among nurses for older people. J Adv Nurs. 2013;69:2185–95.

    Article  PubMed  Google Scholar 

  5. Edvardsson D, Sandman PO, Nay R, Karlsson S. Associations between the working characteristics of nursing staff and the prevalence of behavioral symptoms in people with dementia in residential care. Int Psychogeriatr. 2008;20:764–76.

    Article  PubMed  Google Scholar 

  6. Vasse E, Vernooij-Dassen M. Psychosocial interventions in dementia care. In: McNamara P, editor. Dementia - treatments and developments. Santa Barbara: Praeger; 2011. p. 151–71.

    Google Scholar 

  7. Cohen-Mansfield J, Golander H, Ben-Israel J, Garfinkel D. The meanings of delusions in dementia: a preliminary study. Psychiatry Res. 2011;189:97–104.

    Article  PubMed  Google Scholar 

  8. Bedard A, Landreville P, Voyer P, Verreault R, Vezina J. Reducing verbal agitation in people with dementia: evaluation of an intervention based on the satisfaction of basic needs. Aging Ment Health. 2011;15:855–65.

    Article  PubMed  Google Scholar 

  9. Koopmans RT, Zuidema SU, Leontjevas R, Gerritsen DL. Comprehensive assessment of depression and behavioral problems in long-term care. Int Psychogeriatr. 2010;22:1054–62.

    Article  PubMed  Google Scholar 

  10. Bartholomeyczik S. Some critical remarks on standardised assessment instruments in nursing. Pflege. 2007;20:211–7.

    Article  PubMed  Google Scholar 

  11. Reisberg B, Borenstein J, Salob SP, Ferris SH, Franssen E, Georgotas A. Behavioral symptoms in Alzheimer's disease: phenomenology and treatment. J Clin Psychiatry. 1987;48(Suppl):9–15.

    PubMed  Google Scholar 

  12. Wood S, Cummings JL, Hsu MA, Barclay T, Wheatley MV, Yarema KT, et al. The use of the neuropsychiatric inventory in nursing home residents. Characterization and measurement. Am J Geriatr Psychiatry. 2000;8:75–83.

    Article  CAS  PubMed  Google Scholar 

  13. Cohen-Mansfield J, Marx MS, Rosenthal AS. A description of agitation in a nursing home. J Gerontol. 1989;44:M77–84.

    Article  CAS  PubMed  Google Scholar 

  14. Moniz-Cook E, Woods R, Gardiner E, Silver M, Agar S. The challenging behaviour scale (CBS): development of a scale for staff caring for older people in residential and nursing homes. Br J Clin Psychol. 2001;40:309–22.

    Article  CAS  PubMed  Google Scholar 

  15. Bartholomeyczik S, Holle D, Halek M. Herausforderndes Verhalten bei Menschen mit Demenz verstehen - Die Verbesserung der Versorgung Demenzkranker durch Qualitätsinstrumente. Weinheim und Basel: Beltz Juventa Verlag; 2013.

  16. Zwijsen SA, Smalbrugge M, Eefsting JA, Twisk JW, Gerritsen DL, Pot AM, et al. Coming to grips with challenging behavior: a cluster randomized controlled trial on the effects of a multidisciplinary care program for challenging behavior in dementia. J Am Med Dir Assoc. 2014;15:531.e1–10.

    Article  Google Scholar 

  17. Jackman LJ, Wood-Mitchell A, James IA. Micro-skills of group formulations in care settings. Dementia (London). 2014;13:23–32.

    Article  Google Scholar 

  18. Holle D, Halek M, Mayer H, Bartholomeyczik S. The influence of understanding diagnostics on perceived stress of nurses caring for nursing home residents with dementia. Pflege. 2011;24:303–16.

    Article  PubMed  Google Scholar 

  19. Lichtwarck B, Selbaek G, Kirkevold Ø, Rokstad AMM, Benth JŠ, Myhre J, et al. TIME–ttargeted interdisciplinary model for evaluation and treatment of neuropsychiatric symptoms: protocol for an effectiveness-implementation cluster randomized hybrid trial. BMC Psychiatry. 2016;16:233.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Turner J, Snowdon J. An innovative approach to behavioral assessment and intervention in residential care: a service evaluation. Clin Gerontol. 2009;32:260–75.

    Article  Google Scholar 

  21. Holle D, Roes M, Buscher I, Reuther S, Muller R, Halek M. Process evaluation of the implementation of dementia-specific case conferences in nursing homes (FallDem): study protocol for a randomized controlled trial. Trials. 2014;15:485.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Reuther S, Dichter MN, Buscher I, Vollmar HC, Holle D, Bartholomeyczik S, et al. Case conferences as interventions dealing with the challenging behavior of people with dementia in nursing homes: a systematic review. Int Psychogeriatr. 2012;24:1891–903.

    Article  PubMed  Google Scholar 

  23. Gitlin LN, Winter L, Dennis MP, Hauck WW. A non-pharmacological intervention to manage behavioral and psychological symptoms of dementia and reduce caregiver distress: design and methods of project ACT3. Clin Interv In Aging. 2007;2:695.

    Article  Google Scholar 

  24. Gitlin LN, Winter L, Dennis MP, Hodgson N, Hauck WW. Targeting and managing behavioral symptoms in individuals with dementia: a randomized trial of a nonpharmacological intervention. J Am Geriatr Soc. 2010;58:1465–74.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Desai AK, Desai FG. Management of behavioral and psychological symptoms of dementia. Curr Geriatr Rep. 2014;3:259–72.

    Article  Google Scholar 

  26. Lynn MR. Determination and quantification of content validity. Nurs Res. 1986;35:382–5.

    Article  CAS  PubMed  Google Scholar 

  27. DeVet H, Terwee C, Mokkink L, Knol D. Measurement in medicine: a practical guide. Cambridge: Cambridge University Press; 2011.

    Book  Google Scholar 

  28. Streiner DL, Norman GR, Cairney J. Health measurement scales. A practical guide to their development and use. Oxford: Oxford University Press; 2014.

    Google Scholar 

  29. Halek M, Bartholomeyczik S. Description of the behaviour of wandering in people with dementia living in nursing homes - a review of the literature. Scand J Caring Sci. 2012;26:404–13.

    Article  PubMed  Google Scholar 

  30. Waltz C, Stickland OL, Lenz E, Soeken KL. Validity of measures. In: Waltz C, Stickland O, Lenz E, editors. Measurement in nursing and health research. New York: Springer Publishing Company; 2005. p. 154–89.

    Google Scholar 

  31. Polit DF, Beck CT. The content validity index: are you sure you know what's being reported? Critique and recommendations Res Nurs Health. 2006;29:489–97.

    Article  PubMed  Google Scholar 

  32. Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations Res Nurs Health. 2007;30:459–67.

    Article  PubMed  Google Scholar 

  33. Wynd CA, Schmidt B, Schaefer MA. Two quantitative approaches for estimating content validity. West J Nurs Res. 2003;25:508–18.

    Article  PubMed  Google Scholar 

  34. Fleiss J. Statistical methods for rates and proportions. New York: John Wiley & Sons; 1981.

    Google Scholar 

  35. Cicchetti D, Sparrow S. Developing criteria for establishing interrater reliability of specific items: application to assessment of adaptive behaviour. Am J Ment Defic. 1981;86:127–37.

    CAS  PubMed  Google Scholar 

  36. Kunik M, Martinez M, Snow LA, Beck CK, Cody M, Rapp CG, et al. Determinants of behavioral symptoms in dementia patients. Clin Gerontol. 2003;26:83–9.

    Article  Google Scholar 

  37. Smith M, Gerdner LA, Hall GR, Buckwalter KC. History, development, and future of the progressively lowered stress threshold: a conceptual model for dementia care. J Am Geriatr Soc. 2004;52:1755–60.

    Article  PubMed  Google Scholar 

  38. Cohen-Mansfield J. Nonpharmacological management of behavioral problems in persons with dementia: the TREA model. Alzheimer’s Quarterly. 2000;1:22–34.

    Google Scholar 

  39. Algase DL, Beck CK, Kolanowski AM, Whall AL, Berent S, Richards KC, et al. Need-driven dementia-compromised behavior: an alternative view of disruptive behavior. Am J Alzheimers Dis Other Demen. 1996;11:10–9.

    Article  Google Scholar 

  40. Kovach CR, Noonan PE, Schlidt AM, Reynolds S, Wells T. The serial trial intervention: an innovative approach to meeting needs of individuals with dementia. J Gerontol Nurs. 2006;32:18–25.

    PubMed  Google Scholar 

  41. Volicer L, Hurley AC. Management of behavioral symptoms in progressive degenerative dementias. J Gerontol A Biol Sci Med Sci. 2003;58:M837–45.

    Article  PubMed  Google Scholar 

  42. Finnema E, Dröes R, Ribbe M, Tilburg van W. A review of psychosocial models in psychogeriatrics; implications for care and research. Alzheimer Dis Assoc Disord. 2000;14:68–80.

    Article  CAS  PubMed  Google Scholar 

  43. Kolanowski AM, Litaker M, Buettner L. Efficacy of theory-based activities for behavioral symptoms of dementia. Nurs Res. 2005;54:219–28.

    Article  PubMed  Google Scholar 

  44. Cohen-Mansfield J, Agitation in the elderly. Definitional and theoretical conceptualizations. In: Hay D, Klein D, Hay L, Grossberg G, Kennedy J, editors. Agitations in patients with dementia: a practical guide to diagnosis and management. Washington: American Psychiatric Publishing; 2003. p. 1–21.

    Google Scholar 

  45. Cohen-Mansfield J, Billig N. Agitated behaviors in the elderly. A conceptual review. J Am Geriatr Soc. 1986;34:711–21.

    Article  CAS  PubMed  Google Scholar 

  46. Bartholomeyczik S, Halek M, Sowinski C, Besselmann K, Dürrmann P, Haupt M. Rahmenempfehlungen zum Umgang mit herausforderndem Verhalten bei Menschen mit Demenz in der stationären Altenhilfe. Berlin: Bundesministerium für Gesundheit; 2007.

    Google Scholar 

  47. Palm R, Kohler K, Bartholomeyczik S, Holle B. Assessing the application of non-pharmacological interventions for people with dementia in German nursing homes: feasibility and content validity of the dementia care questionnaire (DemCare-Q). BMC Res Notes. 2014;7:950.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Dikken J, Hoogerduijn JG, Kruitwagen C, Schuurmans MJ. Content validity and psychometric characteristics of the "knowledge about older patients quiz" for nurses using item response theory. J Am Geriatr Soc. 2016;64:2378–83.

    Article  PubMed  Google Scholar 

  49. Hawkins RJ, Swanson B, Kremer MJ, Fogg L. Content validity testing of questions for a patient satisfaction with general anesthesia care instrument. J Perianesth Nurs. 2014;29:28–35.

    Article  PubMed  Google Scholar 

  50. Beckstead JW. Content validity is naught. Int J Nurs Stud. 2009;46:1274–83.

    Article  PubMed  Google Scholar 

  51. Haynes S, Richard D, Kubany E. Content validity in psychological assessment: a functional approach to concepts and methods. Psychol Assess. 1995;7:238–47.

    Article  Google Scholar 

  52. McGartland R, Berg-Weger M, Tebb S, Lee S, Rauch S. Objectifying content validity: conducting a content validity study in social work research. Soc Work Res. 2003;27:94–104.

    Article  Google Scholar 

  53. Squires A. A valid step in the process: a commentary on Beckstead (2009). Int J Nurs Stud. 2009;46:1284–5.

    Article  PubMed  Google Scholar 

  54. DeVon HA, Block ME, Moyle-Wright P, Ernst DM, Hayden SJ, Lazzara DJ, et al. A psychometric toolbox for testing validity and reliability. J Nurs Scholarsh. 2007;39:155–64.

    Article  PubMed  Google Scholar 

  55. Holle D, Kruger C, Halek M, Sirsch E, Bartholomeyczik S. Experiences of nursing staff using dementia-specific case conferences in nursing homes. Am J Alzheimers Dis Other Demen. 2015;30:228–37.

    Article  PubMed  Google Scholar 

  56. Dupuis SL, Wiersma E, Loiselle L. Pathologizing behavior: meanings of behaviors in dementia care. J Aging Stud. 2012;26:162–73.

    Article  Google Scholar 

  57. Kolanowski A, Fick D, Frazer C, Penrod J. It's about time: use of nonpharmacological interventions in the nursing home. J Nurs Scholarsh. 2010;42:214–22.

    Article  PubMed  Google Scholar 

  58. Brechin D, Murphy G, James IA, Codner J. Alternatives to antipsychotic medications: psychological approaches in managing psychological and behavioural distress in people with dementia. 2013. http://www.bps.org.uk/content/alternatives-antipsychotic-medication-psychological-approaches-managing-psychological-and-be. Accessed 11 Nov 2013.

    Google Scholar 

  59. Reuther S, Holle D, Buscher I, Dortmann O, Muller R, Bartholomeyczik S, et al. Effect evaluation of two types of dementia-specific case conferences in German nursing homes (FallDem) using a stepped-wedge design: study protocol for a randomized controlled trial. Trials. 2014;15:319.

    Article  PubMed  PubMed Central  Google Scholar 

  60. Schneekloth U, Wahl HW. Möglichkeiten und Grenzen selbständiger Lebensführung in privaten Haushalten (MUG III). Repräsentativbefunde und Vertiefungsstudien zu häuslichen Pflegearrangements, Demenz und professionellen Versorgungsangeboten. MUG III. 2005. https://www.bmfsfj.de/bmfsfj/selbststaendigkeit-im-alter%2D-mug-iii-%2D-%2Donline-publikation/79358?view=DEFAULT. Accessed 14 Nov 2005.

  61. Mueller CA, Tetzlaff B, Theile G, Fleischmann N, Cavazzini C, Geister C, et al. Interprofessional collaboration and communication in nursing homes: a qualitative exploration of problems in medical care for nursing home residents - study protocol. J Adv Nurs. 2015;71:451–7.

    Article  PubMed  Google Scholar 

Download references

Funding

The study was co-funded by DAN Produkte GmbH, a company developing paper- and software-based documentation for health care. The funders had no role in the design or analysis of the study nor in the decision to submit this publication.

Availability of data and materials

No additional data are available.

Author information

Authors and Affiliations

Authors

Contributions

MH developed the study design and the IdA®, collected and analysed the data, and wrote the manuscript. The paper was part of MH’s doctoral thesis (Halek, M. Entwicklung und Testung eines strukturierten Assessmentbogens zur Erfassung der Auslöser für herausforderndes Verhalten von Menschen mit Demenz in der stationären Altenhilfe. (Doctoral Thesis), Witten/Herdecke University, Witten. https://portal.dnb.de/opac.htm?method=simpleSearch&cqlMode=true&query=idn%3D1005029563. 2010). DH was involved in drafting the manuscript and providing critical revisions. SB supervised the conception, analysis and interpretation of the data and contributed to the development of the IdA®; additionally, she revised the manuscript critically. MH and DH supervised the translation process of IdA® into English (IdA-E®) (Additional files 1 and 2). All authors read and approved the final manuscript.

Corresponding author

Correspondence to Margareta Halek.

Ethics declarations

Ethics approval and consent to participate

The study obtained an ethics committee approval from the German Society of Nursing Association Science (15.10.2007). The caregivers participating in the study were provided with written and oral information on the study. Data protection issues and voluntary participation were ensured.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

English version of IdA® (IdA-E®) (IdA_english_version.pdf). (PDF 2867 kb)

Additional file 2:

Translation process of the German IdA ® into English (IdA-E®) using a forward and backward translation process. (PDF 516 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Halek, M., Holle, D. & Bartholomeyczik, S. Development and evaluation of the content validity, practicability and feasibility of the Innovative dementia-oriented Assessment system for challenging behaviour in residents with dementia. BMC Health Serv Res 17, 554 (2017). https://doi.org/10.1186/s12913-017-2469-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-017-2469-8

Keywords