Feasibility of the Dutch ICF Activity Inventory: a pilot study

Background Demographic ageing will lead to increasing pressure on visual rehabilitation services, which need to be efficiently organised in the near future. The Dutch ICF Activity Inventory (D-AI) was developed to assess the rehabilitation needs of visually impaired persons. This pilot study tests the feasibility of the D-AI using a computer-assisted telephone interview. Methods In addition to the regular intake, the first version of the D-AI was assessed in 20 patients. Subsequently, patients and intake assessors were asked to fill in an evaluation form. Based on these evaluations, a new version of the D-AI was developed. Results Mean administration time of the D-AI was 88.8 (± 41.0) minutes. Overall, patients and assessors were positive about the D-AI assessment. However, professionals and 60% of the patients found the administration time to be too long. All included items were considered relevant and only minor adjustments were recommended. Conclusion The systematic character of the revised D-AI will prevent topics from being overlooked and indicate which needs have the highest priority from a patient-centred perspective. Moreover, ongoing assessment of the D-AI will enhance evaluation of the rehabilitation process. To decrease administration time, in the revised D-AI only the top priority goals will be fully assessed. Using the D-AI, a rehabilitation plan based on individual needs can be developed for each patient. Moreover, it enables better evaluation of the effects of rehabilitation. A larger validation study is planned.


Background
As a result of demographic ageing, the number of people with irreversible visual impairments is expected to rise in the coming decades [1,2]. This will lead to increasing pressure on visual rehabilitation services [3], which need to be efficiently organised in the near future. Therefore, the rehabilitation needs of visually impaired people have to be thoroughly investigated immediately after entering a multidisciplinary rehabilitation centre (MRC). This assessment must be performed based on the needs of the patient, not on the supply of the MRC. A patient-centred goal formulation structure in the rehabilitation process may increase patient motivation and help achieve better outcomes [4][5][6]. Therefore, it is essential that patients point out their own priorities. Many studies have focused on specific problems from the perspective of the visually impaired person, or on global rehabilitation goals and outcome measures (e.g. [7][8][9][10][11][12][13][14][15][16][17][18]). Also in the Netherlands, most studies focus on specific domains of rehabilitation such as mobility, adjustment or reading/fine work, or on other important outcome measures, such as vision-related quality of life [19][20][21][22].
However, MRCs lack an instrument which examines and prioritises specific rehabilitation needs in a systematic way. Such an instrument would also enable more effective evaluation of rehabilitation outcome. Therefore, we developed an adapted and extended Dutch version of the Activity Inventory (AI) by Massof et al. [23,24], the so-called Dutch ICF Activity Inventory (D-AI). This instrument not only prioritises rehabilitation needs at the goal level, but also assesses the difficulty of specific tasks that belong (from the patient's perspective) to relevant goals.
MRCs in the Netherlands are planning to make their intake procedure more structured and objective and showed interest in using the D-AI for this purpose. Because implementation of a new intake method needs to be performed accurately and gradually, an important step in this process is a feasibility study; however, this type of study shows considerable variety in both purpose and methods [25][26][27][28][29][30].
To investigate rehabilitation needs, this limited pilot study (as part of a longer-term investigation): i) tests the feasibility of the D-AI using a computer-assisted telephone interview (CATI), ii) determines whether the most relevant topics are included, and (iii) tests whether all questions and answer categories are clear and satisfactory. For this purpose, patients and assessors were asked about their perceptions of and experiences with the D-AI using a CATI.

Recruitment of the study population
To assess feasibility of the questionnaire in visually impaired elderly, eligible participants had to be at least 50 years old, speak adequate Dutch, and have sufficient cognitive ability to participate. Persons with low vision from any cause were allowed to participate.
Patients were recruited after enrolment at a specific location of an MRC. All patients met the study criteria as defined in the evidence-based guidelines on the referral of visually impaired persons to low-vision services in the Netherlands [3]. Following this Dutch guideline implies that the inclusion of patients was not limited to only those who strictly meet the formal criteria for low vision as defined by the WHO. For example, also patients with substantial visual field loss, problems with low or high light levels, or severe problems with reading were allowed to participate.
A random sample of 32 patients was taken from patients who had enroled between September and November 2007. Data collection took place before the start of rehabilitation (i.e. between November 2007 and January 2008).
The study protocol was approved by the Medical Ethics Committee of the VU University Medical Center Amsterdam and was consistent with the principles of the Declaration of Helsinki. All patients provided written informed consent.
The D-AI was administered by four different assessors, including the first author who trained the other three assessors. These three assessors were employees of the MRC where the pilot study took place and were usually involved in the intake process. After the first interview was conducted by the first author, the other three assessors conducted nine, seven and three interviews, respectively.

Structure and routing of the D-AI
The D-AI consists of many possible rehabilitation goals. Each goal belongs to one of the 'Activity and Participation' chapters of the International Classification of Functioning, Disability and Health (ICF) [31]. Additionally, each goal consists of different tasks that all serve the same goal. For each goal the importance is rated on a scale of 0 (not important) to 3 (very important). Only if the goal is at least 'a little important' (goal importance score: 1) will the difficulty of this goal, caused by the visual impairment, be rated on a scale of 0 (not difficult) to 4 (impossible). Only tasks of goals that are at least 'a little important' (goal importance score: 1) and 'a little difficult' (goal difficulty score: 1) will be fully assessed at the task level. Additionally, these goals are labelled with a priority score higher than zero (priority score = goal importance score * goal difficulty score).
Five hobby-related goals (e.g. creative activities) consist of sub-goals. These sub-goals will only be assessed if the overall goal has a priority score higher than zero. In order to develop the first draft of the D-AI, a literature study was made, and patient records and focus group discussions were employed. This process and the content of the D-AI are described in detail elsewhere by Bruijning et al. [32].

Data collection
The D-AI and its routing (see above) were programmed using Blaise Enterprise 4.7 (Heerlen, the Netherlands), so that it could easily be assessed using a CATI. Depending on the patient's response, the computer automatically displays the following question. Immediately after assessing the D-AI, participants were asked to complete questions about the D-AI and its assessment, with the aim to improve the feasibility of the questionnaire. These questions were: What do you think about the assessment time? How did you feel about completing the questionnaire? What other activities or rehabilitation needs do you suggest should be included? How do you feel about completing the questionnaire by telephone? Do you have any recommendations to improve the questionnaire?
The three assessors who were usually involved in the intake process were also requested to fill out an evaluation form about the D-AI. Additionally, the assessors reported each item which they thought was (possibly) misunderstood. Finally, additional recommendations about the content and wording of the D-AI were requested from one expert (an occupational therapist with a PhD in rehabilitation of visually impaired adults).
Patient characteristics, such as age, visual acuity and eye condition, were collected from the patient's medical files.

Analysis
For all goals, priority scores were calculated, as well as the mean, median and maximum scores of goal importance, goal difficulty and priority scores. Data were analysed using SPSS 15.0. If possible, similar answers on the evaluation forms were combined. Finally, based on the results of this pilot study, each item was discussed and consensus between the first and second authors was reached with the aim to develop an improved version of the D-AI.

Study population
Of the 32 patients randomly selected to be invited to join this pilot study, 20 (62.5%) agreed to participate. Table 1 shows the reasons for not participating. All patients who agreed to participate completed the study. Table 2 presents the characteristics of the patient group.

Evaluation of the D-AI
The mean time to complete the telephone interview for the D-AI was 88.8 (± 41.0; range 35-180) min.
The mean number of main goals and sub-goals fully assessed at the task level (with a priority score > 0) was 26.5 (± 15.3). If sub-goals are excluded, this number decreases to 22.7 (± 13.8). For one participant only eight main/sub-goals were fully assessed. The highest number of main goals and sub-goals that were fully assessed was 62.
Further analysis of the assessment of the D-AI showed that for 21 main goals and for 38 sub-goals the mean priority score was less than one. There were four main goals (and eight sub-goals) for which none of the participants had a priority score of at least one. To give an impression of the data, Table 3 presents the most important characteristics of the goals, e.g. the mean priority score of each goal. Characteristics of the subgoals are presented in Table 4. Some sub-goals were given similar answers by the same participants (e.g. 'painting' and 'drawing').

Feedback from assessors and patients
The three assessors indicated that the D-AI is a more objective way to investigate rehabilitation needs and that it is a practical instrument to systematically assess the most important and difficult goals and tasks for patients. No topics are overlooked and the quality of the intake is less dependent on the characteristics of the intake assessor. Patients and assessors thought there was no need to Table 1 Reasons for non participation in the study (n = 12)

Number of patients (%)
Would take too much effort* 5 (41.7) Seemed to be cognitively unable to participate 3 (25) Died between the enrolment and the invitation.  include or delete any specific item. However, they felt that some questions needed rephrasing to improve the clarity. For example, some assessors reported that in many cases the goal 'social activities and trips' was initially interpreted as 'social events' or as 'going on holiday'. Additionally, 'watching TV or film (recreational)' appeared to be too similar to 'watching TV'. Also, the goal 'using specific ICT tools' was reported to be misunderstood. One assessor suggested that is would be useful to establish what percentage of the interview was completed during the CATI. All three assessors agreed that the D-AI interview took too much time, which was 'uncomfortable' and 'unpractical'. Furthermore, they noted that some questions showed up two or more times, which could be annoying for the participants. Moreover, they suggested to include 'not applicable' as an answer category for the question on goal importance. Additionally, one assessor reported that the systematic character and multiplechoice options of the questionnaire made spontaneous conversation difficult and thought that the patients may not like this.
A summary of the patient evaluation is presented in Table 5.

Adaptations to the D-AI
After analysing the results, some minor changes were made to the order of the items. Furthermore, three main goals, 'social activities and trips', 'watching TV or film (recreational)' and 'using specific ICT tools' were deleted. Some tasks that belonged to these goals were moved to other goals. For example, the item 'watching news programs' initially belonging to the goal 'watching TV or film (recreational)' was moved to the goal  [34]. No Activity and Participation domain of the ICF. GI: goal-importance question score; GD: goal-difficulty question score; PR: priority score. Column 'Item order D-AI': Italic numbers represent a change in the order compared to the D-AI pilot . * means that, after the pilot study, this goal has been deleted or merged with another goal. Some underlying activities were merged with other goals.  'watching TV'. Finally, some tasks were added, deleted or rephrased. A summary of the results is given in Tables 3 and 4.
Furthermore, the answer category 'not applicable' has been added to the question on goal importance. Also, an explanation of this answer category is now included in the introduction of the D-AI, as follows: 'Not applicable' means that you usually do not perform this subject, either because of physical conditions other than your eye condition, or because you are satisfied with the help you get concerning this subject.
Another important change concerns the structure of the D-AI. The new version consists of two parts. First, for all goals the 'importance' question and -if the goal is at least 'a little important' (importance score: 1) -the 'difficulty' question will be assessed. Priority scores of all goals are then determined and a priority list (PL) composed, in which all goals are ranked from the highest to the lowest priority score. The second part of the questionnaire consists of assessing all tasks that belong to the goals that have the same or a higher priority score as the fifteenth goal of the list (TPL-15). This means D-AI pilot : the first draft of the D-AI (Dutch ICF Activity Inventory), which was used for the pilot study. D-AI: the new version of the D-AI (Dutch ICF Activity Inventory), after the adaptations were made. GI: goal-importance question; GD: goal-difficulty question; PR: priority.
Column 'Goal of D-AI pilot ': Bold phrases indicate the main goals that contain sub-goals. * means that this goal has been or merged with another goal. Some underlying activities were merged with other goals.  Opinion about administration time -12 (60) -Too long; Lack of concentration; Physical pain (e.g. due to holding telephone); Tiring (need break).   -2 (10) -Face-to-face would be better; Physical pain (e.g. due to holding telephone); Tiring.
Any activities or rehabilitation needs that were not included?  Any other recommendations? -11 (55) -Some items were administered twice (e.g. reading). Unclear questions reported by patient + 11 (55) -All questions were clear. that not all goals with a priority score of at least one will be fully assessed. However, this is possible if the goal with the lowest priority score has the same score as the fifteenth goal. The mean number of goals (including sub-goals) per person that would have been administered following the new structure (TPL-15) is 18.75 ± 6.3 (range 8-34). Figure 1 shows this new structure and routing.

Feasibility of the D-AI
The results of this study support the feasibility of the extended and adapted D-AI for use with visually impaired older adults, assessed by a CATI. In general, participants and assessors were positive about the content of the D-AI and both groups reported that the topics in the D-AI covered all relevant rehabilitation needs. As was confirmed by the assessors, the D-AI had some additional important positive qualities: they felt that the D-AI is a more objective way to assess rehabilitation needs, it makes the intake less dependent on the individual qualities of the intake assessor, and prevents important topics being overlooked by the patient and/or the intake assessor.
In this population of visually impaired older adults, a CATI was considered to be an appropriate way to assess the D-AI. Moreover, direct data entry limits the nonresponse rate to questionnaire items, because of the enforcement of answers. Additionally, it facilitates compiling large databases for the items, which will increase possibilities for future analyses [33]. Most participants were positive about the assessment by telephone, although one patient preferred a face-to face-interview. In the future, different assessment methods can be applied, depending on the patient's preference. Although most participants were positive about the interview, many of them (as well as all assessors) reported that the D-AI took too long to administer. Two participants even needed a break because they had physical pain (e.g. due to holding the telephone) or could not sustain concentration. The new version of the D-AI, in which only those goals with top priority scores will be fully assessed, is expected to increase feasibility as it will decrease the administration time.
The threshold of the TPL-15 was chosen after comparing the assessment time with the number of assessed goals. We expect that this cut-off will decrease the administration time by 15 min. More important, we expect it to lower the outliers with a long assessment time and, simultaneously, to maintain a suitable amount of detailed information. However, because the exact threshold of 15 was arbitrary, more studies are needed to establish whether assessing only those tasks that belong to the goals that have the same or higher priority score as the fifteenth goal of the list, is sufficient to investigate the rehabilitation needs of the visually impaired elderly.

Content and formulation of the D-AI
All assessors and eleven participants reported that some (similar) task questions were asked two or more times (e.g. reading similar items). Examples are 'reading a TV guide' or 'reading a medicine label'; however, these comparable items belonged to different goals (e.g. 'watching TV' and 'personal healthcare'). Not all goals will be assessed and, because the difficulty level of these similar items (e.g. reading) might differ in another context, these double items have not yet been excluded. In an upcoming validation study with a much larger population, we will examine these similar items with the aim to include only the most distinct items.
Another important change was adding an extra answer category to the goal-importance question. Patients complained that the answer categories did not always reflect how they felt. For instance, the goal 'household tasks' was often described as 'not applicable' because the participant had a housekeeper. Several participants reported that saying 'not important' did not adequately represent their situation.  As can be seen in Table 3, some goals had very low mean priority scores. However, some of these goals had very high individual priority scores, meaning that this goal was relevant for some individual participants. We consider that the sample size in this study was too small to delete goals based on low mean priority scores alone. None of the participants had a priority score for the following main goals: 'guide dog care', 'pet care', 'apply for a job' and 'using a computer at work'. However, in this pilot study the youngest participant was 51 years old and, because these four goals are probably more relevant for younger persons, they have not been excluded.

Future implementation
Dutch MRCs are willing to adopt a more structured and objective intake procedure and showed interest in using the D-AI for this purpose. The systematic character of the D-AI prevents topics from being overlooked and indicates which needs have the highest priority from a patient-centred perspective. The results in this study show a wide variety of personal rehabilitation needs. This supports the use of an extensive and systematic instrument that enables each individual to receive their own personal rehabilitation plan. Moreover, ongoing assessment of the D-AI will enhance evaluation of the rehabilitation process. Successful rehabilitation will result in lower priority scores because, after rehabilitation, the goals will be less difficult and/or less important, e.g. due to newly learned alternatives or because of increased acceptance of vision loss. This enables more effective monitoring of the individual patient's progress, as well as evaluation of specific rehabilitation programmes.
Concerning implementation of the D-AI, we suggest that MRCs link all possible rehabilitation goals to specific rehabilitation programmes and assistive devices. After assessment of the first part of the D-AI (at the goal level), the patient and a professional from the MRC will discuss which rehabilitation options will be feasible and where to start. From this viewpoint, not only the 'Activity and Participation' domains but also the total ICF scheme will be discussed. After this, a rehabilitation plan can be formulated together. At the end of the rehabilitation trajectory, this rehabilitation plan will be evaluated to establish whether rehabilitation is successful and, if necessary, to reformulate the plan.

Conclusions
In conclusion, assessment of the extended and adapted D-AI appears to be a promising way to systematically investigate and evaluate rehabilitation needs immediately after entering an MRC. The results of this feasibility study clearly indicate how to improve the feasibility of the D-AI. The small sample size seemed to be sufficient as the results were relatively homogeneous. The findings in this study support the relevance of further developing the D-AI. The next step in this process is a limited patient file study to compare rehabilitation plans based on the regular intake and on the D-AI. To further improve the D-AI and to be able to report important psychometric properties, a larger validation study will also be conducted.