Table 5 Critical appraisal tool
Screening question | Responses and key prompt questions to help make the decision |
|---|---|
1. FORMAT Is the measure relevant for use in palliative1 and/or end of life care2? | o Yes [] o Can’t tell [] o No [] ➢ Has the measure been developed for use for people with conditions that require palliative and/or end of life care? E.g. including but not limited to cancer, neurological conditions, heart failure, chronic obstructive pulmonary disease 1Palliative care is understood as an approach aiming to improve the quality of life of patients who are facing life-threatening illness, through the prevention, assessment and treatment of pain and other physical, psychosocial and spiritual problems. It is not intended to hasten or postpone death (World Health Organization, 2012). 2 End of life care is care that helps those with advanced, progressive, incurable conditions; adults who may die within 12 months; and those with life-threatening acute condition. It also covers support for the families and carers of people in these groups (NICE, 2013) |
Is the measure administratively manageable? | o Yes [] o Can’t tell [] o No [] ➢ Is the measure freely available? ➢ Is the measure easy to access? ➢ Is the measure easy to follow, use and understand? ➢ Can the measure be completed within a short time frame? (max 15 mins)-check ➢ Is there adequate guidance over how scores should be interpreted? ➢ Can the measure be used in clinical practice? ➢ Will it fit into clinical routines? |
Is the measure short? | o Yes [] o Can’t tell [] o No [] ➢ Is the outcome measure no more than 4 pages of A4 paper? |
2. DATA COLLECTION TIME POINTS Does the measure have a clear baseline and subsequent clear time points for measures to be taken? | o Yes [] o Can’t tell [] o No [] ➢ Is there clarity about when the measure should first be used with patients? ➢ Is there clarity about when the measure should be used after this first time? ➢ Is there clarity about how many times the measure should be used with patients? |
3. VALIDITY TESTING Has the measure been tested for validity? | o Yes* [] o Can’t tell [] o No [] *If Yes: Is there evidence of content validity3? o Yes [] o Can’t tell [] o No [] Is there evidence of criterion validity4? o Yes [] o Can’t tell [] o No [] Is there evidence of construct validity5? o Yes [] o Can’t tell [] o No [] ➢ 3 Content validity: Is it clear what concept is being measured? Does the measure include all items that are relevant to the concept being measured? ➢ 4 Criterion validity: Does the measure correlate with superior measures, considered as a “gold standard” tests? ➢ 5 Construct validity: Does it measure the underlying concept of interest? NB. Depending on what the concept being measured is, you need to look this up to determine whether the items within the measure are adequately representing the overall concept. |
4 RELIABILITY TESTING Has the measure been tested for reliability | o Yes* [] o Can’t tell [] o No [] *If Yes: Is there evidence of test-retest reliability?6 o Yes [] o Can’t tell [] o No [] Is there evidence of internal consistency?7 o Yes [] o Can’t tell [] o No [] Is there evidence of inter-rater reliability?8 o Yes [] o Can’t tell [] o No [] ➢ 6 Test-retest reliability: is there consistency in test results when administered on different occasions? ➢ 7 Internal consistency: do all items in the measure address the same underlying concept? ➢ 8 Inter-rater reliability: does the measure produce similar results when used by different observers? |
5 CLINICAL RESPONSIVENESS Is the measure able to detect clinically significant changes that take place over time? | o Yes [] o Can’t tell [] o No [] ➢ Is the measure able to pick up on changes such as changes in perceptions of care, satisfaction with care, worries about care (if designed to do so). These indications may be given as scores. ➢ Is the measure able to acquire a score rating from the patient before an intervention is given to them? (if designed to do so) ➢ Is the measure able to acquire a score rating from the patient on at least one occasion after an intervention is given to them? (if designed to do so). ➢ Does the measure indicate what is counted as a clinically significant score change? |
6 ACCEPTABILITY AND APPLICABILITY Do the intended population find the measure acceptable to use? | o Yes [] o Can’t tell [] o No [] ➢ Is the evidence that the participants on which the measure was used, have been asked to indicate whether they accept the outcome measure? E.g. giving them a questionnaire to find out what they think about the measure and/or conducting an interview with them to acquire their opinions |
Is the measure applicable to the clinical setting? | o Yes [] o Can’t tell [] o No [] ➢ Is there evidence that the measure has been implemented in clinical settings? ➢ Is there a clear explanation as to how the measure has been or can be implemented in clinical settings? ➢ Is there evidence that healthcare professionals accept the measure? |
7. TRANSLATION Only refer to this if author has indicated that translation of the measure occurred | Has the measure been sufficiently translated? o Yes [] o Can’t tell [] o No [] ➢ Have all items of the measure been translated? ➢ If not all items have been translated, is this justified? Have the meaning behind concepts been sufficiently translated? o Yes [] o Can’t tell [] o No [] ➢ Does the translated measure take into account the context/ culture relevant to the language it is being translated into? ➢ Are steps taken to ensure that meanings behind concepts are relevant to the people who will be using the measure in its new language? ➢ Has the translated measure been tested for validity, reliability, clinical responsiveness, acceptability and clinical applicability (see criteria 3,4,5,6). |