Study | Sample | Purpose / Method | Findings* / Critique | |
---|---|---|---|---|
Setting | n / cohort | |||
Huang, Lin & Lin. (Taiwan) [13] | College of Nursing | 14 / 20 invited panel members accepted; 10 scholars in relevant fields of expertise, 4 clinical nurses. | ● To develop content for a fall-risk checklist ● Framework presented to panel who were asked to review a 4-point Likert scale checklist (from strong agreement to strong disagreement), submit comments & provide revision suggestions ● Likert scale used to calculate content validity index (CVI) score for each item, rated along 3 dimensions i.e. content importance, appropriateness and discreteness ● Scoring calculation method detailed | ● 70% of potential panel members accepted, 3 rounds required, completed over 4-month period ● Response rates: round 1, 78.5% (3 withdrew); 2, 91% (1 withdrew); 3, 100% ● Results of each round reported in summarized format ● Key suggestions & resulting refinements for each round provided ● Changes to domains and checklist processes documented ● CVI scores for each domain along the 3 dimensions and total score (range 0.84 – 1.00) in last review round provided ● Information not provided: complete checklist, criteria for deleting items, variation in responses & scores to individual items (results summarized by domain) |
Morgan et al. (Canada) [19] | 2 independent academic centers | 5 anesthesiologists | ● To develop a simulation performance checklist to evaluate performance of practicing anesthesiologists, using a computer-based Delphi technique ● Checklist items generated by participants after reading 2 pre-prepared scenarios, error weighting assigned to each item based on risk level ● Responses collated anonymously & emailed back to participants asking them to check off items to retain or delete & to (re)assign weightings ● Process repeated until no further items added, deleted or changes to weightings ● A-priori decision to delete responses endorsed by ≤ 20% respondents | ● 100% response rate ● Required four rounds to reach consensus ● Participants generated 104 items for scenario 1 & 99 items for scenario 2 ● Final percentage weightings for checklist items provided ● Small sample size ● Information not provided: variation in error weighting to individual items, key study timeframes e.g. time from survey distribution to response |
Hart & Owen. (Australia) [17] | Anesthesia Department at a tertiary hospital | Not reported - consultants with special interest in obstetric anesthesia | ● To generate checklist items for use prior to commencing non-emergency Cesarean delivery under general anesthesia ● Participants contacted via email and remained anonymous to other participants ● Two questionnaires were circulated ● Two questionnaires were circulated | ● Results of 2 questionnaires informed construction of checklist items ● Items were later divided into four sub-categories ● Key information not reported: sample size; contents of questionnaires; response rates; how responses were used to inform 2nd round questionnaire & construct final checklist items e.g. not known whether pre-defined consensus methods were used, how checklist items were grouped & ordered |
Ursprung et al. (USA) [16] | 20-bed tertiary care medical-surgical neonatal ICU | Not reported - experts in neonatology, pediatrics, health services research, systems engineering, infection control, advanced practice nursing | ● To develop a patient safety audit checklist for PICUs ● Questions formatted into a checklist and refined iteratively by consensus ● Participants responses based on potential clinical impact of mistakes, system failures, perceived frequency ● Checklist reviewed and refined by physicians and nursing staff from study NICU to ensure relevance locally | ● 36 audit questions representing a broad range of errors associated with NICU patient care generated ● Questions later divided into 2 categories ● Information not reported: sample size and participant designations; contents of questionnaire; number of rounds required; method of obtaining consensus; how checklist items were further reviewed and refined for relevance by local PICU staff after consensus was reached; method of categorization |
Pronovost et al. (USA) [18] | 13 adult medical & surgical ICUs in urban teaching & community hospitals | Interviews: 8 nurses & 5 ICU physicians Focus group: not reported | ● Development and pilot testing of daily goals form ● Validity of measures: obtaining agreement from ICU physicians and quality experts who developed the measures; semi-structured interviews with nurses & physicians who piloted the measures ● Face validity: focus group of physicians and nurses from 13 participating ICUs | ● Validity of measures: ICU physicians and quality experts unanimously agreed process measures addressed important aspects of ICU quality ● Focus group: participants believed measures ‘evaluated the domain of quality they intended to measure and identified important opportunities to improve quality’ [18], p.154 ● Information not provided: sample sizes for development of measures and focus group; content for focus group discussion & semi-structured interviews; how qualitative data analyzed and interpreted |