The effectiveness of clinical guideline implementation strategies in oncology—a systematic review

Bora, Ana-Mihaela; Piechotta, Vanessa; Kreuzberger, Nina; Monsef, Ina; Wender, Andreas; Follmann, Markus; Nothacker, Monika; Skoetz, Nicole

doi:10.1186/s12913-023-09189-x

BMC Health Services Research

Table 3 GRADE Evidence Profile Comparison: Multi-component guideline implementation intervention vs no intervention or usual care

From: The effectiveness of clinical guideline implementation strategies in oncology—a systematic review

Certainty assessment							Impact (narrative summary)	Certainty
Study design	N	Risk of bias	Inconsistency	Indirectness	Imprecision	Other considerations	Impact (narrative summary)	Certainty
Overall survival (patient-level outcome)
Randomized controlled trial	2	serious^a	not serious^b	not serious	serious^c	none	The intervention may have little to no effect on overall survival	⨁⨁◯◯ Low
Non-randomized study	NA	NA	NA	NA	NA	NA	NA	NA
Quality of life (patient-level outcome)
Randomized controlled trial	1	serious^a	not serious^d	not serious	serious^c	none	The intervention may have little to no effect on pain reduction and overall quality of life scores	⨁⨁◯◯ Low
Non-randomized study	2	very serious^a	not serious^b	serious^c	very serious^d	none	The intervention may have little to no effect on pain intensity scores, but the evidence is very uncertain	⨁◯◯◯ Very low
Adverse events (patient-level outcome)
Randomized controlled trial	2	serious^a	serious^e	not serious	serious^f	none	The intervention may have little to no effect on adverse events, but the evidence is very uncertain	⨁◯◯◯ Very low
Non-randomized study	NA	NA	NA	NA	NA	NA	NA	NA
Screening (provider-level outcome)
Randomized controlled trial	2	serious^a	not serious	not serious	serious^f	none	The intervention may increase screening rates according to guidelines	⨁⨁◯◯ Low
Non-randomized study	NA	NA	NA	NA	NA	NA	NA	NA
Referral (provider-level outcome)
Randomized controlled trial	2	serious^a	not serious^g	not serious	serious^f	none	The intervention may increase referral rates slightly according to guidelines	⨁⨁◯◯ Low
Non-randomized study	NA	NA	NA	NA	NA	NA	NA	NA
Prescribing behaviour (provider-level outcome)
Randomized controlled trial	1	serious^a	not serious^d	not serious	serious^h	none	The intervention may increase guideline adherence slightly regarding prescribing behaviour	⨁⨁◯◯ Low
Non-randomized study	2	very serious^e	not serious^f	not serious^g	very serious^h	none	The intervention may have little to no effect on prescribing behaviour, but the evidence is very uncertain	⨁◯◯◯ Very low
Attitudes (provider-level outcome)
Randomized controlled trial	2	serious^a	not seriousⁱ	serious^j	very serious^k	none	The intervention may have little to no effect on attitudes, but the evidence is very uncertain	⨁◯◯◯ Very low
Non-randomized study	4	very seriousⁱ	not serious^f	serious^l	very serious^h	none	The intervention may have little to no effect on attitudes, but the evidence is very uncertain	⨁◯◯◯ Very low
Knowledge (provider-level outcome)
Randomized controlled trial	1	serious^a	not serious^d	very serious^l	very serious^m	none	The intervention may have little to no effect on knowledge, but the evidence is very uncertain	⨁◯◯◯ Very low
Non-randomized study	2	extremely serious^k	not serious^f	serious^l	very serious^h	none	The intervention may have little to no effect on knowledge, but the evidence is very uncertain	⨁◯◯◯ Very low

GRADE Working Group grades of evidence
High = Further research is very unlikely to change the confidence in the estimate of effect
Moderate = Further research is likely to have an important impact on the confidence in the estimate of effect and may change the estimate
Low = Further research is very likely to have an important impact on the confidence in the estimate of effect and is likely to change the estimate
Very low = Any estimate of effect is very uncertain
N Number of studies, NRSIs Non-randomized studies of interventions (controlled before-and after studies), NA Not available, other considerations publication bias, CIPN Chemotherapy-induced peripheral neuropathy
Explanations a-m (randomized controlled trials)
^aDowngraded one level for serious risk of bias, due to the lack of blinding of personnel, participants, and outcome assessors
^bResults were not pooled but there could be heterogeneity between results explained by the different follow-ups and outcome definitions
^cDowngraded one level for imprecision due to wide 95% confidence intervals and few participants
^dOnly one study reported this outcome
^eDowngraded one level for inconsistency. Both studies suggested effects in completely opposite directions
^fDowngraded one level for imprecision due to few events, participants and wide 95% confidence intervals
^gResults were not pooled but there could be heterogeneity between results explained by the different outcome definitions, measurements and follow-ups
^hDowngraded one level for imprecision due to few events
ⁱOne study did not report data adequately, therefore study results could not be compared. If there was heterogeneity between results, this may have arisen due to the different outcome definitions and measurements
^jDowngraded one level for serious indirectness, because one study reported this outcome as a combined outcome: attitudes and knowledge, and not consistent with the definition of outcomes in this review (attitudes separately considered from knowledge). The other study measured attitudes and knowledge separately
^kDowngraded two levels for very serious imprecision, due to few participants, few events, and wide 95% confidence intervals. One study did not adequately report the data, therefore an effect could not be further quantified
^lDowngraded two levels for very serious indirectness, because the study reported this outcome as a combined outcome: attitudes and knowledge, and was not consistent with the definition of outcomes in this review (knowledge separately considered from attitudes)
^mDowngraded two levels for very serious imprecision, because of few events and wide 95% confidence intervals
Explanations a-l (non-randomized studies of interventions)
^aDowngraded for very serious risk of bias, due to the lack of control for confounders, deviations from intended intervention in one study, serious limitations in the outcome measurement due to the lack of blinding and poorly reported results (risk for reporting biases) found in these two studies
^bThe direction of the effect differs between studies due to heterogeneity in the population, outcome measurements, and follow-up times
^cDowngraded one level for serious indirectness, because one study report only one symptom impacting quality of life (pain intensity)
^dDowngraded two levels for very serious imprecision, because of few participants and wide 95% confidence intervals reported in both studies
^eDowngraded two levels for very serious risk of bias, due to the lack of randomisation (confounders not controlled), inadequate selection of participants in one study, limitations in the outcome measurement, and due to the lack of blinding and poorly reported results (risk for reporting biases) found in these two studies
^fThe results show similar direction of effects (not pooled). Heterogeneity can be explained due to different outcome definitions and follow-ups
^gThe PICO framework of the studies addressed the review question
^hDowngraded two levels for very serious imprecision, because of few participants, few events, and wide 95% confidence intervals in both studies
ⁱDowngraded two levels due to the lack of randomisation and control for confounders in all NRSIs (one study with critical risk), limitations in the selection of participants of two studies, potential biases due to missing data, and biases in the measurement of outcomes (lack of blinding in all studies) and poorly reported outcomes in all studies
^jDowngraded one level for serious indirectness, because one study measured attitudes and knowledge as a combined outcome. Another study measured attitudes only after intervention so that the comparison was not implemented as expected to see a difference before-and after the intervention
^kDowngraded for extremely serious risk of bias, due to the overall critical risk of bias judged in one study and serious risk in the other study
^lDowngraded one level for serious indirectness, because one study measured attitudes and knowledge as a combined outcome

Back to article page

ISSN: 1472-6963

Contact us

General enquiries: journalsubmissions@springernature.com