Adding non-randomised studies to a Cochrane review brings complementary information for healthcare stakeholders: an augmented systematic review and meta-analysis

Background To reduce the burden of asthma, chronic disease management (CDM) programmes have been widely implemented and evaluated. Reviews including randomised controlled trials (RCTs) suggest that CDM programmes for asthma are effective. Other study designs are however often used for pragmatic reasons, but excluded from these reviews because of their design. We aimed to examine what complementary information could be retrieved from the addition of non-randomised studies to the studies included in a published Cochrane review on asthma CDM programmes, for healthcare stakeholders involved in the development, implementation, conduct or long-term sustainability of such programmes. Methods Extending a previously published Cochrane review, we performed a systematic review (augmented review) including any type of study designs instead of only those initially accepted by Cochrane and the Effective Practice and Organization of Care Review group. After double data selection and extraction, we compared study and intervention characteristics, assessed methodological quality and ran meta-analyses, by study design. Results We added 37 studies to the 20 studies included in the Cochrane review. The applicability of results was increased because of the larger variety of settings and asthma population considered. Also, adding non-randomised studies provided new evidence of improvements associated with CDM intervention (i.e. healthcare utilisation, days off work, use of action plan). Finally, evidence of CDM effectiveness in the added studies was consistent with the Cochrane review in terms of direction of effects. Conclusions The evidence of this augmented review is applicable to a broader set of patients and settings than those in the original Cochrane review. It also strengthens the message that CDM programmes have a beneficial effect on quality of life and disease severity, meaningful outcomes for the everyday life of patients with asthma. Despite the moderate to low methodological quality of all studies included, calling for caution in results interpretation and improvements in CDM evaluation methods and reporting, the inclusion of a broader set of study designs in systematic reviews of complex interventions, such as chronic disease management, is likely to be of high value and interest to patients, policymakers and other healthcare stakeholders. Electronic supplementary material The online version of this article (doi:10.1186/s12913-016-1816-5) contains supplementary material, which is available to authorized users.

Score YES if the unit of allocation was by institution, team or professional and allocation was performed on all units at the start of the study; or if the unit of allocation was by patient or episode of care and there was some form of centralised randomisation scheme, an on-site computer system or sealed opaque envelopes were used. CBA studies should be scored NO. Score UNCLEAR if not specified in the paper.

Baseline characteristics similar (EPOC)
RCT QRCT CBA Score YES if baseline characteristics of the study and control patients are reported and similar. Score UNCLEAR if it is not clear in the paper (e.g. characteristics are mentioned in text but no data were presented). Score NO if there is no report of characteristics in text or tables or if there are differences between control and intervention patients. Note that in some cases imbalance in patient characteristics may be due to recruitment bias whereby the provider was responsible for recruiting patients into the trial.

(EPOC)
RCT QRCT CBA Score YES if performance or patient outcomes were measured prior to the intervention, and no important differences were present across study groups. In RCTs, score YES if imbalanced but appropriate adjusted analysis was performed (e.g. Analysis of covariance). Score NO if important differences were present and not adjusted for in analysis. If RCTs have no baseline measure of outcome, score UNCLEAR. YES (Low risk bias) □ NO (High risk bias) □ UNCLEAR □ Quote/comment :

Confounding unlikely
General judgment about confounding risk. Score NO for all noncontrolled studies and NO if: the main conclusions of the study were based on analyses of treatment rather than intention to treat; or the distribution of known confounders differed between the treatment groups but was not taken into account in the analyses. In non-randomised studies if the effect of the main confounders was not investigated or confounding was demonstrated but no adjustment was made in the final analyses the question should be answered as NO. Score UNCLEAR if the distribution of known confounders in the different treatment groups was not described. all Score YES if allocation was by community, institution or practice and it is unlikely that the control group received the intervention. Also score YES if numbers of switches to other interventions was low and if implementation failure was minor. If not, were adjustments techniques used that are likely to correct for these issues? If yes, then score YES. If not, then score NO. Score NO if it is likely that the control group received the intervention (e.g. if patients rather than professionals were randomized). Score UNCLEAR if professionals were allocated within a clinic or practice and it is possible that communication between intervention and control professionals could have occurred (e.g. physicians within practices were allocated to intervention or control)

Detection bias
Blinding of outcome assessment (COCHRANE, EPOC) all Score YES if the authors state explicitly that the primary outcome variables were assessed blindly, or the outcomes are objective, e.g. length of hospital stay. Score NO if the outcomes were not assessed blindly and outcome measurement is likely to be influenced by lack of blinding. Score UNCLEAR if not specified in the paper.

Attrition bias
Incomplete outcome data addressed (COCHRANE, EPOC) all Score YES if no missing data or missing outcome measures were unlikely to bias the results (e.g. the proportion of missing data was similar in the intervention and control groups or the proportion of missing data was less than the effect size i.e. unlikely to overturn the study result). Score NO if missing outcome data was likely to bias the results. Score UNCLEAR if not specified in the paper (Do not assume 100% follow up unless stated explicitly).