Table 3 Risk of bias
Author & year of publication | Bias due to confounding | Bias in selection of participants into the study | Bias in classification of interventions | Bias due to deviations from intended interventions | Bias due to missing data | Bias in measurement of outcomes | Bias in selection of the reported result |
|---|---|---|---|---|---|---|---|
Cloos, Ndao, Aho, et al., (2020) [23] Overall bias: Low | Low Multivariate regression conducted and controlled for many variables | Low Non-probability sampling at a clinic was employed to recruit uninsured which may have introduced selection bias. Bias reduced by also recruiting from venue-based sampling and social media | Low Clear definition for uninsured | Low Study conducted around the time of Interim Federal Health Program (IFHP) reinstatement which was not addressed. Individuals who were unaware that they were eligible for the IFHP or who had benefited from it in the past but had not been able to extend or renew it were also included. Excluded those with private insurance or IFHP | Low Little missing data-did not exceed 5% for most variables and for the three variables that had 12%, they were excluded | Low Questionnaire was developed using the Trajectory Model and was validated in migrant and general populations. Questionnaire was available in multiple languages. May have limited interviewer bias given that research assistants knew that they were interviewing uninsured individuals. The outcome was self-reported | Low Reported all analysis conducted |
Ridde, Aho, Ndao, et al., (2020) [22] Overall bias: Low | Low Multivariate regression conducted and controlled for many variables | Low Non-probability sampling at a clinic was employed to recruit uninsured which may have introduced selection bias. Bias reduced by also recruiting from venue-based sampling and social media | Low Clear definition of uninsured | Low Study conducted around the time of IFHP reinstatement which was not addressed. Individuals who were unaware that they were eligible for the IFHP or who had benefited from it in the past but had not been able to extend or renew it were also included. Excluded those with private insurance or IFHP | Low Little missing data-did not exceed 5% for most variables and for the three variables that had 12%, they were excluded | Low Questionnaire was developed using the Trajectory Model and was validated in migrant and general populations. Questionnaire was available in multiple languages. May have limited interviewer bias given that research assistants knew that they were interviewing uninsured individuals. The outcome was self-reported | Low Reported all analysis conducted |
Darling, Bennett, Burton, et al., (2019) [47] Overall bias: Low | Moderate Potential for confounders that were not measured and not controlled for in the analysis (little information regarding sociodemographic factors, missing parity or maternal age) | Low Population-based cohort of all midwifery clients who gave birth between 2012 and 2015 | Moderate BORN-Ontario registry does not detail why participants were uninsured. They defined intervention group to be those that did not have Ontario Health Insurance Plan (OHIP) and comparator group to be those with OHIP | Low Excluded those whose insurance status was unclear | Low BORN-Ontario had high completion of data | Low Study used retrospective BORN-Ontario administrative data which had validation checks, but retrospective chart review is problematic due to inaccuracy and inconsistency in recording | Low Reported all analysis conducted and numbers of individuals excluded. The numbers in the text are not the same as those reported in the tables. Confidence intervals are not reported |
Hynie, Ardern, & Robertson (2016) [48] Overall bias: Moderate | Moderate Potential for confounders that were not measured and not controlled for (age and sex were adjusted for but socioeconomic factors were not) | Moderate Administrative dataset used. Looked at main diagnoses for insured and uninsured clients in a 10% subsample-how they selected them not specified | Low Analysis of 9 consecutive years of data reduced the impact of temporality. Uninsured definition as those who were self-paying. Reason for self-paying not addressed | Moderate People would only pay out-of-pocket or pay through insurance. Unclear whether people had private insurance and would get reimbursed later | Moderate Not addressed | Moderate International Classification of Diseases (ICD) codes may have differed across the 9Â years as not all hospitals adopted it in 2002. The software was assessed and said to be valid but there was under-reporting of multiple problems and lower agreement of main problem for those with multiple problems | Low Reported all analysis conducted. Did not report p-values and confidence intervals for all analysis |
Bunn, Fleming, Rzeznikiewiz, et al., (2013) [41] Overall bias: Moderate | Serious Study was descriptive-no modelling conducted. No covariates controlled for | Moderate Small sample size. Had a 1:1 case to control ratio. Did not describe randomization process for selecting controls -Case and controls were different in median income | Low A lot of the uninsured population did not provide reason for being uninsured so unclear why they billed Compassionate Care Program | Moderate Unclear whether all those who billed through Compassionate Care Program had no form of insurance | Moderate Uninsured had greater amount of missing charts and had missing information on income | Moderate Researchers were unsure if every participant was screened for all of the diagnoses investigated. Retrospective chart reviews are problematic due to inaccuracy and inconsistency in recording | Low Reported all analysis conducted |
Gagnon, Merry, & Haase (2013) [42] Overall bias: Moderate | Low Several confounding variables were controlled for. Used regression modelling | Low Sampled from 12 hospitals. Selected individuals from set categories and selected controls with closest date and date of birth to a case | Moderate Used pre-set definitions to identify refugees, asylum seekers, and immigrants. Did not describe health insurance variable | Moderate Study was before cuts made to IFHP. Whether uninsured had federal insurance or private insurance was unclear | Moderate Reduced model presented when data was missing. Did not investigate missing data further | Low Obtained data prospectively through interviews as well as medical records. Questionnaires were available in multiple languages. Data verification protocol was utilized | Moderate Did not present full regression results |
Rousseau, Laurin-Lamothe, Rummens, et al., (2013) [45] Overall bias: Moderate | Moderate Sociodemographic information was not reported. Potential for confounders that were not measured and not controlled for | Moderate The hospitals involved in the study were not randomly selected. The hospitals differed from one another in their population and samples from these hospitals were chosen differently | Low Uninsured were those who didn’t have OHIP or Régie de l'assurance maladie du Québec (RAMQ) as recorded by medical files | Moderate Did not exclude those who may have private insurance or IFHP | Moderate Not addressed | Moderate Hospitals differed in their record-keeping. One hospital in particular had a migrant outpatient and so could be biased in reporting. Retrospective chart reviews can be problematic due to inaccuracy and inconsistency | Low Reported all analysis conducted |
Wilson-Mitchell & Rummens, (2013) [43] Overall bias: Moderate | Serious Potential for confounders that were not measured and not controlled for | Serious The uninsured charts were not sampled randomly Sampling methods were not further explained. Did not match cases and controls on relevant factors so unsure whether they were similar in everything except for insurance status | Low Used hospital payment codes to identify insurance status | Low Uninsured included those without provincial coverage. Excluded homeless women, those with IFHP, private insurance, or insurance from another province. Study predates changes to IFHP | Moderate Not addressed | Moderate Retrospective chart reviews are problematic due to inaccuracy and inconsistency in recording | Low Reported all analysis conducted |
Wiedmeyer, Lofters, & Rashid, (2012) [44] Overall bias: Moderate | Low Confounders included but missing a few such as marital status | Moderate 63 charts identified for review were unable to be retrieved for this study, which could have led to selection bias | Moderate Uninsured and insured definitions not provided | Moderate Study predated changes to IFHP. Not clear whether there was deviation | Moderate Not addressed | Moderate Retrospective chart reviews are problematic due to inaccuracy and inconsistency in recording | Moderate Authors performed further analysis after seeing unexpected results, not specified a priori |
Jarvis, Munoz, Graves, et al., (2011) [46] Overall bias: Serious | Moderate Many confounding variables not included | Moderate Convenience sampling utilized-locations were affiliated with each other but differed in access for uninsured. Control sample was selected randomly from same hospitals (did not specify the randomization process), but demographic variables were not compared between cases and controls to know if the groups were comparable | Moderate Used medical records from initial presentation to identify insurance status. The status could have changed over time | Moderate One of the sites had funding for refugee referrals and provided financial assistance for tests and visits to uninsured. Excluded those with IFHP or private insurance | Low Did not compare those who were lost to follow-up. Did not discuss missing data from medical records | Moderate Retrospective chart reviews are problematic due to inaccuracy and inconsistency in recording | Low Reported all analysis conducted |