Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Using routine health information data for research in low- and middle-income countries: a systematic review

BMC Health Services Research volume 20, Article number: 790 (2020) Cite this article

Abstract

Background

Routine health information systems (RHISs) support resource allocation and management decisions at all levels of the health system, as well as strategy development and policy-making in many low- and middle-income countries (LMICs). Although RHIS data represent a rich source of information, such data are currently underused for research purposes, largely due to concerns over data quality. Given that substantial investments have been made in strengthening RHISs in LMICs in recent years, and that there is a growing demand for more real-time data from researchers, this systematic review builds upon the existing literature to summarize the extent to which RHIS data have been used in peer-reviewed research publications.

Methods

Using terms ‘routine health information system’, ‘health information system’, or ‘health management information system’ and a list of LMICs, four electronic peer-review literature databases were searched from inception to February 202,019: PubMed, Scopus, EMBASE, and EconLit. Articles were assessed for inclusion based on pre-determined eligibility criteria and study characteristics were extracted from included articles using a piloted data extraction form.

Results

We identified 132 studies that met our inclusion criteria, originating in 37 different countries. Overall, the majority of the studies identified were from Sub-Saharan Africa and were published within the last 5 years. Malaria and maternal health were the most commonly studied health conditions, although a number of other health conditions and health services were also explored.

Conclusions

Our study identified an increasing use of RHIS data for research purposes, with many studies applying rigorous study designs and analytic methods to advance program evaluation, monitoring and assessing services, and epidemiological studies in LMICs. RHIS data represent an underused source of data and should be made more available and further embraced by the research community in LMIC health systems.

Peer Review reports

Background

Routine health information systems (RHISs) collect and provide information at regular intervals on services and activities delivered in health facilities [1]. RHISs have been implemented in many low and middle-income country (LMIC) health systems to support resource allocation and day-to-day management decisions at facility, district, provincial, and national levels, as well as to facilitate strategy development and policy-making [2, 3]. Despite the fact that RHISs are being implemented at scale in many LMICs, and that they have been widely recognized as an important component of health systems strengthening [4, 5], prior studies have suggested that researchers continue to prefer using intermittent cross-sectional population-based surveys rather than RHISs data to conduct studies, including the monitoring of health programs and policy evaluations [6,7,8].

In order to improve health system performance, reliable, timely, and transparent data on health services are crucial [9, 10]. RHISs collect such data and thus could provide important insights into health system performance [4]. Substantial investments have been made in the development and strengthening of RHISs in many LMICs over the past two decades [5, 11], and interventions targeting data collection, processing, analysis, and dissemination have increased the accessibility of RHIS data [5, 12]. While early RHISs were established using paper-based health facility reports, newer web-based systems have been adopted in many LMICs over the last decade [13, 14]. The most common of these is the District Health Information System 2 (DHIS 2) platform, which is used as the foundation for the national health management information systems (HMIS) in at least 46 countries and has been piloted in at least another 21 countries [15]. Studies have shown that the implementation of newer information and communication technology systems, along with supportive feedback mechanisms to encourage their use in routine practice, can lead to substantial improvements in RHIS reporting and data quality [5, 13, 16, 17].

Despite the documented improvements in data quality, recent studies have shown a persistent underuse of RHIS data for research purposes in LMICs [8, 18]. A number of factors may contribute to the underuse of RHIS data. Numerous studies and commentators have questioned the usefulness of RHIS-sourced data to monitor and evaluate health services due to data quality concerns, such as incompleteness and inaccuracy [19,20,21,22,23]. Additionally, RHIS data are often not publicly available for secondary analyses, which further limits their use [24]. Due to these concerns, the research community has shown a persistent preference to use data sourced from intermittent cross-sectional population-based surveys rather than data sourced from RHISs to conduct research on health service utilization and policy evaluation in LMICs [8, 18, 25, 26]. However, population-based surveys also have drawbacks, including the fact that they may be costly [26] and are often unable to generate sufficient data at the district or other subnational-levels [27]. In addition, reliance on such data may encourage the use of potentially weak evaluation designs [8] and may make establishing an appropriate baseline challenging when trying to evaluate specific programs [28]. Intermittent cross-sectional population-based surveys themselves also suffer from a number of quality concerns and thus should not be considered the gold standard for estimating service coverage or other population-based estimates [29].

Given the potential of RHISs to play a greater role in the evaluation of health policy and programs and to monitor the performance of health systems, it is important to better understand the extent to which such data are currently being used in research studies. To date, there have been no systematic reviews of the use of RHIS data for research purposes beyond studies that were specific to malaria control [18], a gap this paper seeks to address. Specifically, we systematically reviewed the published literature to identify and describe the different ways in which RHIS data have been used in peer-reviewed research, including the types of health conditions studied. We also summarized the different methodologies that have been used to analyze RHIS data in research and the types of strategies that were applied to circumvent common RHIS data issues, such as incomplete or inaccurate data. It is our goal to provide guidance to other researchers who may be interested in using such data for research purposes by helping them to gain a better understanding on how such data have been successfully used in other contexts.

Methods

This systematic literature review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Peer-reviewed published studies that used data from RHISs in LMICs were included in this study, where RHISs were defined as data systems designed to collect and generate information on services provided by health facilities at regular intervals of a year or less [1]. This included data systems that collect information on clinical service delivery, pharmaceuticals, or diagnostic service provision, as well as routine administrative management. Conversely, systems that collect individual-level data for clinical decision-making purposes and pilot systems to test the implementation of a new data collection component or method were not considered to be RHISs.

Search strategy

Relevant studies were identified through an electronic search of four databases of peer-reviewed literature: PubMed, Scopus, EMBASE, and EconLit -- from inception through February 20, 2019, the date we launched the search. For each database, we identified studies that contained any of the following free text terms in their titles or abstracts: ‘routine health information system’, ‘health information system’, or ‘health management information system’, and any LMIC, as defined by the World Bank’s 2019 classifications (Appendix 1). Articles were included in the study if they met the following criteria: a) full-text article available in English, b) original research, and c) used data from a RHIS in at least one LMIC for research purposes. In order to be considered as having used data from a RHIS for research purposes, studies had to involve an analysis, either descriptive or analytical, of RHIS data, or applied RHIS data to inform their study design. We excluded studies that: a) only described RHISs, b) only described the administrative decision-making process, c) only focused on RHIS data collection issues, or d) only assessed RHIS data quality.

Selection of studies

Figure 1 shows the number of articles identified and retained at each stage of the systematic review process. After removing duplicates from the various database searches, we identified 1459 potential articles. Two reviewers independently screened the search results by title and abstract for inclusion eligibility. When there was insufficient information to determine eligibility at the title and abstract screening stage, the article was included for full-text screening. Full texts of the potentially eligible articles were then obtained and further screened for inclusion eligibility. At both stages, the reasons for excluding individual articles were recorded. The full-texts for all but one article were found. Disagreements at each stage were resolved through discussion. Where an agreement could not be reached, a third reviewer made the final determination.

Fig. 1
figure 1

PRISMA flowchart of study identification and screening process of publications use RHIS data

Full size image

Data extraction and analysis

Two authors extracted data from all included studies using a piloted data extraction form. For each included article, data were extracted on study design, study objective, disease or health condition categories, study sample, description of RHIS data used, use of other data sources, analytic methods of RHIS data, strategies applied to circumvent data quality issues, and study findings. Due to the heterogeneity of the studies in terms of study design, study purpose, health conditions, and analysis methods, we thematically analyzed the studies according to research purpose, types of diseases studied, analytic methods applied, impact factor of journals in which the articles were published, and types of strategies used to circumvent RHIS data quality issues.

Results

Of the 1459 unique articles retrieved from the database search, 132 studies met the inclusion criteria after full-text screening and were thus included in the review. The characteristics of these studies are presented in Table 1. Our review identified studies from 37 different countries. Three quarters of the studies were from Sub-Saharan African countries (74%), followed by South Asia (11%). The vast majority of the studies were published in the last decade, and more than half were published after 2014 (55%), suggesting an increase in the use of RHIS data for research purposes over time. Most of the studies included an analysis of RHIS data (97%), and a few used RHIS data to inform the study but did not describe analysis of RHIS data. One study, for example, used information from RHISs to justify for the selection of the indicators to be used at the individual-level in their study. Among the studies that analyzed RHIS data, most utilized an ecological study design (79%). Of those, more than half included statistical inferences (61%), while the remaining studies only used RHIS data for descriptive purposes (39%). Nearly a fifth of the studies were mixed methods or case studies (18%), a third of which included statistical analyses of RHIS data (33%). A quarter of articles included a description of how they managed missing data (25%), while only a small number of studies described how they detected and dealt with extreme values (14%).

Table 1 Characteristics of research studies that used RHIS data
Full size table

Types of disease and research purpose

Figure 2 shows the different research purposes for which RHIS data were used, along with the health topics investigated. The most common purpose of the studies was program evaluation (51%). RHIS data have been used to evaluate a wide range of interventions, ranging from programs that targeted specific diseases to interventions or policies that affected multiple types of diseases or health services. These included: the effect of malaria control strategies [30,31,32,33,34,35,36], user fee exemption policies [37,38,39,40], health financing schemes [41,42,43,44], interventions on health governance [45,46,47,48,49,50,51,52,53], the administration of new vaccines and vaccination campaigns [54,55,56], as well as community-level interventions such as approaches to enhance community participation and improve referrals from traditional birth attendants in increasing the demand for maternal and child care [57,58,59].

Fig. 2
figure 2

Types of service and research purpose of RHIS data use (n = 132)

Full size image

Additionally, RHIS data were used to monitor or assess service provision (23%) and to describe disease epidemiology (17%). Similar to the program evaluation studies, these studies also investigated a diverse set of health services and the allocation of healthcare resources. Some of these studies found large discrepancies between RHIS data and an estimated disease burden in populations or highlighted the lack of service provision. A few studies also used RHIS data to describe specific programs [60,61,62,63,64], conduct impact evaluations (non-programmatic) [65,66,67,68], and estimate costs [69, 70]. Most of the studies investigated a communicable disease (95%), of which malaria was most studied health condition (24%). A few studies focused on mental health (2%), diabetes mellitus (1%), and permanent tooth extraction (1%). Only two studies used RHIS data to research the health workforce or the equity of funding allocations [71, 72].

Analytic methods using RHIS data

Among articles that conducted statistical analyses using RHIS data (n = 68), time series analyses to test or account for trends were most commonly performed (25%), followed by geostatistical analyses (16%), pre-post comparisons (15%), interrupted time series (ITS) (10%), and difference-in-difference analyses (7%). Other longitudinal analyses (13%), other cross-sectional analyses (12%), and scenario analysis on cost effectiveness (2%) were also conducted. Table 2 presents the range of methodologies identified across studies using RHIS data, as well as the corresponding articles.

Table 2 Types of analytic methods applied among studies that analyzed RHIS data
Full size table

Time series analysis

Time series analysis using RHIS data was most often applied to evaluate programs and identify disease epidemiology, with one study assessing the impact of an infectious disease outbreak on primary health service utilization [82]. Studies analyzed indicators using large quantities of monthly or yearly data to estimate change (range of time units: 5–168). For instance, two-thirds of the studies analyzed three or more years of monthly data. Many of the studies utilized the highly disaggregated nature of the data by using either facility or district level data, with the exception of two studies which modelled national trends [33, 116]. Studies commonly applied strategies to account for temporal autocorrelation and the correlation between geographical units, including generalized linear models [58], multi-level analysis [77, 78], and ordinary least-squares regression with adjustment for seasonality and lag [34, 37, 117]. Among studies that modelled multiple facilities or administrative regions, random effects were commonly applied to account for heterogeneity.

In addition to RHIS data, a number of included studies incorporated data from external sources in their models based on geographical location such as district or region. Studies of malaria, for example, commonly included climate data from satellites in their models to control for important temporal factors, for example precipitation, humidity, and temperature [73, 117]. Other studies incorporated information from other national community surveys, health facility surveys, and program data as covariates [34, 77]. While most studies controlled for potential confounders by including covariates in analytic models, one study on maternal health service applied propensity score matching to further remove biases from differences in covariate distribution [37].

Geostatistical analysis

Geostatistical analyses using RHIS data were predominantly conducted for epidemiological purposes and the monitoring and assessment of service provision by exploiting geospatial information included in the RHIS at the facility or district level. Three of the studies that applied geostatistical analysis were cross-sectional, while the remainder were spatial-temporal. About half of the studies focused on malaria, of which three compared and illustrated various kriging methods to provide a reliable estimate of malaria burden amid missing reporting [105,106,107], and one study applied geostatistical modeling to select the most relevant health facility indicators for severe malaria outcomes [108]. Studies on other topics investigated the spatial or spatial-temporal dynamics of malaria in pregnancy [100], childhood diarrhea [101], clustering of malaria and HIV [102], and meningitis [118]. About half of the studies did not include data from external sources, and others triangulated data sourced from satellite data, Demographic and Health Surveys, national Malaria Indicator Surveys, and Service Delivery Indicator Surveys in their analyses. Studies that included covariates in the geostatistical analysis applied Bayesian hierarchical Poisson models or Bayesian geostatistical negative binomial models [103, 108, 110].

Pre-post comparison analysis

Pre-post comparison was commonly applied among studies that used RHIS data for program evaluation, and several studies used simple descriptive statistics to compare the periods before and after interventions. As pre-post comparison is subject to the limitation of temporal confounders and secular trends, two of the studies included contextual factors in regression modelling [35, 119].

Interrupted time series analysis

Most of the studies that conducted ITS analysis used it to evaluate interventions, and one assessed the impact of an infectious disease outbreak on maternal and child health service use [68]. The studies used large quantities of monthly data to model trend and level change (range of time unit: 44–132). RHIS data were minimally aggregated in these studies, which mostly analyzed facility or district level data, and similar to studies using time series analysis, accounted for autocorrelation through incorporating autoregressive structures or clustered standard errors in their modelling.

As ITS analyses are generally unaffected by confounding variables that do not change over time by design [120], baseline characteristics were typically not included in these models. Nonetheless, ITS analyses can be affected by time-varying confounders that rapidly change and some models included contextual factors from other data sources, such as climate and program data. To strengthen the quasi-experimental design, two studies also included a contrast group of time series to control for contextual changes that occurred at the same time as the interventions [38, 45].

Difference-in-difference analysis

Five studies applied difference-in-difference techniques using a wide range of time periods (range of time units: 4–48) and levels of geographical units (facility, district, provincial). Only one study included contextual characteristics from other data sources in its analysis. Analytic methods varied from descriptive comparison between and within intervention and control groups [41, 59, 87, 88], to ordinary least square regression with propensity score matching [42].

Impact of research using RHIS data

Most of the studies that conducted statistical analyses using RHIS data were published in journals with impact factors (88%, Fig. 3), two-thirds of which were two or higher, and more than a fifth of which were greater than three. Among those studies published in journals with the highest impact factors, most of them focused on program evaluation (53%), followed by monitoring and assessment of service provision (20%), epidemiology (20%) and impact evaluation (7%). These studies encompassed a range of health topics commonly studied using RHIS data.

Fig. 3
figure 3

Distribution of impact factor of journals that published research studies that conducted statistical analysis of RHIS data (n = 68)

Full size image

Strategies to circumvent RHIS data quality issues

Data quality is commonly cited as a barrier to using RHIS data in research, and slightly more than a quarter of the included studies described the strategies that they used to handle missing data and/or identify extreme values (Table 3). These strategies consisted of exclusion, imputation, interpolation, verification, and accounting for missing data in modeling. Exclusion of missing data was the most common practice, and among studies that used this technique, they excluded facilities from the analytic samples [38, 41, 45, 52, 65, 79, 83, 84, 87, 94, 96, 121], restricted the study period based on explicit criteria [54, 122], or applied sensitivity analysis to compare various exclusion criteria [41, 89, 90]. Imputation methods varied from assigning specific values to the missing observation [42, 87, 118, 123,124,125], to various modeling strategies such as conditional autoregressive model [110], generalized linear regression [124], and iterative singular value decomposition [124]. A sensitivity analysis was also conducted to select a specific imputation strategy [124]. Interpolation involves predicting values at unsampled locations. Methods described included the use of space-time kriging [105,106,107], and the adjustment of results by calibrating with other relevant information [52, 53, 55]. Some studies assumed data were missing at random, which was accounted for in specific modeling methods such as mixed-effect models [65, 124]. When the source of data could be reached, some studies also described verifying the missing information using registries where the original data were recorded [39, 73, 97, 111, 122].

Table 3 Strategies applied in research articles to counter issues of RHIS data
Full size table

Slightly fewer articles described methods to identify and handle extreme values in the RHIS data, of which three types of strategies emerged: setting specific thresholds, visual inspection, and analytic assessment. Thresholds were set based on the distribution of the data, such as proportions or standard deviations from univariate regression. Several studies used visual inspection of outliers [38, 107], while the use of jackknifing analysis and the identification of influential points through Cook’s distance statistics were also applied [112, 126]. Upon identification of extreme values, several strategies were utilized: exclusion, replacement with the average value, replacement with the missing value, verification with a data source, or discounting the observation in statistical estimation. However, studies that replaced the extreme value with an explicit value potentially introduced bias into their estimates. A few studies also described the strategies applied to assess the reliability of the RHIS data, some of which were routine processes administered in the health systems [39, 97].

Discussion

In recent years, there have been increased investments made to improve the quality of RHIS data in many LMICs. Over the same time period, we found an increase in published research using RHIS sourced data, especially over the past 5 years, likely due to the increased availability, accessibility, and quality of RHIS data [18]. While these studies have made contributions to the literature, we also found that the total number of studies conducted (n = 132) remains a small part of the overall literature base on health system evaluation and performance in LMICs.

Malaria and maternal health conditions were the most commonly studied health conditions, despite the fact that RHISs collect data on a wide range of other diseases and conditions. In particular, the use of RHIS data for non-communicable diseases (NCDs) research was very limited. As LMICs are undergoing an epidemiologic transition and the importance of NCDs is increasing [127], LMIC health systems face the increasing challenges of addressing the dual burden of communicable and non-communicable diseases [128, 129]. In spite of the limited implementation of non-communicable diseaseinterventions [129], the few studies that used RHIS data for non-communicable disease research mainly analyzed the gap in service provision and estimated disease burden, highlighting the large unmet need for health care in affected populations. A couple of the studies described how their research was limited by data availability and quality, such as the lack of diagnostic categories of the investigated health conditions in the RHIS. Future research should investigate how RHIS data on non-communicable diseases could better help to provide insights on its epidemiology and service provision to address these health conditions.

Our systematic review found that many of the studies took advantage of some of the features of RHIS data, in particular by exploiting the high frequency nature of these data at the level of health facilities, as well as combining external information to enhance estimations and enable assessing new research questions. The triangulation of populational health characteristics, environmental factors, and service coverage strengthens the analysis and the understanding of their influence [130]. In addition, the overlay of different information in analyses of RHIS data allows for the advancement of research methods. For instance, a recent study demonstrated how to assess the effects of facility readiness on severe malaria outcomes through constructing a composite facility readiness index based on health facility characteristics and spatial data, and using RHIS data as the outcome variable [108]. The detailed routine nature of RHIS data and the ability to link with other geographically based information, including data on population, environmental, health behavior, and facility characteristics, can generate high impact research and advance our understanding of disease epidemiology and health improvement efforts in LMICs.

Despite the increasing use of RHIS data for research purposes, the quality of these data remains imperfect and such issues should be identified and addressed in order to limit estimation error and bias. RHIS data quality issues remain a particular concern in some settings [131,132,133], however, other studies have shown that strategies that have been implemented to improve RHIS data across different international contexts can be successful [5, 134]. Multiple strategies were discussed in the articles we reviewed in our paper, including strategies to address common data quality issues such as missingness and data validity, for example the simple exclusion of missing data and various imputation and interpolation methods. However, the majority of the studies that used RHIS data did not describe the extent of the quality issues or the steps they took to overcome them. The use of sensitivity analyses in assessing the effect of specific cut-offs or methods was scarce. Explicit descriptions of the extent of the data quality issues and the reasons for selecting a particular approach should be encouraged in future research.

While our review used major databases and systematic methods, it nevertheless has some limitations that are worth noting. First, we included only peer-reviewed studies that were published in English, and therefore may have overlooked potentially relevant studies published in the grey literature or written in other languages. Additionally, given our focus on original research, we did not search the broader body of literature for books, reports, or grey literature. Our literature search also identified phrases that described health information systems in title and abstracts only, possibly resulting in the exclusion of studies that only mentioned RHIS data use in the full text. Finally, additional variants on these search terms may have generated more articles or a slightly different set of articles.

Conclusions

In this systematic review we summarized the use of data collected from RHISs in LMICs. Overall, we found that researchers are increasingly using data sourced from RHISs to conduct health system planning and evaluation studies in LMIC health systems, however these data likely remain underutilized by the broader research community. As many of the studies included in this review were published in prominent journals and were able to use strong quasi-experimental or geo-spatial methods, we believe this makes the case for greater use of these data for research purposes in the future, which will likely happen as RHIS data become more openly available to the research community. However, there is a need to help build the case to use these data for a broader range of health conditions and to develop more of a consensus on methods to deal with data imperfections, given that our findings underlined the limited use and comparison of these methods. That said, our review clearly demonstrates the feasibility of use RHIS data in conjunction with rigorous study designs and analytic methods in LMICs. We suggest that future program evaluations should consider their use more broadly, to assess an increased variety of health conditions in conjunction with, or as a replacement for, household or facility survey methods.

Availability of data and materials

Not applicable.

Abbreviations

DHIS 2:

District health information system 2

HMIS:

Health management information system

ITS:

Interrupted time series

LMIC:

Low- and middle-income country

NCD:

Non-communicable disease

PRISMA:

Preferred reporting items for systematic reviews and meta-analyses

RHIS:

Routine health information system

References

  1. Hotchkiss DR, Diana ML, Foreit KGF. How can routine health information systems improve health systems functioning in lowand middle-income countries? Assessing the evidence base. Adv Health Care Manag. 2012. https://doi.org/10.1108/S1474-8231(2012)0000012006.

  2. AbouZahr C, Boerma T. Health information systems: the foundations of public health. Bull World Health Organ. 2005;83:578–83.

    PubMed  PubMed Central  Google Scholar 

  3. World Health Organization. Everybody’s business--strengthening health systems to improve health outcomes: WHO’s framework for action; 2007.

    Google Scholar 

  4. Lippeveld T, Sauerborn R, Bodart C, Organization WH. Design and implementation of health information systems; 2000.

    Google Scholar 

  5. Mutale W, Chintu N, Amoroso C, et al. Improving health information systems for decision making across five sub-Saharan African countries: implementation strategies from the African health initiative. BMC Health Serv Res. 2013;13(Suppl 2):1–12.

    Google Scholar 

  6. Hogan DR, Stevens GA, Hosseinpoor AR, Boerma T. Monitoring universal health coverage within the sustainable development goals: development and baseline data for an index of essential health services. Lancet Glob Health. 2018;6(2):e152–68. https://doi.org/10.1016/S2214-109X(17)30472-2.

    Article  PubMed  Google Scholar 

  7. Cutts FT, Claquin P, Danovaro-Holliday MC, Rhoda DA. Monitoring vaccination coverage: defining the role of surveys. Vaccine. 2016;34(35):4103–9. https://doi.org/10.1016/j.vaccine.2016.06.053.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Wagenaar BH, Sherr K, Fernandes Q, Wagenaar AC. Using routine health information systems for well-designed health evaluations in low- and middle-income countries. Health Policy Plan. 2016;31(1):129–35. https://doi.org/10.1093/heapol/czv029.

    Article  PubMed  Google Scholar 

  9. World Health Organization. Monitoring the building blocks of health systems: a handbook of indicators and their measurement strategies. Geneva; 2010. http://www.who.int/healthinfo/systems/WHO_MBHSS_2010_full_web.pdf?ua=1.

  10. World Health Organization. Framework and standards for country health information systems (second edition). Geneva; 2012. http://www.who.int/healthinfo/country_monitoring_evaluation/who-hmn-framework-standards-chi.pdf.

  11. Warren AE, Wyss K, Shakarishvili G, Atun R, de Savigny D. Global health initiative investments and health systems strengthening: a content analysis of global fund investments. Glob Health. 2013;9(1):30. https://doi.org/10.1186/1744-8603-9-30.

    Article  Google Scholar 

  12. Gimbel S, Mwanza M, Nisingizwe MP, et al. Improving data quality across 3 sub-Saharan African countries using the consolidated framework for implementation research (CFIR): results from the African health initiative. BMC Health Serv Res. 2017. https://doi.org/10.1186/s12913-017-2660-y.

  13. Kiberu VM, Matovu JKB, Makumbi F, Kyozira C, Mukooyo E, Wanyenze RK. Strengthening district-based health reporting through the district health management information software system: the Ugandan experience. BMC Med Inform Decis Mak. 2014;14:40. https://doi.org/10.1186/1472-6947-14-40.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Hazel E, Wilson E, Anifalaje A, Sawadogo-Lewis T, Heidkamp R. Building integrated data systems for health and nutrition program evaluations: lessons learned from a multi-country implementation of a DHIS 2-based system. J Glob Health. 2018;8(2):20307. https://doi.org/10.7189/jogh.08.020307.

    Article  Google Scholar 

  15. Dehnavieh R, Haghdoost A, Khosravi A, et al. The district health information system (DHIS2): a literature review and meta-synthesis of its strengths and operational challenges based on the experiences of 11 countries. Heal Inf Manag J. June 2018:1833358318777713. https://doi.org/10.1177/1833358318777713.

  16. Githinji S, Oyando R, Malinga J, et al. Completeness of malaria indicator data reporting via the district health information software 2 in Kenya, 2011–2015. Malar J. 2017;16:344. https://doi.org/10.1186/s12936-017-1973-y.

    Article  PubMed  PubMed Central  Google Scholar 

  17. MEASURE Evaluation. Lessons in health information system strengthening: what worked in the Democratic Republic of the Congo. Chapel Hill; 2019. https://www.measureevaluation.org/resources/publications/wp-19-229.

  18. Ashton RA, Bennett A, Yukich J, Bhattarai A, Keating J, Eisele TP. Methodological considerations for use of routine health information system data to evaluate malaria program impact in an era of declining malaria transmission. Am J Trop Med Hyg. 2017. https://doi.org/10.4269/ajtmh.16-0734.

  19. Rowe AK, Kachur SP, Yoon SS, Lynch M, Slutsker L, Steketee RW. Caution is required when using health facility-based data to evaluate the health impact of malaria control efforts in Africa. Malar J. 2009;8(1):209.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Hazel E, Requejo J, David J, Bryce J. Measuring coverage in MNCH: evaluation of community-based treatment of childhood illnesses through household surveys. PLoS Med. 2013. https://doi.org/10.1371/journal.pmed.1001384.

  21. Lim SS, Stein DB, Charrow A, Murray CJL. Tracking progress towards universal childhood immunisation and the impact of global initiatives: a systematic analysis of three-dose diphtheria, tetanus, and pertussis immunisation coverage. Lancet. 2008;372(9655):2031–46. https://doi.org/10.1016/S0140-6736(08)61869-3.

    Article  PubMed  Google Scholar 

  22. Cutts FT, Izurieta HS, Rhoda DA. Measuring coverage in MNCH: design, implementation, and interpretation challenges associated with tracking vaccination coverage using household surveys. PLoS Med. 2013;10(5):e1001404. https://doi.org/10.1371/journal.pmed.1001404.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Sychareun V, Hansana V, Phengsavanh A, Chaleunvong K, Eunyoung K, Durham J. Data verification at health centers and district health offices in Xiengkhouang and Houaphanh provinces, Lao PDR. BMC Health Serv Res. 2014;14(1):255. https://doi.org/10.1186/1472-6963-14-255.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Gloyd S, Wagenaar BH, Woelk GB, Kalibala S. Opportunities and challenges in conducting secondary analysis of HIV programmes using data from routine health information systems and personal health information. J Int AIDS Soc. 2016. https://doi.org/10.7448/IAS.19.5.20847.

  25. Aqil A, Lippeveld T, Hozumi D. PRISM framework: a paradigm shift for designing, strengthening and evaluating routine health information systems. Health Policy Plan. 2009;24(3):217–28.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Short Fabic M, Choi Y, Bird S. A systematic review of demographic and health surveys: data availability and utilization for research. Bull World Health Organ. 2012. https://doi.org/10.2471/BLT.11.095513.

  27. Fernandes QF, Wagenaar BH, Anselmi L, Pfeiffer J, Gloyd S, Sherr K. Effects of health-system strengthening on under-5, infant, and neonatal mortality: 11-year provincial-level time-series analyses in Mozambique. Lancet Glob Health. 2014. https://doi.org/10.1016/S2214-109X(14)70276-1.

  28. Ratnayake R, Ratto J, Hardy C, et al. The effects of an integrated community case management strategy on the appropriate treatment of children and child mortality in Kono District, Sierra Leone: a program evaluation. Am J Trop Med Hyg. 2017. https://doi.org/10.4269/ajtmh.17-0040.

  29. Bishai DM. Global initiatives in universal childhood immunisation. Lancet. 2008;372(9655):2004–5. https://doi.org/10.1016/S0140-6736(08)61853-X.

    Article  PubMed  Google Scholar 

  30. Seitio-Kgokgwe O, Mashalla Y, Seloilwe E, et al. Developing block wise composite health index in Yavatmal district, Maharashtra state, India: an analysis of available data. Malar J. 2017;16(1 PG):35. https://doi.org/10.4103/2008-7802.160056.

    Article  Google Scholar 

  31. Pagnoni F, Convelbo N, Tiendrebeogo J, Cousens S, Esposito F. A community-based programme to provide prompt and adequate treatment of presumptive malaria in children. Trans R Soc Trop Med Hyg NS. 1997;91(5 PG-512-7):512–7.

    Article  CAS  Google Scholar 

  32. Rehman A, Mann A, Schwabe C, et al. Five years of malaria control in the continental region, Equatorial Guinea. Malar J. 2013;12(PG-154):154. https://doi.org/10.1186/1475-2875-12-154.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Nyarango P, Gebremeskel T, Mebrahtu G, et al. A steep decline of malaria morbidity and mortality trends in Eritrea between 2000 and 2004: the effect of combination of control methods. Malar J. 2006;5(PG-33):33. https://doi.org/10.1186/1475-2875-5-33.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Over M, Bakote’e B, Velayudhan R, Wilikai P, Graves PM. Impregnated nets cannot fully substitute for DDT: the Field effectiveness of alternative methods of malaria prevention in Solomon Islands, 1993–99. World Bank, Policy Res Work Pap Ser 3044, 2003. 2003;(PG) https://libproxy.wlu.ca/login?url=https://search.proquest.com/docview/56217804?accountid=15090NS-.

  35. Okullo AE, Matovu JKB, Ario AR, et al. Malaria incidence among children less than 5 years during and after cessation of indoor residual spraying in northern Uganda. Malar J. 2017;16(1 PG-319):319. https://doi.org/10.1186/s12936-017-1966-x.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Sande S, Zimba M, Mberikunashe J, Tangwena A, Chimusoro A. Progress towards malaria elimination in Zimbabwe with special reference to the period 2003-2015. Malar J. 2017. https://doi.org/10.1186/s12936-017-1939-0.

  37. Manthalu G. User fee exemption and maternal health care utilisation at mission health facilities in Malawi: an application of disequilibrium theory of demand and supply. Health Econ. 2019;(PG). https://doi.org/10.1002/hec.3856.

  38. Nguyen H, Zombré D, Ridde V, De Allegri M. The impact of reducing and eliminating user fees on facility-based delivery: a controlled interrupted time series in Burkina Faso. Health Policy Plan. 2018;33(8 PG-948-956):948–56. https://doi.org/10.1093/heapol/czy077.

    Article  PubMed  Google Scholar 

  39. Njuguna J, Kamau N, Muruka C. Impact of free delivery policy on utilization of maternal health services in county referral hospitals in Kenya. BMC Health Serv Res. 2017. https://doi.org/10.1186/s12913-017-2376-z.

  40. Ridde V, Richard F, Bicaba A, Queuille L, Conombo G. The national subsidy for deliveries and emergency obstetric care in Burkina Faso. Health Policy Plan. 2011. https://doi.org/10.1093/heapol/czr060.

  41. Steenland M, Robyn P, Compaore P, et al. Performance-based financing to increase utilization of maternal health services: evidence from Burkina Faso. SSM - Popul Heal. 2017;3(PG-179-184):179–84. https://doi.org/10.1016/j.ssmph.2017.01.001.

    Article  Google Scholar 

  42. Falisse JB, Ndayishimiye J, Kamenyero V, Bossuyt M. Performance-based financing in the context of selective free health-care: an evaluation of its effects on the use of primary health-care services in Burundi using routine data. Health Policy Plan. 2015. https://doi.org/10.1093/heapol/czu132.

  43. Witter S, Zulfiqur T, Javeed S, Khan A, Bari A. Paying health workers for performance in Battagram district, Pakistan. Hum Resour Health. 2011. https://doi.org/10.1186/1478-4491-9-23.

  44. Ir P, Horemans D, Souk N, Van Damme W. Using targeted vouchers and health equity funds to improve access to skilled birth attendants for poor women: a case study in three rural health districts in Cambodia. BMC Pregnancy Childbirth. 2010. https://doi.org/10.1186/1471-2393-10-1.

  45. Iyer HS, Hirschhorn LR, Nisingizwe MP, et al. Impact of a district-wide health center strengthening intervention on healthcare utilization in rural Rwanda: use of interrupted time series analysis. PLoS One. 2017;12(8 PG-e0182418):e0182418. https://doi.org/10.1371/journal.pone.0182418.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  46. de Lima RT, Fernandes TG, da Balieiro AA, et al. Primary health Care in Brazil and the Mais Médicos (more doctors) program: an analysis of production indicators. Cien Saude Colet. 2016. https://doi.org/10.1590/1413-81232015219.15412016.

  47. Mikkelsen-Lopez I, Cowley P, Kasale H, Mbuya C, Reid G, de Savigny D. Essential medicines in Tanzania: does the new delivery system improve supply and accountability? Heal Syst. 2014. https://doi.org/10.1057/hs.2013.14.

  48. Mphatswe W, Mate K, Bennett B, et al. Improving public health information: a data quality intervention in KwaZulu-Natal, South Africa. Bull World Health Organ. 2012. https://doi.org/10.2471/blt.11.092759.

  49. Newbrander W, Ickx P, Feroz F, Stanekzai H. Afghanistan’s basic package of health services: its development and effects on rebuilding the health system. Glob Public Health. 2014;9(SUPPL.1 PG-S6-S28):S6-S28. https://doi.org/10.1080/17441692.2014.916735.

    Article  PubMed Central  Google Scholar 

  50. Santos LMP, Oliveira A, Trindade JS, et al. Implementation research: towards universal health coverage with more doctors in Brazil. Bull World Health Organ. 2017. https://doi.org/10.2471/BLT.16.178236.

  51. Scott VE, Sanders D. Evaluation of how integrated HIV and TB programs are implemented in South Africa and the implications for rural-urban equity. Rural Remote Health. 2013;13(2):2165.

  52. Vindigni SM, Riley PL, Kimani F, et al. Kenya’s emergency-hire nursing programme: a pilot evaluation of health service delivery in two districts. Hum Resour Health. 2014;12(1 PG). https://doi.org/10.1186/1478-4491-12-16.

  53. Field E, Usurup J, Nathan S, Rosewell A. Contextual factors and health service performance from the perspective of the provincial health administrators in Papua New Guinea. Rural Remote Health. 2018. https://doi.org/10.22605/RRH4484.

  54. Inchauste L, Patzi M, Halvorsen K, Solano S, Montesano R, Iñiguez V. Impact of rotavirus vaccination on child mortality, morbidity, and rotavirus-related hospitalizations in Bolivia. Int J Infect Dis. 2017. https://doi.org/10.1016/j.ijid.2017.06.006.

  55. Nanyunja M, Lewis RF, Makumbi I, et al. Impact of mass measles campaigns among children less than 5 years old in Uganda. J Infect Dis. 2003. https://doi.org/10.1086/368026.

  56. Tafesse B, Tekle E, Wondwossen L, et al. Effects of polio eradication activities on routine immunization: lessons from the 2013 outbreak response in Somali region of Ethiopia. Pan Afr Med J. 2017;27(Suppl 2 PG-4):4. https://doi.org/10.11604/pamj.supp.2017.27.2.10569.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Mochache V, Irungu E, El-Busaidy H, Temmerman M, Gichangi P. “Our voices matter”: a before-after assessment of the effect of a community-participatory intervention to promote uptake of maternal and child health services in Kwale, Kenya 11 medical and health sciences 1117 public health and health services. BMC Health Serv Res. 2018;18(1 PG). https://doi.org/10.1186/s12913-018-3739-9.

  58. Kitui J, Dutton V, Bester D, Ndirangu R, Wangai S, Ngugi S. Traditional birth attendant reorientation and Motherpacks incentive’s effect on health facility delivery uptake in Narok County, Kenya: an impact analysis. BMC Pregnancy Childbirth. 2017;17(1 PG-125):125. https://doi.org/10.1186/s12884-017-1307-7.

    Article  PubMed  PubMed Central  Google Scholar 

  59. Paudel D, Shrestha I, Siebeck M, Rehfuess E. Impact of the community-based newborn care package in Nepal: a quasi-experimental evaluation. BMJ Open. 2017;7(10 PG-e015285):e015285. https://doi.org/10.1136/bmjopen-2016-015285.

    Article  PubMed  PubMed Central  Google Scholar 

  60. Khurmi M, Sayinzoga F, Berhe A, et al. Newborn survival case study in Rwanda - bottleneck analysis and projections in key maternal and child mortality rates using lives saved tool (LiST). Int J MCH AIDS. 2017;6(2 PG-93-108):93–108. https://doi.org/10.21106/ijma.214.

    Article  PubMed  PubMed Central  Google Scholar 

  61. Lindgren TG, Deutsch K, Schell E, et al. Using mobile clinics to deliver HIV testing and other basic health services in rural Malawi. Rural Remote Health. 2011;11(2):1682.

  62. Mapira P, Morgan C. The contribution of church health services to maternal health care provision in Papua New Guinea. P N G Med J. 2011;54(3-4):139–146.

  63. Robertson LJ, Szabo CP. Community mental health services in southern Gauteng: an audit using Gauteng district health information systems data. South African J Psychiatry. 2017;23(1 PG-a1055):a1055. https://doi.org/10.4102/sajpsychiatry.v23i0.1055.

    Article  Google Scholar 

  64. Yellapurkar MV. Health management information system in leprosy control programme. Indian J Lepr. 1984;56(1 PG-86-97):86–97 https://www.scopus.com/inward/record.uri?eid=2-s2.0-0021634802&partnerID=40&md5=9c9d755790bfa4154b2372c12bdeeb37NS-.

    CAS  PubMed  Google Scholar 

  65. Wagenaar BH, Augusto O, Beste J, et al. The 2014–2015 Ebola virus disease outbreak and primary healthcare delivery in Liberia: time-series analyses for 2010–2016. PLoS Med. 2018;15(2):e1002508. https://doi.org/10.1371/journal.pmed.1002508.

    Article  PubMed  PubMed Central  Google Scholar 

  66. Quaglio G, Pizzol D, Bome D, et al. Maintaining maternal and child health services during the Ebola outbreak: experience from Pujehun, Sierra Leone. PLoS Curr. 2016;8. https://doi.org/10.1371/CURRENTS.OUTBREAKS.D67AEA257F572201F835772D7F188BA5.

  67. Sesay T, Denisiuk O, Shringarpure KK, et al. Paediatric care in relation to the 2014–2015 Ebola outbreak and general reporting of deaths in Sierra Leone. Public Heal Action. 2017. https://doi.org/10.5588/pha.16.0088.

  68. Sochas L, Channon AA, Nam S. Counting indirect crisis-related deaths in the context of a low-resilience health system: the case of maternal and neonatal health during the Ebola epidemic in Sierra Leone. Health Policy Plan. 2017;32(suppl_3):iii32–9. https://doi.org/10.1093/heapol/czx108.

    Article  PubMed  Google Scholar 

  69. Cassim N, Coetzee LM, Schnippel K, Glencross DK. Estimating the cost-per-result of a national reflexed Cryptococcal antigenaemia screening program: forecasting the impact of potential HIV guideline changes and treatment goals. PLoS One. 2017;12(8 PG-e0182154):e0182154. https://doi.org/10.1371/journal.pone.0182154.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  70. Novaes H, Itria A, Silva G, Sartori A, Rama C, Soárez P. Annual national direct and indirect cost estimates of the prevention and treatment of cervical cancer in Brazil. Clinics (Sao Paulo). 2015;70(4 PG-289-95):289–95. https://doi.org/10.6061/clinics/2015(04)12.

    Article  Google Scholar 

  71. Namaganda G, Oketcho V, Maniple E, Viadro C. Making the transition to workload-based staffing: using the workload indicators of staffing need method in Uganda. Hum Resour Health. 2015;13(PG-89):89. https://doi.org/10.1186/s12960-015-0066-7.

    Article  PubMed  PubMed Central  Google Scholar 

  72. Kusemererwa D, Alban A, Obua OT, Trap B. An exploratory study on equity in funding allocation for essential medicines and health supplies in Uganda’s public sector. BMC Health Serv Res. 2016;16:453((Obua) Ministry of Health, Pharmacy Division, Lourdel Road, Wandegeya, Kampala, Uganda PG-453). https://doi.org/10.1186/s12913-016-1698-6.

    Article  PubMed  PubMed Central  Google Scholar 

  73. Hajison PL, Mwakikunga BW, Mathanga DP, Feresu SA. Seasonal variation of malaria cases in children aged less than 5 years old following weather change in Zomba district, Malawi. Malar J. 2017. https://doi.org/10.1186/s12936-017-1913-x.

  74. Wright J, Vazé P, Russell G, et al. Seasonal aspects of weight-for-age in young children in Zimbabwe. Public Health Nutr. 2001. https://doi.org/10.1079/phn2000100.

  75. Gunda R, Chimbari M, Shamu S, Sartorius B, Mukaratirwa S. Malaria incidence trends and their association with climatic variables in rural Gwanda, Zimbabwe, 2005–2015. Malar J. 2017;16(1 PG-393):393. https://doi.org/10.1186/s12936-017-2036-0.

    Article  PubMed  PubMed Central  Google Scholar 

  76. De Melo Cunha MAG, Lino PA, Dos Santos TR, Vasconcelos M, Lucas SD, De Abreu MHNG. A 15-year time-series study of tooth extraction in Brazil. Med (United States). 2015;94(47 PG-e1924):e1924. https://doi.org/10.1097/MD.0000000000001924.

    Article  Google Scholar 

  77. Luboga SA, Stover B, Lim T, et al. Did PEPFAR investments result in health system strengthening? A retrospective longitudinal study measuring non-HIV health service utilization at the district level. Health Policy Plan. 2016;31(7 PG-897-909):897–909. https://doi.org/10.1093/heapol/czw009.

    Article  PubMed  PubMed Central  Google Scholar 

  78. Wagenaar B, Gimbel S, Hoek R, et al. Effects of a health information system data quality intervention on concordance in Mozambique: time-series analyses from 2009–2012. Popul Health Metr. 2015;13(PG-9):9. https://doi.org/10.1186/s12963-015-0043-3.

    Article  PubMed  PubMed Central  Google Scholar 

  79. Sibomana H, Rugambwa C, Mwenda JM, et al. Impact of routine rotavirus vaccination on all-cause and rotavirus hospitalizations during the first four years following vaccine introduction in Rwanda. Vaccine. 2018. https://doi.org/10.1016/j.vaccine.2018.01.072.

  80. Larsen DA, Winters A, Cheelo S, et al. Shifting the burden or expanding access to care? Assessing malaria trends following scale-up of community health worker malaria case management and reactive case detection. Malar J. 2017;16(1 PG-441):441. https://doi.org/10.1186/s12936-017-2088-1.

    Article  PubMed  PubMed Central  Google Scholar 

  81. Ngabo F, Tate J, Gatera M, et al. Effect of pentavalent rotavirus vaccine introduction on hospital admissions for diarrhoea and rotavirus in children in Rwanda: a time-series analysis. Lancet Glob Health. 2016;4(2 PG-e129–36):e129–36. https://doi.org/10.1016/S2214-109X(15)00270-3.

    Article  PubMed  Google Scholar 

  82. Wagenaar BH, Augusto O, Beste J, et al. The 2014–2015 Ebola virus disease outbreak and primary healthcare delivery in Liberia: time-series analyses for 2010–2016. PLoS Med. 2018;15(2 PG-e1002508):e1002508. https://doi.org/10.1371/journal.pmed.1002508.

    Article  PubMed  PubMed Central  Google Scholar 

  83. Ruton H, Musabyimana A, Gaju E, et al. The impact of an mHealth monitoring system on health care utilization by mothers and children: an evaluation using routine health information in Rwanda. Health Policy Plan. 2018. https://doi.org/10.1093/heapol/czy066.

  84. Gebrehiwot TG, Sebastian MS, Edin K, Goicolea I. The health extension program and its association with change in utilization of selected maternal health services in Tigray region, Ethiopia: a segmented linear regression analysis. PLoS One. 2015. https://doi.org/10.1371/journal.pone.0131195.

  85. Ir P, Korachais C, Chheng K, Horemans D, Van Damme W, Meessen B. Boosting facility deliveries with results-based financing: a mixed-methods evaluation of the government midwifery incentive scheme in Cambodia. BMC Pregnancy Childbirth. 2015. https://doi.org/10.1186/s12884-015-0589-x.

  86. Landoh E, Tchamdja P, Saka B, et al. Morbidity and mortality due to malaria in Est mono district, Togo, from 2005 to 2010: a times series analysis. Malar J. 2012;11(PG-389):389. https://doi.org/10.1186/1475-2875-11-389.

    Article  PubMed  PubMed Central  Google Scholar 

  87. Assegaai T, Reagon G, Schneider H. Evaluating the effect of ward-based outreach teams on primary healthcare performance in north West Province, South Africa: a plausibility design using routine data. S Afr Med J. 2018;108(4 PG-329-335):329–35. https://doi.org/10.7196/SAMJ.2017.v108i4.12755.

    Article  CAS  PubMed  Google Scholar 

  88. Anwari Z, Shukla M, Maseed B, et al. Implementing people-centred health systems governance in 3 provinces and 11 districts of Afghanistan: a case study. Confl Health. 2015;9(PG-2):2. https://doi.org/10.1186/1752-1505-9-2.

    Article  PubMed  PubMed Central  Google Scholar 

  89. Mugeni C, Levine AC, Munyaneza RM, et al. Nationwide implementation of integrated community case management of childhood illness in Rwanda. Glob Heal Sci Pract. 2014. https://doi.org/10.9745/ghsp-d-14-00080.

  90. phane VS, Jassat W, Bertram MY, et al. Supplementary immunization activities (SIAs) in South Africa: comprehensive economic evaluation of an integrated child health delivery platform. Glob Health Action. 2013. https://doi.org/10.3402/gha.v6i0.20056.

  91. Björkman A, Shakely D, Ali AS, et al. From high to low malaria transmission in Zanzibar - challenges and opportunities to achieve elimination. BMC Med. 2019. https://doi.org/10.1186/s12916-018-1243-z.

  92. Oguttu DW, Matovu JKB, Okumu DC, et al. Rapid reduction of malaria following introduction of vector control interventions in Tororo District, Uganda: a descriptive study. Malar J. 2017. https://doi.org/10.1186/s12936-017-1871-3.

  93. Schoon MG. Impact of inter-facility transport on maternal mortality in the free state province. South African Med J. 2013;103(8 PG-534-537):534–7. https://doi.org/10.7196/SAMJ.6828.

    Article  Google Scholar 

  94. Buchmann EJ, Mnyani CN, Frank KA, Chersich MF, McIntyre JA. Declining maternal mortality in the face of persistently high HIV prevalence in a middle-income country. BJOG An Int J Obstet Gynaecol. 2015. https://doi.org/10.1111/1471-0528.13064.

  95. Kakchapati S, Ardkaew J. Modeling of malaria incidence in Nepal. J Res Health Sci. 2011;1(1 PG-7-13):7–13 https://www.scopus.com/inward/record.uri?eid=2-s2.0-79959717379&partnerID=40&md5=259481ca13eda8f1ef59d22356034ffd NS.

    Google Scholar 

  96. Walsh A, Ndubani P, Simbaya J, Dicker P, Brugha R. Task sharing in Zambia: HIV service scale-up compounds the human resource crisis. BMC Health Serv Res. 2010. https://doi.org/10.1186/1472-6963-10-272.

  97. Njuguna J. Effect of eliminating open defecation on diarrhoeal morbidity: an ecological study of Nyando and Nambale sub-counties, Kenya. BMC Public Health. 2016. https://doi.org/10.1186/s12889-016-3421-2.

  98. Mufunda J, Nyarango P, Usman A, et al. Roll back malaria - an African success story in Eritrea. South African Med J. 2007;97(1):46–50.

  99. Kamuliwo M, Chanda E, Haque U, et al. The changing burden of malaria and association with vector control interventions in Zambia using district-level surveillance data, 2006-2011. Malar J. 2013. https://doi.org/10.1186/1475-2875-12-437.

  100. Kamuliwo M, Kirk K, Chanda E, et al. Spatial patterns and determinants of malaria infection during pregnancy in Zambia. Trans R Soc Trop Med Hyg. 2015;109(8 PG-514-21):514–21. https://doi.org/10.1093/trstmh/trv049.

    Article  PubMed  Google Scholar 

  101. Beyene H, Deressa W, Kumie A, Grace D. Spatial, temporal, and spatiotemporal analysis of under-five diarrhea in southern Ethiopia. Trop Med Health. 2018;46(PG-18):18. https://doi.org/10.1186/s41182-018-0101-1.

    Article  PubMed  PubMed Central  Google Scholar 

  102. Gwitira I, Murwira A, Mberikunashe J, Masocha M. Spatial overlaps in the distribution of HIV/AIDS and malaria in Zimbabwe. BMC Infect Dis. 2018;18(1 PG-598):598. https://doi.org/10.1186/s12879-018-3513-y.

    Article  PubMed  PubMed Central  Google Scholar 

  103. Sartorius B, Cohen C, Chirwa T, Ntshoe G, Puren A, Hofman K. Identifying high-risk areas for sporadic measles outbreaks: lessons from South Africa. Bull World Health Organ. 2013. https://doi.org/10.2471/BLT.12.110726.

  104. Philippon S, Broutin H, Constantin de Magny G, et al. Meningococcal meningitis in Mali: a long-term study of persistence and spread. Int J Infect Dis. 2009. https://doi.org/10.1016/j.ijid.2008.05.1223.

  105. Gething PW, Noor AM, Goodman CA, et al. Information for decision making from imperfect national data: tracking major changes in health care use in Kenya using geostatistics. BMC Med. 2007. https://doi.org/10.1186/1741-7015-5-37.

  106. Gething PW, Noor AM, Gikandi PW, et al. Improving imperfect data from health management information systems in Africa using space-time geostatistics. PLoS Med. 2006;3(6 PG-0825-0831):825–31. https://doi.org/10.1371/journal.pmed.0030271.

    Article  Google Scholar 

  107. Gething P, Atkinson P, Noor A, Gikandi P, Hay S, Nixon M. A local space-time kriging approach applied to a national outpatient malaria dataset. Comput Geosci. 2007;33(10 PG-1337–1350):1337–50. https://doi.org/10.1016/j.cageo.2007.05.006.

    Article  PubMed  PubMed Central  Google Scholar 

  108. Ssempiira J, Kasirye I, Kissa J, et al. Measuring health facility readiness and its effects on severe malaria outcomes in Uganda. Sci Rep. 2018;8(1 PG-17928):17928. https://doi.org/10.1038/s41598-018-36249-8.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  109. Bosomprah S, Dotse-Gborgbortsi W, Aboagye P, Matthews Z. Use of a spatial scan statistic to identify clusters of births occurring outside Ghanaian health facilities for targeted intervention. Int J Gynecol Obstet. 2016. https://doi.org/10.1016/j.ijgo.2016.04.016.

  110. Bennett A, Yukich J, Miller JM, et al. A methodological framework for the improved use of routine health system data to evaluate national malaria control programs: evidence from Zambia. Popul Health Metr. 2014. https://doi.org/10.1186/s12963-014-0030-0.

  111. Pires G, Rosa M, Zangarote M, Chicumbe S. Determinants of stillbirths occurred in health facilities in Zambezia province, Mozambique (2013–2014). Rev Bras Saude Matern Infant. 2016;16(4 PG-415-420):415–20. https://doi.org/10.1590/1806-93042016000400004.

    Article  Google Scholar 

  112. Sebastian MS, Lemma H. Efficiency of the health extension programme in Tigray, Ethiopia: a data envelopment analysis. BMC Int Health Hum Rights. 2010. https://doi.org/10.1186/1472-698X-10-16.

  113. Himanshu M, Källestål C. Regional inequity in complete antenatal services and public emergency obstetric care is associated with greater burden of maternal deaths: analysis from consecutive district level facility survey of Karnataka, India. Int J Equity Health. 2017. https://doi.org/10.1186/s12939-017-0573-3.

  114. Souza ML. Registro de distúrbios mentais no Sistema de Informação da Atenção Básica do Brasil, 2014. Epidemiol e Serv saude Rev do Sist Unico Saude do Bras. 2016;25(2 PG-405-410):405–10. https://doi.org/10.5123/S1679-49742016000200018.

    Article  Google Scholar 

  115. Wanzira H, Muyinda R, Lochoro P, et al. Quality of care for children with acute malnutrition at health center level in Uganda: a cross sectional study in West Nile region during the refugee crisis. BMC Health Serv Res. 2018. https://doi.org/10.1186/s12913-018-3366-5.

  116. Nanyunja M, Lewis RF, Makumbi I, et al. Impact of mass measles campaigns among children less than 5 years old in Uganda. J Infect Dis. 2003;187(SUPPL. 1 PG-S63-S68):S63–8. https://doi.org/10.1086/368026.

    Article  PubMed  Google Scholar 

  117. Gunda R, Chimbari M, Mukaratirwa S. Assessment of burden of malaria in Gwanda District, Zimbabwe, using the disability adjusted life years. Int J Environ Res Public Health. 2016;13(2 PG-244):244. https://doi.org/10.3390/ijerph13020244.

    Article  PubMed  PubMed Central  Google Scholar 

  118. Philippon S, Broutin H, Constantin de Magny G, et al. Meningococcal meningitis in Mali: a long-term study of persistence and spread. Int J Infect Dis. 2009;13(1 PG-103-9):103–9. https://doi.org/10.1016/j.ijid.2008.05.1223.

    Article  PubMed  Google Scholar 

  119. Njuguna J. Effect of eliminating open defecation on diarrhoeal morbidity: an ecological study of Nyando and Nambale sub-counties, Kenya. BMC Public Health. 2016;16(1 PG). https://doi.org/10.1186/s12889-016-3421-2.

  120. Bernal JL, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol. 2017. https://doi.org/10.1093/ije/dyw098.

  121. Heymann DL, Chen L, Takemi K, et al. Global health security: the wider lessons from the west African Ebola virus disease epidemic. Lancet. 2015. https://doi.org/10.1016/S0140-6736(15)60858-3.

  122. Luboga SA, Stover B, Lim TW, et al. Did PEPFAR investments result in health system strengthening? A retrospective longitudinal study measuring non-HIV health service utilization at the district level. Health Policy Plan. 2016. https://doi.org/10.1093/heapol/czw009.

  123. Amouzou A, Kachaka W, Banda B, Chimzimu M, Hill K, Bryce J. Monitoring child survival in “real time” using routine health facility records: results from Malawi. Trop Med Int Health. 2013. https://doi.org/10.1111/tmi.12167.

  124. Inguane CA, Gloyd S, Manuel JL, et al. Assessment of linkages from HIV testing to enrolment and retention in HIV care in Central Mozambique. J Int AIDS Soc. 2016. https://doi.org/10.7448/IAS.19.5.20846.

  125. Wagenaar BH, Gimbel S, Hoek R, et al. Effects of a health information system data quality intervention on concordance in Mozambique: time-series analyses from 2009-2012. Popul Health Metr. 2015. https://doi.org/10.1186/s12963-015-0043-3.

  126. Jat TR, Sebastian MS. Technical efficiency of public district hospitals in Madhya Pradesh, India: a data envelopment analysis. Glob Health Action. 2013. https://doi.org/10.3402/gha.v6i0.21742.

  127. Stanaway JD, Afshin A, Gakidou E, et al. Global, regional, and national comparative risk assessment of 84 behavioural, environmental and occupational, and metabolic risks or clusters of risks for 195 countries and territories, 1990-2017: a systematic analysis for the global burden of disease Stu. Lancet. 2018. https://doi.org/10.1016/S0140-6736(18)32225-6.

  128. Remais JV, Zeng G, Li G, Tian L, Engelgau MM. Convergence of non-communicable and infectious diseases in low- and middle-income countries. Int J Epidemiol. 2013. https://doi.org/10.1093/ije/dys135.

  129. Heller O, Somerville C, Suggs LS, et al. The process of prioritization of non-communicable diseases in the global health policy arena. Health Policy Plan. 2019. https://doi.org/10.1093/heapol/czz043.

  130. Victora CG, Black RE, Boerma JT, Bryce J. Measuring impact in the millennium development goal era and beyond: a new approach to large-scale effectiveness evaluations. Lancet. 2011. https://doi.org/10.1016/S0140-6736(10)60810-0.

  131. Endriyas M, Alano A, Mekonnen E, et al. Understanding performance data: health management information system data accuracy in southern nations nationalities and People’s region, Ethiopia. BMC Health Serv Res. 2019. https://doi.org/10.1186/s12913-019-3991-7.

  132. Ward K, Mugenyi K, Benke A, et al. Enhancing workforce capacity to improve vaccination data quality, Uganda. Emerg Infect Dis. 2017. https://doi.org/10.3201/eid2313.170627.

  133. Maïga A, Jiwani SS, Mutua MK, et al. Generating statistics from health facility data: the state of routine health information systems in eastern and southern Africa. BMJ Glob Health. 2019;4(5):e001849. https://doi.org/10.1136/bmjgh-2019-001849.

    Article  PubMed  PubMed Central  Google Scholar 

  134. Necochea E, Tripathi V, Kim YM, et al. Implementation of the standards-based management and recognition approach to quality improvement in maternal, newborn, and child health programs in low-resource countries. Int J Gynecol Obstet. 2015. https://doi.org/10.1016/j.ijgo.2015.04.003.

Download references

Acknowledgements

We acknowledge that this work was conducted on the Haldimand Tract, traditional territory of the Neutral, Anishinaabe and Haudenosaunee peoples.

Funding

This work was supported by AXA Research Fund. The foundation had no role in the design and implementation of the study, writing of the findings, or decision regarding any of these aspects of the study.

Author information

Authors and Affiliations

  1. University of Waterloo, School of Public Health and Health Systems, Waterloo, Canada

    Yuen W. Hung & Klesta Hoxha

  2. Department of Health Sciences, Wilfrid Laurier University, Waterloo, Canada

    Bridget R. Irwin

  3. Centre for Health Services and Policy Research, The University of British Columbia, Vancouver, Canada

    Michael R. Law

  4. School of Public Health, Hong Kong University, Pok Fu Lam, Hong Kong

    Karen A. Grépin

Authors
  1. Yuen W. Hung
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Klesta Hoxha
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bridget R. Irwin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael R. Law
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Karen A. Grépin
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

YWH, KH, and KAG identified the search strategy. YWH and KH conducted the literature screening, and KAG was consulted when required. YWH, KH, and BRI extracted data from the included studies. YWH wrote the manuscript. KAG and MRL provided feedback and edits on the manuscript. The authors read and approved the final manuscript.

Corresponding author

Correspondence to Karen A. Grépin.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hung, Y.W., Hoxha, K., Irwin, B.R. et al. Using routine health information data for research in low- and middle-income countries: a systematic review. BMC Health Serv Res 20, 790 (2020). https://doi.org/10.1186/s12913-020-05660-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-020-05660-1

Keywords

BMC Health Services Research

ISSN: 1472-6963

Contact us