Improving data quality and supervision of antiretroviral therapy sites in Malawi: an application of Lot Quality Assurance Sampling
© Hedt-Gauthier et al.; licensee BioMed Central Ltd. 2012
Received: 12 January 2012
Accepted: 9 July 2012
Published: 9 July 2012
High quality program data is critical for managing, monitoring, and evaluating national HIV treatment programs. By 2009, the Malawi Ministry of Health had initiated more than 270,000 patients on HIV treatment at 377 sites. Quarterly supervision of these antiretroviral therapy (ART) sites ensures high quality care, but the time currently dedicated to exhaustive record review and data cleaning detracts from other critical components. The exhaustive record review is unlikely to be sustainable long term because of the resources required and increasing number of patients on ART. This study quantifies the current levels of data quality and evaluates Lot Quality Assurance Sampling (LQAS) as a tool to prioritize sites with low data quality, thus lowering costs while maintaining sufficient quality for program monitoring and patient care.
In January 2010, a study team joined supervision teams at 19 sites purposely selected to reflect the variety of ART sites. During the exhaustive data review, the time allocated to data cleaning and data discrepancies were documented. The team then randomly sampled 76 records from each site, recording secondary outcomes and the time required for sampling.
At the 19 sites, only 1.2% of records had discrepancies in patient outcomes and 0.4% in treatment regimen. However, data cleaning took 28.5 hours in total, suggesting that data cleaning for all 377 ART sites would require over 350 supervision-hours quarterly. The LQAS tool accurately identified the sites with the low data quality, reduced the time for data cleaning by 70%, and allowed for reporting on secondary outcomes.
Most sites maintained high quality records. In spite of this, data cleaning required significant amounts of time with little effect on program estimates of patient outcomes. LQAS conserves resources while maintaining sufficient data quality for program assessment and management to allow for quality patient care.
KeywordsHIV treatment programs Malawi Africa Supervision Data quality
In 2004, the Malawi Ministry of Health (MoH) began the rapid expansion of the national HIV treatment program, initiating 271,105 and retaining 198,846 patients on antiretroviral therapy (ART) at 377 sites (279 static and 98 mobile/outreach) by October 2009 . The successful scale-up of the Malawi ART program was made possible through a public health approach . ART eligibility is determined by WHO clinical staging or a low CD4 count. In 2009, these criteria were Stage III or IV disease or CD4 < 250 cells/mm3 (CD4 < 350 cells/mm3 for pregnant women). One standard, fixed-dose first-line regimen (stavudine, lamivudine, nevirapine in 2009) is used for >90% of patients and standard alternative regimens are used for patients who develop side effects to the first-line regimen . Experienced sites (generally district hospitals that have been offering ART for at least three years) also provide standard second-line regimens for eligible patients. Patients visit ART clinics monthly until stable on therapy (usually after six to ten months). Two- or three-monthly appointments are scheduled thereafter.
Two critical components of the national ART program are 1) a standardized system for registering and monitoring patients, including individual patient treatment cards with registration and follow-up data and a clinic register summarizing critical data for each patient  and 2) quarterly supervision at all ART sites in Malawi . Every quarter, ART clinic personnel perform a standardized patient cohort analysis that includes aggregation of case-finding details of patients registered during the previous quarter and since ART was first begun, as well as treatment outcomes for the cumulative cohort. The latter analysis requires a review of all treatment cards in order to update the clinic register and then a tally-score on current patient outcomes. The primary outcome is patient status, namely, alive on ART, died, stopped ART (with clinician’s knowledge), transferred to another ART clinic, or lost to follow-up. Secondary outcomes for patients alive on ART include 1) ART regimen, 2) drug adherence, 3) ART side effects, and 4) current tuberculosis status. A rolling cohort survival analysis is also performed by counting primary outcomes of patients who registered during specified previous quarters .
During quarterly supervision, teams of at least two individuals (including a representative from the MoH national or regional office along with a MoH clinical officer or nurse) visit each site for up to a full day. Additional representatives from the health sector, including other MoH staff or non-governmental organization partners, join teams on occasion. Currently, quarterly supervision requires ten teams to be in the field for two to three weeks each at a cost of approximately $60,000 per quarter (excluding staff salaries). The quarterly supervision activity is coordinated by the monitoring and evaluation (M&E) team at the Department for HIV and AIDS in the MoH and funded through the U.S. President's Emergency Plan for AIDS Relief (PEPFAR) and the Global Fund to Fight AIDS, Tuberculosis and Malaria. These visits provide an opportunity to identify any problems at the sites, to discuss difficult cases with medical staff, and to monitor drug stocks, thus improving the overall quality of care .
The supervision teams also review the patient cohort analyses produced by the ART staff and clean patient data and aggregate quarterly and cumulative reports on 44 data elements. These data are used for drug forecasting, monitoring overall program performance, identifying gaps in care, and as early warning indicators for HIV drug resistance [6–18], and therefore maintaining high quality data is of utmost importance. Like other national treatment programs, Malawi has strived since its inception to provide complete, accurate and up-to-date data. Common errors include incomplete registers, failure to update follow-up outcomes, duplicate entries, and incorrect/missing records . In 2008, Makombe et al reported that unchecked site-produced reports resulted in a 12% undercount of patients on ART, affecting the programs ability to precisely forecast future drug needs .
Currently every patient treatment card is reviewed and compared against the entries in the register during supervision visits to ensure up-to-date and accurate data in the register prior to data extraction. Inconsistencies or incomplete data are updated as appropriate. This process can take many hours at each visit, competing with other important aspects of supervision and mentoring. Further, this full audit of patient records by the supervision team may not yield any changes, especially at sites where experienced clinic staff have correctly updated treatment cards and clinic registers in preparation for the supervision visit.
While the quality of recording for certain key elements (such as outcome status or regimen) seem to have improved with repetition of this intensive data cleaning process, there remain significant challenges in quality of secondary elements, such as pill counts and side effects. The goal of this study was to analyze the types and frequency of data errors currently observed in the national HIV treatment program and to evaluate their impact on management and monitoring. Based on these findings we evaluated the performance of a lot quality assurance sampling (LQAS) algorithm to prioritize ART sites with poor data quality for a full review of their records before aggregation of quarterly reports. We then estimated the potential amount of time saved by omitting an exhaustive review of all treatment cards at sites that pass the LQAS assessment. Further, we estimated secondary outcomes from treatment cards during the sampling process.
From 11 to 22 January 2010, a study team joined supervision teams for the 4th Quarter (Q4) 2009 supervision. Twenty-three sites were chosen. Selection was not random, but purposely sampled to achieve diverse representations in terms of geography, size, length of providing treatment, and public/private sectors. As per MoH standard, the clinics were informed of the visit by the supervision team but were not aware that the team would be accompanied by a study team. Although the study team visited all 23 clinics; four were excluded because they either had not initiated any patients on treatment or the patient treatment card filing system had not yet been created. At each site, the study team recorded every treatment card/register discrepancy identified by the supervision team including the type of discrepancy, the information recorded in the treatment card and register, and the correct (updated) information. The study team randomly sampled from the list of all patients enrolled at the site, found the corresponding treatment cards, and recorded the time required for sampling and selected secondary outcomes of patients still alive and on treatment (side effects and treatment adherence; the latter was defined by MoH as eight or fewer pills remaining since last visit). If a selected treatment card was missing, then the next card was included in the sample. All data were recorded in real time in an electronic spreadsheet.
We evaluated the impact of an LQAS system that classified each site either as high or as low data quality. The proposed system randomly sampled treatment cards and compared the sample of treatment cards to the register. Based on the results from these samples, sites were classified and only sites classified as low data quality would receive an exhaustive review of treatment cards and registers during supervision.
To evaluate the LQAS tool, random sampling of 76 records was done in parallel to the standard supervision process, where complete review and comparisons of treatment cards to registers was performed by the ART supervisors.
We compared basic site characteristics (patient burden, public versus private, length of time providing treatment, certificates of excellence) using frequencies. We also used counts to compare updated versus unchanged patient outcomes and treatment regimens, if alive. Secondary data outcomes (pill count and adherence) were restricted to patients alive and on treatment. These outcomes are reported with binomial confidence intervals at the site level, and aggregated using stratified sampling analysis, weighting by site size. Analyses and graphs were produced in Stata v.11 and R v.2.0.1.
All data were collected as part of standard clinical supervision and monitoring practices. Data were extracted from patient cards and registers and there was no direct contact with patients. This activity was an evaluation and quality improvement activity for the Department for HIV and AIDS. The whole process was reviewed and cleared by the Harvard School of Public Health and the Harvard Medical School IRBs.
Characteristics of sites
National ART program
n = 19
n = 249
100 patients or fewer
101 - 500 patients
501 - 1000 patients
more than 1000 patients
Length of time providing treatment †
3 months or less
3 months to 2 years
2 to 4 years
more than 4 years
Certificate given at last supervision visit
(n = 15)
(n = 204)
A total of 16,561 individuals had ever registered for ART at all sites. The exhaustive comparisons and cleaning of the treatment cards to the register took 28.5 hours total for the 19 sites. Extrapolating to all sites, we estimate 374 team-hours are dedicated to data cleaning at the 249 sites every quarter. During the exhaustive review, the supervision team identified few discrepancies in outcomes and treatment regimens when comparing the treatment card and register (median percent of records with discrepancies, 1.0%; IQR: 0.5%-3.0%). Of the 210 discrepancies between records (1.2%) in outcomes, a majority (58%) were incorrect in both the treatment card and the register, often as a result of a patient who had defaulted treatment and neither record had been updated. For 34% of the outcome errors, the treatment card was correct (with the register not updated), and in the remaining 8% the register was correct.
For the 11,034 patients alive and on treatment at the sampled sites, only 50 (0.4%) had discrepancies in treatment regimens recorded in their treatment card and register. Most often (94%), regimen changes had been recorded in the treatment card, but not updated in the register; the remaining were correct in the register (4%) or neither the treatment card nor the register (2%). For the 2,605 patients who initiated treatment in Q4 2009, 378 (15%) had discrepancies in their registration data (gender, age group, transfer status, TB history). The treatment card was determined correct in 71% of these errors, the register in 21% and neither in 8%. Usually this resulted from incorrect WHO clinical staging determined from the documented clinical conditions (where a person with a Stage IV condition would be listed as Stage III or vice versa), or TB status was incorrectly recorded as current instead of TB in the previous two years.
Impact of LQAS classification systems
System without any review of data quality
Upper threshold = 95%
Upper threshold = 99%
Lower threshold = 85%
Lower threshold = 89%
n (sample size)
d (decision rule)
Number of sites above the upper threshold
Number classified as high data quality
Number of sites below the lower threshold
Number classified as low data quality
Differences between updated and non-updated data
1st Line Rx
Time Required for Review
8 hours;5 minutes
13 hours;19 minutes
At 16 sites, we randomly sampled 76 patient treatment cards. The remaining three had fewer than 76 cards inspected because fewer than 76 patients had ever initiated treatment at that site. At these three sites, the classification was made on the exhaustive review. LQAS classification of sites with the 85%/95% thresholds would have reduced the time needed for record review from 28.5 to 8 hours (see Table 2). Based on the exhaustive review, 16 sites had “high data quality” (95% or more agreement between treatment cards and register entries) and all of these sites would have been correctly classified using the LQAS system (Table 2 and Figure 1). “One site had less than 85% concordance between records, and this site would have been classified as low data quality via LQAS. For the two sites with concordance between 85-95%, one site would have been classified as low data quality. Because only two sites were classified as “low data quality” and thus would have received exhaustive review, there were very few updates to the aggregated data extracted from the register. The only marked improvement was a reduced undercount of numbers on the substituted zidovudine regimen (from 3.6% to 2.8%).
We also investigated the more conservative rules for site classification, with “high data quality” defined as 99% or more concordance between the treatment card and register and “low data quality” defined as 89% or less concordance. The nine sites with greater than 99% concordance based on the exhaustive inspection were correctly classified as “high data quality” by LQAS. The one site with “low data quality” was also correctly classified. The subsequent exhaustive review at the one additional site identified with this more conservative classification rule improved the accuracy of the aggregate measures extracted from the register, halving the percent errors on nearly all measures and reducing the undercount of efavirenz substitutes to only one third of its previous size (from 9.7% undercount to only 3.1% undercount). This increased accuracy required half of the time currently required for chart review at sites, though it increased the time by 50% over the less conservative (85%/95%) LQAS approach.
Secondary analysis of side effects and pill counts
Number alive in sample
Number with side effects recorded
Percent with adverse side effects
95% Confidence interval
Number alive and on 1st line in sample
Number with PC recorded
Percent with good adherence
95% Confidence interval
Because of the extensive use of the national ART program data for management and analysis of patient care, ensuring the quality of this data is critical. Poor data can lead to imprecise budgeting and procurement of drugs, incorrect inference on program performance, and inappropriate adaptations to program implementation and, ultimately, poor patient care. Like other countries, Malawi has strived to improve and maintain data quality. As evident in these results, the overall level of quality is currently high for critical elements, but the system in place to produce this quality is expensive, time consuming, and can be redundant as many sites have already reviewed and cleaned data prior to supervision visits. Excluding travel time, nearly 750 team-hours every three months are spent at sites, and this time burden will only increase as more patients initiate treatment. We attribute half of this time solely to the data cleaning process and reducing the time for data cleaning can translate into budget savings for the ART program or, perhaps more importantly, into reallocation of time during supervision visits to more crucial activities such as mentoring or discussing difficult cases. With the LQAS system, we save time while maintaining the current standards for data quality. Other research highlights the challenges and associated consequences of poor data quality in health programs and each provides a method for measuring data quality, including some that rely on LQAS [19, 22–34]. However, one unique benefit of this LQAS-based system is that the assessment is integrated into routine supervision. The simplicity of the procedure allows the classification to happen on site and immediately links to actions that improve data quality, such as the exhaustive data review in this case.
Complete an exhaustive data cleaning process at sites with fewer than 500 patients ever initiated. These sites should have an exhaustive review because error rates are often higher at smaller, less experienced sites such as these, because they have less practice with forms and are unlikely to have a designated data clerk. Further, the review at small sites is less time consuming, reducing the benefit of a sampling process.
Exhaustive review of all patients initiating in the last quarter at every site. Basic patient characteristics are collected once at registration and had high error rates (15%). Correcting these errors at the first supervision visit after initiation prevents them from recurring across future reports.
For the public sites with more than 500 patients, sample 76 treatment cards from patients who initiated treatment before the last quarter and compare these against the register. Only complete an exhaustive review if 7 or more inconsistencies are identified. The LQAS system with 85%/95% thresholds maintains data quality sufficient for program purposes and minimizes time required for data review/cleaning. We also recommend collecting information on secondary outcomes such as pill count and side effects during the sampling process.
Because time required for the exhaustive review was not separated by new and formerly registered patients, we could not estimate the exact time savings of this system. However, we expect the overall savings to remain significant since less than 16% of records were from new initiates. Further, the estimated time savings attributable to LQAS reviews at the small sites was small relative to the overall time savings (3 of the 20.5 hour reduction). The extrapolation regarding number of team hours taken to clean data each quarter assumed the same average quality of data across all clinics in Malawi, which given the purposive nature of our sampling may not be the case. Another limitation of this study, and perhaps the LQAS system for supervision, is the skipping of the patient treatment cards if the card was not available during the sampling procedure. Out of the records sampled, an average of 1.4 records per site were missing and skipped at the time of the visit. While some of these records were simply lost, often the records were not available because the patient was in the clinic receiving care during the supervision visit. Since most discrepancies were for patients who are no longer alive and on treatment, or who have defaulted, excluding these records likely increases the chance of classifying sites as having poor data quality and following up with a exhaustive data review. An additional limitation of this study (and the proposed supervision system) is the missing data on secondary outcomes from patient records. This likely biases the estimates of secondary outcomes, especially side effects where no record of side effects often suggests none are detected. We believe that active recording of secondary outcomes through the proposed supervision system could actually lead to improved documentation, as clerks and health workers will receive feedback during the process if the data is not recorded. Finally, while we believe the effect to be minimal since sites were not informed of the study team ahead of time, there could be some Hawthorne effect on the rigor of the supervision process. The Hawthorn effect could also affect the speed and accuracy of the ART supervision team, which would impact our estimates of time required for data cleaning as well as time saved using the LQAS system.
Additional concerns are raised when proposing a sampling approach for monitoring data quality. One is that failure to identify low data quality sites translates into inaccurate aggregated reports. The proposed algorithm has been designed to minimize this risk – sites with 85% or lower concordance between the treatment card and register have less than a 5% risk of being classified as high data quality at a single visit. As the data quality gets worse the chance of misclassification becomes even smaller, decreasing to less than 1% if the concordance goes down to 80%. This thus results in a very small potential for bias in the aggregated measures. Another concern is that sites may accumulate errors over time or that the site clerks may become less diligent if the exhaustive review is not completed. However, because the LQAS process is repeated independently each quarter, sites with degrading quality, if not captured at one visit, will likely be captured at future visits. Further, during previous rounds of supervision, clerks and health workers have expressed dissatisfaction at the automatic exhaustive review noting the duplication of their efforts and the impression that the supervision teams do not trust their work. The LQAS system addresses these concerns by prioritizing and conducting exhaustive reviews only at sites with poor data.
In this paper, we describe the development of a simple LQAS system to assess data quality of sites. The LQAS classification is conducted independently at each visit; however the method could be extended so that performance at previous data quality assessments can be incorporated into the decision to conduct exhaustive review at a subsequent visit. Further, the Department for HIV and AIDS may still opt for exhaustive review at sites under special circumstances, such as immediately following guideline changes or changes in personnel at a site. Before any LQAS-based system can be implemented, forms to support the LQAS process must be developed and supervision teams need to be trained. The LQAS system, although straightforward to implement, is more complicated than the current methods that are standardized across all sites. However, supervisors would most likely welcome the system due to the substantial time saved. The LQAS system is unlikely to be a long-term solution for rapidly expanding ART programs such as in Malawi, it would provide a mid-term solution to ensure adequate quality of data until a reliable and sustainable system, such as an electronic record system, could be introduced nationwide . Further, the system illustrated here can be readily adapted by other national programs using paper-based systems to ensure high quality data.
Lot quality assurance sampling
Ministry of Health
Monitoring and evaluation
President's Emergency Plan for AIDS Relief
BH and SM received funding from National Institutes of Health Training Grant T32 AI007358. MP received funding from NIH R01 EB006195 and AI097015.
The supervision of the national HIV treatment program is supported by the U.S. President's Emergency Plan for AIDS Relief (PEPFAR) and the Global Fund to Fight AIDS, Tuberculosis and Malaria.
The authors thank all individuals supporting the supervision of the national treatment program, especially Lucy Chalunda, Thom Chaweza, Absalom Kaunda, Eustice Mhango, Josheph Njala, Sabina Phiri, and MacCloud Pirigu. We also thank Anne Ben-Smith for her assistance in reviewing the paper.
- Malawi Ministry of Health HIV Department: Malawi Antiretroviral Treatment Programme Quarterly Report: Results up to 31st December 2009. 2010, http://www.hivunitmohmw.org/Main/AntiretroviralTherapy.
- Harries AD, Gomani P, Teck R, de Teck OA, Bakali E, Zachariah R, Libamba E, Mwansambo A, Salaniponi F, Mpazanje R: Monitoring the response to antiretroviral therapy in resource-poor settings: the Malawi model. Trans R Soc Trop Med Hyg. 2004, 98: 695-701. 10.1016/j.trstmh.2004.05.002.View ArticlePubMed
- Ministry of Health, Malawi: Treatment of AIDS: Guidelines for use of antiretroviral therapy in Malawi. 2008
- Libamba E, Makombe SD, Harries AD, Schouten EJ, Yu JK, Pasulani O, Mhango E, Aberle-Grasse J, Hochgesang M, Limbambala E, Lungu D: Malawi’s contribution to “3 by 5”: achievements and challenges. Bull World Health Organ. 2007, 85 (2): 156-160. 10.2471/BLT.05.033688.PubMed CentralView ArticlePubMed
- Libamba E, Makombe S, Mhango E, Teck O, Limbambala E, Schouten E, Harries AD: Supervision, monitoring and evaluation of nationwide scale-up of antiretroviral therapy in Malawi. Bull World Health Organ. 2006, 84 (4): 320-326.PubMed CentralPubMed
- Harries AD, Schouten E, Libamba E: Scaling up antiretroviral treatment in resource-poor settings. Lancet. 2006, 367: 1870-1872. 10.1016/S0140-6736(06)68809-0.View ArticlePubMed
- Van Oosterhout JJG, Kumwenda JK, Hartung T, Mhango B, Zijlstra EE: Can the initial success of the Malawi ART scale-up programme be sustained? The example of Queen Elizabeth Central Hospital, Blantyre. AIDS Care. 2007, 19 (10): 1241-1246. 10.1080/09540120701403358.View ArticlePubMed
- Makombe SD, Jahn A, Tweya H, Chuka S, Yu JK, Hochgesang M, Aberle-Grasse J, Pasulani O, Schouten EJ, Kamoto K, Harries AD: A national survey of the impact of rapid scale-up of antiretroviral therapy on health-care workers in Malawi: effects on human resources and survival. Bull World Health Organ. 2007, 85 (4): 310-314.
- Makombe SD, Jahn A, Tweya H, Chuka S, Yu JK, Hochgesang M, Aberle-Grasse J, Thambo L, Schouten EJ, Kamoto K, Harries AD: A national survey of teachers on antiretroviral therapy in Malawi: Access, retention in therapy, survival. PLoS One. 2007, 2 (7): e620-10.1371/journal.pone.0000620.PubMed CentralView ArticlePubMed
- Makombe SD, Jahn A, Tweya H, Thambo L, Yu JK, Hedt B, Weigel R, Nkata A, Schouten EJ, Kamoto K, Harried AD: A national survey of prisoners on antiretroviral therapy in Malawi: access to treatment and outcomes on therapy. J Infect Dev Countries. 2007, 1 (3): 303-307.
- Makombe SD, Harries AD, Yu JK, Hochgesang M, Mhango E, Weigel R, Pasulani O, Fitzgerald M, Schouten EJ, Libamba E: Outcomes of patients with Kaposi’s sarcoma who start antiretroviral therapy under routine program conditions in Malawi. Trop Doct. 2008, 38: 5-7. 10.1258/td.2007.060023.View ArticlePubMed
- Makombe SD, Jahn A, Tweya H, Chuka S, Yu JK, Hedt B, Weigel R, Nkhata A, Schouten EJ, Kamoto K, Harries AD: Antiretroviral therapy in the Malawi police force: access to therapy and treatment outcomes. Malawi Med J. 2008, 20 (1): 23-27.PubMed CentralView ArticlePubMed
- Lowrance D, Filler S, Makombe S, Harries A, Aberle-Grasse J, Hochgesang M, Libamba E: Assessment of a national monitoring and evaluation system for rapid expansion of antiretroviral treatment in Malawi. Trop Med Int Health. 2007, 12 (3): 377-381. 10.1111/j.1365-3156.2006.01800.x.View ArticlePubMed
- Lowrance DW, Makombe S, Harries AD, Shiraishi RW, Hochgesang M, Aberle-Grasse J, Libamba E, Schouten E, Ellerbrock T, Kamoto K: A public health approach to rapid scale-up of antiretroviral treatment in Malawi during 2004–2006. JAIDS. 2008, 49: 287-293. 10.1097/QAI.0b013e3181893ef0.PubMed
- Hedt BL, Wadonda-Kabondo N, Makombe S, Harries AD, Schouten EJ, Limbambala E, Hochgesang M, Aberle-Grosse J, Kamoto K: Early warning indicators for HIV drug resistance in Malawi. Antivir Ther. 2008, 13 (Suppl 2): 69-75.PubMed
- Banda AC, Makombe SD, Jahn A, Tweya H, Chuka S, Yu JK, Hedt B, Weigel R, Nkhata A, Schouten EJ, Kamoto K, Harries AD: Antiretroviral therapy in the Malawi defence force: access, treatment outcomes and impact on mortality. PLoS One. 2008, 3 (1): e1445-10.1371/journal.pone.0001445.PubMed CentralView ArticlePubMed
- Makwiza I, Nyirenda L, Bongololo G, Banda T, Chimzizi R, Theobald S: Who has access to counseling and testing and anti-retroviral therapy in Malawi – an equity analysis. Int J Equity Health. 2009, 8: 13-10.1186/1475-9276-8-13.PubMed CentralView ArticlePubMed
- Teferra TB, Hochgesang M, Makombe SD, Kamoto K, Harries AD: Who is accessing antiretroviral therapy in Malawi? A study of the Southern Region on the occupation category “other”. Malawi Med J. 2007, 19 (4): 138-139.PubMed CentralPubMed
- Makombe SD, Hochgesang M, Jahn A, Tweya H, Hedt B, Chuka S, Yu JK, Aberle-Grasse J, Pasulani O, Bailey C, Kamoto K, Schouten EJ, Harries AD: Assessing the quality of data aggregated by antiretroviral treatment clinics in Malawi. Bull World Health Organ. 2008, 86 (11): 851-857.
- Lwanga S, Lemeshow S: Sample Size Determination in Health Studies: A Practical Manual. 1991, World Health Organization, Geneva
- Valadez JJ: Assessing child survival programs in developing countries: testing lot quality assurance sampling. 1991, Harvard University Press, Cambridge, MA
- Kiragga AN, Castelnuovo B, Schaefer P, Muwonge T, Easterbrook PJ: Quality of data collection in a large HIV observational clinic database in sub-Saharan Africa: implications for clinical research and audit of care. J Int AIDS Soc. 2011, 14: 3-10.1186/1758-2652-14-3.PubMed CentralView ArticlePubMed
- Doherty TM, McCoy D, Donohue S: Health system constraints to optimal coverage of the prevention of mother-to-child HIV transmission programme in South Africa: lessons from the implementation of the national pilot program. Afr Health Sci. 2005, 5 (3): 213-218.PubMed CentralPubMed
- Loveday M, Zweigenthal V: TB and HIV integration: obstacles and possible solutions to implementation in South Africa. Trop Med Int Health. 2011, 16 (4): 431-438. 10.1111/j.1365-3156.2010.02721.x.View ArticlePubMed
- Garrib A, Stoops N, McKenzie A, Dlamini L, Govender T, Rhode J, Herbst K: An evaluation of the District Health Information System in rural South Africa. S Afr Med J. 2008, 98: 549-552.PubMed
- Heunis C, Wouters E, Kigozi G, Engelbrecht M, Tsibolane Y, Merwe S, Motlhanke S: Accuracy of Tuberculosis routine data and nurses’ view of the TB-HIV information system in the Fress State, South Africa. J Assoc Nurses AIDS Care. 2011, 22 (1): 67-73. 10.1016/j.jana.2010.06.003.View ArticlePubMed
- Mate KS, Bennett B, Mphatswe W, Barker P, Rollins N: Challenges for routine health systems data management in a large public programme to prevent Mother-to-Child HIV transmission in South Africa. PLoS One. 2009, 4 (5): e5483-10.1371/journal.pone.0005483.PubMed CentralView ArticlePubMed
- Lowrance DW, Ndamage F, Kayirangwa E, Ndagije F, Lo W, Hoover DR, Hanson J, Elul B, Ayaba A, Ellerbrock T, Rukundo A, Shumbusho F, Nash D, Mugabo J, Assimwe A: Adult clinical and immunologic outcomes of the national antiretroviral treatment program in Rwanda during 2004–2005. J Acquir Immune Defic Syndr. 2009, 52: 49-55. 10.1097/QAI.0b013e3181b03316.View ArticlePubMed
- Hoskins S, Weller I, Jahn A, Kaleebu P, Malyuta R, Kirungi W, Fakoya A, Porter K: An appraisal of indicators used to monitor the treated population in antiretroviral programmes in low-income countries. AIDS. 2010, 24: 2603-2607. 10.1097/QAD.0b013e32833dd0d3.View ArticlePubMed
- Helleringer S, Frimpong JA, Phillips JF, Awooner JK, Yeji F: Operational study of the quality of health data aggregated by community health workers in the Upper East Region of Ghana. American Public Health Association Annual Meeting. 2010, Abstract number 219301
- Mahmood S, Ayub M: Accuracy of primary health care statistics reported by community based lady health workers in district Lahore. J Pak Med Assoc. 2010, 60 (8): 649-653.PubMed
- Ronveaux O, Rickert D, Hadler S, Groom H, Lloyd J, Bchir A, Birmingham M: The immunization data quality audit: verifying the quality and consistency of immunization monitoring systems. Bull World Health Organ. 2005, 83: 503-510.PubMed CentralPubMed
- Stewart JC, Schroeder DG, Marsh DR, Allhasane S, Kone D: Assessing a computerized routine health information system in Mali using LQAS. Health Policy Plan. 2001, 16: 248-255. 10.1093/heapol/16.3.248.View ArticlePubMed
- Galvao L, Kaye K: Using lot quality assessment techniques to evaluate quality of data in a community-based health information system. Trop Doct. 1994, 24: 149-151.PubMed
- Douglas GP, Gadabu OJ, Joukes S, Mumba S, McKay MV, Ben-Smith A, Jahn A, Schouten EJ, Landis Lewsi Z, van Oosterhout JJ, Allain TJ, Zachariah R, Berger S, Harries AD, Chimbwandira F: Using touchscreen electronic medical record systems to support and monitor national scale-up of antiretroviral therapy in Malawi. PLoS Med. 2010, 7 (8): e1000319-10.1371/journal.pmed.1000319.PubMed CentralView ArticlePubMed
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/12/196/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.