New Tool Offers Streamlined Assessment of Routine Malaria Data Quality to Inform Program Improvements
John, a monitoring and evaluation (M&E) officer for a national malaria control program (NMCP), sits in his office trying to determine malaria control performance in high malaria transmission districts in his country. He looks at results from a handful of data quality assessments conducted in the past year to determine completeness and accuracy of data reported on the number of people tested for malaria and the number of confirmed malaria cases. As he looks at the findings across districts, not all the assessments capture comprehensive information on routine malaria data quality. When available tools are not standardized or, in this case, malaria-specific, it is difficult to get an accurate sense of the quality of malaria data reported by health facilities. This also limits the NMCP’s ability to accurately analyze the state of malaria control at the national or subnational levels.
NMCPs regularly conduct data quality assessments during their supportive supervision visits to health facilities, but the available tools are often not standardized or not malaria specific, leaving M&E officers like John and other decision makers with an incomplete picture of the timeliness, completeness, and accuracy of malaria data, making it challenging to improve data quality or use that data to make NMCPs regularly conduct data quality assessments during their supportive supervision visits to health facilities, but the available tools are often not standardized or not malaria specific, leaving M&E officers like John and other decision makers with an incomplete picture of the timeliness, completeness, and accuracy of malaria data, making it challenging to improve data quality or use that data to make informed decisions. To fill this gap, the MEASURE Evaluation project developed the Malaria Routine Data Quality Assessment (MRDQA) tool. The PMI Measure Malaria (PMM) project, an associate award of MEASURE Evaluation, is working to implement the MRDQA with NCMPs of multiple countries.
The MRDQA is intended to be simple and highly customizable, allowing for any country to apply the tool to its context as part of supportive supervision visits or use the tool on its own. Through pilots in April 2020 in Liberia and DRC, PMM’s malaria surveillance, monitoring, and evaluation (SME) resident advisors successfully facilitated adaptation of the tool to the respective local contexts. The malaria SME advisors conducted workshops and trainings to familiarize local M&E staff with the tool and ensured close collaboration and support during its implementation. The tool ultimately enabled both countries to identify and address gaps in data quality and improve the performance of their malaria surveillance systems. The Excel-based tool evaluates timeliness, completeness, reporting accuracy, and cross-checks consistency of reported data over time.
Facilitating countries’ streamlined monitoring of malaria indicators
M&E staff from the NMCP at the central level or subnational officers can focus on several malaria-specific indicators to review during their data quality audits at health facilities. The key indicators focus on the areas of malaria prevention, malaria testing, malaria treatment, malaria commodity availability, and malaria mortality.
The tool provides auto-generated graphs to visualize the data quality, including verification factors for health facilities (comparison of validated or recounted values with initially reported values) and a dashboard for a snapshot of the overall data quality of routine health information system reporting.
Helping country NMCPs to visualize and address data quality issues
Pilot users of the MRDQA tool noted that it is relatively easy to use, customizable to country needs, and captured information as intended. Staff in both Liberia and the DRC found the sections pertaining to completeness, timeliness, and data accuracy aligned well with their data quality metrics and improvement goals. They also appreciated the rapid assessment capabilities of the MRDQA tool, which allow national and subnational staff to have a “snapshot” of the data quality situation at targeted health facilities through the various integrated dashboards. The auto-generated dashboards within the MRDQA tool provide data visualizations for implementing teams to share with decision makers to summarize findings. PMM used feedback on the MRDQA’s gaps and challenges to revise the tool for future use.
Since the initial pilot, NMCPs in Côte d’Ivoire, Cameroon, Mali, and Sierra Leone have implemented the MRDQA with PMM support.
In Côte d’Ivoire, findings from MRDQAs demonstrated that data for insecticide-treated nets distributed to children under 5 treated for malaria was not included in monthly reports, even though it was included in data collection tools. Action items included district teams working with providers to include this indicator in monthly reports in the future.
In the DRC, the MRDQA revealed underreported data for some indicators due to use of non-standardized or incomplete data collection tools. To address this, provincial staff issued reminders on data collection and reporting guidelines to providers and engaged with implementing partners to ensure supplies of up-to-date tools.
Continuing use of the MRDQA tool to improve data quality
The MRDQA tool enables countries to identify data quality challenges and their sources. The tool also provides corrective actions, such as improving the data collection process through standardized forms, incorporating internal validations prior to reporting to districts, and ensuring proper filling of all forms.
This resource makes John’s job easier and allows for confidence when making data-informed recommendations and decisions. By using the MRDQA tool for malaria data quality monitoring, stakeholders like John can use data visualizations of findings to advocate for data quality improvement and changes to resource management and malaria programming.
This is the second story in a series focusing on the data use cycle to strengthen malaria programs. Read more from the series:
Celebrating 15 years of PMI: Focusing on the data use cycle to strengthen malaria programs