Some of the e-Pact Monitoring and Verification (MV) team recently attended the WEDC Conference – an annual international event focused on water, sanitation and hygiene (WASH), organised by the Water, Engineering and Development Centre (WEDC) at Loughborough University. One of the key themes this year was the Sustainable Development Goals (SDGs): what they are, how close we are to achieving them, and how we are going to monitor them. The SDGs are important for PBR programmes because they influence what programmes aspire to achieve and how they measure their progress.
The recent publication of the first report (and effective baseline) on SDG 6, covering drinking water, sanitation and hygiene, marked a watershed. With the shift to understanding universal and equitable access, the inclusion of hygiene and a focus on ‘safely managed’ and ‘affordable access’, the breadth and depth of data we aspire to have on water and sanitation services is unprecedented. But the first SDG progress report also highlights a yawning data gap: for example, estimates for safely managed drinking water are only available for one third of the global population, and we are only starting to get to grips with how to measure affordable levels of service.
As part of the WASH Results Programme, the three consortia are constantly grappling with how to objectively measure complex outputs and outcomes linked to water, sanitation and hygiene. At the same time our MV team is trying to understand how we can verify such measures, and if they are sufficiently robust to make payments against. How do the SDGs influence this process? We have three reflections from our experience of verifying results under the WASH Results Programme:
Reflection 1: the relationship between the SDGs and PBR-programming can be mutually beneficial.
The SDGs help PBR programmes to set ambitious benchmarks
It’s clear that to track progress against the SDGs, the WASH sector is going to have to become a lot better at collecting, managing and analysing an awful lot of data. One of the learning points from the WASH Results Programme is that the verification process requires the consortia (and in-country partners) to take data far more seriously.
Compared to more conventional grant programmes, Monitoring and Verification functions take on the importance of financial reporting. One function of this, is that everyone has more confidence that reported results (whether access to water, number of latrines built or handwashing behaviour) accurately reflect reality. As such, PBR programmes can help focusing peoples’ attention on improving service levels.
Conversely, the SDGs help PBR programmes to set ambitious benchmarks and provide an orientation on how to measure them. This is proving important under the WASH Results Programme, which has, at times, struggled with aligning definitions, targets, indicators and how to measure them.
Reflection 2: some of the SDG targets are hard to incorporate into a PBR programme
Physical evidence of a handwashing facility doesn’t guarantee use at critical times
Measuring hygiene behaviour change illustrates this point neatly: the simplest way to understand if people are washing their hand with soap may appear to be just to go out and ask them. Yet self-reported behaviour indicators are notoriously unreliable. Looking for physical evidence of a handwashing facility (with water and soap) is the approach currently suggested by the WHO/UNICEF Joint Monitoring Program (JMP), but there is no guarantee that people use such facilities at the most critical times, for example, after defecation or before handling food.
Under a PBR programme (where implementers get paid against pre-defined results) the temptation to take the shortest route to success, namely focusing on getting the hardware in place, may be high. Therefore, it may be important to complement this indicator with a knowledge-related indicator to also capture behaviour change albeit in a crude way. This brings along another challenge: how to agree on appropriate, payment-related targets in a situation where experience on how to accurately measure behaviour change is still in its infancy?
Reflection 3: keeping indicators lean is challenging when faced with the breadth and depth of the SDGs
Hygiene behaviour change is just one indicator. Attempting to robustly measure changes across three consortia, eight result areas and two phases (output and outcome) has resulted in the MV team reviewing a large amount of surveys, databases, and supporting evidence since 2014.
Under the WASH Results programme, the sustainability of services is incentivised via payment against outcomes: people continuing to access water and sanitation facilities and handwashing stations for up to two years after they gained access to improved services. In the meantime, between the final MDG report, and the initial SDG report, the number of data sources used by JMP to produce estimates for the water, sanitation and hygiene estimates has more than doubled1. Instead of more traditional household surveys, increasingly, data is obtained from administrative sources such as utilities, regulators and governments.
How to marry these new data ambitions with the necessary goal to keep the number of indicators manageable under a PBR programme will be an interesting challenge going forward.
This blog post was originally published on the e-Pact website: Learning about Payment by Results in Water, Sanitation & Hygiene (WASH)
