Itad has been involved in several TPM contracts across a variety of sectors, ranging from large scale Payment by Result (PbR) programmes in WASH and sexual health, to anti-slavery work in Nigeria and various TPM projects in fragile contexts, including with the UK Conflict, Stability and Security Fund (CSSF).
TPM provides a third-party perspective through collecting and verifying data on the success of a programme. By doing so, it can increase confidence that results are reliable and that it is delivering results for the communities we work in.
In this blog, we share five insights from our TPM work across a range of sectors to help donors and aid agencies who want to learn how to maximise the value of TPM.
1, Supporting longer-term change through learning and adaptation
In our experience, TPM is a useful tool for accountability. However, what is often overlooked, is its ability to help strengthen M&E systems that support longer-term change through learning and adaptation.
TPM can involve regularly assessing monitoring systems, guidance and data to ensure clarity and reliability. When you follow this process with a set of recommendations, it can build strong foundations for learning and can guide conversations about performance. As independent monitors for Achieving Sanitation and Water for All (ASWA) II programme, we followed this process to be able to strengthen our implementing partners’ monitoring and reporting systems.
Including additional analysis in reports about what is happening throughout the programme and why also provides an important learning function. As we’ve seen in our work on the Global Mine Action Programme (GMAP), it also helps justify any adaptation of approaches or targets at the country level.
2, Working with local TPM partners for improved delivery
Local suppliers and personnel are critical to effective TPM. Not only do they enable data collection in hard-to-reach areas, but through their deep and nuanced understanding of context, they support learning and improved delivery.
Local TPM partners can provide insights that are not captured in reports and they can help understand the challenges in-country. As third party monitors of community-focused projects in the Conflict, Stability and Security Fund (CSSF) in Yemen, we ran a learning event together with CSSF, the implementers and our local TPM partner. At the event, our TPM partner was able to share additional insights that helped the CSSF team to better understand the challenges on the ground. This face-to-face exchange helped the CSSF team take shared responsibility to work together and find solutions.
Having TPM personnel in-country who can build relationships of trust with the implementing agencies, leading data collection on the ground and shaping recommendations will mean that they are relevant, practical and realistic. This can greatly increase the legitimacy of our TPM, as we’ve learnt from working on ASWA II and help enhance confidence in results.
3, The risks and benefits of remote data collection
There are risks and benefits to remote data collection under TPM. At Itad, we have gone through a steep learning curve on how to maximise the value of remote methods and mitigate risks over the last year.
Collecting data remotely as part of TPM has its challenges, but the fact that it revolves around working with local partners means that a switch to remote methods was relatively swift – for example by switching from in-person to telephone interviews in certain situations.
However, we found it difficult to remotely collect data in the communities we work with while leaving no one behind. When our local TPM personnel was able to proceed with first-hand data collection at the community level, it proved to be difficult to carry out interviews, as masks hindered free-flowing conversations and checking public facilities led to crowds forming around TPM staff, which had to be carefully managed.
These risks can be mitigated, but data collection will be more time consuming for all stakeholders involved. When moving to remote data collection in health care facilities in the WISH programme, the TPM team led by OPM found that they had to double the data collection time to provide detailed background documents, conduct introductory calls and test the technology (phones, internet, data transfer).
4, The added-value of Payment-by-Results
Some TPM contracts take the form of payment-by-results (PbR) where payment is made on the completion of the successful verification of results.
For PbR to be successful, it needs to focus on results that are comparatively measurable over time.
In the WASH Results Programme, for example, we measured sustained access to latrines, water supply and regular handwashing with soap for up to two years after the implementation phase. Setting targets for sustaining access and hygiene practices and paying for their achievement supported driving forward sustained access to WASH services at scale across 11 countries.
Careful design is also key when it comes to PbR programmes achieving their goals. With the focus being disproportionately on results, considerations such as equity need to be hardwired into the programme from the outset.
Another aspect of the programme that needs clear and detailed guidance from the start is the measurement framework underlying the PbR mechanism. As we’ve learnt during the WASH Results Programme, this can lead to time-consuming negotiations and have hidden risks for implementers. Along with clear guidance, an explicit risk-sharing arrangement can help maximise the benefits of PbR.
5, The importance of coordination to maximise the value of TPM
Coordination between clients, implementers and independent monitors – specifically around sharing programme monitoring data and TPM methodologies – is critical in ensuring TPM investments are useful, timely and avoid duplication.
Sharing lessons throughout a programme can be invaluable towards making processes run smoothly and work for all partners involved.
In the WASH Results Programme, we planned in time for reflection between all parties on what worked well, what didn’t and how to adjust the verification process and the data requirements. This was especially important at the start of the programme.
As TPM agents, we can also help improve coordination and alignment. In GMAP for example, our initial research led to a better alignment of data collection approaches measuring behaviour change around mine risk education across the sector. This has resulted in us being able to facilitate improved coordination between the UK government and the Dutch Ministry of Foreign Affairs, and being invited to develop a common Theory of Change for the two donors.
Looking ahead: The increasing importance of learning and adaptation in TPM
An independent perspective on aid delivery is often critical for ensuring money is spent effectively. However, although TPM can support both accountability and learning, prioritising the former can lead to missed opportunities for the latter.
Aid is often delivered in complex and highly dynamic contexts, exacerbated by COVID-19 and the still unknown impact this is going to have on poverty, conflict and humanitarian emergencies. So, it is important that TPM enables frequent evidence-based reflection, learning and adaptation through combining flexibility and agility with thoroughness.
There is an opportunity for greater coordination between donors and implementers around how TPM is contracted and used, as well as sharing of learning across donor community about how to deliver effective and useful TPM.
If you would like to know more about our work on TPM, please get in touch with Katharina.Welle@itad.com
