The article explored serious concerns raised in the journal BMJ global health about major development funders such as DFID exaggerating the impact of their programmes or ‘gagging’ evaluations when they aren’t as positive as expected.
As a company that does a lot of evaluation for the UK government, including DFID and the FCO, it provoked some discussion. Our sense was that, while we have experienced a healthy amount of back and forth and challenge from DFID on our evaluation findings and recommendations, we have never felt ‘gagged’. Providing we can substantiate what we are saying with a clear and robust evidence base, we have always been able to provide an independent, oftentimes critical, evidence-based voice on what is working and what isn’t across UKAID.
But beyond this immediate reaction, the article got us thinking about the importance of independence in evaluation, and what ‘independence’ means, beyond a black or white, ‘right or wrong’ interpretation.
Independence is crucial to the credibility and integrity of evaluation findings around what works, what doesn’t, why and for whom. When the credibility of an evaluation is diminished, its ultimate usefulness for accountability, learning, programme improvement and resource allocation is lost. However, it is important to recognise that independence isn’t as simple as keeping the funder at arm’s length from the evaluation and then foisting conclusions and recommendations on them at the end of the process. Our experience is that ownership of evaluation findings and buy-in to conclusions and recommendations is a driver of engagement and a catalyst for evidence-informed change.
So, as evaluators, how can we operationalise independence?
By the very nature of our work, we have to be one step removed from the implementation process. At the same time, we remain engaged with the funder providing insight and learning through our work so that they can adapt and, we hope, improve their programming. At Itad, this means we aim to be ‘learning partners’ as well as evaluators – we work with our clients to adapt, co-create and learn – we don’t believe this dilutes or compromises our independence, but it does mean that we need the skills and approaches to navigate complex situations while maintaining high ethical standards.
- For successful adaptive programming, we need to be closer to the implementation team than we would be if we were conducting a more traditional evaluation – something that could compromise our independence. We first blogged about the potential difficulties of adaptive programming back in 2016, and we are still mindful of these even as funders have become more flexible and open to adaptation – but the integration of evaluators can and does improve implementation, so it’s down to us to ensure that we strike the balance between involvement and independence.
- Another part of our work that is becoming more and more common is the co-creation of recommendations. We believe that by working with the end users of evaluations to jointly draft the recommendations and actions, based on the independently collected, analysed and presented findings, we can increase the likelihood that they will be used to inform future programming. This doesn’t undermine our independence – it challenges the funder to think about how they use our work and come up with recommendations that are implementable.
- As evaluators, we need to make sure we are building the most effective relationships and sharing our findings in the most engaging and useful way possible. There is a time and place for us to be uncompromising with our findings, but if we are truly to ensure that our work is used to benefit future programmes, we need to deliver messages in the right way, at the right time and in the right language – this navigation doesn’t mean we are diluting the messages or compromising our independence, it just means we are mindful of the social and political realities in which we operate and want to maximise the usefulness of our work.
We work on a lot of complex development programmes and our independence is central to what we do here at Itad, but it can be tricky to navigate our way through funder needs, public scrutiny and our own ethical standards. Our work is engaging and consultative – we believe this is how we can support our clients and partners to make the biggest difference – but this does mean that we need to have the skills and tools to make sure that we maintain our independence in complex and dynamic contexts. It’s not easy, nor is it straightforward, but it is something we believe is central to use being able to deliver more meaningful, and ultimately more useful, results in development.
