At the core of the discussion was the acknowledgement that the current situation requires us to innovate and adapt, rather than assume that we can return to business as usual. Real-time M&E is required, as scenarios are constantly evolving and changing, and programme priorities including implementation strategies and outcome measures are shifting. As evaluators, we need to be prepared to adapt our M&E design, data collection methods and reporting timelines and criteria.
Ways to adapt
The webinar, as well as the discussions that followed, highlighted five ways we need to adapt and learn now and in the post-COVID-19 world:
- Collecting and sharing information with community and stakeholders. The pandemic has resulted in a restriction on face-to-face data collection and interactions. This has made everyone heavily reliant on ICTs and prioritising non-traditional forms of data collection, such as phone calls or text messages. In other instances, community people have been asked to take photographs and provide verbal and other evidence as part of data collection. Organisations are mainstreaming the use of infographics and other visual presentations to ensure easier dissemination and comprehension.
- Relying on the increased use of third-party monitoring. The restrictions on travel have meant that organisations have been prioritising the development of third-party monitoring strategies. CBOs (community-based organisations) are now increasingly used to collecting data at the local level. These organisations are embedded in communities and bring their knowledge of contextual issues to play. To maximise effectiveness and ensure the availability of high quality, reliable data, CBOs receive regular mentoring and training. In addition, there is a process of cross-checking the data collected using partners on the field.
- Leaving no one behind. The last two months have demonstrated that we can make good use of ICTs and mobile technology. Virtual solutions have proved to be efficient in facilitating interactions and the collection and sharing of data. However, we need to keep in mind that we might not be reaching everybody that needs to be reached. Who are we leaving behind by relying purely on technology and remote data collection? In our new way of working, we need to make sure we take a step back and consider how we reach those with no access to ICTs. Using CBOs and partners on the field has helped us ensure we are leaving no one behind; on SCRAP-C and other projects.
- Ensuring accountability in the use of resources. During a crisis situation like we currently have, there is a tendency for everybody to be ‘doing their own thing’ with limited coordination. MEL professionals can help in ensuring mutual accountability by reporting on issues such as divergence, duplication and poor conceptualisation and targeting of responses. In the Nigerian context, panellists challenged stakeholders to ask questions about the management of the funds mobilised to fight the pandemic in Nigeria. MEL professionals can help improve mutual accountability through helping to make data available on issues like who is providing what resources, how they are used and for what? Who is being reached and with what interventions?
- Ensuring data collection is carried out in an ethical way. Even before COVID-19, the mechanisms to ensure there was ethical approval for data collection in Nigeria have been fairly weak. The use of ICTs and mobile technology for data collection has only exacerbated this situation. Following data protection regulations has also been a challenge in this setting, especially when it comes to transferring data online. As MEL professionals, we need to ensure that ethical issues especially around informed consent, are promoted. On the SCRAP-C project, all staff members and partners, including CBOs and journalists have been trained on ethical issues and they abide by the core ethos of informed consent in eliciting information from citizens.
The new normal
We need to acknowledge that this is a crisis and reassess what we consider “good enough” data. The uncertainty and urgency in the current situation mean that decisions are being made quickly. Having ‘some’ data to support those decisions when they are made is better than waiting for ‘large’ data, which might come too late. A quick interview with a sample of CBOs may be done relatively easily, and results available for immediate decisions. This would probably be more useful than developing and administering a full household survey, which could mean information coming too late.
Looking into the future, we need systems that support continuous programme change by embedding M&E within planning and implementation. We need to adapt our M&E tools to capture what is happening, for example, on how people are being affected and adapting at the local level. This could include CBOs and other grassroots organisations, and the type of support they need.
Itad is the M&E and learning technical partner to an ActionAid-led consortium managing the DFID-funded SCRAP-C (Strengthening Citizens’ Resistance against Prevalence of Corruption) project. As part of SCRAP-C, we’ve been focusing on corruption in the face of Covid-19. Using a network of partners at the local level, we have collected information on the implementation of government palliative measures to citizens in terms of who is reached and with what. Phone calls and text messages have been mostly used in data collection, and where feasible, some community people provide verbal accounts and send photographs, with the support of CBOs. The collated information is regularly shared via several media – print, television, radio and online. There are also interactive sessions on radio and TV, where citizens can dial in and ask questions.
