An evidence-based (or knowledge-based) approach can enhance development cooperation. But managing development assistance at the portfolio level presents unique challenges to knowledge-based decision making, particularly in designing and implementing monitoring, evaluation and learning (MEL).
Recognising this, Norad (the Norwegian Agency for Development Cooperation), has been on a journey to move towards knowledge-based portfolio management and reform their approach.
Throughout this process, Norad has chosen to use the term ‘knowledge’, rather than ‘evidence’. They recognise that diverse forms of knowledge are needed to manage a portfolio: monitoring data, evaluation evidence, research, policy and grey literature, and professional insights and experience. Our work has supported them to build their approach to portfolio management in a way that integrates all these types of knowledge.
We have undertaken several studies, in collaboration with the Christian Michelsen Institute, to help Norad understand progress and suggest ways of moving forward. Our latest evaluation led us to develop a learning brief for Norad which provides practical guidance on putting effective portfolio-level monitoring, evaluation and learning in place. It is highly relevant to anyone working on portfolio-based management. In this blog we summarise some of the key points.
Why portfolio MEL is different
A portfolio is a collection of measures based on a common underlying logic which are designed to contribute to achieving higher-level objectives. Portfolio MEL is integral to understanding what overall results are being achieved and whether progress is being made towards the portfolio objectives. This is different to monitoring or evaluating specific interventions in specific contexts to understand whether and how they are delivering results. Portfolio-level MEL instead involves gathering evidence of results across the range of a portfolio’s interventions, and conducting a higher-level analysis of the extent to which the broader outcomes and objectives of a portfolio are being achieved.
Principles for an integrated approach to portfolio MEL
MEL should serve portfolio management and feed directly into decision making. It should therefore be integrated into the portfolio management cycle and tailored to fit an organisation’s existing systems and touchpoints for decision making. In our work with Norad we identified five main steps to implementing portfolio MEL:
- Develop a portfolio-level theory of change. A portfolio-level theory of change is essential for setting direction, integrating the work of a group of partnerships, and situating their combined contributions within the broader system.
- Define strategic and operational questions for your portfolio. Developing strategic and operational questions aligned with your theory of change will help you to plan a MEL approach that responds to management needs, supports decision making and tracks portfolio progress. Operational questions will help you manage ongoing portfolio work: ‘how can we better support coordination among partners?’. Strategic questions will support high-level, longer term decisions: ‘where else should we intervene in the system?’
- Establish monitoring processes that build a picture of progress and support key decisions. Monitoring works best when you collate different types of knowledge that relate to portfolio progress markers, and regularly create space to reflect on the body of knowledge as a team. Making analytical judgements will give richer, more robust insights into progress than simply aggregating numbers across common indicators. This provides the right insights to make informed operational decisions, while forming one pillar of the evidence base for strategic decision making.
- Plan for evaluations that surface the portfolio’s overall results and progress. Evaluations will provide you with more in-depth, richer analysis of the portfolio’s work, complementing the broader picture established by regular monitoring and assessing results and progress.
- Establish spaces for collective learning and reflection. Learning provides the final component and involves setting up processes for collaborative reflection and sense-making, which allow you to reach judgements about the direction and progress of the portfolio.
The learning brief provides more detail on each of these steps and we recommend you delve in to find out more.
Norad has made great strides in becoming a more evidence-based organisation. We know they are already using these guiding principles to demystify portfolio MEL, and to embed knowledge-based portfolio management, leading to even greater development effectiveness.
