The concept of ‘resilience’ has gained a lot of attention in the development sector recently, attention which has translated into large scale investment in resilience building programmes and partnerships (See DFID BRACED, Global Resilience Partnership, USAID). There has been much lively debate about how best to conceptualise resilience in the context of development.
Only recently have practitioners begun to move beyond the conceptual to the practicalities and realities of actually trying to measure resilience. We may have precise (though often debated) measures and thresholds for what makes a household poor or vulnerable but what makes a resilient person, family, community or system? Other common questions which we address in this blog include: how do we capture the complex, context specific nature of resilience in the face of different types of shock without adopting a reductionist, over-simplified approach?; What is our unit of analysis and at what time scale and frequency should we measure resilience? And; Can we truly measure resilience without testing our methods in the face of a shock?
At the Evaluation Conclave 2015 in Kathmandu, a panel of practitioners from across the region and beyond was convened by Itad and Sambhodi, with the support of the Rockefeller Foundation, to share experiences and stimulate debate about how best to address these questions. The panel members, chaired by Itad’s Climate Resilience Theme Leader, Robbie Gregorowski, were challenged to share practical examples of their resilience measurement efforts in a range of country and thematic contexts. Drawing on our experience in the DFID Building Resilience and Adaption to Climate Extremes and Disasters (BRACED) programme – one of the world’s largest resilience strengthening programmes in terms of investment, geographical coverage and ambition – and examples provided by the panellists from other programmes, we highlight the key emerging messages from the session here.
Resilience as an intermediate outcome characterised by capacities and assets
Rather than a discrete unit to be measured, the panel agreed that resilience should be considered as the combination of capacities, assets and the ‘ability’ of individuals, households, communities, institutions and systems to deploy them to anticipate, avoid, recover from and adapt to shocks and stresses. Therefore, resilience strengthening programmes are focused on building and maintaining these capacities and assets, which can be physical, social and cultural, within a system (household, community, forest, market etc.) in the face of shocks and stresses. The focus therefore should be on measuring the ‘building blocks’ or intermediate level outcomes, of resilience conceptualised as assets, resources, skills and capacities.
This may seem like nothing new – think Sustainable Livelihoods Framework – but the crucial difference is that resilience can only truly be understood in the context of defined and identifiable shocks or stresses. For example, building household assets in itself may begin to lift a family out of poverty, but consider what happens to those assets in the event of a flood. A resilient household will be able to deploy their newly built asset base to absorb the worst impacts by combining information, knowledge and skills (new and existing) to make critical decisions in extraordinary circumstances. We should therefore see little disruption to any development gains in spite of a shock or stress.
The additional utility of this focus on sets of capacities and assets is that they can be tailored to, and made relevant for, all resilience strengthening stakeholders within a system, and also can be translated to work across contexts, scales and over time. This represents an opportunity to apply the same resilience measuring principles, albeit with context and theme specific indicators and metrics, to different resilience strengthening programmes. Indeed, there does seem to be some consensus forming that resilience measurement (broadly and not just climate resilience) should be conceptualized as an intermediate outcome (i.e. as a means not an end in itself) based around sets of capacity and asset indicators. To bring this to life, some panellists presented various examples of using scorecards in a composite index of indicators, others organized a basket of indicators around criteria and principles of forest resilience, as well as one example of using more complex adapted econometrics to measure the resilience of a rural energy network.
The importance of scale & context
It is important to remember that while we in the climate resilience theme at Itad are focused on climate related shocks (e.g. floods) and stresses (e.g. delayed onset of rains) in our work, that these shocks and stresses could be economic (e.g. commodity price changes), related to health (e.g. epidemics like the recent Ebola outbreak) or be socio-political in nature (e.g. conflict). In this way, resilience can cut across these thematic areas or development ‘silos’ as a unifying concept and one which shifts the emphasis from vulnerability of passive ‘beneficiaries’ to active agents with capacities and assets relevant to their position and role within a resilience strengthening system.
Broadening the framework to include resilience capacities and assets of a wider set of actors within a system opens up the opportunity to measure resilience strengthening efforts across scales, contexts and over time. One of the central challenges to resilience measurement, based on experience in a programme like BRACED, is that frameworks need to be coherent across intervention, project, programme and system scales. Additionally, any measurement framework needs to be flexible enough to be relevant across a number of different socio-political, geographical, agro-ecological and climatic contexts, while at the same time retaining robustness and coherence. No mean feat. In the BRACED programme, each project has their own monitoring and evaluation (M&E) approach which has been harmonised through the provision of guidance notes, templates and 1-2-1 support organized around our interpretation of climate resilience (3As framework) and an adapted outcome mapping approach to measuring change using progress markers (Areas of Change).
The remaining challenge – measuring resilience in the face of shocks and stresses
If we accept that resilience is about improving and maintaining the wellbeing of people in the context of shocks and stresses then we will only truly be able to test this if we measure the constituents of that wellbeing during or after shocks and stresses have occurred. If wellbeing is maintained, enhanced or doesn’t suffer in the same way as before then we may consider that community or system to have enhanced resilience. Whether we can then attribute this to the activities and interventions of a programme given the inherent complexity is debatable and beyond the scope of this article. However, there is much more work required to fully understand the practice of measuring resilience and it is of course reliant on the occurrence of shocks and stresses (sadly all too frequent) to test the measurement frameworks which are still largely unproven.
The next steps – equipping M&E practitioners with the tools to measure resilience
In sharing the collective experience of the measuring resilience panel of experts at the Evaluation Conclave 2015, and judging from the response in the room, the issues discussed here clearly resonate and represent the start of wider discussion on this topic. In the field, verifiable evidence of impact among programs seeking to build resilience remains scarce; practitioners often lack the necessary experience and knowledge to use existing resilience measurement tools in ongoing monitoring and evaluation efforts; and they often lack access to resilience knowledge being produced by academics and think-tanks, including new measurement approaches.
In recognition of this, Rockefeller Foundation launched a Resilience Measurement Community of Practice (CoP). The 8 month inception phase is being implemented by the Windward Fund with the objectives of working with a core group of global M&E practitioners and resilience measurement specialists to learn from their experience and to help build the knowledge, experience and skills needed to generate the evidence base for resilience investments.
It aims to engage with M&E practitioners and resilience measurement specialists to:
- Determine the needs of practitioners in understanding, measuring and producing evidence around resilience building programming
- Support practitioners in gaining a fundamental understanding of the concepts and frameworks of resilience and strengthen their skills and experience in monitoring and evaluating resilience projects; and
- Consolidate, build and elevate the evidence base for resilience benefits so that funders, implementing partners and implementers can learn, adjust strategies as needed, and scale up resilience efforts and investments.
Rockefeller Foundation and Windward Fund are committed to a wide reaching consultative process in the inception phase to ensure that the CoP is inclusive and effective. If you are currently engaged in measuring, monitoring and evaluating resilience projects, programs and/or investments, and would like to be part of ongoing exchange of experience and lessons in this new and interesting field, please contact: Dr Maliha Khan, Interim Coordinator, Resilience Measurement-M&E Community of Practice (ResilienceCoP@gmail.com).
Itad is pleased to be part of this global conversation and will continue to share insights and lessons learned through delivering Monitoring, Evaluation and Learning (MEL) work in this area.
Dave Wilson, Robbie Gregorowski & Sarah Standley, February 2016
