Itad has recently completed a Results Measurement Handbook for the Growth & Employment in States (GEMS) project in Nigeria. GEMS is a Nigerian employment initiative supported by Nigeria’s Federal Ministry of Trade and Investment (FMTI) and funded by the World Bank and the UK’s Department for International Development (DFID).
The project is aimed at job creation and increased non-oil growth. DFID is providing a grant of £90 million and the World Bank a concessionary loan of US$160 million. A key target of the project is the creation of at least 100,000 jobs.
The purpose of the handbook is to provide GEMS with a consistent framework for capturing and reporting results and to ensure a coherent approach to results measurement across the project.
The handbook provides a set of definitions for key terms (the poor, ‘income increase’, employment etc) whose meanings we might think we understand, but which quickly unravel when two sets of results using different methods and measurements (and, crucially, different M&E specialists) are compared.
In addition to definitions, it provides a set of measurement standards and guidelines for each category of intervention in the project, whether focused on skills, introduction of new products and services, advocacy, business development services or business environment reforms. This categorisation is a new approach to the quest for standardised practice. It avoids the restrictions of standardised indicators, but still allows for different projects to be held to account in a comparable and rigorous way.
Finally, the handbook has tried to respond to the fundamental problems of two approaches to evaluating M4P projects by combining them both in a new way. Typical M4P monitoring and evaluation practice is to focus on multiple intervention-specific studies which cannot easily deal with displacement and deadweight effects – that is, it cannot easily tell us whether the economic sector a project is working in (e.g. ‘Nigerian construction’, ‘Kenyan tea’) is really improving overall as a result of the project. Sector wide studies, with broad baselines using a randomized sample of firms, on the other hand, cannot easily deliver attribution between any sector change, and an M4P type intervention. This Handbook proposes both, combined in such a way to mitigate the limitations of each approach.
