The Department for Foreign Affairs and Trade (DFAT) commissioned Itad to undertake a review of 2012 operational evaluations to address a need to improve the quality of its evidence in measuring the effectiveness of its aid.
The Review has two objectives: to promote good quality independent evaluations; and to inform the Minister, public, partners and aid program staff of overarching lessons emerging from the findings of independent evaluations.
Our approach
Itad assembled a small team of three consultants with skills in synthesis evaluations, performance assessment and quality assurance. The assignment was delivered in two stages: 1) a review of the quality of all independent evaluations commissioned by DFAT in 2012. This involved developing a quality assurance template which was used to review all 87 operational evaluations and identify key strengths and weaknesses. For a sample of evaluation we then interviewed both the DFAT evaluation mangers and the evaluation teams to understand drivers and barriers to evaluation quality; 2) a synthesis evaluation of a sample of reports to draw out key lessons for DFAT. 64 evaluations that were considered of sufficient robustness in the quality review were taken through into a synthesis that sought to identify the main lessons for DFAT. Based on an initial review of the evaluations a series of hypotheses were developed, these were then further refined and revised based on further reviews of the report by the team. The strength of the evidence supporting the hypothesis was then assessed and the best evidenced hypothesis was taken through into the final synthesis.
Read the final report on the quality of Australian Aid operational evaluation.
Read the synthesis report on learning from Australian Aid operational evaluations.
Contact Rob Lloyd (rob.lloyd@itad.com) if you would like to discuss this project.
