Methodology
What is ex-post impact assessment?
Impact assessment is a type of evaluation that is intended to determine the consequences of an intervention, in terms of outcomes of interest. This analysis can either be ex-ante, conducted before the intervention has been initiated and outcomes have been produced, or ex-post, which measures outcomes that have actually resulted from the intervention to date. This website primarily concerns ex-post impact assessment (epIA) of research activities of the CGIAR.
Ex-post impact assessment is only one component of a comprehensive evaluation package, which also includes ex-ante assessment, programme reviews, performance monitoring and evaluation and process evaluations, among others. In comparison, ex-post impact assessment is “intended to determine more broadly whether the program had the desired effects on individuals, households and institutions and whether those effects are attributable to the program intervention” (Baker, 2000 - pdf).
The relationship between these different forms of evaluation is indicated in figure 1. One factor that principally distinguishes ex-post impact assessment from other forms of evaluation is that it is a summative form of evaluation, whereas all the other types are formative. Formative evaluation is primarily concerned with providing information during the implementation phase on how to improve the research program. Summative evaluation is concerned with the programme’s effectiveness, value or impact and is conducted after programme completion for the benefit of an external audience (MacKay and Horton, 2003).
Who does ex-post impact assessment in the CGIAR?
Ex-post impact assessments are conducted by impact assessment specialists in the International Agricultural Research Centres (IARCs), often in collaboration with national and international experts from partner institutions. Each IARC has an impact assessment focal point who is often positioned within the Office of Director-General or within the Economics/Social Science unit.
Impact assessment is supported at the system-level by the Independent Science and Partnership Council’s (ISPC) Standing Panel on Impact Assessment (SPIA). SPIA’s predecessor, the Impact Assessment and Evaluation Group (IAEG), was initiated in 1995, and reorganised into SPIA in 2000.
Unique methodological challenges for ex-post impact assessment of agricultural research
Attribution among multiple partners
The seminal publication on impact assessment for agricultural research is the book "Science Under Scarcity" by Alston, Norton and Pardey (1998), which outlines some of these issues and how they can be handled when estimating the costs and benefits of investments in agricultural research.
Paper by de Janvry, Dustan and Sadoulet on new impact assessment methods for the CGIAR
This paper was commissioned by SPIA in 2010 and appraises a range of recent methodological innovations that are now available to practitioners when carrying out analyisis of the impact of agricultural research. It argues for greater focus on methods that provide rigorous causal inference. Download the de Janvry et al (2011) paper here (pdf)
Strategic Guidance Document
SPIA publishes a guidance document “Strategic Guidance for ex-post Impact Assessment of Agricultural Research”. This was first published in 2008, and will be regularly supplemented as new methodologies emerge in the literature for practitioners to make use of. The document will be re-versioned periodically.
Download the Strategic Guidance document (PDF)
Search site
Share this page
