Building on an earlier exploratory study, in 2007-2008 the CGIAR's Standing Panel on Impact Assessment (SPIA) undertook an initiative in collaboration with seven CGIAR centers to augment the evidence of policy-oriented research (POR) impacts within the CGIAR system and to further the development of methodologies in this challenging area of impact assessment. Seven case studies were commissioned. This impact brief describes the major results that emerged from the overall study.

This study critically reviews and assesses the large body of evidence on the impacts of agricultural research by the CGIAR and its partners in South Asia. The long history of research, the extensive databases available and the vast literature on impacts that exist in this region provided a fertile ground for this study, which aimed to systematically examine and understand the complexities of how research has led to outputs, uptake, outcomes and impacts, and the distributional consequences of these. Implications and lessons are drawn from this synthesis of the literature to address the issues of the gross (positive and negative) and net payoffs from past investments by the CGIAR and partners (the accountability question), as well as to help shape current and future priorities (the institutional learning question). The study also identifies the knowledge gaps and researchable questions that should improve our understanding of opportunities for, and impediments to, agricultural technology enhancement as a strategy for achieving future CGIAR goals, namely poverty alleviation, food security, and environmental sustainability.

This document provides guidance to impact assessment practitioners and research managers for planning, conducting and managing ex post impact assessments. The primary emphasis in this document is on epIA from the accountability perspective. These guidelines do not attempt to indicate what best practices are but rather looks at the options and discusses their pros and cons as a prelude to offering ‘good practice’ advice at the end of each section. This is in recognition of the fact that it is not possible to be definitive in many instances.

Although policy-oriented research (POR) constitutes a large and growing portion of the portfolio of the CGIAR, it is an area with relatively few documented impacts. SPIA launched a scoping study to review the state of POR impact assessment in the CGIAR and to assess whether new studies are desirable and feasible methodologically. This report is based on a synthesis of 25 ex post impact assessments studies that trace the diffusion (uptake), influence and/or impact of POR outputs. While POR impact assessment is a challenging area, there are also promising signs of progress.

This case documents ICRAF's contribution to the development of soil-fertility management technologies that use on-farm resources to replenish soil fertility, including fertilizer tree fallows. Evidence shows that 27% of households had planted an improved fallow by 2003, after 3 years of development efforts. Furthermore, farmers had planted improved fallows in other collaborating countries. Assuming a 20-year time horizon (1988-2008), the IRR was estimated at only 3.2%. However, if the time period is expanded to 25 years, the IRR rises to 15.2%, and further to 20.8 % for a project ending in 2018.

CIAT, together with NARS, worked with farmers to identify, test and adjust promising natural resource conservation and productivity enhancement cassava technologies. The impact study of the project conducted in 2003 in Vietnam and Thailand shows that the IRR for the project was 41.2%. Various scenario analyses revealed that the rate of return of the R&D investment was indeed a safe bet considering that the most conservative scenarios still yielded an IRR of 20%.

The transfer of irrigation systems management (IMT) responsibilities to farmer organizations is expected to lead to higher productivity and sustainability. IWMI was involved in reviewing and analyzing the experiences and impacts of past IMT processes; advising policy-makers in planning and implementing IMT; technical support to governments implementing IMT programs; the development of generic IMT guidelines; and, for the establishment of water users' associations (WUAs). This case study concentrates on measuring the outcomes of the NRMR for the overall IMT knowledge base and on IMT policy and operations in specific countries. The three areas of outcome of its research and interventions include: (i) awareness of new IMT research results among its developing country clients; (ii) application of better policies; and (iii) better techniques in irrigation management.

The most widely adopted resource-conserving technology in the IGP, particularly in India, is ZT. CIMMYT's role with regard to ZT in India was to make the diffusion process faster and more efficient. CIMMYT facilitated technology introduction by helping with the design experiments for technology testing and local adaptation. Using conservative benefit estimates the program yielded a net present value (NPV) of US$94 million; equivalent to a benefit-cost ratio (BCR) of 39 and an IRR of 57%. The economic surplus primarily benefited consumers, at 65%, compared to producers, at 35%.

ICARDA and collaborating NARS have developed alternative cropping systems that can enhance and stabilize production by supplying animal feed and reducing soil erosion under low rainfall and in marginal lands. Two separate impact studies in Tunisia and Morocco indicate that the increased supply of animal feed led to a reduction in feed costs of 13% and 33%. The economic internal rate of return (EIRR) of the project in Tunisia is estimated at 16%. Simulation analysis for robustness of estimates shows that the chance of getting an IRR of more than 4% was found to be 95%. Likewise, in Morocco, based on a constant adoption rate at the 2005 level, an estimated IRR of 29% was calculated for 2015, the project termination year.

The WorldFish Center with the Department of Fisheries in Malawi, implemented an integrated aquaculture-agriculture (IAA) project, which uses farm waste, crop by-products and other natural resources from the farm environment as nutrient inputs for fishponds. An impact study this NRMR project indicates that, assuming that two-thirds of the observed growth in aquaculture production is attributable to the WorldFish project, the net present value (NPV) of the project is estimated at US$3,056 million, and an internal rate of return of 12.2%. Most of the benefits (60%) go to consumers through lower fish prices.

CIFOR has contributed to the development of criteria and indicators (C&I) for the various dimensions of sustainable forest management and produced a comprehensive set of manuals, decision-support software tools that guide users and other publications. By the end of the overall research program, more than 10 million hectares of forest had been certified worldwide, rising to more than 45 million hectares by October 2004. However, because of the difficulty of quantitative attribution of CIFOR's research contribution to certification standards, this case study commenced on an exercise to trace impact pathways that helped to target clients and their particular needs. It then analyzed the target clients in more detail through documentation, interviews and other means to determine the uptake of CIFOR research outputs.

Between 1987 and 2001, research based on a sample survey of 1,245 households was undertaken by the Bangladesh Institute of Development Studies and IRRI on the impact of Modern Varieties (MVs) of rice on livelihoods in Bangladesh. Results showed that the adoption of MVs had significant positive impacts on crop yields and farm incomes for households with access to land. However, as rice farming accounted for only 20 per cent of total household incomes in 2001, the overall impact on incomes was relatively small. Although the profitability of rice is declining due to falling prices, higher input costs and reduced farm sizes, the crop nevertheless contributes greatly to food security and acts as an entry point to off-farm employment.

A CIMMYT survey of 43 countries showed that between 1988 and 2002, public national research organizations and private companies in the developing world released nearly 1,700 new wheat varieties, of which about 75 percent have some CIMMYT ancestry. Using 2002 adoption data, the additional annual production attributable to CIMMYT wheat breeding research is valued at US$0.5 to $3.9 billion, depending on the assumptions used. Whatever assumptions are used, the sum is equivalent to many times CIMMYT's annual investment in wheat breeding research.

In 1987, the International Center for Living Aquatic Resources Management (ICLARM, now the WorldFish Center) and a number of research agencies launched a collaborative project called Genetic Improvement of Farmed Tilapia, or GIFT. The objective of the GIFT project was to raise fish productivity amongst low-income fishers in order to increase protein consumption in poor populations in developing countries. This brief documents the achievements of the project, including, among others, the systematic collection of improved germplasm, the development of a selected breeding approach, the creation of a genetically diverse base population and adoption of improved tilapia on a wide scale. The development and dissemination of GIFT has proved a worthwhile investment with an internal rate of return (IRR) of more than 70 per cent.

This brief takes stock of CIP's contribution to varietal change in potatoes in 30 developing countries between 1993 and 1999. The countries sampled account for about 85 per cent of developing-country potato production, in terms of both area and volume. Survey results indicated that about 23 per cent of the varieties released during the period were at least partially CIP-derived. CIP-derived material is more common in sub-Saharan Africa where about 40 per cent of the potato-growing area was planted to CIP-derived materials. The internal rate of return to investment is estimated at 15 to 17 per cent.

ICRISAT, in collaboration with national program partners in Asia and sub-Saharan Africa, assessed the spillover potential of sorghum varieties and hybrids across eight Sorghum Research Domains. The study demonstrated that cultivars originating from collaborative national and international research can prove highly transferable across different environments. The spillover of finished products, however, tended to be negatively correlated with national research capability: the stronger the national program, the lower the potential for the direct release of varieties and hybrids.

A Food for Education (FFE) program, which rewarded poorer families in Bangladesh with food in return for regular attendance of their children at school, led to a 20 to 30 per cent increase in school participation rates. Seven years after the start of the program, more than one-quarter of all primary schools or 13 per cent of all primary school students (more than 2 million) were covered by the scheme. The return to IFPRI's involvement in the FFE program, which included IFPRI's positive evaluation of the programs early expansion, was conservatively estimated at 64 to 96 per cent.

Success stories have followed one after another in the decades since the Consultative Group on International Agricultural Research ( CGIAR) became a focus for global investment in research for development. Funding for the CGIAR nevertheless began to stagnate in real terms in 1988, with some Members apparently disturbed that "cherry picking" successes sidestepped the question of the CGIAR's overall productivity and value for money.