Although policy-oriented research (POR) constitutes a large and growing portion of the portfolio of the CGIAR, it is an area with relatively few documented impacts. SPIA launched a scoping study to review the state of POR impact assessment in the CGIAR and to assess whether new studies are desirable and feasible methodologically. This report is based on a synthesis of 25 ex post impact assessments studies that trace the diffusion (uptake), influence and/or impact of POR outputs. While POR impact assessment is a challenging area, there are also promising signs of progress.

This study assesses the impacts of the CGIAR system's investments in NRM research. SPIA initiated the research to respond to the concerns regarding the dearth of credible, documented evidence of natural resource management (NRM) research contribution to the CGIAR's mission of reducing poverty and improving food security while protecting the environment. This brief summarizes the results of seven case studies of CGIAR center's and partner's NRM research impact assessments which were diverse both geographically and in terms of the type of research.

This brief reports on a meta-analysis of evidence of CGIAR and partners' research impact in SSA. The study compared the documented benefits of the CGIAR/NARS research in SSA with a comprehensive estimate of all contributory research costs in the region, rather than those of CGIAR centres alone. In five of the six scenarios considered, the benefit/cost ratio ranged from 1.0 to 1.64. The fact that such a small number of impact assessment case studies (25) results in benefits that exceed the costs of all activities in the region indicates a considerable achievement on the part of the CGIAR centres and their partners.

Between 1987 and 2001, research based on a sample survey of 1,245 households was undertaken by the Bangladesh Institute of Development Studies and IRRI on the impact of Modern Varieties (MVs) of rice on livelihoods in Bangladesh. Results showed that the adoption of MVs had significant positive impacts on crop yields and farm incomes for households with access to land. However, as rice farming accounted for only 20 per cent of total household incomes in 2001, the overall impact on incomes was relatively small. Although the profitability of rice is declining due to falling prices, higher input costs and reduced farm sizes, the crop nevertheless contributes greatly to food security and acts as an entry point to off-farm employment.

A CIMMYT survey of 43 countries showed that between 1988 and 2002, public national research organizations and private companies in the developing world released nearly 1,700 new wheat varieties, of which about 75 percent have some CIMMYT ancestry. Using 2002 adoption data, the additional annual production attributable to CIMMYT wheat breeding research is valued at US$0.5 to $3.9 billion, depending on the assumptions used. Whatever assumptions are used, the sum is equivalent to many times CIMMYT's annual investment in wheat breeding research.

In 1987, the International Center for Living Aquatic Resources Management (ICLARM, now the WorldFish Center) and a number of research agencies launched a collaborative project called Genetic Improvement of Farmed Tilapia, or GIFT. The objective of the GIFT project was to raise fish productivity amongst low-income fishers in order to increase protein consumption in poor populations in developing countries. This brief documents the achievements of the project, including, among others, the systematic collection of improved germplasm, the development of a selected breeding approach, the creation of a genetically diverse base population and adoption of improved tilapia on a wide scale. The development and dissemination of GIFT has proved a worthwhile investment with an internal rate of return (IRR) of more than 70 per cent.

This brief takes stock of CIP's contribution to varietal change in potatoes in 30 developing countries between 1993 and 1999. The countries sampled account for about 85 per cent of developing-country potato production, in terms of both area and volume. Survey results indicated that about 23 per cent of the varieties released during the period were at least partially CIP-derived. CIP-derived material is more common in sub-Saharan Africa where about 40 per cent of the potato-growing area was planted to CIP-derived materials. The internal rate of return to investment is estimated at 15 to 17 per cent.

ICRISAT, in collaboration with national program partners in Asia and sub-Saharan Africa, assessed the spillover potential of sorghum varieties and hybrids across eight Sorghum Research Domains. The study demonstrated that cultivars originating from collaborative national and international research can prove highly transferable across different environments. The spillover of finished products, however, tended to be negatively correlated with national research capability: the stronger the national program, the lower the potential for the direct release of varieties and hybrids.

A Food for Education (FFE) program, which rewarded poorer families in Bangladesh with food in return for regular attendance of their children at school, led to a 20 to 30 per cent increase in school participation rates. Seven years after the start of the program, more than one-quarter of all primary schools or 13 per cent of all primary school students (more than 2 million) were covered by the scheme. The return to IFPRI's involvement in the FFE program, which included IFPRI's positive evaluation of the programs early expansion, was conservatively estimated at 64 to 96 per cent.

This case documents ICRAF's contribution to the development of soil-fertility management technologies that use on-farm resources to replenish soil fertility, including fertilizer tree fallows. Evidence shows that 27% of households had planted an improved fallow by 2003, after 3 years of development efforts. Furthermore, farmers had planted improved fallows in other collaborating countries. Assuming a 20-year time horizon (1988-2008), the IRR was estimated at only 3.2%. However, if the time period is expanded to 25 years, the IRR rises to 15.2%, and further to 20.8 % for a project ending in 2018.

CIAT, together with NARS, worked with farmers to identify, test and adjust promising natural resource conservation and productivity enhancement cassava technologies. The impact study of the project conducted in 2003 in Vietnam and Thailand shows that the IRR for the project was 41.2%. Various scenario analyses revealed that the rate of return of the R&D investment was indeed a safe bet considering that the most conservative scenarios still yielded an IRR of 20%.

The transfer of irrigation systems management (IMT) responsibilities to farmer organizations is expected to lead to higher productivity and sustainability. IWMI was involved in reviewing and analyzing the experiences and impacts of past IMT processes; advising policy-makers in planning and implementing IMT; technical support to governments implementing IMT programs; the development of generic IMT guidelines; and, for the establishment of water users' associations (WUAs). This case study concentrates on measuring the outcomes of the NRMR for the overall IMT knowledge base and on IMT policy and operations in specific countries. The three areas of outcome of its research and interventions include: (i) awareness of new IMT research results among its developing country clients; (ii) application of better policies; and (iii) better techniques in irrigation management.

The most widely adopted resource-conserving technology in the IGP, particularly in India, is ZT. CIMMYT's role with regard to ZT in India was to make the diffusion process faster and more efficient. CIMMYT facilitated technology introduction by helping with the design experiments for technology testing and local adaptation. Using conservative benefit estimates the program yielded a net present value (NPV) of US$94 million; equivalent to a benefit-cost ratio (BCR) of 39 and an IRR of 57%. The economic surplus primarily benefited consumers, at 65%, compared to producers, at 35%.

ICARDA and collaborating NARS have developed alternative cropping systems that can enhance and stabilize production by supplying animal feed and reducing soil erosion under low rainfall and in marginal lands. Two separate impact studies in Tunisia and Morocco indicate that the increased supply of animal feed led to a reduction in feed costs of 13% and 33%. The economic internal rate of return (EIRR) of the project in Tunisia is estimated at 16%. Simulation analysis for robustness of estimates shows that the chance of getting an IRR of more than 4% was found to be 95%. Likewise, in Morocco, based on a constant adoption rate at the 2005 level, an estimated IRR of 29% was calculated for 2015, the project termination year.

The WorldFish Center with the Department of Fisheries in Malawi, implemented an integrated aquaculture-agriculture (IAA) project, which uses farm waste, crop by-products and other natural resources from the farm environment as nutrient inputs for fishponds. An impact study this NRMR project indicates that, assuming that two-thirds of the observed growth in aquaculture production is attributable to the WorldFish project, the net present value (NPV) of the project is estimated at US$3,056 million, and an internal rate of return of 12.2%. Most of the benefits (60%) go to consumers through lower fish prices.

CIFOR has contributed to the development of criteria and indicators (C&I) for the various dimensions of sustainable forest management and produced a comprehensive set of manuals, decision-support software tools that guide users and other publications. By the end of the overall research program, more than 10 million hectares of forest had been certified worldwide, rising to more than 45 million hectares by October 2004. However, because of the difficulty of quantitative attribution of CIFOR's research contribution to certification standards, this case study commenced on an exercise to trace impact pathways that helped to target clients and their particular needs. It then analyzed the target clients in more detail through documentation, interviews and other means to determine the uptake of CIFOR research outputs.

This SPIA study was initiated to find out: (i) the types of impact indicators and assessment methods CGIAR Members most preferred, and (ii) the relative influence of impact studies and other information sources on funding decisions. The study indicated that ex post impact assessments ranked, along with external program and management reviews, as the most important information sources affecting overall funding decisions. Secondly, Members are interested in information related to measurement of poverty impacts and clear assessment of actual socio-economic impacts, rather than evaluations of processes. In addition, "magnitude and distribution of benefits" was selected as the most important determinant of use.

Success stories have followed one after another in the decades since the Consultative Group on International Agricultural Research ( CGIAR) became a focus for global investment in research for development. Funding for the CGIAR nevertheless began to stagnate in real terms in 1988, with some Members apparently disturbed that "cherry picking" successes sidestepped the question of the CGIAR's overall productivity and value for money.

The study examines the poverty reduction implications of the introduction of three different agricultural technologies by government and NGOs in three rural sites across Bangladesh. The first is new vegetable seeds developed by AVRDC introduced in Saturia to women owning small amounts of land by a local NGO, based on a training and credit dissemination approach. The second is polyculture fish technology developed by World Fish Center and introduced by a government extension program based on private fishponds operated mostly by men in Mymensingh. The third is the same polyculture fish technology, but introduced through a local NGO in Jessore based on the arrangement of leased fishponds operated by groups of low income women, supported by training and credit provision. The study found a number of significant poverty impacts. Among the strongest was in the case of vegetable technology, which is targeted toward women in households with relatively small amounts of land and is a -nonlumpy" technology requiring a very low level of investment, but with substantial returns and positive impacts on female empowerment and child nutritional status. The private fishpond technology was less successful in terms of poverty impact, since only better-off households tend to own ponds. This technology, however, had positive effects on the pond and crop profits of these households. The operation of the group fishpond technology, though a potentially beneficial agricultural program for poor households, was significantly undermined by collective action problems. Relative to women who did not have access to this group-based program, female group members appeared to have more mobility, greater likelihood of working for pay, higher off-farm incomes, and better nutritional status. The group fishpond technology was also found to increase vulnerability in a number of ways, such as through the theft of fish from ponds, or through gendered intrahousehold inequalities in technology-related time burdens and access to markets for, and hence income from, the agricultural outputs. The study overall showed a higher level of trust for NGOs as opposed to government services, but it also highlighted the variable performance of NGOs. Political dimensions to NGO activity also emerged as important, and are perceived by some sections of the community to affect the dissemination of technologies and extension support services for the technologies. Quantitative and qualitative data were found to complement each other well in the research across a range of issues. For example, the survey addressed female empowerment adopters by measuring the frequency of women's attendance of meetings, etc., while the focus groups revealed the importance of the nonmonetary exchange of vegetables between households to maintain social networks and reduce vulnerability. There were also gains through the overall use of the sustainable livelihoods framework as a way of sharpening understanding of the different entry points at which technology can affect household wellbeing and vulnerability.

Declining soil fertility and recurrent drought pose serious challenges to farmers in Africa's semi-arid regions, where half of all farmland suffers from erosion and nutrient depletion (Cleaver and Schreiber, 1997). In response, farmers and researchers across the continent have experimented with a broad array of soil and water conserving technologies (Reij et al., 1996). This paper describes the development and dissemination of one such technology from Zambia.

This study documents the use of improved maize germplasm by poor small-scale farmers in lowland tropical Mexico and how it contributes to their well-being. To this end, the direct adoption of improved varieties and their "creolization" process were assessed. Farmers produce what they recognize as "creolized" varieties by exposing improved varieties to their conditions and management, continually selecting seed of these varieties for replanting and, in some cases, promoting their hybridization with landraces, either by design or by accident. Our key hypothesis is that improved germplasm benefits poor farmers through creolization, which provides them with new options. In creolization, farmers take an improved technology generated by the formal research system and deliberately modify it to suit their needs. Different methodologies such as participatory methods, ethnographic case studies, household surveys, collection of maize samples, and agronomic evaluation of those samples were applied in this study, which was conducted in two areas: the coast of Oaxaca and La Frailesca, in the states of Oaxaca and Chiapas, two of the poorest in Mexico. The study areas are contrasting-one subsistence-oriented and the other commercial-but extreme poverty is pervasive in both. Maize continues to play a key role in the livelihoods of the poor in both areas. Results show that different maize germplasm types, such as improved varieties and, particularly, creolized varieties, are planted in both areas. The impacts of different types of improved maize germplasm are defined and analyzed based on how well they supply farmers with traits they consider important, and the trade-offs they entail. Results also show that creolized varieties occupy a niche that shifts according to the availability of improved germplasm and the orientation of farmers' maize production.

This case study explores the development, dissemination, adoption, and impact of improved tree fallows in rural western Kenya. The processes of technology development and dissemination throughout the region are described and analyzed. To analyze adoption and impact, the paper applies a variety of different data collection methods as well as samples from both pilot areas where researchers maintained a significant presence and non-pilot areas where farmers learned of the technologies through other channels. Sample sizes for the quantitative analysis ranged from almost 2,000 households for measuring the adoption process to just over 100 households for measuring impact indicators. Qualitative methods included long-term case studies for 40 households and focus group discussions involving 16 different groups. The paper describes the ways in which farmers used and modified improved fallow practices. Discussion also examines the types of households using fallows and benefiting from their use. Empirical results suggest that improved fallows almost always double on-farm maize yields. In addition, the data indicates that poor households use improved fallows at much greater rate (about 30 percent) than they do fertilizer (8 percent), though, on average, the size of fallow plots remains small, at 440m2. As a result, despite these promising signs, the improved fallow systems were not found to be linked to improved household level food security or poverty indicators primarily, primarily because the size of the fields under the agroforestry systems was on average, quite small. Conclusion: To conclude, improved fallows represent a technically effective and financially profitable technology that is attractive to poor households with little cash available for investment. They are being used and adopted by a significant proportion of households in areas of western Kenya where they had been disseminated in the late 1990s. On the other hand, farm sizes are small and the ability of farmers to set aside land, even for a season is limited. Hence, the average size improved fallow is small among adopting farmers. Looking to the future, it is best to view improved fallows as a component of a broader integrated soil fertility management strategy for farmers. Farmers will also use manure, compost and to some degree, fertilizer. Improved fallows have a comparative advantage in that they are relatively labor saving over manure or compost, they have a low risk of failure in supplying nutrients on the farm (because of extensive rooting systems), and they offer some by-products. Our evidence suggests that they may serve as an entry point for improved soil fertility management for farmers who had previously not invested in soil fertility management. Nonetheless, the scaling out of improved fallows to other areas will face challenges. It is not a traditional practice and therefore must be learned. There are several stages involved in managing fallows - choice of species, fallow establishment, and cutting and incorporation. Technical backstopping may be important for each of these stages. Yet ensuring that it is available is no easy feat with scattered NGOs and sub-optimal extension services. These dissemination problems affect the scaling out of most technologies, but particularly affect the more knowledge-intensive technologies such as improved fallows. Making germplasm available is also a challenge, though many fallow species are prolific seeders and farmers can in theory become self-sufficient quite easily. Moving germplasm into new areas is more problematic. Markets for seed may play a limited role in this because farmers need a high quantity of seed, but do not care much about quality of these -input trees- and therefore are not willing to pay much, if anything for the seed.

A recent study commissioned by the CGIAR Science Council, Standing Panel on Impact Assessment (Raitzer, 2003) analyses the effectiveness of the overall investment in the CGIAR System, and produces evidence of impressive efficiency. The study finds overall benefit-cost ratios could plausibly range from nearly two to over 17, with a plausible scenario of benefits to date producing a ratio of 9 to 1. The latter translates into $69 billion (1990 US Dollars) of benefits over the history of the CGIAR.

This volume reports on the productivity impacts of varietal improvement research conducted at a number of international centres affiliated with the Consultative Group on International Agricultural Research. Such centres have been at the forefront of a 40-year effort to breed new varieties of the world’s staple food crops. The volume is the main product of a study initiated and supported by the Impact Assessment and Evaluation Group (now Standing Panel on Impact Assessment) of the CGIAR. Descriptive data and econometric models are used to evaluate the impact of research on eleven crops and in three country case studies (Brazil, China and India).

This pamphlet reports and collates information on a variety of CGIAR impacts, including new crop varieties, no till agricultural techniques, and other technologies.