DFID’s Support to Agricultural Research

1. Introduction

1.1 The Independent Commission for Aid Impact (ICAI) is the independent body responsible for scrutinising UK aid. We focus on maximising the effectiveness of the UK aid budget for intended beneficiaries and on delivering value for money for UK taxpayers. We carry out independent reviews of aid programmes and of issues affecting the delivery of UK aid. We publish transparent, impartial and objective reports to provide evidence and clear recommendations to support UK Government decision-making and to strengthen the accountability of the aid programme. Our reports are written to be accessible to a general readership and we use a simple ‘traffic light’ system to report our judgement on each programme or topic we review.

1.2 We wish to undertake a review of DFID’s support to agricultural research. DFID aims to develop new agricultural products and technologies, encourage their uptake by farmers and thus increase food supply and reduce hunger and malnutrition. Our evaluation will focus on the current £330 million Agriculture Research Programme (2011-15) and review evaluations of earlier investments, in order to assess impact.

1.3 These Terms of Reference outline the purpose and nature of the review and identify the main themes that it will investigate. A detailed methodology will be developed during an inception phase.

2. Background

2.1 DFID’s Research and Evidence Division (RED) was established in 2009. It is responsible for delivering DFID’s priority to be more systematic in using evidence as a basis for how best to reduce global poverty and provide high-quality, relevant evidence to others.¹ Much of RED’s work aligns with wider UK priorities, such as support to international climate change, biodiversity and desertification conventions and work on global issues by research councils.

2.2 RED has a £1.16 billion budget over 2011-15.² Research expenditure is budgeted at £334 million in 2013-14, rising to £356 million in 2014-15.³ The RED programme has three main functions:

• Commissioning research: commissioning and disseminating research evidence under each Millennium Development Goal (MDG) theme. The five focus themes mirror DFID’s priorities: human development; economic growth; climate, environment and governance; conflict and social development; and agriculture;
• Making evidence and evaluation results accessible: synthesising, analysing and disseminating evidence and evaluation results, to improve aid delivery and drive value for money; and
• Increasing the skills of professional cadres and using evidence across DFID: increasing the professional skills and impact of DFID technical cadres across the organisation. This includes establishing a Quality Assurance Unit, to ensure the best use of evidence in DFID business cases.

DFID’s agricultural research work

2.3 DFID’s agricultural research work accounts for 29% of RED’s expenditure under its ‘commissioning research’ activity. As shown in Figure 1, DFID plans a substantial increase in research expenditure over the current Operational Plan (2011-15). Expenditure on agricultural research will decrease as a share of the total, however, from 39% to 24%.

2.4 DFID’s agricultural research work has three objectives:⁶

• to develop new agricultural products sustainably to intensify agriculture to meet the challenges of global population growth and climate change;⁷
• to increase understanding of agricultural innovation and ensure proven new technologies and products are used as quickly as possible and improve the livelihoods, food security and nutrition of the poorest people; and
• to deepen understanding of the complex economic, social and political contexts required to increase the likelihood of adoption of new products and technologies.

2.5 The programme currently comprises 13 on-going projects, most costing under £10 million. Four projects cost £30 million or more:

• Support to the Consultative Group on International Agricultural Research (CGIAR), £120 million (2012-15): CGIAR is a consortium of 15 agriculture research centres around the globe,⁸ of which DFID finances 7-10% each year.⁹ This places the UK in a position of influence as one of the top five donors to the system.¹⁰ CGIAR’s 2012 budget was £568 million ($900 million).¹¹ Its research has underpinned global agricultural development since the green revolution.¹² Over 60% of modern plant varieties grown in developing countries have CGIAR ancestry and 30% of global yield growth between 1965 and 1998 has been attributed to plant genetic improvement by CGIAR.¹³ Recent reforms to CGIAR aim to streamline and consolidate research management and funding. They are also intended to provide new impetus to CGIAR’s research effort, with a strong focus on Africa;
• Support to other international agriculture research centres (which are not members of CGIAR), £40 million (2011-15): these are the Centre for Agricultural Bioscience International (CABI), the International Centre for Insect Physiology and Ecology (ICIPE), the World Vegetable Centre¹⁴ and the Global Forum for Agriculture Research;
• The Bill and Melinda Gates Foundation, £33 million (2011-15): research to develop new technologies and make them accessible to small farmers and others along the commodity chain from the farm to the consumer; and
• Scaling up nutritionally improved food crops through HarvestPlus, £30 million (2012-15): support to the delivery of six nutritionally improved (or biofortified) food crops to 3 million farming households in seven countries in Africa and Asia. HarvestPlus is a CGIAR programme but this project is in addition to the main CGIAR project (above).

2.6 The Research into Use Programme, which ended in 2012, also cost over £30 million.¹⁵ It investigated approaches to scaling up successful innovations from earlier DFID agricultural research in Africa and Asia.

DFID’s delivery of agricultural research

2.7 DFID adopts a range of delivery mechanisms for its agricultural research, including:

• World Bank-managed multi-donor trust funds for CGIAR and a regional research organisation – the Association for Strengthening Agricultural Research in Eastern and Central Africa (ASARECA);
• bilateral relationships with other international organisations, underpinned by memoranda of understanding. These include the West and Central Africa Council for Agricultural Research and Development (WECARD) and specialist research institutes such as the African Agricultural Technology Foundation (AATF) and ICIPE. DFID generally co-finances these research centres with other donors, contributing between 5% and 75% of the total funds;¹⁶
• accountable grants to academic and not-for-profit organisations. These include projects co-funded with the UK Biotechnology and Biological Sciences Research Council (BBSRC) and grants to organisations to translate existing technologies from public or private sectors into products which can be licenced in developing country markets. These organisations include the not-for-profit Global Alliance for Livestock Vaccines (GALVMed), which aims to make vaccines, medicines and diagnostics accessible and affordable to livestock keepers in developing countries; and
• direct contracts let following competitive tender.¹⁷

2.8 The breakdown of funds distributed through these channels is shown in Figure 2.

2.9 The International Development Committee (IDC) noted recently that a significant share of RED expenditure is channelled through multilateral and international organisations without market testing.¹⁸ This applies to the majority of RED’s agriculture research expenditure. DFID seeks to ensure value for money of this expenditure by being represented on CGIAR’s Fund Council and by careful monitoring of progress of all projects against performance frameworks. Although the initial financing decision is not market tested, many of these organisations do allocate funds internally through competition.

2.10 DFID defines two types of risk:

• Management risk: the risk that research organisations are poorly managed and do not deliver research programmes effectively. DFID estimates this as a medium risk for its agricultural research work;¹⁹ and
• Research risk: the risk that research will not be successful i.e. that well-conducted research may not deliver a positive result. Overall, DFID estimates that its agricultural research work is low-medium risk in terms of the types of research funded. Recently, it has placed greater emphasis on commissioning high risk, high pay-off research as part of the portfolio. This includes projects on controlling African Trypanosomiasis disease and investing in nitrogen fixation in cereals.²⁰ DFID recognises that it needs to develop a more explicit balance between risk and pay-off across the programme.²¹

3. Purpose of the evaluation

3.1 To assess the value for money and effectiveness of DFID’s support to agricultural research and its impact on poor people in developing countries.

4. Relationship to other evaluations and studies

4.1 DFID’s agricultural research is guided by the 2011 foresight study on the Future of Global Food and Farming.²² This study, commissioned by the UK Government’s Chief Scientist and co-sponsored by DFID and the Department for Environment, Food and Rural Affairs (DEFRA), assesses the threats facing the global food system and identifies key areas for future research.

4.2 The foresight report analysed the pressures on global food systems between now and 2050 and identified key policies and actions needed to ensure that the global population can be fed sustainably. The global food system, which is currently failing to provide adequate food and nutrition for 2 billion people, will experience unprecedented pressures. On the demand side, the global population will rise from 7 billion to 9 billion people and incomes will increase. This will raise demand for cereals and other foods and, increasingly, animals. On the production side, competition for land, water and energy will intensify, while climate change impacts will increase. The foresight study notes that many food production systems are currently unsustainable. New systems of sustainable agriculture are needed which produce more output from the same area of land, with diminishing adverse impacts on the environment. Sustainable intensification is needed.²³

4.3 The challenge for agricultural researchers is to find ways to raise agricultural productivity while using fewer resources (water, nitrogen fertiliser, energy). New science and technology are needed to raise the limits of sustainable agriculture production, through improved crops and farming systems. Research is needed to tackle threats, including new pests and diseases and increasing droughts and floods due to climate change. Efforts are also needed to improve storage and reduce food wastage.²⁴ Stronger evidence is also needed to support better agricultural policies. A number of other recent studies have come to similar conclusions.²⁵

4.4 There is a comprehensive body of evidence demonstrating the global impact of agricultural research over four decades. A recent analysis of 11 published studies assessing the impact of CGIAR research demonstrated strong returns to investments in agricultural research and substantial impacts on poverty through higher food production resulting in lower food prices.²⁶ These positive returns largely resulted from the ‘green revolution’ in rice and wheat productivity in Asia and parts of Latin America. We need to understand better how to achieve impacts on food and nutrition security for poor people in the complex farming systems common in Africa. Related to this, we need to know how research outcomes can be taken to scale in different contexts.

4.5 In 2005, DFID commissioned an evaluation of its Renewable Natural Resources Research Strategy (1993-2005). It noted that very few studies of impact had been carried out.²⁷ Since 2005, DFID has increasingly emphasised the importance of developing a strong evidence base to improve development outcomes for the poorest people. This led, in 2009, to the establishment of RED with a high priority placed on research and evaluation. Since then, a number of impact evaluations of agricultural research have been undertaken, notably by the Research into Use Programme on initiatives to scale up successful agricultural research innovations. These impact evaluations are useful but, because baseline studies had not been undertaken, their scope is limited.

4.6 DFID reports that it is integrating impact evaluation into its new generation of agricultural research projects.²⁸ CGIAR’s reforms, supported by DFID, are designed to strengthen the consortium’s capacity to undertake comprehensive impact evaluations. It is expected that these changes will result in robust assessments of the impacts of CGIAR’s research on food security, poverty alleviation and the sustainable management of natural resources.²⁹

5. Analytical approach

5.1 Our evaluation will review DFID’s agricultural research work and how it contributes to and is guided by the overall RED strategy.³⁰ We will assess how DFID balances the need to research and develop products for use by farmers in the short to medium term (two to five years; putting ‘research into action’), with high risk advanced research, which takes longer to generate benefits for farmers (5-15 years) but has potentially high payoffs.

5.2 The aim of this review is to assess the impact of DFID’s agricultural research on intended beneficiaries, that is, smallholder farmers and hungry and malnourished people in developing countries.³¹ We will assess what works and what doesn’t work and how poor people benefit from the programme. We will do this by examining:

• the objectives of the current research portfolio and choice of delivery channels, bidding processes, the approach to risk management and the resultant balance of risk;
• how well DFID uses agricultural research findings to encourage uptake and good policies, both internally and by other organisations;
• the extent to which the agricultural research has led or is likely to lead to improved food and nutrition security for poor people through the scaling up of new technologies and their widespread adoption by smallholders; and
• the extent to which the research agenda targets the priorities of poor farmers in developing countries and is set by them.

5.3 The grant to CGIAR accounts for 50% of the total agricultural research programme cost under the current Operational Plan (2011-15). The £30 million CGIAR HarvestPlus project accounts for a further 12%. In view of this, an examination DFID’s work with CGIAR will form an important part of this review. We will examine CGIAR’s on-going reform programme, examine DFID’s management of the grant and assess the impact of CGIAR research on the ground in Africa. We will select HarvestPlus and trace the impacts that follow from generating new technologies in the laboratory to improved livelihoods and nutrition for poor farmers. This work will be conducted in Uganda.

5.4 In addition to the assessment of CGIAR, we will review:

• two research projects implemented by other international research organisations;
• one of the ‘advanced science’ projects with the UK Biotechnology and Biological Science Research Council (BBSRC); and
• two market-focussed projects designed to put research products into use, including a research project examining the social, political and economic factors that affect new product adoption.

5.5 We recognise that most of DFID’s current agricultural research projects have only been operational for a few years, making an assessment of their impact difficult. In view of this, we will use DFID and CGIAR impact evaluations of earlier research projects, reviewing them to assess their quality and testing the sustainability of results through further fieldwork. We will also test the extent to which the evaluation results from these studies influenced DFID’s research and programming choices. Additionally, we will focus on the quality of the evaluations that DFID and its partner organisations currently undertake. A major aim of CGIAR’s reform process, supported by DFID and other donors, is to strengthen its ability to measure impact. We will assess its progress in this area.

5.6 A key priority of RED is to produce evidence, through high quality research, that can inform policy-making. This is intended to support the development of new policies by the international development community, national governments and other stakeholders. We will assess the extent to which DFID’s agricultural research work contributes to generating policy relevant evidence on the links between agriculture, hunger and nutrition; and agriculture and climate change.

6. Indicative questions

6.1 This review will use as its basis the standard ICAI guiding criteria and evaluation framework, which are focussed on four areas: objectives, delivery, impact and learning. A detailed methodology will be developed during the inception phase, setting out the evaluation questions and the methods to be used for answering them. Likely evaluation questions will include:

6.2 Objectives

6.2.1 Does DFID’s agricultural research work (including DFID’s engagement with CGIAR, with other international research organisations and with academic and not-for-profit bodies) have clear, relevant and realistic objectives that focus on defined short, medium and long-term impacts? Do DFID and other donors take a holistic approach?

6.2.2 Is there a clear and convincing plan, with evidence and assumptions, to show how DFID’s agricultural research work – and individual projects – will achieve impact for the smallholder farmers and poor people and minimise the risk that benefits become monopolised by others (e.g. private seed companies or large-scale farmers)?

6.3 Delivery

6.3.1 Is the choice of funding and delivery options appropriate? Does DFID consider a wide enough range of potential delivery partners in its bidding processes?

6.3.2 Do programme design and implementation take into account the needs and priorities of the intended beneficiaries?

6.3.3 How efficient and effective are the different delivery channels?

6.3.4 Are risks to the achievement of the objectives identified and managed effectively? Is the balance of low, medium and high risk research projects appropriate?

6.4 Impact

6.4.1 Is the current research likely to deliver clear, significant and timely research outputs, with the potential to be taken to scale and with positive development outcomes for the intended beneficiaries?

6.4.2 Did earlier DFID investments in agricultural research deliver significant and timely benefits for intended beneficiaries?

6.4.3 Have there been long-term and sustainable impacts from the earlier DFID investments in agricultural research? Are similar impacts likely to be provided by the current programme?

6.5 Learning

6.5.1 Are there appropriate arrangements for monitoring inputs, processes, outputs and results and ensuring rigorous impact evaluation?

6.5.2 Have lessons been learned from both the successes and the challenges of earlier investments? If so, were they used to refocus the research programme?

6.5.3 Is there evidence of innovation in carrying out good research, in encouraging research uptake and in the use of global best practice?

6.5.4 How well does DFID share research findings and encourage take-up within the organisation and with external partners? How well are research results disseminated at the country level?

7. Methodology

7.1 The methodology for this review will be developed during the inception phase. It will involve a number of elements, including:

• an overall review of the current portfolio of agricultural research programmes in the context of DFID’s wider policies and priorities;
• a review of DFID’s engagement with CGIAR, including the effectiveness of CGIAR’s on-going reform programme and its likely future impact on the intended beneficiaries of DFID’s agricultural research work;
• a review and synthesis of the evidence available internationally on the adoption of agricultural research outputs and their impacts. This will include DFID’s earlier investments in agricultural research as well as those programmes supported by other development agencies (e.g. the World Bank, the United States Agency for International Development (USAID) and the Bill and Melinda Gates Foundation); and
• analysis of the financial reports of DFID’s partner organisations to assess how DFID funds are used and how costs are allocated, with an emphasis on management and administrative costs.

7.2 The review team will conduct interviews in the UK, Kenya and Uganda. It will interview CGIAR staff in Washington (CGIAR Fund), Montpellier (CGIAR Consortium HQ) and Rome (Independent Science and Partnership Council and Independent Evaluation Arrangement). The CGIAR interviews will take place by telephone unless it proves necessary to visit one of these locations.

7.3 As part of the review process, we will conduct a range of interviews with stakeholders and intended beneficiaries, to include:

• DFID staff and consultants that worked on the design, implementation, monitoring and evaluation of the agricultural research projects;
• scientists and other staff of implementing organisations, including CGIAR, other international research organisations, UK-based research organisations (e.g. BBSRC) and national researchers in developing countries;
• senior government officials in Kenya, where DFID has made significant investments in agricultural research;
• DFID agricultural advisors in different countries, to understand how they incorporate research findings into their regular agriculture development programmes;
• third-party experts in agricultural research and development, who have not been involved directly in DFID’s agricultural research work; and
• the intended beneficiaries of the programme in Kenya and Uganda – smallholder farmers and other poor people.

7.4 In Kenya and Uganda, the review team will work with local researchers to build on the findings and conclusions of two completed impact assessments, by re-visiting the areas involved to interview smallholder farmers, poor consumers and other stakeholders to assess the sustainability of results. The review team will visit randomly selected villages and will conduct focus group discussions and interviews with key individuals at each location. The other stakeholders interviewed will include small agri-business firms and government agricultural advisors working with farmers.

8. Timing and Deliverables

8.1 The review will be overseen by Commissioners and implemented by a small team from ICAI’s consortium. The lead Commissioner will be John Githongo. The review will take place during the second and third quarters of 2013 and will be published in the final quarter of 2013.

Footnotes

1. ↑ Operational Plan 2011-2015, DFID Research and Evidence Division, DFID, June 2012, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/67359/rsch-evi-div-2011.pdf.
2. ↑ Operational Plan 2011-2015, DFID Research and Evidence Division, DFID, June 2012, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/67359/rsch-evi-div-2011.pdf.
3. ↑ These figures relate to RED’s research expenditure and do not include the budgets for RED’s evaluation department or professional cadres which command a total budget of £31.4 million over the Operational Plan period (2011-15).
4. ↑ Operational Plan 2011-2015, DFID Research and Evidence Division, DFID, June 2012, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/67359/rsch-evi-div-2011.pdf. These figures are for research which accounts for 95% of RED’s overall budget. They do not include expenditure on the evaluation department and professional cadres.
5. ↑ Operational Plan 2011-2015, RED Agriculture Research Team, DFID, February 2013.
6. ↑ Programme Review and Priorities Paper (draft), DFID Agriculture Research Team, March 2011.
7. ↑ These new products include crops, livestock breeds, vaccines and farming systems.
8. ↑ Information note for ICAI Secretariat, DFID Agriculture Research Team, 2013.
9. ↑ DFID’s contribution each year is fixed. Its share of the overall budget each year depends on the exchange rate and other donors’ contributions.
10. ↑ Business Case and Intervention Summary: Support to CGIAR, DFID, 2010, see: http://projects.dfid.gov.uk/project.aspx?Project=202211.
11. ↑ In these Terms of Reference, we have used pounds sterling figures provided by DFID wherever possible. Where figures are only available in US dollars, unless otherwise stated, we have translated into pounds sterling using the applicable average annual exchange rate, see: http://www.oanda.com/currency/average.
12. ↑ Business Case and Intervention Summary: Support to CGIAR, DFID, 2010, see: http://projects.dfid.gov.uk/project.aspx?Project=202211.
13. ↑ The impacts of CGIAR research: A review of recent evidence, Renkow and Byerlee, Food Policy, 2010, Volume 35, Issue 5, 391–402.
14. ↑ The complete name is AVRDC/World Vegetable Centre; AVRDC is an acronym for the previous name of the centre.
15. ↑
16. ↑ Information provided by DFID Agriculture Research Team, March 2013.
17. ↑ The only such project since 2008-09 was the Research Into Use Project, which was implemented by a consultancy firm.
18. ↑ Department for International Development’s Annual Report and Accounts 2011-12, Ninth Report of Session 2012-13, House of Commons, International Development Committee, January 2013, http://www.publications.parliament.uk/pa/cm201213/cmselect/cmintdev/751/751.pdf.
19. ↑ E-mail from the DFID Agriculture Research Team to the review team leader, 25 March 2013.
20. ↑ Programme Review and Priorities Paper (draft), DFID Agriculture Team, March 2011.
21. ↑ Programme Review and Priorities Paper (draft), DFID Agriculture Team, March 2011.
22. ↑ The Future of Food and Farming: Challenges and choices for global sustainability, Foresight Final Project Report, The Government Office for Science, London, 2011, Chapter 4, http://www.bis.gov.uk/assets/foresight/docs/food-and-farming/11-546-future-of-food-and-farming-report.pdf.
23. ↑ The Future of Food and Farming: Challenges and choices for global sustainability, Foresight Final Project Report, The Government Office for Science, London, 2011, Chapter 4, http://www.bis.gov.uk/assets/foresight/docs/food-and-farming/11-546-future-of-food-and-farming-report.pdf.
24. ↑ Global Food – Waste Not Want Not, Institute of Mechanical Engineers, London, 2012.
25. ↑ For example, Scenarios and Challenges for Feeding the World in 2050, Agrimonde, CIRAD/INRA, Montpellier/Paris, 2009, http://www.cirad.fr/en/content/download/3796/35063/version/7/file/1209Agrimonde_SummaryReport.pdf; and Reaping the benefits: Science and the sustainable intensification of global agriculture, Royal Society, London, 2009, http://royalsociety.org/uploadedFiles/Royal_Society_Content/policy/publications/2009/4294967719.pdf.
26. ↑ Benefit-cost meta-analysis of investment in the international agricultural research centers of the CGIAR, Raitzer and Kelley, Agricultural Systems, 2008, Vol. 96 (1-3), p108-123.
27. ↑ Evaluation of DFID Renewable Natural Resources Research Strategy, LTS International, 2005, http://www.oecd.org/derec/unitedkingdom/35168885.pdf.
28. ↑ Telephone conversation between DFID Agriculture Research Team and review team leader, 26 March 2013.
29. ↑ Business Case and Intervention Summary: Support to CGIAR, DFID, 2010, see: http://projects.dfid.gov.uk/project.aspx?Project=202211.
30. ↑ Operational Plan 2011-2015, DFID Research and Evidence Division, DFID, June 2012, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/67359/rsch-evi-div-2011.pdf.
31. ↑ DFID’s RED Operational Plan 2011-15 does not explicitly define the intended beneficiaries of the programme but it does mention smallholder farmers and people who are hungry and malnourished (page 2). The same groups are also the focus of the UK Government’s recent policy statement (22 March 2013) on hunger and malnutrition, see: https://www.gov.uk/government/policies/reducing-hunger-and-malnutrition-in-developing-countries.