DFID’s Contribution to Improving Nutrition

1. Introduction

1.1 The Independent Commission for Aid Impact (ICAI) is the independent body responsible for scrutinising UK aid. We focus on maximising the effectiveness of the UK aid budget for intended beneficiaries and on delivering value for money for UK taxpayers. We carry out independent reviews of aid programmes and of issues affecting the delivery of UK aid. We publish transparent, impartial and objective reports to provide evidence and clear recommendations to support UK Government decision-making and to strengthen the accountability of the aid programme. Our reports are written to be accessible to a general readership and we use a simple ‘traffic light’ system to report our judgement on each programme or topic we review.

1.2 We have decided to review the impact and value for money of the Department for International Development’s (DFID’s) aid programme supporting nutrition. The aim of DFID’s nutrition programme (2011-15) is to reach¹ 20 million children under 5 years of age with nutrition interventions.² This will be achieved through:

• nutrition-specific interventions, which address the immediate causes of under-nutrition by, for example, supplementing children’s diets with key vitamins and minerals and deworming;
• nutrition-sensitive interventions, which address underlying causes of under-nutrition, such as poverty and access to safe drinking water and sanitation facilities;
• generating evidence on what works and does not work, as a basis for action; and
• encouraging a global effort to tackle under-nutrition by donors working in partnership with country governments.

1.3 This Inception Report sets out the assessment questions, methodology and work plan for the review. It is, however, intended that the methodology and work plan be flexible enough to allow new questions and lines of inquiry to emerge over the course of the assessment.

2. Background

2.1 The background to this review is described in the Terms of Reference.³

3. Purpose of this review

3.1 Under-nutrition is a major challenge to human development and future economic prosperity in Sub-Saharan Africa and South Asia. Despite this, donors and developing countries have historically placed relatively little emphasis on nutrition.

3.2 This changed following the sudden food price increases in 2007-08, which severely affected poor people in developing countries and resulted in international commitments to mobilise financing to achieve global food and nutrition security.⁴

3.3 Since 2009, DFID has put in place an ambitious programme to encourage the international community to continue efforts to tackle under-nutrition and to reach 20 million children under 5 years of age with its nutrition programmes.

3.4 This review will assess whether DFID’s approach to tackling under-nutrition is likely to achieve its ambitious targets. We will ask whether the scope and balance of the different components is appropriate and whether the programme is being implemented at the necessary scale and pace to achieve its objectives for intended beneficiaries.

3.5 The purpose of the review is to examine whether DFID’s support to nutrition through bilateral and multilateral programming is strategic and coherent and is achieving – or is on track to achieve – meaningful results for its intended beneficiaries. We will assess whether DFID is likely to achieve its 2015 targets, given the scope, pace and effectiveness of its programme.

3.6 The review will focus on seven core questions:

• Are the DFID-funded approaches to nutrition coherent in their support of the strategic objective of (i) reaching 20 million children under-five and (ii) saving the lives of 50,000 women during pregnancy and 250,000 new born babies by 2015?
• Does the programme have a robust theory of change with sound evidence and credible assumptions on how its activities will lead to the desired outcomes and impact?
• Does the design of the portfolio respond to beneficiary needs and were intended beneficiaries involved appropriately in all stages of the programme from design to implementation and evaluation?
• Is the pace of implementation reasonable given the level of funding and high priority DFID gives to the work?
• Are activities managed so as to maximise effectiveness for intended beneficiaries and value for money for UK taxpayers?
• How effectively is DFID working through multilateral agencies as a key pillar of its approach and how effectively are the multilateral partners delivering and measuring and reporting on results?
• Are the four components of DFID’s nutrition programme (nutrition-specific interventions, nutrition-sensitive interventions, influencing the international agenda, and research and evidence) well-coordinated, complementary globally and with country programmes and sufficient to achieve its strategic objective? Does DFID use its influence effectively globally?

4. Relationship to other reviews

4.1 ICAI has recently reviewed DFID’s work on agricultural research⁵ and DFID’s contributions to the reduction of child mortality in Kenya.⁶ Our current review will examine the links between DFID’s nutrition programme and its work in these other two areas. We will minimise any duplication with these reviews by examining only those programmes which explicitly aim to achieve nutritional outcomes.

4.2 Other relevant ICAI reviews include: DFID’s Health Programmes in Burma;⁷ DFID’s work with UNICEF;⁸ DFID’s Bilateral Aid to Pakistan;⁹ DFID’s Humanitarian Emergency Response in the Horn of Africa;¹⁰ DFID’s Support for Health and Education in India;¹¹ and DFID’s Support to the Health Sector in Zimbabwe.¹²

5. Methodology

What is distinctive about our approach?

5.1 In addition to reviewing documents and interviewing DFID staff, its partners and third party experts, the review will include four distinct elements. We will carry out:

• a series of interviews with those engaged in global efforts to combat under-nutrition to evaluate DFID’s contribution to the process (e.g. staff and experts from a wide range of UN organisations, international foundations, NGOs, the private sector and other governments);
• comprehensive assessments of DFID’s work to combat under-nutrition in two of its priority countries: Zambia and India. We will examine DFID’s bilateral, multilateral and civil society projects, funded centrally and through its country programmes. We will also review DFID staff’s work in seeking to influence governments and other donors;
• beneficiary surveys in each country to assess how intended beneficiaries have participated in DFID supported projects and benefitted from them; and
• a workshop with DFID staff to review its theory of change, with key experts selected by the ICAI team, who are familiar with the research evidence.

What will we do?

5.2 To obtain the information we require to address the core questions of the review, we will examine DFID’s overall programme and portfolio of interventions to assess whether it is coherent and addresses the key issues. This will involve:

• a detailed briefing by DFID London followed by face-to-face and video/telephone interviews with DFID staff, DFID partners and third-party experts in the UK and other countries; and
• review and analysis of the DFID project documentation to assess:
  • whether the programme is on track and whether the pace of implementation is adequate to achieve DFID’s targets;
  • the extent to which beneficiaries were involved in the design and implementation of the programme;
  • the robustness of programme monitoring and evaluation; and
  • the extent to which DFID has developed innovative ways of integrating the different components of its programme to maximise its impact.

5.3 We will also assess the robustness of the evidence DFID used in designing its nutrition programme and the research and impact evaluation activities it has commissioned to provide additional evidence needed to improve the programme. This will be conducted by ICAI review team members who are researchers at a leading UK university.

5.4 In addition, we will examine DFID’s theories of change to assess whether they are based on sound evidence and credible assumptions on how its activities will lead to the desired outcomes and impact. The ICAI review team will organise a workshop with DFID staff and third party experts to assess the robustness of the theories of change for DFID’s nutrition specific projects, nutrition sensitive projects and international influencing activities. The analysis of theories of change will draw especially on the assessment of evidence.

5.2.1 Also as part of our review, we will undertake detailed reviews of the efficiency, effectiveness and likely impact of the DFID programme in two countries: Zambia and India, focussing on the state of Madhya Pradesh. These countries were selected by ICAI from a group of 10 countries where DFID considers its nutrition work is likely to have the greatest impact and/or is especially innovative. In each country we will:

• assess the total size of UK investments in nutrition by mapping DFID’s investments through its bilateral, multilateral (e.g., through UNICEF, WFP, EU, World Bank) and civil society programmes. Some of these interventions will be financed through DFID’s country programme, whereas others will be financed centrally;
• conduct a detailed review of three projects in each country, to assess their efficiency, effectiveness, value for money and likely impact. We will examine one nutrition specific, one nutrition sensitive and one other project (either research/evidence or influencing). A least one of the projects will be implemented by a multilateral organisation to provide evidence on how well DFID works through multilaterals. The ICAI review team will interview DFID staff, staff of partner organisations implementing the projects and intended beneficiaries and other key local level stakeholders;
• contract a local research organisation to conduct detailed interviews with intended beneficiaries and other stakeholders. These will be undertaken in randomly selected programme districts – two where under-nutrition rates have declined and another two where they remain materially unchanged. These interviews will aim to assess whether the projects are adequately addressing beneficiary needs and the extent to which intended beneficiaries were involved in project design and implementation; and
• assess the extent to which DFID has been able to influence the nutrition policies and programmes of the country and of multilateral, commercial and other organisations working on nutrition. We will also examine the issue of food waste and losses in transportation and storage.

5.2.2 Our review will also examine the effectiveness of DFID’s global work, which is designed to influence the nutrition agenda through global movements such as SUN (the Scaling Up Nutrition movement), the G-8, multilateral organisations, civil society and the private sector. This will involve interviews with key actors and organisations in the UK and global centres (e.g. Geneva) and with third party experts experienced in influencing the international development agenda. These will include civil society leaders and experts with experience in influencing work in the UN and related fora.

6. Roles and responsibilities

6.1 The Team Leader will be the primary point of contact with DFID. KPMG will provide oversight of this review under the overall leadership of the ICAI Project Director. Supplementary analysis and peer review will be provided by KPMG staff.

6.2 The team will comprise the following members:

Team Leader (Independent)

He is a rural livelihoods expert with over 30 years’ experience in research and consultancy on agricultural and rural development, food security and social protection. He led the design of a number of DFID’s and the World Bank’s flagship rural poverty programmes and has also worked on more than a dozen impact evaluations of rural livelihoods programmes in Asia and Africa. He has a good knowledge of food security and nutrition issues. He was an expert on the ICAI evaluation of the Western Orissa Rural Livelihoods Project in 2012 and led the ICAI review of DFID’s agricultural research in 2013.

Team Member 1 (Independent)

She has over 30 years’ experience as a public health consultant in Africa and Asia. Her work focussed on delivery of evidence-based essential reproductive, maternal, neonatal and child health services, including nutrition services and health systems strengthening. It has involved targeting poor women and children in a wide range of countries and contexts, including fragile states. She has worked for DFID, UNICEF and Save the Children Fund.

She will be the nutrition expert on the review. She will assess the effectiveness and impact of DFID’s nutrition-specific and (with the Team Leader) nutrition sensitive interventions.

Team Member 2 (KPMG)

She is a Chartered Accountant with a Masters in Development Studies and has over seven years’ experience with KPMG working across public sector audit. She has also worked at the European Commission’s Humanitarian Aid Office and at the Institute of Development Studies. Within KPMG, she has worked for two years as part of the internal audit team at King’s College Hospital NHS Foundation Trust and has experience of auditing charitable funds at a range of healthcare organisations. She will assess DFID’s financial and programme management capacity in the context of its nutrition programme. She will assess the capacity of implementing partners and examine the delivery chain of DFID’s programmes.

Team Member 3 (Independent)

He is a social scientist and specialist in monitoring and evaluation. He has five years’ experience and has worked in six eastern and southern African countries. He has a thorough understanding of a wide range of evaluation techniques, including experimental, quasi-experimental approaches and theory-based approaches. He has worked on the evaluation of private sector development, governance and food security projects.

He will be the impact assessment specialist on the review. He will review completed impact evaluations by DFID and others, working closely with the Team Leader. Additionally, he will contribute to the design and supervision of field research by local consultants.

Team Member 4 – Peer Reviewer (Independent)

He is a professor with over 30 years’ of nutrition research experience, in Africa and Asia. He has a PhD and DSc in biophysical anthropology. He has worked as a nutrition consultant to DFID poverty reduction projects in Bangladesh and Nepal. He will provide quality assurance of the impact evaluation and theory of change work-stream for this review. He will also supervise the consultant undertaking the literature review to assess the quality of evidence used and generated by DFID.

Team Member 5 – Literature review (Independent)

She has 15 years’ experience as a nutrition researcher and is currently a Research Fellow at a UK university. She will assess the strength of the evidence DFID used in designing its nutrition programme and assess the appropriateness of its on-going and planned research on nutrition. She has a PhD in nutrition. She will also contribute to the theory of change work-stream for this review.

Team Member 6

He is the Managing Director of an innovative consultancy firm based in Bangalore. He has over 20 years’ of experience in managing consultancy and research projects, including extensive experience in the health sector. He has expertise in research design, quality control and delivery. He has worked in India and over 15 other countries in Asia and Africa.

He will be the Project Director of the field research study in Madhya Pradesh. He will be responsible for designing the studies, overseeing implementation, data analysis and report writing.

Team Member 7

He is the Managing Director of a consultancy firm based in Harare, Zimbabwe, which operates across southern and eastern Africa. He has over 20 years’ experience in managing consultancy and research projects, including extensive experience in the social protection and livelihoods sectors. He has expertise in research design, quality control and delivery. He has worked in over 15 other countries in Asia and Africa, including Zambia.

He will be the Project Director of the field research study in Zambia. He will be responsible for designing the studies, overseeing implementation, data analysis and report writing.

7. Management and reporting

7.1 A first draft report will be produced for review by the ICAI Secretariat and Commissioners by w/c 31 March 2014, with time for subsequent revision and review prior to completion and sign off in w/c 16 June 2014.

8. Expected outputs and time frame

8.1 The main deliverables will be:

9. Risks and mitigation

9.1 The following sets out the key risks and mitigating actions for this review.

10. How this ICAI review will make a difference

10.1 This review will examine the impact of DFID’s nutrition programme on under-nourished babies, children, adolescents and adults. It will focus on Zambia and India, especially the state of Madhya Pradesh. Findings will support direct improvements in the ability of DFID’s portfolio of nutrition projects to deliver impact for undernourished people in these and other developing countries.

10.2 Our review will scrutinise the extent to which DFID’s nutrition agenda targets the priorities of poor families in Zambia and India and is set by them. We will examine financial management and performance assessment as part of our review of the overall efficiency, effectiveness and value for money of the programme. We will also assess the effectiveness of programme management in respect of objective-setting, the balance of risk and the use of monitoring and evaluation to feed back into future programme design. Our review will focus especially on the theories of change assumed in the design of the programme and the evidence on which these were based.

10.3 In Zambia and India, we will focus in detail on DFID’s bilateral programmes and its support to multilateral programmes in Zambia and India. We will examine how the four components of DFID’s programme complement each other and assess the likely impact of DFID’s programme on nutrition in these two countries. At the portfolio level, we will assess whether DFID’s programme is being delivered at the scale and pace needed to achieve its objectives.

10.4 This ICAI review will also look explicitly at how effective DFID has been in supporting and influencing global efforts to combat under-nutrition. Lessons learned will be of wider interest to DFID programmes and those of other donors.

Footnotes

1. DFID does not define ‘reach’ in its 2011 Scaling-Up Nutrition position paper. We will review this when examining DFID’s theory of change for the programme.
2. This is in addition to those reached through humanitarian response.
3. Terms of Reference: DFID’s contribution to nutrition, ICAI, http://icai.independent.gov.uk/wp-content/uploads/2014/01/ICAI-Nutrition-TORs-final.pdf
4. See Terms of Reference (footnote 3).
5. DFID’s support to Agricultural Research, ICAI, 2013, http://icai.independent.gov.uk/wp-content/uploads/2013/12/ICAI-Agricultural-Research-report-FINAL.pdf.
6. DFID’s contributions to the reduction of child mortality in Kenya, ICAI, forthcoming in 2014.
7. DFID’s Health Programmes in Burma, ICAI, 2013, http://icai.independent.gov.uk/wp-content/uploads/2010/11/16-July-2014-ICAI-Burma-Health-Report-FINAL.pdf.
8. DFID’s work with UNICEF, ICAI, 2013, http://icai.independent.gov.uk/wp-content/uploads/2010/11/ICAI-report-DFIDs-work-with-UNICEF.pdf.
9. DFID’s Bilateral Aid to Pakistan, ICAI, 2012, http://icai.independent.gov.uk/wp-content/uploads/2011/11/ICAI-Pakistan-Report_P1.pdf.
10. DFID’s Humanitarian Emergency Response in the Horn of Africa, ICAI, 2012, http://icai.independent.gov.uk/wp-content/uploads/2010/11/ICAI-report-FINAL-DFIDs-humanitarian-emergency-response-in-the-Horn-of-Africa11.pdf.
11. DFID’s support for Health and Education in India, ICAI, 2012, http://icai.independent.gov.uk/wp-content/uploads/2010/11/ICAI-Evaluation-of-DFIDs-Support-for-Health-and-Education-in-India-Final-Report.pdf.
12. DFID’s support to the Health sector in Zimbabwe, ICAI, 2011, http://icai.independent.gov.uk/wp-content/uploads/2010/11/DFIDs-Support-to-the-Health-Sector-in-Zimbabwe.pdf.