DFID contributions to the reduction of child mortality in Kenya

1. Introduction

1.1 The Independent Commission for Aid Impact (ICAI) is the independent body responsible for scrutinising UK aid. We focus on maximising the effectiveness of the UK aid budget for intended beneficiaries and on delivering value for money for UK taxpayers. We carry out independent reviews of aid programmes and of issues affecting the delivery of UK aid. We publish transparent, impartial and objective reports to provide evidence and clear recommendations to support UK Government decision-making and to strengthen the accountability of the aid programme. Our reports are written to be accessible to a general readership and we use a simple ‘traffic light’ system to report our judgement on each programme or topic we review.

1.2 Reducing under-five mortality is one of the Millennium Development Goals and requires a complex mix of interventions in health and other sectors. In this review, we will assess whether DFID’s package of support to achieve this goal in Kenya is strategic in nature, managed appropriately and achieving its intended impact. We will assess the strategic coherence of the channels that DFID supports, including bilateral programmes and funding through multilateral organisations. We will focus on newer global initiatives, sometimes known as ‘vertical funds’, particularly the GAVI Alliance (formerly the ‘Global Alliance for Vaccines and Immunisation’) and the Global Fund. We will compare and contrast a sample of funding channels, to assess whether they are delivering relevant and sustainable outcomes for their intended beneficiaries. These will include the support to childhood immunisation provided by GAVI − to which DFID is a major contributor − and the provision of insecticide-treated bed nets through DFID’s bilateral programme and by the Global Fund.

1.3 These Terms of Reference outline the purpose and nature of the review and the main themes that it will investigate. A detailed methodology will be developed during the inception phase.

2. Background

2.1 The Millennium Development Goal on child mortality (MDG 4) is to reduce the under-five mortality rate by two thirds between 1990 and 2015. This has three indicators:

• indicator 4.1: under-five mortality rate;
• indicator 4.2: infant mortality rate (during the first year of life); and
• indicator 4.3: proportion of one-year-old children immunised against measles.

2.2 Global deaths among children under age five have fallen from more than 12 million per year in 1990 to 7.6 million in 2010 and 6.9 million in 2011. The decline in deaths has accelerated in the past decade, from 1.9% a year on average in the 1990s to 2.5% a year from 2000 to 2011. Despite this progress, however, most developing countries − particularly those in sub-Saharan Africa − will not achieve MDG4. Also, as Figure 1 on page 2 sets out, progress in reducing child mortality in East and Southern Africa has been significantly faster than in West and Central Africa.

2.3 Whilst aggregate mortality is falling, this is not uniform either across or within countries. In Kenya, under-five mortality ranges from 51 to 149 deaths per 1,000 live births in different provinces (see Figure 2). As under-five mortality reduces, the role of neonatal mortality (within the first 28 days of life) becomes more important. The proportion of neonatal deaths in all deaths among children under the age of five in Kenya increased from 27% to 42% over the decade to 2008-09. Understanding these variations and trends is important for maintaining a strong poverty focus and for future strategies for equitable mortality reduction.

2.4 Kenya has been chosen for this study because it is a substantial recipient of both bilateral and multilateral funding for child health. It has been a location for the piloting of innovative approaches by multilateral organisations. It has seen reductions in child mortality but this masks the significant regional variations within the country. Current moves towards devolution provide opportunities to reduce these disparities, but also create new risks. This review provides an opportunity to learn about the factors behind these regional differences as well as to recommend strategies to improve the overall performance of DFID’s approach to reducing under-five mortality in Kenya.

2.5 Reducing under-five mortality is a major component of DFID’s health sector strategies. £929 million (22%) of DFID’s bilateral spending in 2010-11 was in the health sector, an increase of 12% compared to 2009-10.⁴ The areas of expenditure that are particularly relevant to under-five mortality include spending on malaria, nutrition and HIV; and broader investments in the strengthening of health systems and water and sanitation. It is not possible to assess from centrally held data how much of DFID’s expenditure is specifically spent on child health and on health education for parents. This will need to be done by analysis of the budgets within Kenya.

2.6 DFID makes substantial contributions to a wide range of multilateral organisations and global initiatives that work to reduce child mortality, including GAVI, UNICEF and the Global Fund. UK disbursements to GAVI from 2000 to 2012 totalled £259 million,⁵ which was 13.6% of its donor funding. The UK has been the fourth-largest donor after the Gates Foundation, the USA and Norway. The UK is also the largest donor to the International Finance Facility for Immunisation (IFFIm), an innovative funding mechanism of GAVI to which it has pledged £1.63 billion over 23 years. In the inception phase, we will deepen our analysis of the role of DFID’s multilateral channels for reducing child mortality in the context of the Kenya programme.

3. Purpose of this review

3.1 To assess whether DFID’s bilateral and multilateral support for the reduction of under-five mortality in Kenya offers a strategic, coherent, well-managed, sustainable and effective package of support for the intended beneficiaries.

4. Relationship to other evaluations and studies

4.1 DFID undertakes regular evaluations of its work related to under-five mortality, including studies of the health sector. We will review the range of DFID annual reviews and evaluation material which is available related to Kenya. This also applies to relevant studies from the National Audit Office, UK Audit Commission and the House of Commons (including the International Development Committee and Public Accounts Committee).

4.2 Each of the global initiatives and multilateral organisations produces its own evaluations. GAVI has an Evaluation Advisory Committee, which commissions and reviews a range of independent evaluations and impact assessments. The GAVI Secretariat has also produced progress reports every year since 2004 and annual financial statements since 2005. Donors to GAVI generally refrain from conducting their own evaluations, relying instead on those commissioned by the GAVI committee. We will extract relevant data from GAVI studies and those of other funds and multilateral organisations, including UN agencies, related to Kenya.

4.3 Relevant ICAI reviews include: DFID’s Health Programmes in Burma;⁶ DFID’s work with UNICEF;⁷ DFID’s Bilateral Aid to Pakistan;⁸ DFID’s Humanitarian Emergency Response in the Horn of Africa;⁹ DFID’s Support for Health and Education in India;¹⁰ and DFID’s Support to the Health Sector in Zimbabwe.¹¹

4.4 We will also review other relevant research from The Lancet, the Organisation for Economic Cooperation and Development, the Overseas Development Institute and other sources.

5. Analytical approach

5.1 This ICAI review provides an opportunity to assess the coherence of a range of interventions that DFID undertakes, directly and through multilateral channels, across a range of sectors in pursuit of the Millennium Development Goal of reducing under-five mortality in Kenya. During the inception phase, we will research both DFID and non-DFID theories of change and other models that relate to the reduction of under-five mortality at both the global and country levels. We will assess whether the theory of change is reflected by DFID’s approach, priorities and financial allocations.

5.2 We will visit Kenya to meet key officials, examine programmes and visit field locations. We will review DFID’s bilateral programmes in Kenya in health and other sectors which aim to promote child survival (MDG 4), to include objectives, the apportionment of funds and how these programmes are related and co-ordinated. We will also review the application of DFID procedures and guidance governing project identification, management of activities, monitoring, evaluation and learning. In addition, we will review evidence of beneficiary involvement and impact. Our review will cover the past five years.

5.3 Our review will focus on five core questions:

• Are the DFID-funded approaches coherent in their support of the strategic objective of reducing under-five mortality?
• Are activities managed so as to maximise effectiveness for intended beneficiaries and value for money for UK taxpayers?
• Are the programmes performing and achieving sustainable impact?
• How effectively is DFID leveraging and working through multilateral agencies as a key element of its portfolio?
• Do we see evidence that vertical interventions, such as immunisation programmes, are effectively co-ordinated with health system strengthening activities?

5.4 The first question requires us to assess the coherence of interventions to reduce under-five mortality at two levels:

• Coherence for the intended beneficiary: through beneficiary surveys, the study will assess the overall level of beneficiary satisfaction with DFID-funded services to reduce child mortality, including quality, equity and ease of access and the specific performance of programmes to deliver immunisation and other services in support of child survival; and
• Coherence of DFID’s portfolio: this will assess the performance of DFID’s portfolio for addressing child mortality in Kenya. This portfolio includes bilateral support, as well as multilateral organisations and global initiatives supported by DFID, and DFID’s influencing work in the context of government, other donors’ and civil society approaches. DFID-funded work will be set in the context of the Government of Kenya’s overall approach to reducing child mortality.

5.5 The views of intended beneficiaries will be central to this review. We will review the potential for undertaking a representative sample survey of intended beneficiaries or for focussing on a number of specific locations. The surveys will be undertaken by a local research organisation. They will focus on the views of parents and carers as proxies for the ultimate beneficiaries, who are children under the age of five. Questions for intended beneficiaries will include their assessment of their own needs and priorities relevant to reducing under-five mortality and how these relate to services provided. We will explore the types of services delivered, the quality of services and the ease of access for intended beneficiaries, particularly poor women and children, to these services. We will ask whether intended beneficiaries see any major gaps in the services provided and whether they were involved in any discussions with service providers on their needs and priorities. The results of the survey will be cross-checked against aggregate data on the utilisation of the key services relevant to reducing under-five mortality.

5.6 We will examine the processes by which DFID influences the overall strategy to reduce under-five mortality in Kenya. This will include a review of the mechanisms for donor co-ordination and alignment and the influence that DFID has on a sample of global initiatives and multilateral organisations to which it makes contributions at both Head Office and Country Office level. We will review DFID’s relationship with the Ministry of Health at the national and county levels.

5.7 In reviewing the work of multilateral organisations and global initiatives, we will undertake two case studies; the first on immunisation and the role of GAVI and the second on insecticide-treated bed nets which are delivered through the DFID bilateral programme and by the Global Fund. These will include consultations with intended beneficiaries as well as analysis of the design and implementation of the programmes. In reviewing the GAVI programme, we will examine the implementation process for immunisation and how GAVI works with its own partners, such as the World Health Organisation and UNICEF, at country level. We will review the effects of ‘vertical funds’ on aid priorities.

5.8 We will undertake a financial analysis of the programmes to assess their efficiency and cost effectiveness. This will include comparisons with alternative delivery channels for multilateral and bilateral delivery of equivalent services. We will examine the level of overheads involved in alternative delivery arrangements. We will review the procedures in place to prevent fraud and consider any specific cases brought to our attention.

6. Indicative assessment questions

6.1 This review will use as its basis the standard ICAI guiding criteria and assessment framework, which are focussed on four areas: objectives, delivery, impact and learning. A detailed methodology will be developed during the inception phase, setting out the assessment questions and the methods to be used for answering them. Likely questions will include:

6.2 Objectives

6.2.1 Is there a clear and convincing plan underlying DFID’s approach to achieving the MDGs and specifically child survival interventions, with evidence and assumptions, to show how the programme activities will lead to the desired impact (a theory of change)? Is there a coherent and strategic focus by DFID on under-five mortality in Kenya?

6.2.2 Do DFID interventions complement the efforts of government and other aid providers to fill gaps and avoid duplication? Does DFID use its influence in Kenya to promote a coherent response from the international community as a whole? Do the various programmes demonstrate shared goals and common indicators for measuring progress?

6.2.3 Are the programmes well designed, with appropriate objectives and choices of funding and delivery options? Has beneficiary feedback contributed to the design?

6.3 Delivery

6.3.1 Does programme roll-out produce a coherent and integrated package of services which addresses the needs of the intended beneficiaries?

6.3.2 Are risks the achievement of the objectives, including corruption, identified and managed effectively?

6.3.3 Are the delivery systems for the programmes operating in a cost-effective and efficient manner? How do different delivery channels compare with regard to value for money?

6.3.4 How much do the delivery systems used engage with local implementing partners to reinforce sustainability and ownership?

6.4 Impact

6.4.1 Are the programmes delivering against their agreed results? How do different channels compare in this regard?

6.4.2 Are the programmes delivering clear, significant, sustainable and timely benefits for the intended beneficiaries? How do different channels compare in this regard?

6.4.3 Are the programmes focussed on geographical areas with high child mortality? Are they reaching the poorest and most vulnerable children so that access and equity issues are being addressed?

6.4.4 Is there evidence of lasting impact and sustained improvement?

6.5 Learning

6.5.1 Do the measurement systems allow programme impact to be measured on an ongoing basis to drive decision-making?

6.5.2 Do DFID child survival interventions in Kenya demonstrate innovation, response to changing evidence, shared learning across countries and use of global best practice?

6.5.3 Is there transparency and accountability to intended beneficiaries, donors and UK taxpayers and is there learning from their feedback?

6.5.4 Have lessons about the objectives, design and delivery of the programmes been learned and shared effectively across different organisations, regions and sectors?

7. Methodology

7.1 The review will involve a number of elements, including:

Desk review

• a review of literature at the global level to include:
  • evidence of theories of change, approaches and best practice for reducing under-five mortality;
  • the policies and strategies of major multilateral organisations and global initiatives working in this area; and
  • documentation on DFID’s approach to achieving MDGs and specifically the reduction of under-five mortality.
• a review of literature at the country level for Kenya, to include:
  • country data on progress towards MDG 4;
  • government policies and strategies for child health;
  • strategies and financial allocations related to child health;
  • relevant beneficiary studies, evaluations and annual reviews; and
  • a review of DFID’s strategic priorities and pillars in Kenya for: Reproductive, Maternal and Newborn Health; Malaria; HIV/AIDS; Other health; Water and Sanitation; and Poverty, Hunger and Vulnerability.
• a review of DFID’s contributions to global initiatives and multilateral organisations working in Kenya to reduce under-five mortality including:
  • a mapping of DFID’s financial contributions; and
  • a review of spending patterns and types of activities and whether they indicate strategic use of resources.

Preliminary Consultations

• meetings with DFID HQ staff whose work is related to child health, multilaterals and/or Kenya, including research;
• meetings with GAVI and other multilateral officials responsible for strategy and policy in Kenya; and
• meetings with representatives of UK non-governmental organisations involved in child health for their views on DFID’s support to child health in general and in Kenya specifically.

In-country review

• meetings with DFID staff, government officials and development partners to assess how support to under-fives is aligned, co-ordinated and harmonised;
• an assessment of how DFID uses its influencing efforts to support its goals through government and multilateral programmes;
• a review of relevant DFID bilateral programmes and multilateral programmes to include activity reporting and results data, two case studies including immunisation and the role of the GAVI Alliance, field visits to a range of project sites and meetings with intended beneficiaries; and
• beneficiary surveys at a range of locations in Kenya to assess priorities for intended beneficiaries, service delivery standards, equity and ease of access.

8. Timing and deliverables

8.1 The review will be overseen by Commissioners and implemented by a small team from ICAI’s consortium. The lead Commissioner will be Mark Foster.

8.2 The review will start in September 2013, with a final report available during the first quarter of 2014.

Footnotes

1. Building a Future for Women and Children – The 2012 Report, Countdown 2015, see: http://www.countdown2015mnch.org/documents/2012Report/2012-part-2.pdf.
2. The State of the World’s Children, UNICEF, 2013, http://www.unicef.org/sowc2013/.
3. Kenya Demographic and Health Survey, 2008-09, Kenya National Bureau of Statistics, http://www.measuredhs.com/pubs/pdf/FR229/FR229.pdf.
4. Statistics on International Development, DFID, 2012, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/67317/SID-2012.pdf.
5. We use pounds sterling figures provided by DFID wherever possible. Where figures are only available in a foreign currency, unless otherwise stated, we translate these into pounds sterling using the applicable average annual exchange rate, see: http://www.oanda.com/currency/average.
6. DFID’s Health Programmes in Burma, ICAI, July 2013 http://icai.independent.gov.uk/wp-content/uploads/2010/11/16-July-2014-ICAI-Burma-Health-Report-FINAL.pdf.
7. DFID’s work with UNICEF, ICAI, March 2013, http://icai.independent.gov.uk/wp-content/uploads/2010/11/ICAI-report-DFIDs-work-with-UNICEF.pdf.
8. DFID’s Bilateral Aid to Pakistan, ICAI, October 2012, http://icai.independent.gov.uk/wp-content/uploads/2011/11/ICAI-Pakistan-Report_P1.pdf.
9. DFID’s UK’s Humanitarian Emergency Response in the Horn of Africa, ICAI, September 2012, http://icai.independent.gov.uk/wp-content/uploads/2010/11/ICAI-report-FINAL-DFIDs-humanitarian-emergency-response-in-the-Horn-of-Africa11.pdf.
10. DFID’s support for Health and Education in India, ICAI, May 2012, http://icai.independent.gov.uk/wp-content/uploads/2010/11/ICAI-Evaluation-of-DFIDs-Support-for-Health-and-Education-in-India-Final-Report.pdf.
11. DFID’s support to the Health sector in Zimbabwe, ICAI, November 2011, http://icai.independent.gov.uk/wp-content/uploads/2010/11/DFIDs-Support-to-the-Health-Sector-in-Zimbabwe.pdf.