DFID’s Health Programme in Zimbabwe terms of reference
1. Introduction
1.1 The Independent Commission for Aid Impact (ICAI) is the independent body responsible for scrutinising UK aid. We focus on maximising the effectiveness of the UK aid budget for intended beneficiaries and on delivering value for money for UK taxpayers. We carry out independent reviews of aid programmes and of issues affecting the delivery of UK aid. We publish transparent, impartial and objective reports to provide evidence and clear recommendations to support UK Government decision-making and to strengthen the accountability of the aid programme. Our reports are written to be accessible to a general readership and we use a simple ‘traffic light’ system to report our judgement on each programme or topic we review.
1.2 We wish to assess UK-funded support for healthcare in Zimbabwe. These Terms of Reference outline the purpose and nature of the evaluation and identify the main themes that it will investigate. A detailed methodology will be developed during an inception phase.
2. Background
2.1 Zimbabwe is a country twice the area of the UK with a population of 12 million. It once had a well-developed infrastructure and financial system but its economy declined rapidly from the late 1990s as its political situation deteriorated. HIV/AIDS prevalence rose to be amongst the highest in the world during the 1990s and a series of failed harvests during the 2000s increased rural poverty and malnutrition. Zimbabwe faced a health crisis in 2007-09 with a cholera epidemic, record inflation which caused many doctors and nurses to leave and a serious drugs shortage. By 2009, HIV/AIDS accounted for half of the disease burden in the country. Life expectancy had fallen to 34 years from over 60 only a decade earlier.
2.2 Total UK aid to Zimbabwe is planned to be £80 million in 2011-12, with ‘the prospect of further increases if the political situation is favourable’.1 The Department for International Development (DFID) expects to disburse a total of £353 million in the period 2011-15. Support to the health sector is expected to be 35% of DFID’s total bilateral spending in Zimbabwe. Much of this seeks to compensate for failures in local health systems, for instance by providing emergency medicines and supporting maternal and newborn health. The largest area of assistance is the response to and prevention of HIV/AIDS, with a total of £56 million spent from 2004-11.
2.3 We recognise that spending in fragile states such as Zimbabwe is becoming a larger proportion of UK aid. We also recognise the challenge of providing effective aid that demonstrates value for money in such contexts. We wish to evaluate the provision of health support in Zimbabwe, given that similar programmes are likely to be increasingly important in the future.
3. Purpose of this evaluation
3.1 To assess how effectively DFID provides support for health care in Zimbabwe.
4. Relationships to other evaluations/studies
4.1 The International Development Committee praised DFID for its work in the health sector in Zimbabwe in its March 2010 report on the country.2 As well as commending the UK aid programme’s response to HIV/AIDS, it concluded that:
‘DFID support is making a significant difference to the availability and quality of health care available in Zimbabwe. The retention scheme for health workers is an important intervention which is making a contribution to addressing the lack of trained staff and supporting committed staff to continue to work in health care..The Vital Medicines programme has ensured that all health facilities in the country have basic drugs and medical supplies. We commend DFID’s work in the health sector to date and recommend that it continue to give priority to supporting the rebuilding of health services.’
4.2 A 2008 Organisation for Economic Co-operation and Development (OECD) Development Assistance Committee report3 highlights the importance of choosing delivery models (e.g. whether to use government or other partners), the type of aid and the importance of prioritising and bundling services packages in ways that make sense given institutional failures. It also notes the importance of joint working and balancing long-term and short-term impacts. A more recent background paper for the 2011 World Development Report4 similarly assessed global lessons on how to deliver services in fragile and conflict-affected states. It concluded that seeking to support services in such contexts ‘inevitably means taking risks: working more as a venture capital fund than a traditional aid agency’.
4.3 Our evaluation will take into account these reports and other learning to date.
5. Analytical Approach
5.1 This evaluation will seek to draw conclusions on the value for money, impact and effectiveness of UK health assistance to Zimbabwe in the period 2004-11. It will take particular account of the impact of the context on the delivery and performance of the UK’s aid. It will draw conclusions and derive lessons specifically for the country programme and also for other similar work elsewhere.
5.2 The study will involve impact and delivery evaluations of elements of the health programme. Due to constraints of time, it will primarily draw evidence from currently-available data, particularly impact monitoring and evaluation information provided as part of programme reporting. It will, where possible, seek to validate such monitoring evidence using third-party assessments.
6. Indicative Questions
6.1 The following indicative evaluation questions will be elaborated in the inception report:
6.2 Delivery
6.2.1 Are intended beneficiaries involved in the development, roll-out and evaluation of the programme? If so, how?
6.2.2 How has the context influenced DFID’s choices in delivery of healthcare support to Zimbabwe? What is unique about this context that affects value for money?
6.2.3 Is DFID’s programme complementary with that of other organisations (locally, nationally, internationally, including the private sector)?
6.2.4 Are resources being leveraged to maximise impact and provide value for money?
6.2.5 How do DFID’s implementation partners demonstrate effectiveness and value for money?
6.2.6 How is sound financial management maintained?
6.2.7 Is there good governance at all levels and what are the steps being taken to avoid corruption?
6.2.8 To what extent do DFID staff and partners engage the population in the field as part of the rolling out of this programme? How many visits, by whom, when?
6.2.9 Is the programme delivering against its objectives? Have amendments been made if required?
6.2.10 What is the cost of delivery (at each stage of the delivery process, examining the delivery chain)?
6.2.11 How are risks managed and mitigated?
6.2.12 Is technology being used to increase effectiveness of delivery and reporting?
6.2.13 What are the linkages to other assistance provided by DFID? How holistic is the DFID approach in-country?
6.3 Impact
6.3.1 How is effectiveness assessed given the context is high-risk?
6.3.2 How is the impact being measured, both in terms of quantitative outputs (e.g. the number of people reached) and qualitative outcomes (such as changes to individual behaviour or government policies)?
6.3.3 Is it possible to identify the broader and long-term social and economic impacts of the support for healthcare for Zimbabwe’s people?
6.3.4 Are there identifiable impacts that are clear to recipient communities? Are local people involved in community health training? How? If not, why not?
6.3.5 Is there transparency and accountability to intended recipients?
6.3.6 How has UK aid supported the strengthening of Zimbabwe’s overall health system?
6.3.7 Has there been sustainable policy change on the part of the Zimbabwe government?
6.3.8 How will the long-term and sustainable impact of the programme be assured (in the context of global targets on development and aid effectiveness)?
6.3.9 What would happen if DFID left next month?
6.4 Other
6.4.1 Is this assistance demand-driven?
6.4.2 What evaluations or reports have been done in the past six years and how have their key recommendations been followed up?
6.4.3 What are the lessons learned to date from this programme?
6.4.4 How is DFID sharing lessons from/into this programme across its activities globally?
6.4.5 How is DFID applying international lessons in its delivery?
6.4.6 Are there actions that will improve the effectiveness and value for money?
7. Outline Methodology
7.1 The evaluation will have a number of elements:
- a contextual analysis of Zimbabwe, drawing upon available literature and informed by key experts, focussing in particular on the risks, constraints and opportunities created by the country context;
- a review of DFID’s documentation and files on the health programme in Zimbabwe, focussing on delivery mechanisms and evidence of performance;
- a review of independent and third-party assessments of health impact data for Zimbabwe;
- detailed analysis of costs, benefits and rates of return at each stage of the delivery chain, including involvement of the private sector;
- interviews with intended beneficiaries in Zimbabwe (both direct and representative, both urban and rural);
- interviews with Zimbabwe-based respondents from government, DFID, civil society and peer organisations;
- structured discussions with global experts in service delivery in fragile states; and
- direct observation of delivery.
8. Timing and Deliverables
8.1 The review will be overseen by Commissioners and implemented by a small team from ICAI’s consortium. The review will take place during the third to fourth quarter of 2011, with a final report available by the end of November 2011.
Footnotes
1 DFID Zimbabwe Operational Plan 2011-15, DFID, May 2011, www.dfid.gov.uk/Documents/publications1/op/Zimbabwe-2011.pdf.
2 DFID’s Assistance to Zimbabwe, International Development Committee, March 2010, www.publications.parliament.uk/pa/cm200910/cmselect/cmintdev/252/252i.pdf.
3 Service Delivery in Fragile Situations: key concepts, findings and lessons, OECD, 2008, www.oecd.org/dataoecd/17/54/40886707.pdf.
4 M. Baird, Service Delivery in Fragile and Conflict-Affected States, World Development Report 2011 Background Paper, March 2010, http://wdr2011.worldbank.org/sites/default/files/pdfs/WDR_Service_Delivery_Baird.pdf.