DFID’s Support for Health and Education in India

1. Introduction

1.1 The Independent Commission for Aid Impact (ICAI) is the independent body responsible for scrutinising UK aid. We focus on maximising the effectiveness of the UK aid budget for intended beneficiaries and on delivering value for money for UK taxpayers. We carry out independent reviews of aid programmes and of issues affecting the delivery of UK aid. We publish transparent, impartial and objective reports to provide evidence and clear recommendations to support UK Government decision-making and to strengthen the accountability of the aid programme. Our reports are written to be accessible to a general readership and we use a simple ‘traffic light’ system to report our judgement on each programme or topic we review.

1.2 We wish to assess the impact and value for money of UK-funded health and education programmes in India. These Terms of Reference outline the purpose and nature of the evaluation and identify the main themes that it will investigate. A detailed methodology for the evaluation will be developed during an inception phase.

2. Background

2.1 India’s 1.16 billion people and the 28 States and 7 Union territories they live in are diverse. Whilst overall income per person remains low at US$1,180 per year (one fortieth of that for the UK), the country is in the process of rapid economic and social development. India has seen economic growth rates over 8% for much of the last decade and the percentage of the population living in extreme poverty fell from 60% in 1981 to 42% in 2005. Yet, as overall wealth has increased, huge inequalities remain; 456 million people (equivalent to the combined populations of Russia, Germany, Turkey, France, the UK and Poland) still lived on less than US$1.25 a day in 2005.1 There are also considerable regional inequalities; most of the extreme poor are concentrated in the northern states. At current rates, India will only achieve its Millennium Development Goal (MDG) poverty targets by 2043. One fifth of all child deaths in the world are of Indian children. Girls are particularly at risk. As Save the Children has noted, ‘if India fails to achieve the MDGs so does the world’.2

2.2 India has always been a key partner for the UK aid programme. It receives the largest annual amount in bilateral assistance: £295 million in 2009-10. In 2011-12, health and education together will represent 65% of annual expenditure, with the remaining 35% largely taken up with spending on growth, governance, rural livelihoods and urban management. The current Operational Plan for 2011-153 shows that health and education will make up a declining share of the Department for International Development (DFID) budget for India. DFID reports that it is ‘moving away from large financial transfers to the centrally-sponsored schemes of the national government, to a tighter focus on India’s poorest states’. By 2014-15, the share of the total DFID India programme accounted for by health, nutrition and education will be 32% (12% on health, 7% on nutrition and 13% on education).

2.3 In the past, DFID India has operated under an overall Country Strategy and then, for each of its partner states, a State Strategy. DFID has generally supported national programmes run by the Government of India (GoI) to improve health and education, which then are applied at State level. It is these national programmes that have accounted for the majority of DFID India expenditure. In the future, DFID will target its support at the level of the priority states.

2.4 Assistance to India is, however, controversial. There is some scepticism in the UK about the appropriateness of providing aid to a country which has seen such strong recent economic growth. The wider policy issue of whether to provide assistance to one of the most rapidly growing middle-income countries in the world is not a matter for ICAI. We wish to assess whether the assistance provided is having the desired impact and providing the optimum value for money: is DFID providing the best forms of aid given the rapidly changing context and is there an appropriate balance in the programme between financial support, technical assistance and knowledge transfer?

2.5 This study will consider DFID’s health and education programmes in the context of DFID’s commitment ‘to move from an aid-based relationship [with India] to one based on shared contributions to global development goals, such as climate change’.4

3. Purpose of this evaluation

3.1 To assess whether DFID is achieving impact and value for money in India through an evaluation of its health and education programmes in the state of Bihar.

4. Relationships to other evaluations/studies

4.1 The review will take place in the context of the International Development Committee’s (IDC’s) June 2011 report The Future of DFID’s Programme in India.5 A key motivation for the IDC’s investigation was the public concerns mentioned above. IDC concluded that the UK should continue to support India until 2015 but noted the need for DFID assistance to add value:

‘Given current high levels of poverty in India we agree with the Government’s decision to maintain an aid programme in India until 2015 provided it can make a difference. Not every DFID project currently does this and DFID must be more rigorous in its choices over the next four years, funding only projects which have a clear development benefit and which national or state governments would not otherwise fund. DFID rightly focuses on catalytic, demonstration projects which can be replicated and scaled up. This approach should continue.’

4.2 Witnesses to IDC commented that DFID could improve how it targeted activities, indeed some implied that the amount of assistance meant that a focus on value was reduced: ‘Dr Eyben, from the Institute of Development Studies, thought that if DFID had a smaller budget it might be less concerned with spending the money and more with ensuring that the funding and the relationships it built would support far reaching change.’

4.3 IDC made a series of specific recommendations about aspects of the programme, among them calling for strengthened approaches to private sector and civil society engagement. Its overall conclusion, however, was positive. The Committee recognised that the relationship between the two countries was changing through, as the Secretary of State for International Development made clear in evidence, a new phase of mutual engagement. It concluded:

‘…we support the UK’s continued development assistance to India for the period up to 2015. However after this the development relationship must change fundamentally to one based on mutual learning and technical assistance where requested.’

4.4 IDC made a particular recommendation for ICAI:

‘We recommend that the newly created Independent Consortium [sic] on Aid Impact (ICAI) undertake a study of the opportunity cost of DFID continuing to provide £280 million per year to India, or to other countries with a lower GDP per person or slower progress towards the Millennium Development Goals, as a means of demonstrating to us and the British public whether investing in India represents value for money.’

4.5 The assessment of the ‘opportunity cost’ of providing such assistance includes wider policy considerations and lies outside our remit. By considering in more detail, however, the issues of impact and value for money, using examples from significant areas of DFID’s programme, we can provide important information that others may use if they wish to make such an assessment.

4.6 DFID’s country programme was evaluated in 2006, covering the period 2000-05.6 A key element of our evaluation will be to see what the response to the recommendations of that earlier evaluation has been. In particular, the 2006 evaluation recommended more rigorous use of impact evaluation, ensuring stronger monitoring of progress and improving lesson learning. These elements will be a focus for our evaluation, as well as recommendations which are relevant to the programme in Bihar.

4.7 The Programme in West Bengal was evaluated in 2007 using DFID’s methodology for a country programme evaluation.7 The evaluation particularly considered health and education activities:

‘The Health Sector Development Initiative (HSDI) – sector budget support to the health sector … was the first such sector programme in India, and a highly relevant response. It had a long design period – slowed by political, technical and bureaucratic factors from DFID and the Government of West Bengal (GoWB). Progress has been made towards the majority of milestones – these are process-oriented in the first two years, with the aim of leading to successful outcomes for service delivery and health status and positive impacts on poverty towards the end of the programme….

In the education sector, a number of District Primary Education Programme (DPEP) 1 and 2 outcomes were successfully achieved: enrolment increased to 95%, virtual equity with respect to gender reached, a significant increase in the numbers of Scheduled Caste (SC) and Scheduled Tribe (ST) children studying in school, and strong community demand for education engendered. However, progress was affected by institutional and policy constraints which resulted in slow implementation and led to DPEP being placed under ‘special watch’ status by the Government of India (GoI). Lessons learnt from DPEP 1 led to improved performance in DPEP 2….’

4.8 Among its conclusions, it found that:

‘In the education sector, and more widely, state and national programmes would benefit from more interaction. The state programme in running lean and ‘hands-off’ programmes, and the national programmes in using state level knowledge, relationships and experience to gain traction in slow moving state-level issues. It is recommended that ways of working that better facilitate synergy between state and national teams are sought.

Performance assessment needs to be improved across the programme. One option is to follow the new CAP Guidance, and develop a new-style CAP Performance Framework and Delivery Plan for the state. Producing this document would be a useful reflective exercise for the final year of the current SAP.’

4.9 These findings will provide useful background for us to consider performance in Bihar.

4.10 India scored 3.1 in Transparency International’s Corruption Perceptions Index 2011, where the index ranges from 0 (‘highly corrupt’) to 10 (‘very clean’).8 An active media in both the UK and India pursues allegations of corruption. In June 2010, an article in the News of the World suggested that there had been substantial corruption in a key part of the UK programme. Sarva Shiksha Abhiyan II (SSA II) is the second phase of a country-wide GoI primary education programme. 90% of the programme funding comes from GoI. The balance is provided by DFID, the World Bank and the European Commission. UK expenditure was £210 million in the first phase of SSA (2003-07) and it has disbursed £137 million so far in the second phase (2008-10), 2% of SSA’s total funding. An internal audit inquiry noted that there was adequate scrutiny by Indian oversight bodies such as the Comptroller and Auditor General and that effective assurance measures were in place (in the GoI and DFID). A public statement from DFID noted that:

‘In conclusion, it is impossible to ensure that every rupee will be spent properly and effectively within a project as large and complex as SSA. However, the risk to UK funds has been adequately managed by DFID making payment only on the basis of valid externally-audited expenditure. DFID also gains direct assurance on the programme’s financial and operational performance through its participation in the Joint Review Missions and through exercises such as independent procurement reviews and external evaluations. Where audits and reviews have detected weaknesses in SSA’s financial management, there is evidence that the GoI has taken or is taking action to address these, and action to strengthen tracking of SSA’s overall exposure to fraud and corruption will provide greater comfort that all irregularities and allegations of fraud and other abuse are being dealt with effectively.’9

4.11 Our evaluation will wish to establish whether this approach is in place and working effectively.

5. Analytical Approach

5.1 The overall evaluation will seek to identify impact and value for money, using the key questions set out below. The approach should be geographical, focussing on one state – Bihar – where both education and health are supported by DFID.

5.2 Comparative assessments will be undertaken of the results and approaches of a small number of education and health programmes. For each of the key questions, comparisons will be made between the programmes, highlighting key differences in inputs, outputs and outcomes. Whilst an understanding of the context will be necessary, given that all programmes are delivered in the same state, a detailed assessment of the contextual factors in other states is not essential. This removes some of the variables that might make the comparative approach problematic and provides a greater chance of producing meaningful findings.

5.3 While this approach will result in findings that are specifically applicable to Bihar, they may or may not have wider applicability. It should be noted that, since DFID is only one funder out of many and provides only a small proportion of overall expenditure, there will also be issues of attribution that may be hard to assess.

6. Indicative Questions

6.1 This review will use as its basis the standard ICAI guiding criteria and evaluation framework, which are focussed on four areas: objectives, delivery, impact and learning. The questions outlined below comprise those questions in our standard evaluation framework which are of particular interest in this review, as well as other pertinent questions we want to investigate. The full, finalised list of questions that we will consider in this review will be set out in the inception report.

6.2 It will be understood that the questions will be answered for both health and education programmes, with the differences and similarities between the responses being used to provide better definitions of factors that have resulted in outcomes.

6.3 Objective

6.3.1 Is there a clear and convincing plan, with evidence and assumptions, to show how the programme will work?

6.3.2 Is the assistance based on a strategic, realistic and well evidenced assessment of local needs? If not, why not?

6.4 Delivery

6.4.1 Is there good governance at all levels, with sound financial management and adequate steps being taken to avoid corruption?

6.4.2 How do managers ensure the efficiency and effectiveness of the delivery chain?

6.4.3 Are resources being leveraged so as to work best with others and maximise impact?

6.4.4 Do programme design and roll-out involve and take into account the needs of the intended beneficiaries?

6.4.5 How is current expenditure affected by DFID’s commitment to move from an aid-based relationship with India to a partnership approach?

6.5 Impact

6.5.1 Is the aid delivering clear, significant and timely benefits for the intended beneficiaries?

6.5.2 How are the programme impacts being measured? What is the basis of the attribution of UK funds?

6.5.3 Is there transparency and accountability to intended beneficiaries, donors and UK taxpayers?

6.5.4 To what extent are DFID personnel seeing the impact of the programme first hand?

6.5.5 Is DFID providing additional value that could not otherwise be achieved? Is DFID acting as a magnet and catalyst for other money?

6.6 Learning

6.6.1 What mechanisms are in place to assure knowledge transfer?

6.6.2 What mechanisms are in place to assure sustainable local ownership and sustainability after 2015?

6.7 Given that this is a comparative evaluation, running through each question is the dimension of assessing what the critical factors are that result in differential achievement between health and education (if indeed there is a difference).

7. Outline Methodology

7.1 The review will have a number of elements. These will include:

  • a literature review, focussing on examining past evaluations;
  • a review of evidence from DFID’s files and information systems, including financial information;
  • meetings with intended beneficiaries in India;
  • interviews with Indian-based respondents from government, DFID, civil society and peer organisations;
  • interviews with UK and internationally-based experts,
  • an assessment of the partnerships that DFID uses to implement its programmes;
  • cross reference to other relevant ICAI reviews; and
  • follow-up of relevant IDC recommendations.

8. Timing and Deliverables

8.1 The review will be overseen by Commissioners and implemented by a small team from ICAI’s consortium. It will begin in November 2011, with a final report available in the second quarter of 2012.

Footnotes

  1. ↑ The developing world is poorer than we thought, but no less successful in the fight against poverty, World Bank Development Research Group, Policy Research Working Paper 4703, August 2008, http://econ.worldbank.org/external/default/main?pagePK=64165259&piPK=64165421&theSitePK=469382&menuPK=64216926&entityID=000158349_20100121133109.
  2. ↑ Written evidence submitted by Save the Children to the International Development Committee report ‘The Future of DFID’s Programme in India’, March 2011, www.publications.parliament.uk/pa/cm201011/cmselect/cmintdev/writev/616/m15.htm.
  3. ↑ DFID India Operational Plan 2011-15, DFID, October 2010, www.dfid.gov.uk/Documents/publications1/op/india-2011.pdf.
  4. ↑ The Future of DFID’s Programme in India: Government Response to the Committee’s Eighth Report of Session 2010–12, International Development Committee, September 2011, www.publications.parliament.uk/pa/cm201012/cmselect/cmintdev/1486/1486.pdf.
  5. ↑ The Future of DFID’s Programme in India Volume 1, International Development Committee, June 2011, HC 616, www.publications.parliament.uk/pa/cm201012/cmselect/cmintdev/616/616.pdf.
  6. ↑ An Evaluation of DFID’s India Programme 2000-2005, DFID, EV670, August 2006, www.dfid.gov.uk/Documents/publications1/evaluation/ev670.pdf.
  7. ↑ Evaluation of DFID country programmes: West Bengal State Programme, Final Report, DFID, EV681, October 2007, www.dfid.gov.uk/Documents/publications1/evaluation/ev681.pdf.
  8. ↑ Corruption Perceptions Index 2011, Transparency International, 2011, www.transparency.org/content/download/64426/1030807.
  9. ↑ Inquiry regarding UK funding for SSA II, DFID Internal Audit Department, August 2010, www.dfid.gov.uk/Documents/publications1/India-SSA-Inquiry-Final-report.pdf.