DFID’s education programme in Nigeria: the community perspective

1. Introduction

1.1 The Independent Commission for Aid Impact (ICAI) is the independent body responsible for scrutinising UK aid. We focus on maximising the effectiveness of the UK aid budget for intended beneficiaries and on delivering value for money for UK taxpayers. We carry out independent reviews of aid programmes and of issues affecting the delivery of UK aid. We publish transparent, impartial and objective reports to provide evidence and clear recommendations to support UK Government decision-making and to strengthen the accountability of the aid programme. Our reports are written to be accessible to a general readership and we use a simple ‘traffic light’ system to report our judgement on each programme or topic we review.

1.2 We have decided to review the Department for International Development’s (DFID’s) education programmes in Nigeria, with a particular, innovative focus on gathering community perspectives on the impact that UK aid has on pupil experiences in primary and junior secondary education. In line with our stated core value of innovation, this review will provide an opportunity to examine the education experience and outcomes enabled by aid through focussing on the experiences of intended beneficiaries. These Terms of Reference outline the purpose and nature of the review and identify its main themes. A detailed methodology will be developed during an inception phase.

2. Background

2.1 Nigeria is Africa’s most populous country, with 158 million¹ people. It is also one of Africa’s most diverse countries, with over 250 different ethnic and religious groups. The country is Africa’s largest oil producer; this has accounted for nearly 80% of government revenue in recent years² and has contributed to an average annual growth rate of 4% in Nigeria’s Gross Domestic Product (GDP) since 1961.³ Despite its oil wealth, Nigeria is not a rich country, with an economy about one-twentieth the size of the UK’s despite having nearly double the population. More than 100 million people, around two-thirds of Nigeria’s population, survive on less than £1 a day – representing a quarter of Africa’s extreme poor.⁴ The country has suffered from systemic weak governance and corruption; Transparency International’s Corruption Perceptions Index placed Nigeria at 143 out of 182 countries.⁵

2.2 In 2009, total Official Development Assistance (ODA) to Nigeria equated to 1% of its Gross National Income (GNI), the lowest proportion of all African countries.⁶ In terms of UK aid, Nigeria is DFID’s sixth-largest bilateral aid recipient, receiving £141 million last year.⁷ DFID’s Nigeria Operational Plan⁸ 2011-15 sets out its plans to scale up its aid programme in Nigeria significantly to £305 million by 2014-15, when it is forecast to be one of DFID’s largest bilateral aid programmes. The Operational Plan consists of six focus areas: health, wealth creation, poverty reduction, water and sanitation, governance and education.

2.3 The Nigerian Government’s Nigeria Vision 20: 2020 plan outlines the country’s ambitions, opportunities for growth and policies for poverty reduction, providing a framework for co-operation with DFID and other donors.⁹ Achieving the country’s ambitious Millennium Development Goal (MDG) targets is an area where DFID seeks to partner with the Government of Nigeria. Assistance to Nigeria, however, can be challenging: ethnic violence throughout the 1990s in the Niger Delta and more recent sectarian conflict in the north-east of the country have made delivering aid difficult in some areas.

Nigeria’s education sector

2.4 Nigeria’s education system has suffered from years of under-investment, with renewed strategy and funding commitments made by the Government of Nigeria only recently. Legislation was passed in 2004 that mandated free universal primary education and stated that every child should receive at least nine years of education between the ages of 6 and 15. Nigeria remains off-track, however, on the education elements of the MDGs: that is, the goal of universal primary education and the target to eliminate gender disparity in access to education at all levels. Nigeria has 10% of the world’s out-of-school population and, in 2007, about nine million children (37% of its primary school-age child population)¹⁰ were not attending school because of social and economic circumstances. A majority of these are girls, who are often prevented from advancing their formal education for cultural and economic reasons (such as assuming traditional female roles within the community). There are also stark regional inequalities in access to education; for example, 60% of girls aged 6-17 in northern Nigeria are out of school.¹¹

2.5 The Nigerian Government, however, has made a commitment to transform its education system. Its Nigeria Vision 20: 2020 plan places education at the heart of the country’s future growth and success. The country is dedicated to achieving the Millennium Development Goals, aiming to deliver universal basic education for all children and to promote gender equity in education services, with specific targets to ‘increase net primary enrolment from 61.5 per cent to 75 per cent’ and attain ’40 per cent gender parity’ by 2013.¹² The Nigerian Government remains focussed, therefore, on input targets, rather than targets related to attainment or learning.

2.6 DFID’s education programme in Nigeria is targeted on interventions in some of the poorer states, with a focus on gender parity. DFID spent £24 million on education in Nigeria in 2010-11; this is planned to rise to a peak of £47 million in 2013-14, falling to £42 million in 2014-15.¹³ DFID currently runs two large education programmes, covering 10 of Nigeria’s 36 states (outlined in the Figure 1 below).

2.7 DFID has set clear indicators that, by the end of financial year 2014-15, its programmes should support:

• 800,000 more children into education in northern Nigeria, including 600,000 girls; and
• 5,000 women from rural areas of northern Nigeria to attend teacher training.¹⁷

2.8 DFID’s approach to beneficiary impact has been to focus on interventions that aim to be more structural and sustainable (such as professional and teacher development, building schools, budgeting and strategic planning), as well as supporting education service delivery. Nigeria, however, faces weak governance, corruption, poor teaching quality and low levels of pupil enrolment. According to a recent ESSPIN-funded study, ‘of primary and junior secondary teachers working in government schools, only 75 out of 19,000 teachers surveyed achieved the minimum standards.’¹⁸

3. Purpose

3.1 To examine how effectively DFID’s education programmes in Nigeria deliver impact, focussing in particular on the community perspective of pupil experiences in primary and junior secondary education.

4. Relationship to other evaluations/studies

4.1 In July 2011, DFID completed an internal mid-term review of its ESSPIN programme, which found that:¹⁹

• the programme has been effective in forming working relationships at the state level where it operates, working to bring about better governance within basic education, however, ESSPIN has not, to a great degree, pursued the same goals at the federal government level;
• ESSPIN’s pilot programme was effective in working with around 2000 schools and that initial teaching and learning impacts could be seen;
• the programme will need to monitor intended beneficiary impact closer as the programme matures; and
• ESSPIN’s new strategy aims to improve over 10,000 schools over the coming years.

4.2 In 2009, the International Development Committee (IDC) produced a report on DFID’s bilateral aid programme in Nigeria, where they examined the education sector and the initial set-up and planning of the ESSPIN programme. Additionally, they studied ESSPIN’s predecessor, the Capacity for Universal Basic Education programme. IDC recognised the ‘multi-dimensional crisis’ in Nigeria’s education system, with very poor access to quality education. It praised DFID’s programmes and goals, particularly DFID’s focus on building an education system and not running one – helping states to recognise that they are responsible for the delivery of basic, quality education.²⁰

4.3 The IDC report provides valuable benchmarks for our team to examine ESSPIN further. Specifically, IDC recognised the poor management information and reporting on education indicators at the state level. ESSPIN aims to strengthen state-level education governance and capacity building and is engaged in supporting and planning education management information systems at the federal and state levels.

4.4 IDC also praised the innovative approach that DFID has taken in promoting and integrating secular and religious education in northern Nigeria’s Islamiyya schools. It praised the pilot programme that aims to incorporate more ‘core curriculum by offering the incentive of assistance with provision of teachers, books and materials’. It cautioned DFID, however, that this innovative approach will ‘require careful monitoring’.

4.5 As a result of the IDC report, DFID agreed to take IDC’s comments into account.²¹ DFID is specifically supporting additional work for the annual school census, which should provide better evidence and management information, thereby helping to measure the impact of the ESSPIN programme.

4.6 The National Audit Office undertook a study of bilateral support to primary education in 2010, which looked at DFID programmes in Ghana, Kenya, Ethiopia and India.²² At the subsequent Public Accounts Committee hearing, DFID was criticised for not placing enough emphasis on value for money in deciding where and how to spend and not going beyond enrolment to look at attainment and outcomes for children.²³

4.7 The review will also compare and contrast the experience of recent ICAI reviews of education in India (Bihar), Pakistan (Punjab) and East Africa.

5. Analytical approach

5.1 This evaluation will seek to examine the effectiveness and impact of the GEP and ESSPIN education programmes in Nigeria from when they started (2004 and 2009 respectively) to the present. It will focus particularly on gathering community perspectives on the pupil experience in schools. As a result of this focus, our review is not intended to cover the entirety of DFID’s education programmes in Nigeria but rather to examine in depth the education experience in particular schools and parts of the country.

5.2 In developing the methodology during the inception phase, this review will seek to cover both a range of states for each programme and a range of pupil cohorts in those states. The review will seek to document the community perspective on the impact of DFID’s ESSPIN and GEP programmes through formal and informal discussions (including focus groups) with pupils, teachers, parents, local education officers, civil society organisations and local community leaders. The perspective of young people who are not attending school will also be included, either directly or through community organisations. The interviews and focus groups will balance their contact between students and other stakeholders.

5.3 While this qualitative work will not be representative of the entirety of DFID’s education work in Nigeria, it will specifically target schools and communities where DFID programming should have affected pupils’ educational experience. This will allow us to take a snapshot of whether money spent by DFID has made any meaningful difference to the lives of individual pupils and communities from their own perspectives, rather than using the traditional lens of donor targets.

5.4 The review will support this innovative approach with:

• a succinct review of DFID’s management of these programmes, including objective-setting, delivery and learning processes, including approaches to monitoring and evaluation;
• a focus on identifying, at the community level, any corruption or impropriety, using open questions to interviewees to build a picture of any concerns they have about either the use of resources or the behaviour of education officials. Our aim is to identify patterns of any alleged economic loss or improper behaviour, not to pursue individual cases. We will pass specific cases on to DFID in line with our established reporting responsibilities;
• evidence from currently-available data, particularly impact monitoring and evaluation information provided as part of programme reporting; and
• a consideration of the impact of DFID’s programmes on the quality and capacity of the education system in Nigeria. This will focus on technical assistance to improve teaching, management skills, budgeting and management information, particularly in schools and at state level.

6. Indicative questions

6.1 This review will use as its basis the standard ICAI guiding criteria and evaluation framework, which are focussed on four areas: objectives, delivery, impact and learning. The questions outlined below comprise those questions in our standard evaluation framework which are of particular interest in this review, as well as other pertinent questions we want to investigate. The full, finalised list of questions that we will consider in this review will be set out in the inception report. The emphasis is on gathering the community voice to gauge the pupil experience, rather than on quantitative data collection and analysis.

6.2 Objectives

6.2.1 Does the programme have clear, relevant and realistic objectives that focus on the desired impact?

6.2.2 Is there a clear and convincing plan, with evidence and assumptions, to show how the programme will work?

6.2.3 Who are the intended beneficiaries? How were they identified?

6.3 Delivery

6.3.1 Is the choice of funding and delivery options appropriate and based on the best existing evidence?

6.3.2 Was the programme set up with long-term impact on the communities in mind? Did it plan community involvement in designing the programme, in measuring impact and/or in achieving the planned outcomes? If so, in what way?

6.3.3 Is there good governance at all levels, with sound financial management and adequate steps being taken to avoid corruption?

6.4 Impact

6.4.1 Is the programme reaching the intended beneficiaries? Are the local communities involved in ensuring that the intended beneficiaries are reached and, if so, how?

6.4.2 What is the experience of individual students and recipient communities? Can community members identify any perceived impacts of the programme and, if so, what are they?

6.4.3 Has the programme generated any outputs or outcomes that were not intended or not included in the stated objectives?

6.4.4 What systems have been put in place to monitor the programme and measure impact? Are communities involved and, if so, how? With whom is the relevant information shared?

6.4.5 Which indicators of impact are being measured and how? What is the basis of the attribution of the benefits of UK funds to intended beneficiaries?

6.4.6 What is the potential for the programme’s long-term sustainability? What are the prospects for improvement, including assuring financial sustainability and local ownership?

6.4.7 Is there an appropriate exit strategy involving effective transfer of ownership of the programme?

6.5 Learning

6.5.1 Is DFID innovative in its approach? If so, in what way is it different to other similar interventions?

6.5.2 What lessons have been learned, including ways of improving delivery? What systems are in place to incorporate them a) in its Nigeria programme; and b) where relevant, in other DFID education programmes?

7. Outline methodology

7.1 The review will involve a number of elements, including:

• recording the voices and experiences of intended beneficiaries and community members;
• a literature review on past relevant evaluations;
• a review of evidence from the financial and information systems of DFID’s education programmes in Nigeria, including DFID’s partners (such as UNICEF and Cambridge Education Consultants);
• a review of DFID’s monitoring and evaluation framework and techniques;
• interviews with local, state and federal government education officials and DFID staff;
• triangulation of views from stakeholders, including the International Development Committee, civil society and peer organisations in the UK and Nigeria; and
• cross-reference to other relevant ICAI reports.

7.2 In particular, to capture the community perspective on impact, we will carry out:

• focus groups, including with pupils, teachers, education associations, civil society organisations and community leaders;
• field visits (unannounced where possible) to schools in both rural and urban areas; and
• one-to-one interviews with intended beneficiaries and community members to record their views and experiences.

7.3 The interviews will be carried out by experienced local people, mainly Hausa-speaking teachers who will be sensitive to working with young children and the range of issues affecting their participation and performance, at school. The fieldwork will also pilot the use of school-aged girls as interviewers. The interviewers will be supported by recent graduates from a local university who will record the discussions. To ensure the integrity of the fieldwork, and the safeguarding of the participants, the interviewers will comply with a ‘human subjects’ protocol which ensures that each interview takes place with the informed consent of the individual, is voluntary, is confidential and can be stopped at any time. The overall approach will be in compliance with ICAI’s policy on dealing with interviewees (including the protection of children).

8. Timing and deliverables

8.1 The review will be overseen by Commissioners and implemented by a small team from ICAI’s consortium – including the Center for Effective Global Action (CEGA) at the University of California at Berkeley, who are specialists in qualitative research. The review will take place during the second quarter of 2012, with a final report available in the fourth quarter of 2012.

Footnotes

1. World Bank Data: Nigeria, the World Bank Group, 2010, http://data.worldbank.org/country/nigeria.
2. Country Overview: Nigeria, The Extractive Industries Transparency Initiative, 2011, http://eiti.org/Nigeria.
3. World Bank Data: GDP Growth (Annual %), the World Bank Group, 2011, http://data.worldbank.org/indicator/NY.GDP.MKTP.KD.ZG.
4. DFID Nigeria Operational Plan 2011-2015, DFID, 2011, http://www.dfid.gov.uk/Documents/publications1/op/nigeria-2011.pdf.
5. Corruption Perceptions Index 2011, Transparency International, 2011, http://cpi.transparency.org/cpi2011/results/.
6. World Bank Data: Net ODA received (% of GNI), the World Bank Group, 2011, http://data.worldbank.org/indicator/DT.ODA.ODAT.GN.ZS.
7. UK Bilateral Aid Review, DFID, 2011, http://www.dfid.gov.uk/Documents/MAR/FINAL_BAR%20TECHNICAL%20REPORT.pdf.
8. DFID Nigeria Operational Plan 2011-2015, DFID, 2011, http://www.dfid.gov.uk/Documents/publications1/op/nigeria-2011.pdf.
9. Nigeria Vision 20: 2020, The First National Implementation Plan (2010 – 2013), Government of Nigeria: National Planning Commission, 2010, http://www.npc.gov.ng/vault/files/NV2020-NIP-Volume-II-Original-document_edited__versioin3_10_06_2010.pdf.
10. Out-of-School Children: New Data Reveal Persistent Challenges, UNESCO Institute for Statistics, June 2011, http://www.uis.unesco.org/FactSheets/Documents/FS12_2011_OOSC_EN.pdf.
11. DFID Nigeria Operational Plan 2011-2015, DFID, 2011, http://www.dfid.gov.uk/Documents/publications1/op/nigeria-2011.pdf.
12. Nigeria Vision 20: 2020, The First National Implementation Plan (2010 – 2013), Government of Nigeria: National Planning Commission, 2010, http://www.npc.gov.ng/vault/files/NV2020-NIP-Volume-II-Original-document_edited__versioin3_10_06_2010.pdf.
13. DFID Nigeria Operational Plan 2011-2015, DFID, 2011, http://www.dfid.gov.uk/Documents/publications1/op/nigeria-2011.pdf.
14. Information provided to ICAI by DFID.
15. DFID Project Details: Education Sector Support Programme in Nigeria, DFID, http://projects.dfid.gov.uk/project.aspx?Project=104200.
16. DFID Project Details: Girls Education Project, DFID, http://projects.dfid.gov.uk/project.aspx?Project=104199.
17. DFID Nigeria Operational Plan 2011-2015, DFID, 2011, http://www.dfid.gov.uk/Documents/publications1/op/nigeria-2011.pdf.
18. Johnson, David and Sergij Gabrscek, Education Sector Support Programme in Nigeria (ESSPIN) – An Assessment of the Development Needs of Teachers in Nigeria Kwara State Case Study: Report No. KW 301, 2008, http://www.esspin.org/index.php/resources/download/52/KW%20301%20An%20Assessment%20Of%20the%20Development%20Needs%20of%20Teachers%20in%20Nigeria%20(Revised).
19. Mid Term Review of the Education Sector Support Programme in Nigeria (ESSPIN), July 2011, http://www.esspin.org/index.php/resources/download/329/ESSPIN%2520Mid%2520Term%2520Review&sa=U&ei=C760T9yKMMen8QONubgj&ved=0CBgQFjAC&sig2=oTVDlNr3o5MuWGMW4sZNlw&usg=AFQjCNEM3yPerWJLH64x_9NlOtK_VsnxzQ
20. DFID’s Programme in Nigeria: Volume 1, House of Commons International Development Committee, HC 840-1, 2009, http://www.publications.parliament.uk/pa/cm200809/cmselect/cmintdev/840/840i.pdf.
21. DFID’s Programme in Nigeria: Government Response to the Committee’s Eighth Report of Session 2008-09, House of Commons International Development Committee, HC 250, 2009, http://www.publications.parliament.uk/pa/cm200910/cmselect/cmintdev/250/250.pdf.
22. DFID: Bilateral Support to Primary Education, National Audit Office, June 2010, http://www.nao.org.uk//idoc.ashx?docId=134fd82e-fdaf-4a38-8f58-1a0515f67d2f&version=-1.
23. Examination of Witnesses: the Department for International Development’s bilateral support to primary education, Public Accounts Committee, December 2010, www.publications.parliament.uk/pa/cm201011/cmselect/cmpubacc/594/10111002.htm.