Accessing, staying and succeeding in basic education – UK aid’s support to marginalised girls
Unsatisfactory achievement in most areas of effectiveness and value for money, with mixed achievement on relevance
DFID has made a major commitment to supporting girls’ education. As developing countries move closer to universal enrolment in basic education, continuing progress involves addressing the many barriers to girls’ education – both those applying to all girls and those leading to the marginalisation of particular groups of girls. In most of DFID’s priority countries, it has set itself objectives on girls’ education and developed targeted interventions. Its centrally managed, multicountry Girls’ Education Challenge is the largest ever donor programme targeting marginalised girls. Notwithstanding this commitment, we found that DFID does not have a coherent strategy for addressing girls’ marginalisation in education, and that its various activities are not well joined up. We identified a clear pattern of DFID programmes losing their focus on marginalised girls through the implementation process, leading to disappointing results. Furthermore, DFID’s approach to achieving value for money has not been updated to reflect the challenges of programming for hard-to-reach groups, and risks working against the “leave no one behind” commitment. We have awarded DFID an amber-red score for its performance in addressing the marginalisation of girls in education, indicating that significant improvement is required for DFID to achieve its ambitions.
Around the world, many girls face barriers that are preventing them from achieving a basic education. Globally there are 61.2 million primary and lower-secondary aged girls out of school. Others drop out early or fail to reach functional literacy and numeracy. The barriers are complex and context-specific, but range from early marriage and pregnancy to cultural biases and the lack of basic facilities for girls in schools. Some relate to all girls, others to particular groups of girls, resulting in intersecting causes of disadvantage. As developing countries progress towards universal primary education, addressing the marginalisation of girls in education has emerged as a major challenge.
DFID has been committed for many years to prioritising the education of girls to reduce gender inequality. In 2010, DFID’s Education Strategy for 2010-15 reflected the Millennium Development Goals on education and set out priorities for educating girls and other marginalised groups. In 2013, DFID stated that one of its three core priorities for education programming was to “keep girls in school, helping the most marginalised girls stay in school and learning for longer”. In 2015, reflecting its commitment to the Global Goals, it pledged to “leave no one behind” in its development assistance – that is, to prioritise those who may not be reached by “business as usual” development programming.
In this review, we assess DFID’s support for girls who are marginalised in education. Some DFID programmes support education as a whole, including for girls, while others contain specific objectives around tackling gender inequalities in education. Some programmes also target particular groups of girls who are marginalised in education because of the way gender intersects with other forms of disadvantage, such as remoteness, poverty, ethnicity or disability. Our review covers this spectrum of programming for girls’ education. The topic enables us to examine whether DFID has found effective ways of supporting hard-to-reach girls, despite the greater challenges and cost involved. It also enables us to explore the implications of the “leave no one behind” commitment for DFID’s approach to achieving value for money.
DFID has been a major donor in the education sector for many years, and its commitment to promoting gender equality in education goes back to at least 2000. Given this long-standing engagement, we have conducted a performance review to assess how successful DFID has been at delivering on its commitments over the 2011 to 2015 period. We looked in detail at DFID’s basic education portfolio in six countries (Kenya, Lebanon, Malawi, Pakistan’s Khyber Pakhtunkhwa province, South Sudan and Tanzania). We carried out visits to two of these countries: Kenya and Tanzania. In each country, we explored DFID’s contribution to girls’ education as a whole and its efforts to reach marginalised girls.
Box 1: Examples of DFID’s support for marginalised girls
Leonard Cheshire Disability’s “Empowering Pioneering Inclusive Education Strategies for Disabled Girls in Kenya” project
Through its flagship girls’ education programme, the Girls’ Education Challenge, DFID funds a £2.8 million project implemented by Leonard Cheshire Disability in Kenya. Before the project, many disabled children were confined to their homes because of the stigma attached to disability and a lack of equipment and transport. As a result of the project, a group of disabled girls now attend mainstream schools. The project is working with education officials to revise the teacher training curriculum to include inclusive education, and demonstrating how to support disabled girls.
DFID South Sudan’s “Girls’ Education in South Sudan” programme
This £61.4 million programme responds to the pressing need for education in conflict-torn South Sudan. At present, there are only seven girls for every ten boys in primary school. The barriers to girls’ education, including economic pressures and early marriage, are exacerbated by the conflict. DFID’s
programme is providing cash incentives to households to offset the costs of keeping girls in school, cash grants to schools to improve the learning environment, and behaviour change communications designed to change the way people think about girls’ education.
How relevant is DFID’s approach to supporting the education of marginalised girls?
DFID has adopted a prominent position internationally on girls’ education, through its policy commitments, its international advocacy and its investments in research. We found that its commitment to gender equality in education is reflected in its country operational plans and across its basic education portfolio. Many of its country operational plans include objectives for girls’ education. DFID country offices have developed a range of interventions to support girls’ education. Of the 63 basic education country programmes we reviewed, 32 included one or more activities specifically targeting girls and eight of the programmes were directed primarily at girls.
To increase its focus on marginalised girls, DFID launched the centrally managed Girls’ Education Challenge (£355 million in 2012-17). This flagship investment is designed to reach at least a million marginalised girls across 18 countries in Africa and Asia. It is an innovative programme and it is making a significant contribution to learning on what works in supporting marginalised girls’ education. Taking into account both the Girls’ Education Challenge projects and DFID country programmes, we found that DFID’s level of focus on girls broadly corresponded with the level of need in terms of the number of girls out of school.
While DFID has made a clear commitment to girls’ education and has backed this with significant resources, we found that the department needs to strengthen the coherence of its approach to tackling the marginalisation of girls in national education systems. During the review period, DFID did not set out detailed objectives or guidance on addressing different types and causes of marginalisation. At the country level, we found that DFID’s policy dialogue, system building and targeted interventions for particular groups of marginalised girls were not joined up into a coherent whole. DFID’s flagship initiative, the Girls’ Education Challenge, works through civil society or the private sector, rather than directly through national governments. While this fosters innovation, DFID is not yet clear how successful innovations will be scaled up or replicated. Evidence from the country portfolios we reviewed suggests a need for better coordination between Girls’ Education Challenge interventions and country programmes, and for better use of learning to inform DFID’s policy dialogue with governments. DFID informs us that it is putting in place measures to improve coordination.
At the programme level, we found that DFID’s girl-focused programmes included a good mix of intervention types, focusing most commonly on financial barriers, social norms and teaching practices. Many of the programmes address multiple barriers, which the literature suggests is likely to be the most effective approach. However, a few common causes of girls’ marginalisation, such as early pregnancy and violence against girls in schools, are not consistently addressed. There is also little coordination between DFID’s programming for girls’ education and its work in other sectors, such as health.
The stakeholders consulted reported that DFID is making an important contribution to generating new research on girls’ education. However, we judged that the use of evidence and analysis in DFID’s girl-focused programming was of mixed quality. Around half of the programmes in our sample included robust analysis of the needs of girls and specific factors that cause marginalisation in education. In some instances, baseline research was conducted but not used to guide intervention design. There is also limited evidence of takeup by country offices of research, guidance or other knowledge resources about what works in addressing disadvantage in education.
Overall, we have awarded DFID a green-amber score for relevance. This reflects its strong commitment to girls’ education, its good range of girl-focused programming and the groundbreaking work of the Girls’ Education Challenge. Many of the most important causes of girls’ marginalisation are being addressed within DFID’s portfolio. However, we found a need for better coordination across different delivery channels and sectors, and a clearer understanding of how to combine policy dialogue, system building and targeted interventions to achieve transformative impact. We share the concern of some external stakeholders that DFID’s major investment in the Girls’ Education Challenge, although undoubtedly positive, has served as a substitute for a clear strategic approach.
How effectively is DFID delivering results for marginalised girls?
Under its global results framework, DFID set itself the target of supporting 11 million children in basic education in the period 2011 to 2015. This included 5.2 million girls. DFID reported reaching 11.3 million children, including 5.3 million girls. There was no specific targeting of marginalised girls.
We measured DFID’s performance in supporting girls’ education against the objectives and targets set out in its country operational plans and programme design documents. In some cases, these targets were later revised downwards. We focused on the original objectives and targets in order to assess whether DFID’s commitment to girls’ education was being sustained over time.
Four of our case study countries included objectives on girls’ education in their operational plans. We found a pattern of underperformance against those objectives, including targets that were not met or abandoned. At the programme level, DFID’s business cases often included objectives on girls’ education that were not reflected in programme designs or logframe targets. While the programmes generally scored well in annual reviews for their overall performance, we found that achievement against the original objectives on girls’ education often lagged behind. This was particularly the case where girl-focused interventions were included in broader education programmes. Where programmes focused primarily on girls, their performance tended to be better. There is a marked tendency for DFID programmes to lose their focus on girls’ education over time. We found that a series of recurrent factors were contributing to this loss of focus, including:
- Girls’ education objectives being displaced by competing priorities such as improving overall access to basic education.
- A lack of influence by DFID on the focus of government education programmes.
- Poor design of girl-focused interventions, inhibiting their ability to address barriers to girls’ education or to target marginalised girls.
- A lack of expertise in girls’ education on the part of delivery partners.
- Difficulties with implementing in challenging environments
If it is to meet its commitment to “leaving no one behind”, DFID needs to do better at addressing the factors contributing to the marginalisation of girls across its education portfolio and at ensuring that this focus is not lost during delivery.
We identified examples of both weak and strong targeting of marginalised girls. For a third of the programmes we reviewed, the geographical focus was driven by practical or political considerations rather than solely by a focus on marginalisation. For example, for the Girls’ Education Challenge iMlango project in Kenya, an e-learning programme that aimed to target marginalised girls, accessibility and electricity supply were strong factors in the selection of project sites. We found several programmes that were failing to reach their intended beneficiaries as a result of poor targeting, such as the voucher component of the Khyber Pakhtunkhwa Education Sector Programme. We also found some strong examples, such as Camfed’s Girls’ Education Challenge project (“A New Equilibrium for Girls”) in Tanzania, which works with and through communities to identify marginalised girls.
In 2015, DFID reported that the Girls’ Education Challenge (“the Fund”) was performing strongly, based on the output data then available. Our assessment, based on emerging mid-term outcomes data, interviews with DFID and implementer staff and detailed review of four projects, found that the Fund was behind on three of its four outcome milestones, on attendance, learning and sustainability. The Fund itself has identified that some of its interventions may not be sufficient to bring participating girls up to the level of functional literacy and numeracy, given lower than expected initial learning levels. There are also concerns about sustainability across the portfolio, given the model of working through non-state implementers and the challenges of influencing national education systems.
DFID has responded to these concerns by extending the Fund, including enabling the current cohort of beneficiaries to be supported through a full cycle of education and to reach a new cohort of highly marginalised girls. The extension is also aiming to accelerate progress with learning outcomes.
Overall, we are concerned about a recurring loss of focus on girls within DFID’s broader education programmes. In these programmes, performance against original girl-focused objectives is weaker than the average performance across all objectives. Focus is better sustained within programmes that are solely or mainly devoted to girls’ education. We have identified what appear to be recurrent or systemic reasons to explain this loss of focus. If DFID is successfully to deliver on its “leave no one behind” commitment, it needs to overcome these barriers. We have awarded DFID an amber-red score for effectiveness.
Is DFID ensuring value for money in supporting marginalised girls’ education?
DFID has increased its use of value for money analysis across its education portfolio in response to a 2010 National Audit Office report. It has guidance on value for money assessment in education, based on the standard UK government “‘3 Es” model of economy, efficiency and effectiveness. At the programme level, we saw evidence that value for money analysis is embedded in project management, although it tends to focus on economy and efficiency rather than effectiveness.
DFID is yet to adapt its education value for money framework to reflect the “leave no one behind” commitment. Providing education to marginalised groups involves additional challenges and higher unit costs. Many stakeholders consulted for this review, both within and outside DFID, suggested that DFID’s framework needed to include explicit consideration of equity. If it does not do so, there is a risk that the value for money approach creates disincentives to focus on marginalised groups. A notable example of this was the use of a cost-per-girl metric in the selection criteria for Girls’ Education Challenge projects. Although some higher cost projects were selected, this approach attached priority to reaching more beneficiaries, rather than “the most marginalised”, who were the target group.
DFID is yet to develop guidance on how value for money considerations apply when targeting hard-to-reach beneficiaries through education programmes. On a positive note, the Girls’ Education Challenge has generated evidence that could be used to inform such guidance. The programme is building up a set of efficiency and cost-effectiveness metrics for girls’ education interventions. DFID informed us that it is using this evidence for internal learning.
We assessed whether DFID had clear value for money criteria for its choice of delivery channels, including working with state or non-state education providers. For the time being, it lacks a basis for making informed judgments as to what combination offers the best value for money in different contexts. Support for low-cost private schools through the Girls’ Education Challenge, though controversial among some stakeholders, will contribute to the evidence base.
We have awarded DFID an amber-red score for its approach to value for money in this area. DFID has made efforts to strengthen value for money analysis across the education portfolio and has begun to collect the data required for more sophisticated assessments, particularly through the detailed work on value for money metrics carried out through the Girls’ Education Challenge. However, its education value for money framework has not yet been developed to take account of the prioritisation of marginalised groups and the “leave no one behind” commitment. As a result, it is not well suited to ensuring value for money in tackling the marginalisation of girls in education, and risks creating disincentives to focus on the most marginalised.
Conclusions and recommendations
We recognise and support DFID’s substantial commitment to tackling the marginalisation of girls in education. However, it is clear from this review that a “business as usual” approach in DFID’s education work will not be sufficient to deliver on its commitment to “leaving no one behind”. We have identified three major deficiencies in its approach:
• The lack of a clear strategic approach that would help promote coherence and complementarity across the various strands of its work, including through different delivery channels, kinds of interventions and sectors.
• A pattern of the focus on girls being lost through the implementation phase of its programmes.
• The fact that its value for money framework for education programmes is yet to be updated to reflect its commitment to “leave no one behind”.
The following recommendations are intended to address these issues:
DFID should develop country-specific strategies for marginalised girls’ education, based on detailed knowledge of the barriers in each context and learning from successful interventions. Its strategies should combine policy dialogue, system building and targeted interventions, and should identify opportunities for cross-sectoral working.
In its delivery plans and monitoring of programmes with objectives around girls’ education and marginalisation, DFID should make it a priority to ensure that this focus is not lost during implementation.
DFID should specify how to approach value for money analysis when targeting marginalised and harder-to-reach groups, emphasising equity as well as cost-effectiveness.
The UK government has made a strong commitment to promoting the education of girls in developing countries. Globally there are 61.2 million girls who should be in basic education (primary and lower secondary school) but are not. Despite important progress over the past decade in reducing gender inequality in education, many girls still face a range of barriers that prevent them from accessing, staying and succeeding in basic education.
In this review, we assess DFID’s efforts to tackle the marginalisation of girls in education. The topic is timely, given DFID’s commitment to leaving no one behind. This pledge, which applies across the aid programme, will require DFID to pay more attention to the targeting of its programmes in order to reach not just the poorest but also remote, marginalised and excluded groups. This has significant implications for how DFID designs and delivers its programmes and how it approaches value for money.
Education has been a major area of assistance for DFID for many years. Its commitment to promoting gender equality in education goes back to at least 2000. This is therefore an appropriate subject for a performance review, which looks at how well DFID has delivered on its commitments.
Box 2: What is an ICAI performance review? ICAI performance reviews take a rigorous look at the efficiency and effectiveness of UK aid delivery, with a strong focus on accountability. They also focus on core DFID business processes and explore whether its systems, capacities and practices are robust enough to deliver effective assistance with good value for money. Other types of ICAI review include impact reviews, which examine results claims made for UK aid to assess their credibility and their significance for the intended beneficiaries, and learning reviews, which explore how knowledge is generated in novel areas and translated into credible programming. Our review covers all of DFID’s bilateral programming on basic education, including in-country and centrally managed programmes, operational during the period of DFID’s Result Framework 2011-2015. We have not assessed the performance of DFID’s multilateral funding, but we have taken account of The 2010 Education for All Global Monitoring Report defined marginalisation in education as “a form of acute and persistent disadvantage rooted in underlying social inequalities”. The causes of marginalisation in education vary widely according to context, and can impact on enrolment, retention, progression and learning. Some DFID programmes support education as a whole, including for girls, while others contain specific objectives around tackling gender inequalities in education. Some programmes also target particular groups of girls who are marginalised in education because of the way gender intersects with other forms of disadvantage, such as remoteness, poverty, ethnicity or disability. Our review covers this spectrum of programming for girls’ education.
DFID’s work with the Global Partnership for Education – a multilateral partnership that supports the preparation and implementation of national education plans in developing countries.
Box 2: What is an ICAI performance review?
ICAI performance reviews take a rigorous look at the efficiency and effectiveness of UK aid delivery, with a strong focus on accountability. They also focus on core DFID business processes and explore whether its systems, capacities and practices are robust enough to deliver effective assistance with good value for money.
Other types of ICAI review include impact reviews, which examine results claims made for UK aid to assess their credibility and their significance for the intended beneficiaries, and learning reviews, which explore how knowledge is generated in novel areas and translated into credible programming.
Our review covers all of DFID’s bilateral programming on basic education, including in-country and centrally managed programmes, operational during the period of DFID’s Result Framework 2011-2015. We have not assessed the performance of DFID’s multilateral funding, but we have taken account of
The 2010 Education for All Global Monitoring Report defined marginalisation in education as “a form of acute and persistent disadvantage rooted in underlying social inequalities”. The causes of marginalisation in education vary widely according to context, and can impact on enrolment, retention, progression and learning. Some DFID programmes support education as a whole, including for girls, while others contain specific objectives around tackling gender inequalities in education. Some programmes also target particular groups of girls who are marginalised in education because of the way gender intersects with other forms of disadvantage, such as remoteness, poverty, ethnicity or disability. Our review covers this spectrum of programming for girls’ education.
Table 1: Our review questions
|Review criteria and questions||Sub-questions|
|1. Relevance: How relevant are DFID’s basic education programmes given what is known about the causes of marginalisation and barriers to access, retention and learning for marginalised girls?||• To what extent are programmes addressing the different barriers to girls’ basic education?
• Is the design of programmes based on good evidence/learning on what works and why?
|2. Effectiveness: How well are DFID’s education programmes performing in delivering results for marginalised girls?||• What results is DFID reporting?
• How effectively is DFID targeting interventions to support the education of marginalised girls?
|3. Value for money: How does DFID ensure value for money in delivering education for marginalised girls?||• Are delivery mechanisms cost-effective in getting marginalised girls into school and supporting their retention and learning?
• How efficient are different mechanisms (state and non-state) in addressing the education needs of marginalised girls?
There are three main methodological elements to our review.
i. Our literature review covered recent academic works and publications by international organisations. It synthesised analysis of the barriers to education for marginalised girls across different country contexts and experience in attempting to overcome them. This provided a reference point for assessing the relevance and effectiveness of DFID’s programmes.
ii. We carried out a strategy review of DFID’s approach to supporting basic education for marginalised girls. This contributed to our assessment of the relevance of DFID’s approach and portfolio as a whole and DFID’s approach to value for money in the context of a focus on marginalisation. It included:
- Review of DFID strategy documents and guidance
- Consultation with key stakeholders in the UK, including DFID and a range of external partners, and a workshop with practitioners
- Mapping of the level of focus on marginalised girls’ education in DFID country operational plans over the review period
- High-level mapping of all bilateral education programmes in scope.
iii. We carried out country assessments of DFID’s education portfolio in six countries (Kenya, Lebanon, Malawi, Pakistan’s Khyber Pakhtunkhwa province, South Sudan and Tanzania), selected to cover a range of contexts for girls’ education (see Figure 1) and types of programming (see Annex 2 for a list of the programmes we assessed in these countries). We assessed how DFID’s approach in each country was informed by local conditions, whether there was a relevant and coherent approach to girls’ education across the portfolio, whether DFID was achieving its objectives on girls’ education and the strength of programme approaches for maximising value for money. For two countries (Kenya and Tanzania), the review team conducted visits, allowing for more in-depth assessment. We also assessed the Girls’ Education Challenge as a portfolio. Country assessments were guided by a standard Country Assessment Framework specifying the information that should be collected and providing standard criteria for evaluative judgements.
The methodology is explained in more detail in Annex 2 and in full in our Approach Paper. Both our methodology and this report were independently peer reviewed.
Box 3: Limitations of our methodology
Our country case studies were purposively selected to reflect a range of contexts and programme types. The sample may not be fully representative, and our findings may not always be applicable to the portfolio as a whole. We relied primarily on DFID’s own reported results data to judge programme effectiveness, triangulated to the extent possible through key stakeholder interviews and other data sources. This may introduce a bias into the analysis. We inevitably gathered more in-depth information for the case study countries that we visited, so examples from these countries are more prominent in this report.
The challenge of achieving gender equality in education
Recent decades have seen important progress towards gender equality in education around the world. From 2000 to 2015, over the period of the Millennium Development Goals, the number of countries with gender parity in both primary and secondary enrolment increased from 36 to 62 (of a total of 145 countries with available data). However, most countries are still some distance away from gender equality in enrolment in education, and disparities are often pronounced at sub-national level. Across DFID’s partner countries, more girls than boys of primary-school age are out of school (17.9 million girls, compared to 13.6 million boys, in those countries for which DFID has data). There are also gender disparities in completion and attainment in many countries, and these disparities can be particularly stark for marginalised girls.
Investing in girls’ education offers significant economic, health and social benefits – for the girls themselves, their families and their communities. Educating girls reduces child mortality, early marriage, fertility and the risks of HIV infection and early pregnancy. It also improves maternal health outcomes and raises the potential earnings of women.
The causes of marginalisation of girls in education are complex and vary according to context, but there are some common barriers that many girls face. These include the costs to households of educating girls relative to the perceived likely benefits, the lack of basic facilities for girls in schools, violence against girls in school and the prevalence of early pregnancy and marriage. Girls in school can face barriers to successful learning, including both in-school factors (such as teachers’ attitudes towards girls) and out-of-school factors (such as the heavy burden of domestic chores on girls).
Box 4: Stakeholders views on girls’ marginalisation in education from our case study countries
“Parents want to send girls to a place where they feel safe, do not suffer from harassment and are not vulnerable to terrorist or militant attacks. In a school environment, especially for girls, a lot of the province is not considered safe, with too many perceived dangers. The more marginalised girls tend to be in the more conservative districts. Khyber Pakhtunkhwa has some very remote and inaccessible areas. These are very mountainous areas where there is also a lack of proper roads and infrastructure. This often goes hand in hand with a certain conservativism and religiousness.” Pakistan
“Girls from most pastoralist communities are regarded as family assets and that’s why early and forced marriage is common in South Sudan.”
“The whole country is marginalised and we are back to base one.” South Sudan
“Girls from the displaced settlements where the refugees live (the “camps”) have more challenges to access education. These include social barriers from early marriage to taking care of small babies or even child labour.”
“There are huge numbers of out-of-school children and they are facing many barriers. There are gender specific barriers for boys and for girls, we have to tackle both.”
“Children with disabilities and suffering from the impact of conflict are the ones I am the most concerned about. There is an inclusive education strategy but it is not operational yet.” Lebanon
“Most of the schools do not have sanitary facilities, which disadvantages girls more than boys because girls need more privacy… Inadequate number of female teachers to serve as role models for the girls… Some communities or households practise early marriages… Poverty is a big thing that plays out when it comes to marginalisation of girls… In female-headed families, it is a common practice in Malawi that a girl child is more engaged in domestic calls to support other siblings, to the extent that girls would easily give in when it comes to their right to education for the sake of their siblings, boys… Families are more prepared to send boys to school than a girl child, especially in times of economic difficulties.” Malawi
“In terms of sexual abuse and harassment, it clearly goes on… It’s not just acute physical or sexual violence. It’s the whole esteem, talking down, putting young girls and women in their place.” Tanzania
“Disabled girls suffer double or triple what the boys suffer. They will not be given any attention in their family.” Kenya
Addressing marginalisation of girls in education is a complex challenge. The literature suggests that this requires the promotion of inclusive education systems and targeted interventions to address specific barriers faced by girls. Creating an effective package of interventions requires detailed knowledge of local contexts, as well as a willingness to challenge entrenched norms and practices.
Box 5: Conclusions from the literature on the causes of marginalisation and how they can be addressed
• There is no agreed definition in the literature of marginalisation in education, or the marginalisation of girls in or through education.
• The roots of girls’ marginalisation in education lie in the political economy and the socio-cultural conditions of schooling and education policy formation, and are therefore difficult to address.
• Gender intersects with other factors such as poverty, location, ethnicity and disability to produce heightened disadvantage for some girls.
• Which groups of girls in a country are the most marginalised in education varies according to context. Different forms of marginalisation will have different implications for the types of interventions required.
• Some disentangling is needed as to whether addressing marginalisation is about working with some effects – for example girls out of school – or whether engaging with marginalisation entails work on deeper causes, which require not simply a list of “factors” causing marginalisation but an assessment of the historical, structural gender dynamics.
• There is limited evidence about which interventions to support girls’ education work over the longer term. Most of the evaluations that have been carried out look only at short-term effects. However, a range of interventions on girls’ education are well evaluated in the short term and have been seen to work together with addressing lack of resources (eg conditional cash transfers), inequitable institutions (eg training teachers in gender-sensitive teaching) and a range of norms (eg girls’ clubs).
• A mix of interventions is likely to be more effective than trying to identify a “silver bullet” solution.
DFID’s commitments on marginalised girls’ education
DFID has been committed to prioritising girls’ education since 2000. The Millennium Development Goals included the objective of achieving gender parity at all levels of education. DFID was also a signatory to the Education for All Goals, which included achieving gender equality by 2015, with a particular focus on ensuring that girls have access to basic education of good quality. The theme of marginalised girls’ education now features in two of the Global Goals (“Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all” and “Achieve gender equality and empower all women and girls”).
These commitments are echoed in DFID’s policies and strategies from 2005 onwards. Successive strategies and policy statements have had a strong emphasis on girls accessing and remaining in education (see Figure 2). The former International Development Secretary, Justine Greening, was a strong advocate of the rights of girls.
As access to basic education expanded in DFID’s partner countries, it became apparent that achieving gender equality in education would require targeted measures to address groups who were marginalised or hard to reach. In November 2015, in keeping with the Global Goals, the UK made a commitment to “leaving no one behind” in its development assistance. It pledged that “people who are furthest behind, who have least opportunity and who are the most excluded will be prioritised”.
DFID’s programming on marginalised girls’ education
Education is one of DFID’s major areas of development assistance, receiving around 12% of UK bilateral aid. Over our review period, DFID spent £2.6 billion on country programmes relating to basic education (the programmes had a combined lifetime budget of £4.3 billion). DFID is the second largest bilateral donor in basic education.
There is no way of separately identifying DFID’s expenditure on girls’ education or on tackling marginalisation because such interventions are usually part of broader programmes. However, we classified DFID’s country basic education programmes into those that include at least one activity targeting barriers to girls’ education (we call these “girl-focused programmes”) and those where the majority of interventions target girls (“highly girl-focused programmes”). See Box 6 for the definitions we used and Figure 3 for the number of programmes and associated expenditure in each category over the review period.
Box 6: Definitions of different types of programme
For this review, we classified DFID’s education programmes according to their degree of focus on girls’ education:
• Girl-focused programmes – those which include at least one activity which targets barriers specific to girls’ education.
For example, DFID’s Khyber Pakhtunkhwa Province Education Sector Programme supports implementation of the Education Sector Plan in this province in Pakistan. DFID funding supports a mixture of interventions, with some specifically targeting girls, including stipends for girls conditional on adequate school attendance.
• Highly girl-focused programmes – those where we judged that the majority of activity was targeting girls, rather than merely including some girl-focused activities within a broader programme.
For example, the Keeping Girls in School programme in Malawi supports a series of interventions to support girls in school, including increasing numbers of female teachers, bursaries and cash transfers, the provision of girls’ sanitation facilities, training Mother Groups to provide pastoral support to girls, and technical assistance to the Malawian government to strengthen its capacity to support girls’ education.
In our analysis we classified the centrally managed Girls’ Education Challenge as a highly girl focused programme.
• Non-girl-focused programmes – those which do not contain any activities targeting barriers specific to girls’ education.
The Education Support in Malawi programme funds infrastructure, as well as improvements in the teaching of early literacy (for example through teacher training), but with no explicit focus on girls.
It is important to note that programmes that we have categorised as girl-focused may involve only a small proportion of activity targeting barriers specific to girls’ education.
We also draw a distinction according to whether bilateral programmes are managed centrally by DFID’s UK-based teams (referred to in this report as centrally managed programmes) or by DFID’s country offices (country programmes).
We note that DFID’s investments in building education systems through non-girl-focused education programmes also benefit marginalised girls. Furthermore, interventions like policy dialogue and advocacy, with no specific budgets attached, may also be important in addressing the needs of marginalised girls.
In addition to its country programmes, DFID has made a major financial commitment to supporting marginalised girls through its centrally managed Girls’ Education Challenge. This programme was established to boost DFID’s results on girls’ education and hard-to-reach and marginalised girls. It spent £249 million during the review period, with a budget of £355 million for its first phase (2012-17). A further phase has been approved. Ten other centrally managed programmes with a focus on girls’ education spent £23.7 million during the review period.
The Global Partnership for Education is a multilateral partnership that supports the preparation and implementation of national education plans in developing countries. Between 2011 and 2014, this partnership spent between £275 and £314 million per year across 50 countries. DFID was the largest donor, at £296 million (2011-14).
Box 7: The Girls’ Education Challenge
DFID launched the Girls’ Education Challenge (the Fund) in 2012 in order to increase its focus on the education of marginalised girls. The business case argued that DFID’s existing education programmes were not responsive enough to reach the most marginalised girls. The Fund has adopted a working definition of marginalised girls: “girls (age: 6 to 19) who have not been enrolled in or have dropped out of school (whether living in slums, remote areas, ethnic/religious minorities, girls with disabilities, girls who become pregnant, girls affected by conflict) or are in danger of doing so”. This highlights a range of categories of girls who experience some form of particular disadvantage that may lead to marginalisation from education. The identification of these categories was supported by evidence presented in the business case.
The Fund has a budget of £355 million for its first phase, which ends in June 2017. It has supported 37 individual projects across 18 countries in Africa and Asia through a competitive process. The Fund is managed for DFID by a consortium led by PricewaterhouseCoopers LLP.
The projects were selected under three funding windows:
• Step change: scaling up successful interventions that are already having a positive impact.
• Innovation: applying new interventions such as technological innovations, developing new partnerships, adapting proven solutions for new geographies, communities or age groups.
• Strategic partnerships: creating new partnerships with the private sector, including Discovery Communications, The Coca Cola Company, Avanti Communication and Ericsson.
The Girls’ Education Challenge approach rested on the assumption that supporting and expanding nonstate provision – including non government organisation-led programmes, consortia of low-cost private schools and new public-private partnerships – would prove better able to reach marginalised girls than traditional support for national education systems.
DFID’s business case stated that the Fund will:
• Allocate funding to projects that will enable approximately 650,000 marginalised girls to complete a full six-year cycle of primary school or approximately one million marginalised girls to complete three years of junior secondary school.
• Focus on getting girls into primary and secondary education, keeping them there, and ensuring they learn.
• Support girls through at least one level of education.
• Work with new partners and in new ways to drive innovative approaches to educating girls.
• Only fund non-state organisations.
• Leverage private sector involvement and resources.
• Complement existing DFID (and other donor and national government) activity.
DFID has also used the Fund to test results-based contracting, whereby implementers are paid in proportion to the results they deliver.
DFID has recently announced plans to extend the Girls’ Education Challenge through two new funding windows:
• A girls’ transition window, to extend and expand cost-effective interventions from the first phase that accelerate girls’ learning outcomes, retention, progression and completion at the next level of education.
• A leave no girl behind window, to fund targeted “catch-up” or accelerated learning programmes, providing literacy, numeracy and skills for life and work for highly marginalised girls, particularly adolescent girls that have dropped out of or never attended school.
In this section, we assess the relevance of DFID’s approach to tackling the marginalisation of girls in education. We look at the level of commitment to girls’ education in its country operational plans and its basic education programmes, and the extent to which these programmes respond to the specific causes of marginalisation of girls in education found in each of our case study countries. We look at the approach of DFID’s flagship centrally managed fund on girls’ education, the Girls’ Education Challenge, and whether DFID’s country portfolios work as a coherent whole.
DFID lacks an explicit approach to addressing marginalisation of girls in education
Many of the non governmental organisations, academics and donor stakeholders we consulted for this review regard DFID as a global champion for girls’ education. Stakeholders from all three groups told us that its high-profile policy commitments, significant financial commitment to the Girls’ Education Challenge and international advocacy at events such as the 2016 Girls’ Education Forum have given DFID a substantial international profile in the area.
While its education targets are disaggregated by gender, DFID has not articulated how its objectives to support marginalised girls’ education and the “leave no one behind” commitment apply to its country education portfolios. There is no overall theory of change or detailed analysis about how to tackle the different causes of girls’ marginalisation in education through policy dialogue and programming. Its policy documents do not distinguish between the marginalisation of girls as a whole and groups of girls facing particular forms of disadvantage. The Girls’ Education Challenge identified categories of girls who were more likely to experience disadvantage in education, but DFID has only recently developed its thinking on overlapping causes of marginalisation. Current Girls’ Education Challenge projects are left to adopt their own definitions of marginalisation, although DFID informed us that evidence from the mid-term evaluations is being used to inform targeting in the next phase of the Fund.
Most of DFID’s country operational plans include a focus on girls’ education
Many of DFID’s country operational plans include commitments and objectives relating to girls’ education. Over the review period, 22 of DFID’s 28 country offices were active in basic education. Of these, 18 included a focus on girls’ education in their 2011-12 country operational plans and 11 had a strong focus.
The emphasis on girls’ education in country operational plans was broadly in line with need, measured in terms of the number of girls out of school. Two exceptions were Pakistan (at least up until 2014), where DFID informed us that national counterparts expected its education programmes to address the needs of both girls and boys, and Yemen.
As the operational plans were updated over the review period, the focus on girls’ education increased up until 2014, before receding again between 2014 and 2016 (although all the plans retain at least some reference to girls’ education). This trend coincides with an increased focus on a broader girls’ empowerment agenda (eg reducing gender-based violence and tackling discriminatory norms), as well as on marginalised groups generally.
Girls’ education continues to feature to a greater or lesser extent in country operational plans. However, we found uneven attention being paid to identifying specific groups of marginalised girls to be prioritised through education programming. The few positive examples include direct targeting of the poorest girls in ethnic minority states in Myanmar, and geographic targeting of the poorest states and districts in India or hard-to-reach slums and arid areas in Kenya.
A commitment to educating girls runs through the portfolio, but with varying depth and focus
Of the 63 country education programmes that we reviewed, a large majority included a commitment to promoting gender equality within their design documents.
We identified 32 country programmes with activities specifically targeting girls (girl-focused) and, within that group, eight programmes with the majority or all of their activities targeting girls (highly girl-focused). The 32 girl-focused programmes had a combined lifetime budget of £2.2 billion, with the eight highly girl-focused programmes representing £327 million of the total.
The remaining 31 programmes had no specific activities for girls. 20 of them claimed that they would nonetheless promote gender equality in education – for example by stating that interventions not specifically targeting girls would benefit girls more than boys, or that the partner country or implementing partner had a strong gender focus. In half of these cases, we concluded that these claims were not substantive, based on the programme descriptions in the business cases.
Fewer than half of the 32 girl-focused country education programmes in our sample specified particular groups of marginalised girls amongst their target groups. The majority of these focused on girls in particular geographical areas, while six included a focus on disabled girls.
Across our six country case studies, interviews and documentary evidence point to a number of factors that have worked against a stronger and more consistent focus on marginalised girls’ education. These include:
• Competing educational priorities for DFID country offices, evident in five of the case study countries assessed (Malawi was the exception).
• A perceived risk of political and community resistance to an overt focus on girls, reported in four of the country case studies – Kenya, Tanzania, South Sudan and Pakistan.
• A lack of fine-grained data to analyse the causes of (and potential solutions to) girls’ marginalisation, evident in DFID’s response to the education crisis among Syrian refugees in Lebanon and also acknowledged by DFID Kenya. DFID Kenya’s education portfolio includes an emphasis on action research and impact evaluations, to help identify what works.
Box 8: How well does DFID target the specific needs of marginalised girls in each country?
Malawi’s biggest education challenge for girls is low transition rates from primary to secondary school. This is being tackled in one highly girl-focused country programme (Keeping Girls In School), which includes interventions relating to affordability, teaching practices, community attitudes and gender-based violence. Other general education country programmes complement this by addressing other barriers facing girls and boys. These include investments in school infrastructure and early reading improvement.
South Sudan faces large discrepancies between girls’ and boys’ access to schools. DFID South Sudan has several country education programmes that aim to tackle these discrepancies, mostly through cash transfers to girls, changing community attitudes, improving teaching and learning, and building infrastructure. There is thus a coordinated focus on girls across the education programmes.
Kenya has a relatively large number of programmes tackling marginalised girls’ education, with a strong focus on new research and evidence. However, there is a need for greater coherence across country and Girls’ Education Challenge projects in targeting marginalised groups or areas and sharing evidence. There is limited work to support institutional change related to girls’ education – for example through the Global Partnership for Education.
Pakistan (Khyber Pakhtunkhwa province)
One programme (the Khyber Pakhtunkhwa Education Sector Programme) has a focus on girls and out-of-school children and addresses some of the most relevant barriers to girls’ education (affordability and lack of schools). Khyber Pakhtunkhwa province is a conflict-affected area and girls’ education is a potential factor in the causes of conflict. DFID is sensitive to this context and does not focus too explicitly on girls.
Tanzania has two Girls’ Education Challenge projects, including a Camfed initiative supporting marginalised girls in secondary schools. However, DFID’s country programmes are not addressing girls’ education issues at the secondary level, which is where girl-specific barriers, such as the management of menstruation and early pregnancy and marriage, become most apparent.
Lebanon has the heaviest burden of refugees per inhabitant of any country. Among refugee children, only just over half of 6 to 14-yearolds are in school, with little difference between boys and girls. DFID’s education programming in Lebanon has so far focused on refugee children as a group, rather than on specific barriers faced by girls (such as early marriage, which is a common coping strategy among vulnerable households), although a new programme gives more consideration to girls and includes interventions designed to address barriers to their education.
The Girls’ Education Challenge was established to increase DFID’s focus on marginalised girls’ education
Early in the review period, DFID assessed that its existing bilateral and multilateral support was not reaching the most marginalised girls. In 2012, it launched the centrally managed Girls’ Education Challenge (“the Fund”) in order to boost its investment in marginalised girls’ education. With a lifetime budget of £355 million in its first phase (2012-17), the Fund has supported 37 projects across 18 countries in Africa and Asia, working solely through non-state implementing organisations.
The Girls’ Education Challenge is a significant investment, supporting a million marginalised girls in its first phase. It is the largest ever aid programme dedicated solely to girls’ education. Its high profile has helped to signal to partner countries and other development partners the importance of this issue. Stakeholders (internal and external to DFID) saw the Fund as helping to position DFID as a global leader in the area. It is also seen as a driver of innovation, able to test out new approaches to combatting the marginalisation of girls.
As a fund dedicated to supporting marginalised girls in education through partnership with non governmental organisations and private sector organisations, the Girls’ Education Challenge has greater flexibility to bypass some partner countries’ reluctance to dedicate resources specifically to girls’ education. By contrast, some of DFID’s country offices told us that bilateral country programmes delivered in partnership with national governments are under pressure to address the needs of the population as a whole.
DFID has not prioritised marginalised girls in policy dialogue and system-building
DFID’s decision to promote girls’ education through a separate, centrally managed fund may have made it more difficult to develop a coherent approach to marginalised girls’ education in each of its partner countries.
Girls’ Education Challenge projects are relatively short-term (in line with the Fund’s initial four-year funding cycle). While they support girls within state schools, they are not delivered through state actors. We found that impact is generally limited to the girls who are reached directly, rather than via changes to education systems, and that little attention is given to the sustainability of the interventions. The Fund is now being extended to a second phase with the aim, amongst other things, of enabling girls who are currently supported to complete a full cycle of primary or secondary education, and of sustaining and leveraging future investment in girls’ education.
To achieve wider and longer-lasting impact, DFID needs to go beyond direct funding of services for marginalised girls and do more to promote their interests in national education policies and systems. This is the territory of many of DFID’s country programmes and its work with the Global Partnership for Education, which supports national education strategies. DFID informed us that, during the latter part of the review period, they have taken steps to improve coordination between the Girls’ Education Challenge and other programming. However, we saw limited evidence of learning or innovation from Girls’ Education Challenge projects being used to inform DFID’s policy dialogue in our case study countries or system-building programmes. Stakeholders (including DFID and implementer staff) confirmed that coordination between the Girls’ Education Challenge and DFID country offices and programmes has been mixed.
In our case study countries, we found that most DFID country education programmes were not pursuing system-wide change in relation to girls’ marginalisation. One notable exception was the Keeping Girls in School programme in Malawi, which planned to strengthen the education ministry’s strategy for girls’ education and effect change at both national and district government levels.
In two of our case study countries (Kenya and Tanzania), the majority of external stakeholders we interviewed reported that DFID did not promote girls’ education within the Global Partnership for Education at the country level. The 2014 annual review of the Global Partnership for Education also commented on the need to improve communication between the central partnership team and DFID country offices.
At the global level, however, DFID has supported initiatives to improve the Global Partnership for Education’s work related to marginalised girls, including through the secondment of an education advisor. DFID has also informed us that, over the past 12-18 months, it has engaged with the Global Partnership for Education to develop a results framework, which includes gender-disaggregated targets. It is too early to assess whether these initiatives will lead to a stronger focus on marginalised girls in the country education sector plans.
DFID’s girl-focused programming includes a good range of complementary interventions, with some gaps
Given the complex causes of girls’ marginalisation in education, the literature concludes that multipronged interventions and cross-sectoral working are required to enhance girls’ participation in school and the quality of education they receive.
To assess the relevance of DFID’s programming choices, we drew on our literature review to develop a typology of interventions related to common causes of marginalisation. Positively, across the portfolio of girl-focused education programmes, we found that programmes with a combination of intervention types were the norm.
Some of the most common types of intervention in DFID’s girl-focused programmes aimed to tackle barriers to girls’ education related to:
• Gender inequalities in teaching and learning practices in schools – for example increasing the number of female teachers in order to provide role models for girls and providing gender-sensitive teacher training.
• Gender inequalities in community practices and norms – for example mobilising communities in support of girls’ education, such as through tackling negative perceptions about the value of educating girls.
• Affordability – for example stipends, scholarships and conditional cash transfers to reduce the economic barriers to girls’ education (including opportunity costs to households, for instance from lost labour).
DFID’s highly girl-focused programmes, including Girls’ Education Challenge projects, generally take a more holistic approach to tackling the causes of girls’ marginalisation. They include more intervention types and demonstrate a greater understanding of the links between education and girls’ empowerment.
Across the country portfolios we reviewed, we found that school-related gender-based violence and health barriers were relatively neglected. Expert stakeholders identified a need for increased focus on early child marriage and pregnancy. We found some positive examples of projects that are helping girls manage menstruation and avoid pregnancy. Table 2 contains examples from Kenya and Tanzania. DFID has identified these areas as a focus for the next phase of the Girls’ Education Challenge.
Table 2: Programmes tackling gender-based violence and reproductive health issues
|Gender-based violence in the community||School Development Fund, part of the Kenya Essential Education Programme (country education programme, Kenya)
The School Development Fund planned, amongst other interventions, to implement girls’ clubs across target schools to raise awareness of issues such as early child marriage, and encourage reporting to and intervention by local officials to enforce legislation.
|School-related genderbased violence||The Safe Schools for Girls project, part of the Kenya Essential Education Programme (country education programme, Kenya)
The project plans to develop gender-sensitive school plans, train boards of management on national laws and policies, and form girls’ clubs to support girls.
|Sexual and reproductive health||A Community Based Approach: Supporting Retention, Re-entry and Improving Learning (Girls’ Education Challenge project implemented by BRAC, Tanzania)
The project planned to introduce study clubs for in-school and outof-school girls, including coverage of sexual and reproductive health issues with the aim of improving girls’ understanding and awareness of pregnancy, sexually transmitted diseases and marriage.
Gaps in programming relating to sensitive issues like reproductive health and violence against girls reflect the complexity of changing deeply rooted social inequalities and cultural norms. Tackling such barriers requires well-connected, multi-level interventions, working with both government and society. DFID and external expert stakeholders recognise the need for better coordination between education and other programmes, such as on gender-based violence. However, they remained unsure about how to achieve greater coherence in practice. DFID Kenya noted the difficulty of coordination across multiple programmes with different objectives and time periods. Overall, we found limited coordination between girl-focused country education programmes and interventions in other sectors.
Use of evidence is of mixed quality
Within the six case study countries, we found that around half of the girl-focused education programmes were based on a robust analysis of the barriers to education facing girls. As would be expected, highly girl-focused programmes (including Girls’ Education Challenge projects) showed better use of evidence and analysis of barriers facing girls in their designs.
In some cases, further baseline research was carried out during inception, to obtain a better understanding of the barriers facing girls. However, while such research helped to establish baselines for logframe targets, it was not necessarily used to refine the design of interventions (in the case of some Girls’ Education Challenge projects, due to time pressures on delivery).The majority of girl-focused education programmes within case study countries were found to be
The majority of girl-focused education programmes within case study countries were found to be based on weak evidence of what works in addressing barriers to education, although highly girl focused programmes (including Girls’ Education Challenge projects) performed better. For example, the business case for DFID Pakistan’s Khyber Pakhtunkhwa Education Sector Programme presents very limited evidence on what works in girls’ education.
Box 9: DFID’s investments in research and evidence on marginalised girls’ education
DFID has made significant progress in consolidating the existing evidence on what works for girls’ education and inclusive education. At the beginning of the review period, there was limited robust research and evaluation about what works for marginalised girls. In 2014, DFID published a rigorous literature review on girls’ education and gender equality, which consolidated the available evidence, identified gaps and proposed an indicative theory of change.
There is a general consensus among UK development non government organisations, academic experts and other donors consulted for the review that DFID is at the forefront of generating new research on girls’ education within the donor community. In particular, the evaluation work underway through the
Girls’ Education Challenge is expected to make a major contribution. However, across the eight wider research programmes that we reviewed, gender equality and marginalisation were not always consistent areas of focus. DFID’s Education Research Team reported that it has adopted a Gender Action Plan (2015), which aims to ensure that gender issues are adequately covered in DFID’s future education research.
Conclusion on relevance
DFID is widely regarded by external stakeholders as a global leader on girls’ education, as a result of its policy commitments and international advocacy. It has also made a strong commitment to investing in research, to understand the barriers to girls’ education and what interventions work.
Many of its country operational plans in DFID priority countries include objectives on girls’ education, while around half of its country education programmes include at least some activities that specifically target marginalised girls. The Girls’ Education Challenge is a major investment that has helped to increase DFID’s focus on marginalised girls and raise the profile of the issue internationally. It is encouraging that, when the Girls’ Education Challenge is taken into account, DFID’s girl-focused programming is broadly concentrated in countries where the need is greatest. It also contains a good range of interventions addressing different causes of girls’ marginalisation.
While DFID’s commitment to and investment in marginalised girls’ education is substantial, our conclusion is that DFID needs a more strategic approach. It has not yet identified how to combine different initiatives – such as policy dialogue, system building and interventions targeting particular groups of marginalised girls – into a coherent strategy for addressing marginalisation, either at the central level or in specific countries. While Girls’ Education Challenge projects are innovative in design and targeted at the most marginalised, they cannot achieve transformative impact unless they also influence national education policies and systems. DFID has recognised this need for greater coherence and coordination. At the central level, it has also supported Global Partnership for Education actions to increase its focus on girls’ education.
Overall, we have awarded DFID a green-amber for relevance. This recognises its significant commitment of resource and effort in girls’ education, the groundbreaking work of the Girls’ Education Challenge. Many of DFID’s individual interventions are well designed. However, the lack of a clear strategy bringing together the different strands of DFID’s work is an obvious gap, resulting in inconsistencies in approach and insufficient complementarity across different types of programming.
In this section, we examine how DFID’s education programmes have performed against their objectives on supporting girls’ education. We look at DFID’s delivery against its global results target, at whether country offices have delivered the results anticipated in their country operational plans, and whether girl-focused programmes have achieved their objectives.
We have assessed DFID’s country operational plans and programmes against their original stated objectives. We recognise that objectives and targets are often revised. However, we are interested in the question of whether DFID sustains the focus of its programming on girls’ education during implementation, and the reasons for any changes.
DFID has met its global target for basic education
Under its overall results framework for 2011-15, DFID set itself the target of supporting 11 million children in primary and lower secondary education, including 5.2 million girls. This target was achieved. In 2015, DFID reported that it had supported 11.3 million children, including 5.3 million girls.
This global results framework only includes results from a subset of DFID’s country education programmes. It also excludes the Girls’ Education Challenge, which has a target of improving attendance and learning for up to a million marginalised girls. Therefore DFID has demonstrated that the global results target was achieved, but has not attempted to calculate the full extent of its reach in education (see Box 10 for the challenges it faced in doing so). Importantly, the reach indicator only measures enrolment, rather than attendance or whether children are actually learning. While these results give an indication of the scale of DFID’s education programming, including for girls, they do not help us identify whether marginalised girls are being supported.
Box 10: DFID’s global results target for basic education 2011-2015
DFID’s global results target on education measures the number of children who are in school only because of DFID support. The primary method of estimating this is to calculate the proportion of DFID’s financial support in the total education budget for a given country, region or group of schools, and multiply that by the number of children enrolled. However, over the review period, DFID moved away from supporting education primarily through budget support programmes towards more focused education programmes. DFID staff informed us that they found it difficult to summarise the results of these programmes in the form of a single reach indicator. The methods and assumptions they used to do this vary across programmes and countries.
We checked DFID’s reported result back to the supporting calculations and, on a sample basis, to the results data from individual programmes. In some instances, we were able to identify the source of the data. In other cases, the calculations were difficult to reproduce without a detailed investigation of each
Performance against the original targets in DFID’s country operational plans was poor or mixed
DFID country operational plans set out the results it aimed to achieve in each country. These include headline results targets and a discussion of related objectives. All country operational plans included education targets, although often with different measures from the global target in DFID’s Results Framework. Four of our six case study countries (Kenya, Malawi, South Sudan and Tanzania) included objectives and targets for girls’ education in their earliest country operational plans in the review period. In some instances, these objectives were revised or removed when the operational plans were renewed. When we assessed the four countries against their original objectives, we found that their performance was poor or mixed (see Table 3).
Table 3: Performance against expectations for girls in case study country operational plans
|Country||Positive performance against expectations||Negative performance against expectations|
|Kenya||Target to support 160,000 girls in primary education exceeded (236,383 girls supported).||Proportion of girls reported to have been supported (48%) was lower than targeted (53%).
Number of beneficiaries for a programme targeting adolescent girls with interventions (including in education) was reduced from 15,000 to 6,000. Revised target not met by the end of 2015 due to challenges of working in remote rural areas
|Malawi||Target for number of girls supported in primary school exceeded (242,098 against a target of 233,000). Target for number of girls supported with bursaries in secondary school exceeded (16,242 against a target of 15,000).||Target for mothers’ groups (3,000 groups) not met (implementation had not started by March 2015, which was the end of the period of this country operational plan).
Target for female teachers trained (380) not met (DFIDfunded teacher training colleges were not completed in time to train teachers to meet this target).
Target for toilets built in schools (4,200 toilets) not met (1,180 toilets built).
|South Sudan||Target for children supported in primary education significantly exceeded (1,430,000 children against an original target of 240,000).||Proportion of girls much lower than originally planned (33% compared to 75%) due to an adjustment in programming over the review period.
The original operational plan stated that DFID’s interventions would target a number of social norm barriers to girls’ education, but DFID-funded community mobilisation activities may not have been effective tools for transforming these norms.
|Tanzania||Targets nearly met for the number of girls supported to complete primary school (73,751 girls against a target of 75,000).||Target not met for number of girls supported in primary and lower secondary school (the target of 200,000 girls each year for four years was met in 2011-12 and 2012-13, but not for the other two school years).
Target not met for the number of girls supported to complete secondary school (22,685 girls supported against a target of 40,000).
Commitment to focus on girls in funding of improvements in the quality of secondary education not achieved.
Slower than expected progress on addressing violence in schools.
The performance of interventions on girls’ education was generally poor, particularly in broader education programmes
We assessed the performance of individual education programmes against their expected contribution to girls’ education. These expectations include formal targets recorded in logframes and other explicit commitments to activities or results. The majority of girl-focused programmes in our sample failed to meet the original expectations for their girl-specific components, as set out in their design documents (11 out of 19). Our assessments were based on DFID’s own reporting, interviews with the responsible DFID staff and implementing partners and, for six programmes in the two countries we visited, site visits.
DFID’s business cases often contain ambitions around supporting girls’ education that are not carried through into logframes and detailed programme designs. For example, DFID Tanzania’s Human Development Innovation Fund was designed to support seven results areas, including improved educational access for girls and vulnerable groups. The business case stated that it would “focus on and be weighted towards supporting innovations which will particularly benefit girls and women”. However, these commitments were not represented in logframe outcome or output targets. This works against effective monitoring of progress on girls’ education, including when programme plans are revised to reflect changing contexts.
Importantly, our rating of how DFID programmes performed on girls’ education does not necessarily reflect their performance in other areas. The majority of the programmes in our sample were rated by DFID as performing strongly. It is telling that their commitments and results on girls’ education have lagged behind their overall performance.
We also found that highly girl-focused programmes (including Girls’ Education Challenge projects) performed considerably better than other girl-focused programmes. This supports the conclusion that, where activities in support of girls’ education are part of broader education programmes, the focus on girls often gets lost. If DFID is to meet its commitment to “leave no one behind”, it needs to do better at mainstreaming and maintaining a focus on marginalised girls across its education portfolio.
Box 11: A loss of focus on girls in DFID’s South Sudan education programme
Ongoing conflict has made South Sudan one of the most underdeveloped regions in the world. South Sudan has some of the worst educational indicators worldwide, with outcomes for girls particularly poor, resulting from insecurity and conflict, as well as cultural practices such as early marriage.
DFID’s Girls’ Education in South Sudan programme has a budget of £64 million. The business case has a strong focus on girls. The expected result is “acceleration of girls’ enrolment, retention, completion and improved learning outcomes at primary and secondary levels of education in South Sudan.”
While output targets in the logframe have largely been met, we found that the focus on girls’ education has been insufficient to meet the programme’s objectives. Indeed, the latest annual review of the programme concludes that implementation needs to “better reflect the focus on girls”.
There has been good progress on providing girls with cash transfers and schools with capitation grants (payments per pupil); the milestones in these areas are likely to be exceeded. However, the annual review notes that the programme is not ensuring that schools use their grants to promote girls’ education, as originally intended. A plan in the business case to increase the capitation grant in respect of female pupils has not been implemented.
The programme relied on radio programmes and community mobilisation activities to transform social norms and behaviours that impede girls’ education. The evidence is that some of these are not having the intended effect. The annual review relates this to a lack of expertise on girls’ education and gender equality on the part of a member of the implementing consortium.
Targets around increasing the knowledge and evidence available to policy makers about what works to promote girls’ education in South Sudan have been met in quantitative terms, but the latest annual review recommends “a far greater focus on girls in all research”.
Recurrent factors across the portfolio are causing DFID to fall short of its objectives on supporting marginalised girls’ education
Across the 19 girl-focused programmes in our sample (including four Girls’ Education Challenge projects), we identified various factors impeding performance against expectations for girls’ education. These factors were commonly associated with a loss of focus on girls. Together, these suggest that DFID faces systemic problems in carrying through on its commitments on girls’ education, which is relevant to its “leave no one behind” pledge.
• Girls’ education objectives being displaced by competing priorities within the programmes, such as the need to boost access as a whole. This was an issue impeding programme performance for four of the programmes we assessed.
Example: DFID Lebanon’s £5 million Low Cost Innovative Non Formal Education and Skills for Lebanese and Syrian Children and Youth programme (2015-16), implemented by UNICEF, was intended to reach equal numbers of girls and boys, and included tailored interventions to address barriers to education specific to each gender. While its gender-disaggregated reach targets were met, and some limited interventions to address barriers to girls’ education were implemented, a member of DFID’s Lebanon team told us that DFID is challenging UNICEF to do more to tackle gender-specific barriers for both boys and girls. The programme stakeholders that we interviewed stressed that the large numbers of Syrian children out of school meant that the first focus had been on bringing as many children as possible to school, rather than
tackling gender-based marginalisation.
• DFID’s lack of control/influence over a focus on girls in government education programmes that it helps to fund. This affected three of the programmes we assessed.
Example: The design of DFID Tanzania’s £186 million Growth and Poverty Reduction Grant programme (2012-15) stated that policy dialogue in the education sector would focus, amongst other things, on improving learning outcomes for girls at primary level to improve transition to and achievement in secondary school. However, a breakdown in dialogue between donors and government impeded what influence DFID Tanzania might have had on the gender focus of government programming in education.
• Poor design of girl-focused components within programmes, which inhibited their targeting of girls or their effectiveness in addressing barriers to girls’ education. This affected five of the programmes we assessed.
Example: One component of DFID Kenya’s £32.9 million Kenya Essential Education Programme (2012-16) provided cash transfers for education, with the intention of supporting more girls than boys. However, the implementer adopted a programme model without a focus on girls. The programme targets were then changed to accommodate this, with targets of twice as many boys to benefit as girls.
• Lack of expertise of delivery partners in girls’ education, affecting their ability to deliver for girls. This was an issue in four of the programmes we assessed.
Example: The £13.8 million Girls’ Education Challenge iMlango project in Kenya (2013- 17) is implemented by a consortium led by Avanti, a company specialising in satellite communications. We found that a lack of development, gender and education expertise among its programme staff had weakened the project’s ability to deliver for girls – a finding confirmed in the most recent annual review. The Fund Manager has provided support to the project in this area.
• Challenges in implementing programmes in difficult operating environments. This was an issue for four of the programmes we assessed.
Example: The £10 million Kenya Adolescent Girls Initiative: Action Research Programme (2013-18) is a highly girl-focused country programme. Its performance in supporting marginalised girls’ education has been negatively affected by delays in implementation in the arid county of Wajir, which presents a particularly challenging operating environment, with great distances involved in reaching the participating villages.
Box 12: A shift in objectives away from girls in the Education Quality Improvement Programme in Tanzania
The Education Quality Improvement Programme in Tanzania is a programme with a budget of £60 million, scheduled to run between 2013 and 2018. We classified the programme as a girl-focused country programme since it has a number of activities focused on girls. Its expected outcome is “better quality education, especially for girls, and an approach ready for national scale-up by the Tanzania government”.
It is being implemented in seven regions in Tanzania, chosen on the basis of relative educational disadvantage. Severe gender disparities are apparent in primary school exam pass rates and transition to secondary school in the regions where it operates. For example, in Mara Region, 48% of boys passed
their exams at the end of primary school, compared to 28% of girls.
The programme was judged by DFID to be performing well, scoring an A in its latest annual review. The national and local government stakeholders we interviewed were also positive about the programme. However, with regard to girls’ education, we found that the programme had not met DFID’s original
Few outputs in the logframe relate specifically to girl-focused activities and, for those that do, the latest targets were not achieved. An infrastructure component that included building separate toilets for girls was mentioned in the business case but dropped before implementation. Furthermore, the business case
described an expected outcome of “27,500 more girls able to make the transition to secondary schools”, but this expectation is not represented in the logframe and has not been monitored.
On the girl-focused aspects of the programme, the latest annual review found the programme was lagging on supporting gender equality and social inclusion. This was confirmed during our visit to implementation sites. Both DFID and the implementing partner acknowledge that support for girls had been de-prioritised due to:
• A shift in the focus of the programme to early primary education (which is not where the barriers for girls’ education are most apparent in the Tanzanian context) related to changing DFID and Government of Tanzania priorities.
• A need to establish more basic aspects of the programme first, and to have been seen to have delivered tangible support, before tackling issues around girls’ education.
The implementing partner has begun to revise its gender strategy and plans to roll out further girl focused activities.
We found both strong and weak examples of targeting of marginalised girls
The effectiveness of DFID’s girl-focused programmes in targeting marginalised girls was mixed. For a third of the programmes we examined, the geographical focus was influenced by practical or political considerations, rather than by the sole consideration of relative marginalisation. Furthermore, some programmes were failing to reach the marginalised girls who were their intended beneficiaries due to a lack of effective targeting.
Box 13: Poor targeting practice in a Girls’ Education Challenge Kenya project
The £13.8 million iMlango programme in Kenya is implemented by Avanti (a private sector provider of satellite data communications) as the lead partner. The project involves, amongst other things, providing high-speed internet connections to schools in order to provide access to online educational content.
Programme documentation produced by Avanti did not outline a clear and coherent selection strategy for the locations and schools that it worked in. Our interviews with project stakeholders and review of project documentation revealed that practical issues such as school accessibility, access to electricity
and proximity to regional hubs predominated in decisions about which locations and schools to work in. One school that we visited did not appear to be relatively marginalised, given that it was performing well for both girls and boys before the start of the programme, as acknowledged by the head teacher and as
demonstrated by its national exam results.
More positively, some girl-focused programmes had a clear idea of which groups of marginalised girls they wished to support and how to reach them. For example, the Camfed Girls’ Education Challenge project, A New “Equilibrium” for Girls, which is operating in Tanzania and Zimbabwe with a combined budget of £29.3 million, targets individual girls with tailored packages of support. Criteria for selection of the most marginalised girls were drawn up with communities and refined in consultation with a broad range of stakeholders. The selection process was community-led. External audits of the selection process judged it to be fair, an assessment which was corroborated by local stakeholders whom we interviewed during a visit to one of the programme schools in Tanzania.
Box 14: Poor targeting of out-of-school girls in Pakistan
The Khyber Pakhtunkhwa Education Sector Programme (a £238 million girl-focused country programme) includes a component that provides out-of-school children with vouchers allowing them to enrol in private schools. The initiative supports the objective of providing all girls in the province with access to education. The 2014 DFID annual review, produced before the scale-up of this voucher scheme, recommended that DFID reviewed the pilot, including its targeting of out-of-school girls. A review was conducted but did not focus on this issue. Following scale-up, the vouchers were distributed to a relatively low proportion of out-of-school girls compared to out-of-school boys. According to DFID staff, only 43% of vouchers were distributed to girls, despite girls making up a much higher proportion of out-of-school children in the province. DFID informed us that the distribution of vouchers was to some extent dependent on the supply of private schools. There are more private schools for boys than for
The Girls’ Education Challenge is making an important contribution to girls’ education, but its outcomes are falling short of its ambitions
The Girls’ Education Challenge aims to improve the life chances of marginalised girls by enabling them to complete a full cycle of education and demonstrate learning (Figure 4 illustrates reported outputs). We found evidence, including from emerging mid-term results data, that outcomes are falling short of ambitions.
We assessed that DFID had met one of its four outcome milestones, on attracting “matching” funding contributions from other sources, but had missed the other three, on attendance, learning and sustainability.
Table 4: Performance of the Girls’ Education Challenge against DFID’s latest outcome milestone targets
|Outcome||Our assessment of performance against targets|
|Outcome 1: attendance |
Weighted average percentage improvement in girls’ attendance
The overall result for the improvement in girls’ attendance for Step Change Window projects was negative, and therefore did not meet the March 2016 milestone target of a 2.1% improvement. The overall improvement in girls' attendance in schools for Innovation Window projects (6.8%) exceeded the milestone target (3.5%). Given that the Step Change Window projects have received the majority of funding, we assess that overall this target has not been met. We note, however, that the Fund Manager has urged caution in using this indicator because of issues with the quality of attendance data.
|Outcome 2: learning|
Number of marginalised girls supported by the Girls’ Education Challenge with improved learning outcomes (defined as improvements in literacy and numeracy by Girls’ Education Challenge-supported girls that exceeds improvements by non-Girls’ Education Challenge-supported girls, to an agreed target).
DFID reports that (across all of the windows) over 500,000 girls have achieved significantly higher learning outcomes than their peers and met the set targets. Therefore, the milestone target of 739,300 girls was not met.
|Outcome 3: match funding|
Additional match funding secured alongside DFID Girls’ Education Challenge funds.
The milestone target of £37.9 million was exceeded, with £45.3 million secured.
|Outcome 4: sustainability|
Number of projects that have mechanisms in place to enable marginalised girls to complete a full cycle of education.
The milestone target for March 2016 was for all 37 Girls’ Education Challenge projects to have these mechanisms in place. DFID’s latest Annual Review of the Girls’ Education Challenge (September 2015) assessed that all but one project had these mechanisms in place. However, this assessment is not supported by the underlying documentation.
DFID realised that some Girls’ Education Challenge projects might not be sufficient to enable the beneficiaries to become functionally literate and numerate, even if they achieved their targets, due to low initial learning levels. It is not yet clear how successful the Fund’s response to this concern has been. Based on DFID’s experience during the first phase, DFID plans that the Girls’ Education Challenge extension will focus on accelerating girls’ learning outcomes, so that all beneficiaries achieve functional levels of literacy and numeracy.
Sustainability has been a weakness for the Girls’ Education Challenge. The business case stated that the Fund would prioritise projects able to support girls through a full cycle of education. It argued that, without this sustained support, not only would the individual beneficiary be let down, but the developmental benefits of educating them would not be fully realised.
In practice, sustainability was identified by both DFID and implementer staff as a weakness. DFID staff told us that many projects were unable to articulate either their approach to, or their performance on, sustainability and have been challenged to improve. The evaluation manager’s baseline research concluded that sustainability had not been given nearly enough attention in project design. Despite this, DFID and the Fund Manager have asserted that all 37 projects have plans in place to enable marginalised girls to complete a full cycle of education, although DFID acknowledges that this may not always be achieved.
Across the four Girls’ Education Challenge projects that we reviewed in depth, we found mixed performance on sustainability. The plans for BRAC Tanzania’s project to sustain activities beyond the life of the Fund and support girls through a full cycle of education were not convincing. The Avanti project in Kenya has also found sustainability a major challenge. By contrast, in its Tanzania project, the non governmental organisation Camfed has developed its sustainability plans more fully and is committed to supporting the education of girls up to the end of lower secondary, by raising support from other sources if Girls’ Education Challenge funding should cease.DFID informs us that the extension to the Girls’ Education Challenge now provides an opportunity for
DFID informs us that the extension to the Girls’ Education Challenge now provides an opportunity for projects to make proposals to continue supporting the current cohort of girls.
The Girls’ Education Challenge was designed to reach the “most marginalised” girls, but has not always done so
The Girls’ Education Challenge business case identifies the target group for the programme as “marginalised girls” and also “the most marginalised”.
Some of its projects have reached very marginalised groups. For example, the Kenya Equity in Education Project is working with girls in northern Kenya in and around the Dadaab and Kakuma refugee camps. Disabled girls were explicitly targeted in five of the 37 projects. However, reports produced by the evaluation manager provide evidence that those reached were not always the most marginalised. One report concluded that:
DFID has informed us that one third of the portfolio targets the most marginalised, including disabled girls, those at risk of early marriage and those living in remote rural locations. We also found from interviews with DFID staff and a report by the evaluation manager that the goal of reaching the most marginalised competed with other priorities, including a focus on low unit costs, ambitious beneficiary targets and DFID’s desire to test payment-by-results approaches. This suggests that some of the processes DFID uses to ensure accountability for results and value for money may work against reaching the most marginalised.
Key stakeholders from the Girls’ Education Challenge consortium agreed that the approach to targeting marginalised girls needed to improve, given the greater understanding that the programme has now gained on the causes of marginalisation in different contexts. In the business case for the second phase, DFID sets out different categories of marginalisation as “easier”, “harder” or “hardest” to reach and acknowledges that supporting the “hardest to reach” girls will be more expensive.
Conclusions on effectiveness
Overall, we have given DFID an amber-red score for the effectiveness of its education programming in addressing the marginalisation of girls. We found that, against the expectations that DFID set itself in both country operational plans and the business cases of its girl-focused programmes, its performance has been inconsistent and, at times, poor. We also found a tendency for objectives and targets on marginalisation to be downgraded or abandoned in successive iterations of country operational plans, in the progression from business cases to programme designs and over the course of programme implementation. The focus is better sustained in programmes that are solely or mainly devoted to girls’ education, and is most likely to be lost for girl-focused components within wider education programmes.
Importantly, we have found what appear to be recurrent or systemic reasons why DFID loses focus on girls’ education during programme implementation. These include competing priorities, conflicting preferences from country partners, poor programme design, lack of expertise on the part of implementing partners and the difficulties of working in challenging locations. If DFID is successfully to deliver on its “leave no one behind” commitment, it needs to overcome these challenges.
In this section, we have assessed DFID’s approach to ensuring value for money in its support for marginalised girls’ education. We examined its value for money assessment methods and how well value for money approaches are integrated into programme management. DFID’s usual framework for assessing value for money does not directly address equity
A 2010 National Audit Office report on DFID’s bilateral education portfolio provided the initial impetus for DFID to begin applying value for money analysis to its programming. DFID developed specific guidance for value for money in the education sector, based on the “3 Es” model of economy, efficiency and effectiveness, which is used across the UK government. The guidance centres on the long-term development returns of investing in education, although it does not discuss the specific benefits of educating girls for communities and households. The guidance stresses the need to consider value for money through the programme cycle, and to embed value for money analysis into programme management.
The guidance is designed to be applied in a flexible manner. It makes it clear that achieving value for money does not necessarily mean choosing the cheapest option. However, there is limited discussion of circumstances where DFID may need to choose more expensive options. The guidance considers cost drivers associated with delivery to remote or insecure locations, but makes no reference to the fact that the costs of engaging and supporting marginalised groups are likely to be higher, regardless of their location.
Many stakeholders consulted for this review, both within and outside DFID, expressed the view that this guidance needs to be adapted in light of DFID’s commitment to marginalised girls’ education and the “leave no one behind” pledge, to ensure that the marginalised are not excluded by value for money considerations, This would entail giving due weight to equity (sometimes cited as the fourth “E” in the value for money framework). At present, DFID does not routinely require this for education programmes.
In a workshop we held with practitioners, some viewed the lack of a clear framework for addressing equity objectives in education programmes as a flaw in DFID’s approach to value for money, with the potential to weaken programme design. A notable example is the use of a cost-per-girl metric as one of the selection criteria for Girls’ Education Challenge projects. Although some higher-cost projects were selected, stakeholders reported that this emphasised the need to reach more beneficiaries rather than the “most marginalised”. This is despite recognition by DFID that cost-per-pupil is a poor measure of value for money, given the need to improve the quality and effectiveness of education interventions. These stakeholders highlighted that this may have constrained the Fund’s ability to address the needs of girls that are harder to reach. Implementation of the “leave no one behind” promise will require DFID to develop new methods of bringing equity into its value for money framework for education programmes.
DFID has begun to generate the data needed for more robust value for money analysis
The National Audit Office report mentioned above recommended that DFID “take a tougher, clearer stance on the importance of cost and service performance information; particularly on indicators of education delivery and attainment”. From 2011, DFID began collecting a set of standard data across its priority countries to enable comparisons between countries and over time. It covers both input costs (such as average primary teacher salaries and textbook costs) and learning metrics (such as pupil attendance and completion). In principle, the data provides DFID with a method for driving improvement in value for money across the portfolio. The data was also shared with the Public Accounts Committee until 2013.
By 2014, however, the practice had fallen away. DFID informs us that this was because there was little variance in the data across years. These metrics are also considered less relevant, given the shift away from budget support towards more focused interventions. DFID now only collates and reports on a single measure: cost per pupil. The Girls’ Education Challenge has developed project-level efficiency metrics (cost per output) and cost-effectiveness metrics (cost per learning outcomes), allowing it to generate a set of comparable data from across its portfolio. DFID expects that these metrics will also be adopted by country education programmes.
Based on its mid-term and final results, the Girls’ Education Challenge will be able to assess the costs associated with achieving different levels of educational attainment for marginalised girls, which can be factored into future value for money assessments. A centrally managed fund is well suited to this kind of measurement. While there is at present no similar initiative for country education programmes, the Girls’ Education Challenge may in due course provide DFID with some of the information it needs to begin applying value for money analysis to its “leave no one behind” commitment.
DFID has also placed a strong emphasis on understanding what drives quality of learning across its portfolio. It is supporting three initiatives that aim to improve the quality and availability of education data: the UNESCO Institute of Statistics, the Programme for International Student Assessment and the Global Education Monitoring Report. These initiatives are developing new methods of measuring the quality and equity of learning outcomes – both important issues in tackling the marginalisation of girls.
Value for money analysis is embedded in project management, but focuses mainly on economy and efficiency
For bilateral country programmes, value for money assessment is the responsibility of programme managers. They were provided with training on the implementation of the value for money guidance. In our country case studies, we found that staff were confident in applying it. While the level of specialist knowledge available in country offices varies, back-up support is available elsewhere when needed (for example, from DFID education economists in the UK). Value for money assessments are often reviewed by specialists or carried out by external advisors.
In our review of girl-focused country programme documents (business cases and annual reviews), we found a consistent focus on economy (input costs) and efficiency (the relationship between outputs and inputs). The majority of programmes pay close attention to identifying cost drivers, and in some cases we saw evidence that these are tracked over time. In a few cases, this extends to the use of “trigger points”: that is, if costs rise above a predetermined level, this triggers further investigation to understand the causes.
For the majority of these programmes, we saw examples of DFID achieving cost savings, largely related to efficiencies in delivery. For example, one programme in Tanzania reduced the cost of teacher training by changing its delivery model and using community-based accommodation, rather than hotels. In a Kenya programme, costs were reduced by sharing procurement and transportation among subcontractors and by scrutinising prices for building materials. A Girls’ Education Challenge project partner achieved greater cost-efficiency by tailoring its support to the needs of individual girls, enabling it to reach more girls with its budget. There were also examples of cost-overruns highlighted in annual reviews.
Box 15: Monitoring cost drivers in a girl-focused programme
In the £10 million Kenya Adolescent Girls Initiative (2013-18), the member of the implementing consortium carrying out value for money research requires implementing partners to send detailed data on their key costs and the drivers of these costs. This data enables the programme to compare its cost structure with a similar programme in Zambia, and to compare cost structures and drivers across its partners. By investigating the causes of any outliers, the programme can ensure that both inefficiencies and good practices are identified and shared, so as to drive up value for money over time.
While important, unit cost analysis is only one part of the value for money equation for programmes that seek to reach marginalised girls. The stakeholders we spoke to, both in DFID and among its partners, recognised that not enough is known about the drivers of effectiveness and, in particular, which interventions lead to the best learning outcomes. Assessing effectiveness in education is a complex undertaking and DFID’s education research team has launched a number of studies to explore which mix of inputs generates the best learning outcomes. The Girls’ Education Challenge is also tracking learning outcomes more systematically, in order to inform assessment of cost effectiveness. The data required for a more holistic value for money assessment is therefore gradually being assembled.
The Girls’ Education Challenge has also been trialling the use of outcome-based payment by results, whereby suppliers are paid partly in proportion to their achievement of specified results, rather than for the cost of their inputs. Some stakeholders told us that this has supported a focus on learning and continuous improvement amongst delivery partners. However, not everybody is in favour of the approach. Many stakeholders highlighted the difficulty of reconciling DFID’s desire for innovation with its pressure on suppliers for guaranteed results. Some stakeholders told us that smaller civil society partners are unable to take on the financial risks associated with payment by results. This may have a negative impact on DFID’s access to specialist expertise and contextual knowledge.
The relative merits of state and non-state education are not yet clear
As part of our review, we explored the value for money considerations at stake in DFID’s choice of delivery channels. One of the key choices is between working with national education systems or private sector providers. While DFID does both, for the time being it lacks the evidence to make informed judgments as to what combination offers the best value for money in which contexts.
While DFID has moved away from budget support in its education portfolio, most of the country education programmes that we reviewed work closely with and through governments as the primary partner for delivering education. In some instances, this constrains DFID’s ability to focus on girls’ education, owing to the competing priorities of its national counterparts. By contrast, the Girls’ Education Challenge works through civil society and private sector delivery partners to support girls in public schools. This has enabled more targeted and innovative approaches, but at the expense of reach and sustainability. There is inadequate evidence from our review of programmes and the wider literature to draw conclusions about which delivery mechanisms work best for marginalised girls, but many of the stakeholders we consulted thought that a combination is likely to be required.
Several stakeholders argued that DFID should place a greater emphasis on supporting governments to develop more inclusive education systems and allocate more funds to education for girls, in order to achieve transformative and sustainable change. However, the trade-offs involved in working through governments were also noted. A key dilemma when working through government is whether to single out marginalised girls for additional support when there is not enough capacity in the education system as a whole to provide quality education for the majority. At present, DFID lacks a strong evidence base to guide its decisions on whether to support private schools
In three of the countries we assessed (Pakistan, Tanzania and Kenya), DFID supports low-cost private schools. These have expanded rapidly in recent years, providing a possible alternative means for reaching marginalised girls. However, for the time being, DFID lacks evidence on the effectiveness and value for money of working through private schools.
Many stakeholders consulted during this review questioned the value of DFID’s work through private schools for marginalised girls. Their concerns centred on whether private provision is equitable and whether it has a detrimental effect on public education.
A rigorous review commissioned by DFID of the research into private schools in developing countries concluded that the evidence on whether private schools are equally accessible to boys and girls is inconsistent. Given this uncertainty, it is important that DFID’s work with low-cost private schools generates evidence regarding access and learning outcomes, including for marginalised girls.
Conclusion on value for money
We have given DFID an amber-red score for its efforts to ensure value for money in its programming for marginalised girls. We found that DFID has a workable framework for value for money across its education portfolio, and that there has been some progress in integrating this framework into programme management, though we note it remains stronger at measuring economy and aspects of efficiency rather than effectiveness. While we saw that DFID has begun to invest in collecting the standardised data required for more sophisticated value for money assessment through its Girls’ Education Challenge programme, this practice is not yet consistent across its whole portfolio, where DFID currently only collates and reports on a single measure: cost per pupil. DFID also lacks the evidence to make informed judgments on the relative cost-effectiveness of different delivery channels in its programming for marginalised girls.
Finally, DFID has not yet updated its value for money approach for education programming to take account of the importance of working with hard-to-reach groups and the “leave no one behind” commitment. There is a need for DFID to consider how to incorporate equity considerations more comprehensively into the framework and to develop guidance for country offices and projects.
DFID’s “leave no one behind” commitment has important implications for many areas of the aid programme, and particularly for education. As DFID’s partner countries move closer to achieving universal basic education, the challenges of bringing marginalised groups into the education system, and ensuring that they succeed in gaining a useful education, come to the fore. DFID has committed itself to supporting these marginalised groups. This will require a concerted effort to improve on current performance.
This review has identified three main deficiencies in DFID’s approach. First, while DFID has committed itself to supporting the education of marginalised girls and has followed this up with significant investments, it lacks a coherent approach to achieving its goals. As a result, its efforts are inconsistent in their depth and focus, and poorly joined up.
Second, we identified recurrent factors across the portfolio that have resulted in a loss of focus on marginalised girls during implementation. As a result, DFID’s record on effectiveness is at best mixed, and at times poor. DFID will need to overcome these challenges if it is to deliver on its “leave no one behind” commitment.
Finally, DFID has yet to adapt its education value for money framework to reflect its commitments on tackling marginalisation. DFID should use emerging practice from the Girls’ Education Challenge to inform its approach to equity and value for money. Without this, there is a risk that the drive for economy and efficiency will create disincentives for DFID and its partners to focus on marginalised girls.
We welcome DFID’s commitment to supporting the education of marginalised girls and the contribution made by its programmes. The following recommendations are intended to help DFID strengthen its efforts to ensure that the commitment to leaving no one behind can be delivered.
DFID should develop country-specific strategies for marginalised girls’ education, based on detailed knowledge of the barriers in each context and learning from successful interventions. Its strategies should combine policy dialogue, system building and targeted interventions, and identify opportunities for cross-sectoral working.
• DFID does not give sufficient attention to identifying the challenges facing girls as a whole and the disadvantages experienced by specific groups of marginalised girls, as well as how different dimensions of disadvantage interact to create barriers to education.
• DFID lacks mechanisms for ensuring that research and evidence from learning about what works is consistently used to inform programming.
• DFID has not identified how to combine different kinds of intervention at the country level (policy dialogue, system building and targeted programming) into a coherent approach to tackling the marginalisation of girls.
• While various causes of girls’ marginalisation in education are addressed, there are inconsistencies across the portfolio and a number of important factors (for example health issues and genderbased violence) are not given sufficient attention.
In its delivery plans and monitoring of programmes with objectives around girls’ education and marginalisation, DFID should introduce measures to ensure that this focus is not lost during implementation.
• Many education programmes have performed poorly against their original ambitions for supporting girls’ education.
• There is a clear tendency for the focus on marginalised girls to be lost during programme implementation.
• The most common reasons impeding programme performance against expectations for girls’ education were:
• Lack of expertise of delivery partners in girls’ education.
• Girls’ education objectives being deprioritised because of competing priorities within the programmes.
• DFID’s lack of control/influence over a focus on girls in government education programmes that DFID helps to fund.
• Challenges in implementing programmes in difficult operating environments.
• Poor design of programme components.
DFID should specify how to approach value for money analysis when targeting marginalised groups and harder-to-reach groups, emphasising equity as well as cost-effectiveness.
• DFID has a sound general framework for assessing value for money in education programmes, but this does not explicitly address equity for marginalised groups and the commitment to leaving no one behind.
• The common emphasis on cost-efficiency within DFID’s value for money analysis may work against a focus on the most marginalised.
• There is limited evidence on the relative cost-effectiveness of different delivery channels to help achieve better value for money in DFID programming for marginalised girls.
• Learning from the Girls’ Education Challenge is not being fully utilised to inform the overall value for money framework for education.
Question 1: Relevance Green/Amber
DFID has made a substantial commitment to promoting the education of girls and to tackling the range of barriers that lead to girls being marginalised in education. It has invested in generating evidence and learning on how to tackle marginalisation. There are programmes in most DFID priority countries that prioritise girls’ education and include targeted interventions. The innovative, centrally managed, multi-country Girls’ Education Challenge is the largest ever donor programme aimed at the most marginalised girls. However, DFID needs to strengthen its strategy for addressing the causes of marginalisation, both globally and in particular countries. The different strands of its work in this area – including country programmes, the Girls’ Education Challenge, the Global Partnership for Education and relevant work in other sectors such as health – are not sufficiently joined up. We rated performance on relevance as green-amber due to the significant investment in girls’ education programming, while recognising that greater strategic coherence and coordination will be needed to achieve transformative impact.
Question 2: Effectiveness Amber/Red
DFID met its overall results target for supporting girls in primary and lower secondary school, although this target relates to all girls and not the most marginalised. Many DFID country plans and programmes have performed poorly against their original objectives for tackling the marginalisation of girls in education, and there is clear evidence that the focus on marginalised girls is often lost during programme implementation. We identified a number of factors that were causing this, including competing priorities, a lack of expertise among implementers and political constraints in partner countries. DFID has reported strong performance of the Girls’ Education Challenge, although it is yet to deliver on its overall ambition for improving girls’ learning and has struggled with sustainability. We rated performance on effectiveness as amber-red due to the frequent loss of focus on girls and a pattern of poor performance against expectations for girls in education programming.
Question 3: Value for money Amber/Red
DFID has a framework for assessing the value for money of its education programmes, which is widely used for project management. However, this framework has not been adapted to include equity objectives and is therefore difficult to apply to programming for hard-to-reach groups. DFID has made some progress towards collecting the data required for better value for money analysis in education programming, but remains stronger at measuring economy and aspects of efficiency, rather than effectiveness. We have rated DFID’s performance amber-red on value for money to reflect the fact that its framework has not been adjusted to reflect its commitment to marginalised girls and leaving no one behind.
Overall Score – Amber/Red
Overall, we have awarded DFID an amber-red score. DFID has made a strong commitment to tackling girls’ education and has a good range of programming directed towards marginalised girls. However, we find that its various activities do not add up to a coherent strategic approach to tackling marginalisation. We have identified a series of factors that are causing DFID programmes to lose their focus on marginalised girls through the implementation process. In light of its promise to marginalised girls’ education and the “leave no one behind” pledge, this is a serious shortcoming. Furthermore, DFID has not yet adjusted its approach to value for money to reflect this commitment.
Our methodology was structured to facilitate assessment of DFID’s strategic approach to supporting basic education for marginalised girls (covering expenditure, strategies, guidance and contribution to the evidence base) and to examine delivery (through assessment at the country and programme levels). There were three main components to the methodology: (1) a literature review, (2) a strategic review and (3)
1. Our literature review covered recent published material in academic journals, books and the publications of international organisations. It synthesised analysis of the barriers to education for marginalised girls across different country contexts and experience in attempting to overcome them. This provided a reference point for assessing the relevance and effectiveness of DFID’s programmes.
2. We carried out a strategic review of DFID’s overall approach to supporting basic education for marginalised girls. We examined the appropriateness and capacity of DFID’s strategic approach to supporting interventions that aim to benefit marginalised girls. This included the quality of guidance for those responsible for programming and use of research and evaluation outputs to support continual improvement. Specific aspects of this review included:
- Review of DFID strategy documents and guidance, including value for money guidance.
- Mapping of the level of focus on marginalised girls’ education in DFID country operational plans over the review period.
- High-level mapping of all bilateral education programmes in scope, including their level of focus on girls’ education, what girl-focused interventions they included, and whether they targeted particular groups.
- An overall assessment of the Girls’ Education Challenge (GEC), through document review and interviews with relevant DFID staff and staff from the consortium delivering the programme.
- Consultation with a range of key stakeholders in the UK, including interviews with DFID staff and external stakeholders from academia, civil society and other donor organisations, and a workshop with practitioners.
3. We carried out country assessments of DFID’s relevant activity in six countries, purposively selected to cover a range of country contexts (including situations of fragility and different degrees of gender inequality in education) and types of programming (ensuring that the countries selected had a good range of girl-focused country programmes, GEC projects and Global Partnership for Education activity). These assessments included detailed programme reviews of girl-focused centrally managed and country basic education programmes in these countries, and fewer in-depth reviews of other country basic education programmes (see Figure 5 below for a full list of the programmes reviewed). For four of these countries (Lebanon, Malawi, Pakistan [Khyber Pakhtunkhwa province] and South Sudan), these assessments were conducted remotely. For the other two countries (Kenya and Tanzania), we conducted country visits. As part of our assessment of the GEC, we carried out detailed programme reviews of four GEC projects in Kenya and Tanzania, selected to cover examples of projects from all three of the funding windows.
This approach reflects the importance of understanding the aspects of context that influence marginalisation and which may affect the development, implementation and performance of programmes. This approach provides a basis for detailed examination of the ways in which DFID’s education programming supports marginalised girls’ education, looking across the country education portfolio as a whole. This focus on the country level allows for an assessment of the coherence of DFID’s efforts on supporting marginalised girls’ education.
Country assessments were guided by the use of a standard Country Assessment Framework specifying the information that should be collected and providing standard criteria for evaluative judgements.
This framework covered:
- The relevance of the bilateral basic education portfolio for supporting marginalised girls’ education in the particular country context.
- The effectiveness of programming for supporting marginalised girls’ education, and constraints on and success factors for effectiveness.
- The strength of programme approaches to maximising value for money
Programme assessments involved examining programme documents and conducting structured interviews with key people involved with the programme, including (as appropriate):
- The responsible DFID adviser and/or programme manager.
- The responsible manager of the implementing agency.
Beyond interviews conducted as part of programme reviews, country assessments were informed by:
- Structured telephone interviews with DFID Education Advisors, as well as a limited number of key stakeholders in those countries, including government officials and staff from other donor organisations and civil society.
- Review of relevant documentation including education data and academic and donor reports.
- In the case of visited countries, face-to-face interviews with a broader range of stakeholders were conducted, including further DFID country office staff and staff from implementing agencies working at different levels. In these countries we also visited programme sites. Site visits informed our assessment of relevance, effectiveness and of aspects of value for money including sustainability. These site visits also allowed us to gain a better understanding of the context in which programmes are being implemented.
Collection of information from different sources on DFID’s support to girls’ education allowed triangulation of findings, including through seeking out critical voices.