Accessing, staying and succeeding in basic education – UK aid’s support to marginalised girls

ICAI Score

Unsatisfactory achievement in most areas of effectiveness and value for money, with mixed achievement on relevance

DFID has made a major commitment to supporting girls’ education. As developing countries move closer to universal enrolment in basic education, continuing progress involves addressing the many barriers to girls’ education – both those applying to all girls and those leading to the marginalisation of particular groups of girls. In most of DFID’s priority countries, it has set itself objectives on girls’ education and developed targeted interventions. Its centrally managed, multicountry Girls’ Education Challenge is the largest ever donor programme targeting marginalised girls. Notwithstanding this commitment, we found that DFID does not have a coherent strategy for addressing girls’ marginalisation in education, and that its various activities are not well joined up. We identified a clear pattern of DFID programmes losing their focus on marginalised girls through the implementation process, leading to disappointing results. Furthermore, DFID’s approach to achieving value for money has not been updated to reflect the challenges of programming for hard-to-reach groups, and risks working against the “leave no one behind” commitment. We have awarded DFID an amber-red score for its performance in addressing the marginalisation of girls in education, indicating that significant improvement is required for DFID to achieve its ambitions.

Executive Summary

Around the world, many girls face barriers that are preventing them from achieving a basic education. Globally there are 61.2 million primary and lower-secondary aged girls out of school.[1] Others drop out early or fail to reach functional literacy and numeracy. The barriers are complex and context-specific, but range from early marriage and pregnancy to cultural biases and the lack of basic facilities for girls in schools. Some relate to all girls, others to particular groups of girls, resulting in intersecting causes of disadvantage. As developing countries progress towards universal primary education, addressing the marginalisation of girls in education has emerged as a major challenge.

DFID has been committed for many years to prioritising the education of girls to reduce gender inequality. In 2010, DFID’s Education Strategy for 2010-15 reflected the Millennium Development Goals on education and set out priorities for educating girls and other marginalised groups. In 2013, DFID stated that one of its three core priorities for education programming was to “keep girls in school, helping the most marginalised girls stay in school and learning for longer”.[2] In 2015, reflecting its commitment to the Global Goals, it pledged to “leave no one behind” in its development assistance – that is, to prioritise those who may not be reached by “business as usual” development programming.

“People who are furthest behind, who have least opportunity and who are the most excluded will be prioritised.”

Leaving no one behind: Our promise, DFID, November 2015.

“Marginalisation in education is ‘a form of acute and persistent disadvantage rooted in underlying social inequalities’. “

Education for All Global Monitoring Report, UNESCO, 2010.

In this review, we assess DFID’s support for girls who are marginalised in education. Some DFID programmes support education as a whole, including for girls, while others contain specific objectives around tackling gender inequalities in education. Some programmes also target particular groups of girls who are marginalised in education because of the way gender intersects with other forms of disadvantage, such as remoteness, poverty, ethnicity or disability. Our review covers this spectrum of programming for girls’ education. The topic enables us to examine whether DFID has found effective ways of supporting hard-to-reach girls, despite the greater challenges and cost involved. It also enables us to explore the implications of the “leave no one behind” commitment for DFID’s approach to achieving value for money.

DFID has been a major donor in the education sector for many years, and its commitment to promoting gender equality in education goes back to at least 2000. Given this long-standing engagement, we have conducted a performance review to assess how successful DFID has been at delivering on its commitments over the 2011 to 2015 period. We looked in detail at DFID’s basic education portfolio in six countries (Kenya, Lebanon, Malawi, Pakistan’s Khyber Pakhtunkhwa province, South Sudan and Tanzania). We carried out visits to two of these countries: Kenya and Tanzania. In each country, we explored DFID’s contribution to girls’ education as a whole and its efforts to reach marginalised girls.

How relevant is DFID’s approach to supporting the education of marginalised girls?

DFID has adopted a prominent position internationally on girls’ education, through its policy commitments, its international advocacy and its investments in research. We found that its commitment to gender equality in education is reflected in its country operational plans and across its basic education portfolio. Many of its country operational plans include objectives for girls’ education. DFID country offices have developed a range of interventions to support girls’ education. Of the 63 basic education country programmes we reviewed, 32 included one or more activities specifically targeting girls and eight of the programmes were directed primarily at girls.

To increase its focus on marginalised girls, DFID launched the centrally managed Girls’ Education Challenge (£355 million in 2012-17). This flagship investment is designed to reach at least a million marginalised girls across 18 countries in Africa and Asia.[3] It is an innovative programme and it is making a significant contribution to learning on what works in supporting marginalised girls’ education. Taking into account both the Girls’ Education Challenge projects and DFID country programmes, we found that DFID’s level of focus on girls broadly corresponded with the level of need in terms of the number of girls out of school.

While DFID has made a clear commitment to girls’ education and has backed this with significant resources, we found that the department needs to strengthen the coherence of its approach to tackling the marginalisation of girls in national education systems. During the review period, DFID did not set out detailed objectives or guidance on addressing different types and causes of marginalisation.[4] At the country level, we found that DFID’s policy dialogue, system building and targeted interventions for particular groups of marginalised girls were not joined up into a coherent whole. DFID’s flagship initiative, the Girls’ Education Challenge, works through civil society or the private sector, rather than directly through national governments. While this fosters innovation, DFID is not yet clear how successful innovations will be scaled up or replicated. Evidence from the country portfolios we reviewed suggests a need for better coordination between Girls’ Education Challenge interventions and country programmes, and for better use of learning to inform DFID’s policy dialogue with governments. DFID informs us that it is putting in place measures to improve coordination.

At the programme level, we found that DFID’s girl-focused programmes included a good mix of intervention types, focusing most commonly on financial barriers, social norms and teaching practices. Many of the programmes address multiple barriers, which the literature suggests is likely to be the most effective approach. However, a few common causes of girls’ marginalisation, such as early pregnancy and violence against girls in schools, are not consistently addressed. There is also little coordination between DFID’s programming for girls’ education and its work in other sectors, such as health.

The stakeholders consulted reported that DFID is making an important contribution to generating new research on girls’ education. However, we judged that the use of evidence and analysis in DFID’s girl-focused programming was of mixed quality. Around half of the programmes in our sample included robust analysis of the needs of girls and specific factors that cause marginalisation in education. In some instances, baseline research was conducted but not used to guide intervention design. There is also limited evidence of takeup by country offices of research, guidance or other knowledge resources about what works in addressing disadvantage in education.

Overall, we have awarded DFID a green-amber score for relevance. This reflects its strong commitment to girls’ education, its good range of girl-focused programming and the groundbreaking work of the Girls’ Education Challenge. Many of the most important causes of girls’ marginalisation are being addressed within DFID’s portfolio. However, we found a need for better coordination across different delivery channels and sectors, and a clearer understanding of how to combine policy dialogue, system building and targeted interventions to achieve transformative impact. We share the concern of some external stakeholders that DFID’s major investment in the Girls’ Education Challenge, although undoubtedly positive, has served as a substitute for a clear strategic approach.

How effectively is DFID delivering results for marginalised girls?

Under its global results framework, DFID set itself the target of supporting 11 million children in basic education in the period 2011 to 2015.[5] This included 5.2 million girls.[6] DFID reported reaching 11.3 million children, including 5.3 million girls. There was no specific targeting of marginalised girls.

We measured DFID’s performance in supporting girls’ education against the objectives and targets set out in its country operational plans and programme design documents. In some cases, these targets were later revised downwards. We focused on the original objectives and targets in order to assess whether DFID’s commitment to girls’ education was being sustained over time.

Four of our case study countries included objectives on girls’ education in their operational plans. We found a pattern of underperformance against those objectives, including targets that were not met or abandoned. At the programme level, DFID’s business cases often included objectives on girls’ education that were not reflected in programme designs or logframe targets. While the programmes generally scored well in annual reviews for their overall performance, we found that achievement against the original objectives on girls’ education often lagged behind. This was particularly the case where girl-focused interventions were included in broader education programmes. Where programmes focused primarily on girls, their performance tended to be better. There is a marked tendency for DFID programmes to lose their focus on girls’ education over time. We found that a series of recurrent factors were contributing to this loss of focus, including:

• Girls’ education objectives being displaced by competing priorities such as improving overall access to basic education.
• A lack of influence by DFID on the focus of government education programmes.
• Poor design of girl-focused interventions, inhibiting their ability to address barriers to girls’ education or to target marginalised girls.
• A lack of expertise in girls’ education on the part of delivery partners.
• Difficulties with implementing in challenging environments

If it is to meet its commitment to “leaving no one behind”, DFID needs to do better at addressing the factors contributing to the marginalisation of girls across its education portfolio and at ensuring that this focus is not lost during delivery.

We identified examples of both weak and strong targeting of marginalised girls. For a third of the programmes we reviewed, the geographical focus was driven by practical or political considerations rather than solely by a focus on marginalisation. For example, for the Girls’ Education Challenge iMlango project in Kenya, an e-learning programme that aimed to target marginalised girls, accessibility and electricity supply were strong factors in the selection of project sites. We found several programmes that were failing to reach their intended beneficiaries as a result of poor targeting, such as the voucher component of the Khyber Pakhtunkhwa Education Sector Programme. We also found some strong examples, such as Camfed’s Girls’ Education Challenge project (“A New Equilibrium for Girls”) in Tanzania, which works with and through communities to identify marginalised girls.

In 2015, DFID reported that the Girls’ Education Challenge (“the Fund”) was performing strongly, based on the output data then available. Our assessment, based on emerging mid-term outcomes data, interviews with DFID and implementer staff and detailed review of four projects, found that the Fund was behind on three of its four outcome milestones, on attendance, learning and sustainability. The Fund itself has identified that some of its interventions may not be sufficient to bring participating girls up to the level of functional literacy and numeracy, given lower than expected initial learning levels. There are also concerns about sustainability across the portfolio, given the model of working through non-state implementers and the challenges of influencing national education systems.

DFID has responded to these concerns by extending the Fund, including enabling the current cohort of beneficiaries to be supported through a full cycle of education and to reach a new cohort of highly marginalised girls. The extension is also aiming to accelerate progress with learning outcomes.

Overall, we are concerned about a recurring loss of focus on girls within DFID’s broader education programmes. In these programmes, performance against original girl-focused objectives is weaker than the average performance across all objectives. Focus is better sustained within programmes that are solely or mainly devoted to girls’ education. We have identified what appear to be recurrent or systemic reasons to explain this loss of focus. If DFID is successfully to deliver on its “leave no one behind” commitment, it needs to overcome these barriers. We have awarded DFID an amber-red score for effectiveness.

Is DFID ensuring value for money in supporting marginalised girls’ education?

DFID has increased its use of value for money analysis across its education portfolio in response to a 2010 National Audit Office report.[7] It has guidance on value for money assessment in education, based on the standard UK government “‘3 Es” model of economy, efficiency and effectiveness.[8] At the programme level, we saw evidence that value for money analysis is embedded in project management, although it tends to focus on economy and efficiency rather than effectiveness.

DFID is yet to adapt its education value for money framework to reflect the “leave no one behind” commitment. Providing education to marginalised groups involves additional challenges and higher unit costs. Many stakeholders consulted for this review, both within and outside DFID, suggested that DFID’s framework needed to include explicit consideration of equity. If it does not do so, there is a risk that the value for money approach creates disincentives to focus on marginalised groups. A notable example of this was the use of a cost-per-girl metric in the selection criteria for Girls’ Education Challenge projects. Although some higher cost projects were selected, this approach attached priority to reaching more beneficiaries, rather than “the most marginalised”, who were the target group.

DFID is yet to develop guidance on how value for money considerations apply when targeting hard-to-reach beneficiaries through education programmes. On a positive note, the Girls’ Education Challenge has generated evidence that could be used to inform such guidance. The programme is building up a set of efficiency and cost-effectiveness metrics for girls’ education interventions. DFID informed us that it is using this evidence for internal learning.

We assessed whether DFID had clear value for money criteria for its choice of delivery channels, including working with state or non-state education providers. For the time being, it lacks a basis for making informed judgments as to what combination offers the best value for money in different contexts. Support for low-cost private schools through the Girls’ Education Challenge, though controversial among some stakeholders, will contribute to the evidence base.

We have awarded DFID an amber-red score for its approach to value for money in this area. DFID has made efforts to strengthen value for money analysis across the education portfolio and has begun to collect the data required for more sophisticated assessments, particularly through the detailed work on value for money metrics carried out through the Girls’ Education Challenge. However, its education value for money framework has not yet been developed to take account of the prioritisation of marginalised groups and the “leave no one behind” commitment. As a result, it is not well suited to ensuring value for money in tackling the marginalisation of girls in education, and risks creating disincentives to focus on the most marginalised.

Conclusions and recommendations

We recognise and support DFID’s substantial commitment to tackling the marginalisation of girls in education. However, it is clear from this review that a “business as usual” approach in DFID’s education work will not be sufficient to deliver on its commitment to “leaving no one behind”. We have identified three major deficiencies in its approach:

• The lack of a clear strategic approach that would help promote coherence and complementarity across the various strands of its work, including through different delivery channels, kinds of interventions and sectors.
• A pattern of the focus on girls being lost through the implementation phase of its programmes.
• The fact that its value for money framework for education programmes is yet to be updated to reflect its commitment to “leave no one behind”.

The following recommendations are intended to address these issues:

Recommendation 1

DFID should develop country-specific strategies for marginalised girls’ education, based on detailed knowledge of the barriers in each context and learning from successful interventions. Its strategies should combine policy dialogue, system building and targeted interventions, and should identify opportunities for cross-sectoral working.

Recommendation 2

In its delivery plans and monitoring of programmes with objectives around girls’ education and marginalisation, DFID should make it a priority to ensure that this focus is not lost during implementation.

Recommendation 3

DFID should specify how to approach value for money analysis when targeting marginalised and harder-to-reach groups, emphasising equity as well as cost-effectiveness.

Introduction

The UK government has made a strong commitment to promoting the education of girls in developing countries. Globally there are 61.2 million girls who should be in basic education (primary and lower secondary school) but are not.[9] Despite important progress over the past decade in reducing gender inequality in education, many girls still face a range of barriers that prevent them from accessing, staying and succeeding in basic education.

In this review, we assess DFID’s efforts to tackle the marginalisation of girls in education. The topic is timely, given DFID’s commitment to leaving no one behind.[10] This pledge, which applies across the aid programme, will require DFID to pay more attention to the targeting of its programmes in order to reach not just the poorest but also remote, marginalised and excluded groups. This has significant implications for how DFID designs and delivers its programmes and how it approaches value for money.

Education has been a major area of assistance for DFID for many years. Its commitment to promoting gender equality in education goes back to at least 2000.[11] This is therefore an appropriate subject for a performance review, which looks at how well DFID has delivered on its commitments.

Table 1: Our review questions

Methodology

There are three main methodological elements to our review.

i. Our literature review covered recent academic works and publications by international organisations. It synthesised analysis of the barriers to education for marginalised girls across different country contexts and experience in attempting to overcome them. This provided a reference point for assessing the relevance and effectiveness of DFID’s programmes.

ii. We carried out a strategy review of DFID’s approach to supporting basic education for marginalised girls. This contributed to our assessment of the relevance of DFID’s approach and portfolio as a whole and DFID’s approach to value for money in the context of a focus on marginalisation. It included:

• Review of DFID strategy documents and guidance
• Consultation with key stakeholders in the UK, including DFID and a range of external partners, and a workshop with practitioners
• Mapping of the level of focus on marginalised girls’ education in DFID country operational plans over the review period
• High-level mapping of all bilateral education programmes in scope.

iii. We carried out country assessments of DFID’s education portfolio in six countries (Kenya, Lebanon, Malawi, Pakistan’s Khyber Pakhtunkhwa province, South Sudan and Tanzania), selected to cover a range of contexts for girls’ education (see Figure 1) and types of programming (see Annex 2 for a list of the programmes we assessed in these countries). We assessed how DFID’s approach in each country was informed by local conditions, whether there was a relevant and coherent approach to girls’ education across the portfolio, whether DFID was achieving its objectives on girls’ education and the strength of programme approaches for maximising value for money. For two countries (Kenya and Tanzania), the review team conducted visits, allowing for more in-depth assessment. We also assessed the Girls’ Education Challenge as a portfolio. Country assessments were guided by a standard Country Assessment Framework specifying the information that should be collected and providing standard criteria for evaluative judgements.

The methodology is explained in more detail in Annex 2 and in full in our Approach Paper.[13] Both our methodology and this report were independently peer reviewed.

Background

The challenge of achieving gender equality in education

Recent decades have seen important progress towards gender equality in education around the world. From 2000 to 2015, over the period of the Millennium Development Goals, the number of countries with gender parity in both primary and secondary enrolment increased from 36 to 62 (of a total of 145 countries with available data).[14] However, most countries are still some distance away from gender equality in enrolment in education, and disparities are often pronounced at sub-national level. Across DFID’s partner countries, more girls than boys of primary-school age are out of school (17.9 million girls, compared to 13.6 million boys, in those countries for which DFID has data).[15] There are also gender disparities in completion and attainment in many countries, and these disparities can be particularly stark for marginalised girls.[16]

Investing in girls’ education offers significant economic, health and social benefits – for the girls themselves, their families and their communities. Educating girls reduces child mortality, early marriage, fertility and the risks of HIV infection and early pregnancy. It also improves maternal health outcomes and raises the potential earnings of women.[17]

The causes of marginalisation of girls in education are complex and vary according to context, but there are some common barriers that many girls face. These include the costs to households of educating girls relative to the perceived likely benefits, the lack of basic facilities for girls in schools, violence against girls in school and the prevalence of early pregnancy and marriage.[18] Girls in school can face barriers to successful learning, including both in-school factors (such as teachers’ attitudes towards girls) and out-of-school factors (such as the heavy burden of domestic chores on girls).[19]

Addressing marginalisation of girls in education is a complex challenge. The literature suggests that this requires the promotion of inclusive education systems and targeted interventions to address specific barriers faced by girls. Creating an effective package of interventions requires detailed knowledge of local contexts, as well as a willingness to challenge entrenched norms and practices.

DFID’s commitments on marginalised girls’ education

DFID has been committed to prioritising girls’ education since 2000. The Millennium Development Goals included the objective of achieving gender parity at all levels of education. DFID was also a signatory to the Education for All Goals, which included achieving gender equality by 2015, with a particular focus on ensuring that girls have access to basic education of good quality.[24] The theme of marginalised girls’ education now features in two of the Global Goals (“Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all” and “Achieve gender equality and empower all women and girls”).

These commitments are echoed in DFID’s policies and strategies from 2005 onwards. Successive strategies and policy statements have had a strong emphasis on girls accessing and remaining in education (see Figure 2). The former International Development Secretary, Justine Greening, was a strong advocate of the rights of girls.

As access to basic education expanded in DFID’s partner countries, it became apparent that achieving gender equality in education would require targeted measures to address groups who were marginalised or hard to reach. In November 2015, in keeping with the Global Goals, the UK made a commitment to “leaving no one behind” in its development assistance. It pledged that “people who are furthest behind, who have least opportunity and who are the most excluded will be prioritised”.[25]

DFID’s programming on marginalised girls’ education

Education is one of DFID’s major areas of development assistance, receiving around 12% of UK bilateral aid.[26] Over our review period, DFID spent £2.6 billion on country programmes relating to basic education (the programmes had a combined lifetime budget of £4.3 billion). [27]DFID is the second largest bilateral donor in basic education.[28]

There is no way of separately identifying DFID’s expenditure on girls’ education or on tackling marginalisation because such interventions are usually part of broader programmes. However, we classified DFID’s country basic education programmes into those that include at least one activity targeting barriers to girls’ education (we call these “girl-focused programmes”) and those where the majority of interventions target girls (“highly girl-focused programmes”). See Box 6 for the definitions we used and Figure 3 for the number of programmes and associated expenditure in each category over the review period.

We note that DFID’s investments in building education systems through non-girl-focused education programmes also benefit marginalised girls. Furthermore, interventions like policy dialogue and advocacy, with no specific budgets attached, may also be important in addressing the needs of marginalised girls.

In addition to its country programmes, DFID has made a major financial commitment to supporting marginalised girls through its centrally managed Girls’ Education Challenge. This programme was established to boost DFID’s results on girls’ education and hard-to-reach and marginalised girls. It spent £249 million during the review period, with a budget of £355 million for its first phase (2012-17). A further phase has been approved. Ten other centrally managed programmes with a focus on girls’ education spent £23.7 million during the review period.[29]

The Global Partnership for Education is a multilateral partnership that supports the preparation and implementation of national education plans in developing countries. Between 2011 and 2014, this partnership spent between £275 and £314 million per year across 50 countries.[30] DFID was the largest donor, at £296 million (2011-14).

How relevant is DFID’s approach to supporting marginalised girls’ education?

In this section, we assess the relevance of DFID’s approach to tackling the marginalisation of girls in education. We look at the level of commitment to girls’ education in its country operational plans and its basic education programmes, and the extent to which these programmes respond to the specific causes of marginalisation of girls in education found in each of our case study countries. We look at the approach of DFID’s flagship centrally managed fund on girls’ education, the Girls’ Education Challenge, and whether DFID’s country portfolios work as a coherent whole.

DFID lacks an explicit approach to addressing marginalisation of girls in education

Many of the non governmental organisations, academics and donor stakeholders we consulted for this review regard DFID as a global champion for girls’ education. Stakeholders from all three groups told us that its high-profile policy commitments, significant financial commitment to the Girls’ Education Challenge and international advocacy at events such as the 2016 Girls’ Education Forum have given DFID a substantial international profile in the area.

While its education targets are disaggregated by gender, DFID has not articulated how its objectives to support marginalised girls’ education and the “leave no one behind” commitment apply to its country education portfolios. There is no overall theory of change or detailed analysis about how to tackle the different causes of girls’ marginalisation in education through policy dialogue and programming. Its policy documents do not distinguish between the marginalisation of girls as a whole and groups of girls facing particular forms of disadvantage. The Girls’ Education Challenge identified categories of girls who were more likely to experience disadvantage in education, but DFID has only recently developed its thinking on overlapping causes of marginalisation. Current Girls’ Education Challenge projects are left to adopt their own definitions of marginalisation, although DFID informed us that evidence from the mid-term evaluations is being used to inform targeting in the next phase of the Fund.

Most of DFID’s country operational plans include a focus on girls’ education

Many of DFID’s country operational plans include commitments and objectives relating to girls’ education. Over the review period, 22 of DFID’s 28 country offices were active in basic education. Of these, 18 included a focus on girls’ education in their 2011-12 country operational plans and 11 had a strong focus.

The emphasis on girls’ education in country operational plans was broadly in line with need, measured in terms of the number of girls out of school. Two exceptions were Pakistan (at least up until 2014), where DFID informed us that national counterparts expected its education programmes to address the needs of both girls and boys, and Yemen.

As the operational plans were updated over the review period, the focus on girls’ education increased up until 2014, before receding again between 2014 and 2016 (although all the plans retain at least some reference to girls’ education).[32] This trend coincides with an increased focus on a broader girls’ empowerment agenda (eg reducing gender-based violence and tackling discriminatory norms), as well as on marginalised groups generally.

Girls’ education continues to feature to a greater or lesser extent in country operational plans. However, we found uneven attention being paid to identifying specific groups of marginalised girls to be prioritised through education programming. The few positive examples include direct targeting of the poorest girls in ethnic minority states in Myanmar, and geographic targeting of the poorest states and districts in India or hard-to-reach slums and arid areas in Kenya.

A commitment to educating girls runs through the portfolio, but with varying depth and focus

Of the 63 country education programmes that we reviewed,[33] a large majority included a commitment to promoting gender equality within their design documents.

We identified 32 country programmes with activities specifically targeting girls (girl-focused) and, within that group, eight programmes with the majority or all of their activities targeting girls (highly girl-focused). The 32 girl-focused programmes had a combined lifetime budget of £2.2 billion, with the eight highly girl-focused programmes representing £327 million of the total.

The remaining 31 programmes had no specific activities for girls. 20 of them claimed that they would nonetheless promote gender equality in education – for example by stating that interventions not specifically targeting girls would benefit girls more than boys, or that the partner country or implementing partner had a strong gender focus. In half of these cases, we concluded that these claims were not substantive, based on the programme descriptions in the business cases.

Fewer than half of the 32 girl-focused country education programmes in our sample specified particular groups of marginalised girls amongst their target groups. The majority of these focused on girls in particular geographical areas, while six included a focus on disabled girls.

Across our six country case studies, interviews and documentary evidence point to a number of factors that have worked against a stronger and more consistent focus on marginalised girls’ education. These include:

• Competing educational priorities for DFID country offices, evident in five of the case study countries assessed (Malawi was the exception).
• A perceived risk of political and community resistance to an overt focus on girls, reported in four of the country case studies – Kenya, Tanzania, South Sudan and Pakistan.
• A lack of fine-grained data to analyse the causes of (and potential solutions to) girls’ marginalisation, evident in DFID’s response to the education crisis among Syrian refugees in Lebanon and also acknowledged by DFID Kenya. DFID Kenya’s education portfolio includes an emphasis on action research and impact evaluations, to help identify what works.

The Girls’ Education Challenge was established to increase DFID’s focus on marginalised girls’ education

Early in the review period, DFID assessed that its existing bilateral and multilateral support was not reaching the most marginalised girls. In 2012, it launched the centrally managed Girls’ Education Challenge (“the Fund”) in order to boost its investment in marginalised girls’ education. With a lifetime budget of £355 million in its first phase (2012-17), the Fund has supported 37 projects across 18 countries in Africa and Asia, working solely through non-state implementing organisations.

The Girls’ Education Challenge is a significant investment, supporting a million marginalised girls in its first phase. It is the largest ever aid programme dedicated solely to girls’ education. Its high profile has helped to signal to partner countries and other development partners the importance of this issue. Stakeholders (internal and external to DFID) saw the Fund as helping to position DFID as a global leader in the area. It is also seen as a driver of innovation, able to test out new approaches to combatting the marginalisation of girls.

As a fund dedicated to supporting marginalised girls in education through partnership with non governmental organisations and private sector organisations, the Girls’ Education Challenge has greater flexibility to bypass some partner countries’ reluctance to dedicate resources specifically to girls’ education. By contrast, some of DFID’s country offices told us that bilateral country programmes delivered in partnership with national governments are under pressure to address the needs of the population as a whole.

DFID has not prioritised marginalised girls in policy dialogue and system-building

DFID’s decision to promote girls’ education through a separate, centrally managed fund may have made it more difficult to develop a coherent approach to marginalised girls’ education in each of its partner countries.

Girls’ Education Challenge projects are relatively short-term (in line with the Fund’s initial four-year funding cycle). While they support girls within state schools, they are not delivered through state actors. We found that impact is generally limited to the girls who are reached directly, rather than via changes to education systems, and that little attention is given to the sustainability of the interventions. The Fund is now being extended to a second phase with the aim, amongst other things, of enabling girls who are currently supported to complete a full cycle of primary or secondary education, and of sustaining and leveraging future investment in girls’ education.[36]

To achieve wider and longer-lasting impact, DFID needs to go beyond direct funding of services for marginalised girls and do more to promote their interests in national education policies and systems. This is the territory of many of DFID’s country programmes and its work with the Global Partnership for Education, which supports national education strategies. DFID informed us that, during the latter part of the review period, they have taken steps to improve coordination between the Girls’ Education Challenge and other programming. However, we saw limited evidence of learning or innovation from Girls’ Education Challenge projects being used to inform DFID’s policy dialogue in our case study countries or system-building programmes. Stakeholders (including DFID and implementer staff) confirmed that coordination between the Girls’ Education Challenge and DFID country offices and programmes has been mixed.

In our case study countries, we found that most DFID country education programmes were not pursuing system-wide change in relation to girls’ marginalisation. One notable exception was the Keeping Girls in School programme in Malawi, which planned to strengthen the education ministry’s strategy for girls’ education and effect change at both national and district government levels.

In two of our case study countries (Kenya and Tanzania), the majority of external stakeholders we interviewed reported that DFID did not promote girls’ education within the Global Partnership for Education at the country level. The 2014 annual review of the Global Partnership for Education also commented on the need to improve communication between the central partnership team and DFID country offices.

At the global level, however, DFID has supported initiatives to improve the Global Partnership for Education’s work related to marginalised girls, including through the secondment of an education advisor. DFID has also informed us that, over the past 12-18 months, it has engaged with the Global Partnership for Education to develop a results framework, which includes gender-disaggregated targets. It is too early to assess whether these initiatives will lead to a stronger focus on marginalised girls in the country education sector plans.

DFID’s girl-focused programming includes a good range of complementary interventions, with some gaps

Given the complex causes of girls’ marginalisation in education, the literature concludes that multipronged interventions and cross-sectoral working are required to enhance girls’ participation in school and the quality of education they receive.[37]

To assess the relevance of DFID’s programming choices, we drew on our literature review to develop a typology of interventions related to common causes of marginalisation. Positively, across the portfolio of girl-focused education programmes, we found that programmes with a combination of intervention types were the norm.

Some of the most common types of intervention in DFID’s girl-focused programmes aimed to tackle barriers to girls’ education related to:

• Gender inequalities in teaching and learning practices in schools – for example increasing the number of female teachers in order to provide role models for girls and providing gender-sensitive teacher training.

• Gender inequalities in community practices and norms – for example mobilising communities in support of girls’ education, such as through tackling negative perceptions about the value of educating girls.

• Affordability – for example stipends, scholarships and conditional cash transfers to reduce the economic barriers to girls’ education (including opportunity costs to households, for instance from lost labour).

DFID’s highly girl-focused programmes, including Girls’ Education Challenge projects, generally take a more holistic approach to tackling the causes of girls’ marginalisation. They include more intervention types and demonstrate a greater understanding of the links between education and girls’ empowerment.

Across the country portfolios we reviewed, we found that school-related gender-based violence and health barriers were relatively neglected. Expert stakeholders identified a need for increased focus on early child marriage and pregnancy. We found some positive examples of projects that are helping girls manage menstruation and avoid pregnancy. Table 2 contains examples from Kenya and Tanzania. DFID has identified these areas as a focus for the next phase of the Girls’ Education Challenge.

Table 2: Programmes tackling gender-based violence and reproductive health issues

Gaps in programming relating to sensitive issues like reproductive health and violence against girls reflect the complexity of changing deeply rooted social inequalities and cultural norms. Tackling such barriers requires well-connected, multi-level interventions, working with both government and society.[38] DFID and external expert stakeholders recognise the need for better coordination between education and other programmes, such as on gender-based violence. However, they remained unsure about how to achieve greater coherence in practice. DFID Kenya noted the difficulty of coordination across multiple programmes with different objectives and time periods. Overall, we found limited coordination between girl-focused country education programmes and interventions in other sectors.

Use of evidence is of mixed quality

Within the six case study countries, we found that around half of the girl-focused education programmes were based on a robust analysis of the barriers to education facing girls. As would be expected, highly girl-focused programmes (including Girls’ Education Challenge projects) showed better use of evidence and analysis of barriers facing girls in their designs.

In some cases, further baseline research was carried out during inception, to obtain a better understanding of the barriers facing girls. However, while such research helped to establish baselines for logframe targets, it was not necessarily used to refine the design of interventions (in the case of some Girls’ Education Challenge projects, due to time pressures on delivery).The majority of girl-focused education programmes within case study countries were found to be

The majority of girl-focused education programmes within case study countries were found to be based on weak evidence of what works in addressing barriers to education, although highly girl focused programmes (including Girls’ Education Challenge projects) performed better. For example, the business case for DFID Pakistan’s Khyber Pakhtunkhwa Education Sector Programme presents very limited evidence on what works in girls’ education.[39]

Conclusion on relevance

DFID is widely regarded by external stakeholders as a global leader on girls’ education, as a result of its policy commitments and international advocacy. It has also made a strong commitment to investing in research, to understand the barriers to girls’ education and what interventions work.

Many of its country operational plans in DFID priority countries include objectives on girls’ education, while around half of its country education programmes include at least some activities that specifically target marginalised girls. The Girls’ Education Challenge is a major investment that has helped to increase DFID’s focus on marginalised girls and raise the profile of the issue internationally. It is encouraging that, when the Girls’ Education Challenge is taken into account, DFID’s girl-focused programming is broadly concentrated in countries where the need is greatest. It also contains a good range of interventions addressing different causes of girls’ marginalisation.

While DFID’s commitment to and investment in marginalised girls’ education is substantial, our conclusion is that DFID needs a more strategic approach. It has not yet identified how to combine different initiatives – such as policy dialogue, system building and interventions targeting particular groups of marginalised girls – into a coherent strategy for addressing marginalisation, either at the central level or in specific countries. While Girls’ Education Challenge projects are innovative in design and targeted at the most marginalised, they cannot achieve transformative impact unless they also influence national education policies and systems. DFID has recognised this need for greater coherence and coordination. At the central level, it has also supported Global Partnership for Education actions to increase its focus on girls’ education.

Overall, we have awarded DFID a green-amber for relevance. This recognises its significant commitment of resource and effort in girls’ education, the groundbreaking work of the Girls’ Education Challenge. Many of DFID’s individual interventions are well designed. However, the lack of a clear strategy bringing together the different strands of DFID’s work is an obvious gap, resulting in inconsistencies in approach and insufficient complementarity across different types of programming.

How effective are DFID’s education programmes in delivering results for marginalised girls?

In this section, we examine how DFID’s education programmes have performed against their objectives on supporting girls’ education. We look at DFID’s delivery against its global results target, at whether country offices have delivered the results anticipated in their country operational plans, and whether girl-focused programmes have achieved their objectives.

We have assessed DFID’s country operational plans and programmes against their original stated objectives. We recognise that objectives and targets are often revised. However, we are interested in the question of whether DFID sustains the focus of its programming on girls’ education during implementation, and the reasons for any changes.

DFID has met its global target for basic education

Under its overall results framework for 2011-15,[40] DFID set itself the target of supporting 11 million children in primary and lower secondary education, including 5.2 million girls.[41] This target was achieved. In 2015, DFID reported that it had supported 11.3 million children, including 5.3 million girls.[42]

This global results framework only includes results from a subset of DFID’s country education programmes. It also excludes the Girls’ Education Challenge, which has a target of improving attendance and learning for up to a million marginalised girls. Therefore DFID has demonstrated that the global results target was achieved, but has not attempted to calculate the full extent of its reach in education (see Box 10 for the challenges it faced in doing so). Importantly, the reach indicator only measures enrolment, rather than attendance or whether children are actually learning.[43] While these results give an indication of the scale of DFID’s education programming, including for girls, they do not help us identify whether marginalised girls are being supported.

Performance against the original targets in DFID’s country operational plans was poor or mixed

DFID country operational plans set out the results it aimed to achieve in each country. These include headline results targets and a discussion of related objectives. All country operational plans included education targets, although often with different measures from the global target in DFID’s Results Framework. Four of our six case study countries (Kenya, Malawi, South Sudan and Tanzania) included objectives and targets for girls’ education in their earliest country operational plans in the review period. In some instances, these objectives were revised or removed when the operational plans were renewed. When we assessed the four countries against their original objectives, we found that their performance was poor or mixed (see Table 3).

Table 3: Performance against expectations for girls in case study country operational plans

The performance of interventions on girls’ education was generally poor, particularly in broader education programmes

We assessed the performance of individual education programmes against their expected contribution to girls’ education. These expectations include formal targets recorded in logframes and other explicit commitments to activities or results. The majority of girl-focused programmes in our sample failed to meet the original expectations for their girl-specific components, as set out in their design documents (11 out of 19). Our assessments were based on DFID’s own reporting, interviews with the responsible DFID staff and implementing partners and, for six programmes in the two countries we visited, site visits.

DFID’s business cases often contain ambitions around supporting girls’ education that are not carried through into logframes and detailed programme designs. For example, DFID Tanzania’s Human Development Innovation Fund was designed to support seven results areas, including improved educational access for girls and vulnerable groups. The business case stated that it would “focus on and be weighted towards supporting innovations which will particularly benefit girls and women”. However, these commitments were not represented in logframe outcome or output targets. This works against effective monitoring of progress on girls’ education, including when programme plans are revised to reflect changing contexts.

Importantly, our rating of how DFID programmes performed on girls’ education does not necessarily reflect their performance in other areas. The majority of the programmes in our sample were rated by DFID as performing strongly. It is telling that their commitments and results on girls’ education have lagged behind their overall performance.

We also found that highly girl-focused programmes (including Girls’ Education Challenge projects) performed considerably better than other girl-focused programmes. This supports the conclusion that, where activities in support of girls’ education are part of broader education programmes, the focus on girls often gets lost. If DFID is to meet its commitment to “leave no one behind”, it needs to do better at mainstreaming and maintaining a focus on marginalised girls across its education portfolio.

Recurrent factors across the portfolio are causing DFID to fall short of its objectives on supporting marginalised girls’ education

Across the 19 girl-focused programmes in our sample (including four Girls’ Education Challenge projects), we identified various factors impeding performance against expectations for girls’ education. These factors were commonly associated with a loss of focus on girls. Together, these suggest that DFID faces systemic problems in carrying through on its commitments on girls’ education, which is relevant to its “leave no one behind” pledge.

• Girls’ education objectives being displaced by competing priorities within the programmes, such as the need to boost access as a whole. This was an issue impeding programme performance for four of the programmes we assessed.

Example: DFID Lebanon’s £5 million Low Cost Innovative Non Formal Education and Skills for Lebanese and Syrian Children and Youth programme (2015-16), implemented by UNICEF, was intended to reach equal numbers of girls and boys, and included tailored interventions to address barriers to education specific to each gender. While its gender-disaggregated reach targets were met, and some limited interventions to address barriers to girls’ education were implemented, a member of DFID’s Lebanon team told us that DFID is challenging UNICEF to do more to tackle gender-specific barriers for both boys and girls. The programme stakeholders that we interviewed stressed that the large numbers of Syrian children out of school meant that the first focus had been on bringing as many children as possible to school, rather than
tackling gender-based marginalisation.

• DFID’s lack of control/influence over a focus on girls in government education programmes that it helps to fund. This affected three of the programmes we assessed.

Example: The design of DFID Tanzania’s £186 million Growth and Poverty Reduction Grant programme (2012-15) stated that policy dialogue in the education sector would focus, amongst other things, on improving learning outcomes for girls at primary level to improve transition to and achievement in secondary school. However, a breakdown in dialogue between donors and government impeded what influence DFID Tanzania might have had on the gender focus of government programming in education.

• Poor design of girl-focused components within programmes, which inhibited their targeting of girls or their effectiveness in addressing barriers to girls’ education. This affected five of the programmes we assessed.

Example: One component of DFID Kenya’s £32.9 million Kenya Essential Education Programme (2012-16) provided cash transfers for education, with the intention of supporting more girls than boys. However, the implementer adopted a programme model without a focus on girls. The programme targets were then changed to accommodate this, with targets of twice as many boys to benefit as girls.

• Lack of expertise of delivery partners in girls’ education, affecting their ability to deliver for girls. This was an issue in four of the programmes we assessed.

Example: The £13.8 million Girls’ Education Challenge iMlango project in Kenya (2013- 17) is implemented by a consortium led by Avanti, a company specialising in satellite communications. We found that a lack of development, gender and education expertise among its programme staff had weakened the project’s ability to deliver for girls – a finding confirmed in the most recent annual review. The Fund Manager has provided support to the project in this area.

• Challenges in implementing programmes in difficult operating environments. This was an issue for four of the programmes we assessed.

Example: The £10 million Kenya Adolescent Girls Initiative: Action Research Programme (2013-18) is a highly girl-focused country programme. Its performance in supporting marginalised girls’ education has been negatively affected by delays in implementation in the arid county of Wajir, which presents a particularly challenging operating environment, with great distances involved in reaching the participating villages.

We found both strong and weak examples of targeting of marginalised girls

The effectiveness of DFID’s girl-focused programmes in targeting marginalised girls was mixed. For a third of the programmes we examined, the geographical focus was influenced by practical or political considerations, rather than by the sole consideration of relative marginalisation. Furthermore, some programmes were failing to reach the marginalised girls who were their intended beneficiaries due to a lack of effective targeting.

More positively, some girl-focused programmes had a clear idea of which groups of marginalised girls they wished to support and how to reach them. For example, the Camfed Girls’ Education Challenge project, A New “Equilibrium” for Girls, which is operating in Tanzania and Zimbabwe with a combined budget of £29.3 million, targets individual girls with tailored packages of support. Criteria for selection of the most marginalised girls were drawn up with communities and refined in consultation with a broad range of stakeholders. The selection process was community-led. External audits of the selection process judged it to be fair, an assessment which was corroborated by local stakeholders whom we interviewed during a visit to one of the programme schools in Tanzania.

The Girls’ Education Challenge is making an important contribution to girls’ education, but its outcomes are falling short of its ambitions

The Girls’ Education Challenge aims to improve the life chances of marginalised girls by enabling them to complete a full cycle of education and demonstrate learning (Figure 4 illustrates reported outputs). We found evidence, including from emerging mid-term results data, that outcomes are falling short of ambitions.

We assessed that DFID had met one of its four outcome milestones, on attracting “matching” funding contributions from other sources, but had missed the other three, on attendance, learning and sustainability.

Table 4: Performance of the Girls’ Education Challenge against DFID’s latest outcome milestone targets

[46][47]

DFID realised that some Girls’ Education Challenge projects might not be sufficient to enable the beneficiaries to become functionally literate and numerate, even if they achieved their targets, due to low initial learning levels. It is not yet clear how successful the Fund’s response to this concern has been. Based on DFID’s experience during the first phase, DFID plans that the Girls’ Education Challenge extension will focus on accelerating girls’ learning outcomes, so that all beneficiaries achieve functional levels of literacy and numeracy.

Sustainability has been a weakness for the Girls’ Education Challenge. The business case stated that the Fund would prioritise projects able to support girls through a full cycle of education. It argued that, without this sustained support, not only would the individual beneficiary be let down, but the developmental benefits of educating them would not be fully realised.

In practice, sustainability was identified by both DFID and implementer staff as a weakness. DFID staff told us that many projects were unable to articulate either their approach to, or their performance on, sustainability and have been challenged to improve. The evaluation manager’s baseline research concluded that sustainability had not been given nearly enough attention in project design. Despite this, DFID and the Fund Manager have asserted that all 37 projects have plans in place to enable marginalised girls to complete a full cycle of education, although DFID acknowledges that this may not always be achieved.

Across the four Girls’ Education Challenge projects that we reviewed in depth, we found mixed performance on sustainability. The plans for BRAC Tanzania’s project to sustain activities beyond the life of the Fund and support girls through a full cycle of education were not convincing. The Avanti project in Kenya has also found sustainability a major challenge. By contrast, in its Tanzania project, the non governmental organisation Camfed has developed its sustainability plans more fully and is committed to supporting the education of girls up to the end of lower secondary, by raising support from other sources if Girls’ Education Challenge funding should cease.DFID informs us that the extension to the Girls’ Education Challenge now provides an opportunity for

DFID informs us that the extension to the Girls’ Education Challenge now provides an opportunity for projects to make proposals to continue supporting the current cohort of girls.

The Girls’ Education Challenge was designed to reach the “most marginalised” girls, but has not always done so

The Girls’ Education Challenge business case identifies the target group for the programme as “marginalised girls” and also “the most marginalised”.

Some of its projects have reached very marginalised groups. For example, the Kenya Equity in Education Project is working with girls in northern Kenya in and around the Dadaab and Kakuma refugee camps. Disabled girls were explicitly targeted in five of the 37 projects. However, reports produced by the evaluation manager provide evidence that those reached were not always the most marginalised. One report concluded that:[48]

“If the intention is to systematically target the most marginalised girls through project targeting strategies and designs then this should be a more explicit programme focus in the future.”

DFID has informed us that one third of the portfolio targets the most marginalised, including disabled girls, those at risk of early marriage and those living in remote rural locations. We also found from interviews with DFID staff and a report by the evaluation manager[49] that the goal of reaching the most marginalised competed with other priorities, including a focus on low unit costs, ambitious beneficiary targets and DFID’s desire to test payment-by-results approaches. This suggests that some of the processes DFID uses to ensure accountability for results and value for money may work against reaching the most marginalised.

Key stakeholders from the Girls’ Education Challenge consortium agreed that the approach to targeting marginalised girls needed to improve, given the greater understanding that the programme has now gained on the causes of marginalisation in different contexts. In the business case for the second phase, DFID sets out different categories of marginalisation as “easier”, “harder” or “hardest” to reach and acknowledges that supporting the “hardest to reach” girls will be more expensive.

Conclusions on effectiveness

Overall, we have given DFID an amber-red score for the effectiveness of its education programming in addressing the marginalisation of girls. We found that, against the expectations that DFID set itself in both country operational plans and the business cases of its girl-focused programmes, its performance has been inconsistent and, at times, poor. We also found a tendency for objectives and targets on marginalisation to be downgraded or abandoned in successive iterations of country operational plans, in the progression from business cases to programme designs and over the course of programme implementation. The focus is better sustained in programmes that are solely or mainly devoted to girls’ education, and is most likely to be lost for girl-focused components within wider education programmes.

Importantly, we have found what appear to be recurrent or systemic reasons why DFID loses focus on girls’ education during programme implementation. These include competing priorities, conflicting preferences from country partners, poor programme design, lack of expertise on the part of implementing partners and the difficulties of working in challenging locations. If DFID is successfully to deliver on its “leave no one behind” commitment, it needs to overcome these challenges.

How does DFID ensure value for money in supporting marginalised girls’ education?

In this section, we have assessed DFID’s approach to ensuring value for money in its support for marginalised girls’ education. We examined its value for money assessment methods and how well value for money approaches are integrated into programme management. DFID’s usual framework for assessing value for money does not directly address equity

A 2010 National Audit Office report on DFID’s bilateral education portfolio provided the initial impetus for DFID to begin applying value for money analysis to its programming.[50] DFID developed specific guidance for value for money in the education sector, based on the “3 Es” model of economy, efficiency and effectiveness, which is used across the UK government. The guidance centres on the long-term development returns of investing in education, although it does not discuss the specific benefits of educating girls for communities and households. The guidance stresses the need to consider value for money through the programme cycle, and to embed value for money analysis into programme management.

The guidance is designed to be applied in a flexible manner. It makes it clear that achieving value for money does not necessarily mean choosing the cheapest option. However, there is limited discussion of circumstances where DFID may need to choose more expensive options. The guidance considers cost drivers associated with delivery to remote or insecure locations, but makes no reference to the fact that the costs of engaging and supporting marginalised groups are likely to be higher, regardless of their location.

Many stakeholders consulted for this review, both within and outside DFID, expressed the view that this guidance needs to be adapted in light of DFID’s commitment to marginalised girls’ education and the “leave no one behind” pledge, to ensure that the marginalised are not excluded by value for money considerations, This would entail giving due weight to equity (sometimes cited as the fourth “E” in the value for money framework). At present, DFID does not routinely require this for education programmes.[51]

In a workshop we held with practitioners, some viewed the lack of a clear framework for addressing equity objectives in education programmes as a flaw in DFID’s approach to value for money, with the potential to weaken programme design. A notable example is the use of a cost-per-girl metric as one of the selection criteria for Girls’ Education Challenge projects. Although some higher-cost projects were selected, stakeholders reported that this emphasised the need to reach more beneficiaries rather than the “most marginalised”. This is despite recognition by DFID that cost-per-pupil is a poor measure of value for money, given the need to improve the quality and effectiveness of education interventions. These stakeholders highlighted that this may have constrained the Fund’s ability to address the needs of girls that are harder to reach. Implementation of the “leave no one behind” promise will require DFID to develop new methods of bringing equity into its value for money framework for education programmes.

DFID has begun to generate the data needed for more robust value for money analysis

The National Audit Office report mentioned above recommended that DFID “take a tougher, clearer stance on the importance of cost and service performance information; particularly on indicators of education delivery and attainment”. From 2011, DFID began collecting a set of standard data across its priority countries to enable comparisons between countries and over time. It covers both input costs (such as average primary teacher salaries and textbook costs) and learning metrics (such as pupil attendance and completion). In principle, the data provides DFID with a method for driving improvement in value for money across the portfolio. The data was also shared with the Public Accounts Committee until 2013.

By 2014, however, the practice had fallen away. DFID informs us that this was because there was little variance in the data across years. These metrics are also considered less relevant, given the shift away from budget support towards more focused interventions. DFID now only collates and reports on a single measure: cost per pupil. The Girls’ Education Challenge has developed project-level efficiency metrics (cost per output) and cost-effectiveness metrics (cost per learning outcomes), allowing it to generate a set of comparable data from across its portfolio. DFID expects that these metrics will also be adopted by country education programmes.

Based on its mid-term and final results, the Girls’ Education Challenge will be able to assess the costs associated with achieving different levels of educational attainment for marginalised girls, which can be factored into future value for money assessments. A centrally managed fund is well suited to this kind of measurement. While there is at present no similar initiative for country education programmes, the Girls’ Education Challenge may in due course provide DFID with some of the information it needs to begin applying value for money analysis to its “leave no one behind” commitment.

DFID has also placed a strong emphasis on understanding what drives quality of learning across its portfolio. It is supporting three initiatives that aim to improve the quality and availability of education data: the UNESCO Institute of Statistics, the Programme for International Student Assessment and the Global Education Monitoring Report. These initiatives are developing new methods of measuring the quality and equity of learning outcomes – both important issues in tackling the marginalisation of girls.

Value for money analysis is embedded in project management, but focuses mainly on economy and efficiency

For bilateral country programmes, value for money assessment is the responsibility of programme managers. They were provided with training on the implementation of the value for money guidance. In our country case studies, we found that staff were confident in applying it. While the level of specialist knowledge available in country offices varies, back-up support is available elsewhere when needed (for example, from DFID education economists in the UK). Value for money assessments are often reviewed by specialists or carried out by external advisors.

In our review of girl-focused country programme documents (business cases and annual reviews), we found a consistent focus on economy (input costs) and efficiency (the relationship between outputs and inputs). The majority of programmes pay close attention to identifying cost drivers, and in some cases we saw evidence that these are tracked over time. In a few cases, this extends to the use of “trigger points”: that is, if costs rise above a predetermined level, this triggers further investigation to understand the causes.

For the majority of these programmes, we saw examples of DFID achieving cost savings, largely related to efficiencies in delivery. For example, one programme in Tanzania reduced the cost of teacher training by changing its delivery model and using community-based accommodation, rather than hotels. In a Kenya programme, costs were reduced by sharing procurement and transportation among subcontractors and by scrutinising prices for building materials. A Girls’ Education Challenge project partner achieved greater cost-efficiency by tailoring its support to the needs of individual girls, enabling it to reach more girls with its budget. There were also examples of cost-overruns highlighted in annual reviews.

While important, unit cost analysis is only one part of the value for money equation for programmes that seek to reach marginalised girls. The stakeholders we spoke to, both in DFID and among its partners, recognised that not enough is known about the drivers of effectiveness and, in particular, which interventions lead to the best learning outcomes. Assessing effectiveness in education is a complex undertaking and DFID’s education research team has launched a number of studies to explore which mix of inputs generates the best learning outcomes. The Girls’ Education Challenge is also tracking learning outcomes more systematically, in order to inform assessment of cost effectiveness. The data required for a more holistic value for money assessment is therefore gradually being assembled.

The Girls’ Education Challenge has also been trialling the use of outcome-based payment by results, whereby suppliers are paid partly in proportion to their achievement of specified results, rather than for the cost of their inputs. Some stakeholders told us that this has supported a focus on learning and continuous improvement amongst delivery partners. However, not everybody is in favour of the approach. Many stakeholders highlighted the difficulty of reconciling DFID’s desire for innovation with its pressure on suppliers for guaranteed results. Some stakeholders told us that smaller civil society partners are unable to take on the financial risks associated with payment by results. This may have a negative impact on DFID’s access to specialist expertise and contextual knowledge.

The relative merits of state and non-state education are not yet clear

As part of our review, we explored the value for money considerations at stake in DFID’s choice of delivery channels. One of the key choices is between working with national education systems or private sector providers. While DFID does both, for the time being it lacks the evidence to make informed judgments as to what combination offers the best value for money in which contexts.

While DFID has moved away from budget support in its education portfolio, most of the country education programmes that we reviewed work closely with and through governments as the primary partner for delivering education. In some instances, this constrains DFID’s ability to focus on girls’ education, owing to the competing priorities of its national counterparts. By contrast, the Girls’ Education Challenge works through civil society and private sector delivery partners to support girls in public schools. This has enabled more targeted and innovative approaches, but at the expense of reach and sustainability. There is inadequate evidence from our review of programmes and the wider literature to draw conclusions about which delivery mechanisms work best for marginalised girls, but many of the stakeholders we consulted thought that a combination is likely to be required.

Several stakeholders argued that DFID should place a greater emphasis on supporting governments to develop more inclusive education systems and allocate more funds to education for girls, in order to achieve transformative and sustainable change. However, the trade-offs involved in working through governments were also noted. A key dilemma when working through government is whether to single out marginalised girls for additional support when there is not enough capacity in the education system as a whole to provide quality education for the majority. At present, DFID lacks a strong evidence base to guide its decisions on whether to support private schools

In three of the countries we assessed (Pakistan, Tanzania and Kenya), DFID supports low-cost private schools. These have expanded rapidly in recent years, providing a possible alternative means for reaching marginalised girls. However, for the time being, DFID lacks evidence on the effectiveness and value for money of working through private schools.

Many stakeholders consulted during this review questioned the value of DFID’s work through private schools for marginalised girls. Their concerns centred on whether private provision is equitable and whether it has a detrimental effect on public education.

A rigorous review commissioned by DFID of the research into private schools in developing countries concluded that the evidence on whether private schools are equally accessible to boys and girls is inconsistent.[52] Given this uncertainty, it is important that DFID’s work with low-cost private schools generates evidence regarding access and learning outcomes, including for marginalised girls.

Conclusion on value for money

We have given DFID an amber-red score for its efforts to ensure value for money in its programming for marginalised girls. We found that DFID has a workable framework for value for money across its education portfolio, and that there has been some progress in integrating this framework into programme management, though we note it remains stronger at measuring economy and aspects of efficiency rather than effectiveness. While we saw that DFID has begun to invest in collecting the standardised data required for more sophisticated value for money assessment through its Girls’ Education Challenge programme, this practice is not yet consistent across its whole portfolio, where DFID currently only collates and reports on a single measure: cost per pupil. DFID also lacks the evidence to make informed judgments on the relative cost-effectiveness of different delivery channels in its programming for marginalised girls.

Finally, DFID has not yet updated its value for money approach for education programming to take account of the importance of working with hard-to-reach groups and the “leave no one behind” commitment. There is a need for DFID to consider how to incorporate equity considerations more comprehensively into the framework and to develop guidance for country offices and projects.

Conclusions & recommendations

DFID’s “leave no one behind” commitment has important implications for many areas of the aid programme, and particularly for education. As DFID’s partner countries move closer to achieving universal basic education, the challenges of bringing marginalised groups into the education system, and ensuring that they succeed in gaining a useful education, come to the fore. DFID has committed itself to supporting these marginalised groups. This will require a concerted effort to improve on current performance.

This review has identified three main deficiencies in DFID’s approach. First, while DFID has committed itself to supporting the education of marginalised girls and has followed this up with significant investments, it lacks a coherent approach to achieving its goals. As a result, its efforts are inconsistent in their depth and focus, and poorly joined up.

Second, we identified recurrent factors across the portfolio that have resulted in a loss of focus on marginalised girls during implementation. As a result, DFID’s record on effectiveness is at best mixed, and at times poor. DFID will need to overcome these challenges if it is to deliver on its “leave no one behind” commitment.

Finally, DFID has yet to adapt its education value for money framework to reflect its commitments on tackling marginalisation. DFID should use emerging practice from the Girls’ Education Challenge to inform its approach to equity and value for money. Without this, there is a risk that the drive for economy and efficiency will create disincentives for DFID and its partners to focus on marginalised girls.

Recommendations

We welcome DFID’s commitment to supporting the education of marginalised girls and the contribution made by its programmes. The following recommendations are intended to help DFID strengthen its efforts to ensure that the commitment to leaving no one behind can be delivered.

Recommendation 1:

DFID should develop country-specific strategies for marginalised girls’ education,  based on detailed knowledge of the barriers in each context and learning from successful interventions. Its strategies should combine policy dialogue, system building and targeted interventions, and identify opportunities for cross-sectoral working.

Problem statements:
• DFID does not give sufficient attention to identifying the challenges facing girls as a whole and the disadvantages experienced by specific groups of marginalised girls, as well as how different dimensions of disadvantage interact to create barriers to education.
• DFID lacks mechanisms for ensuring that research and evidence from learning about what works is consistently used to inform programming.
• DFID has not identified how to combine different kinds of intervention at the country level (policy dialogue, system building and targeted programming) into a coherent approach to tackling the marginalisation of girls.
• While various causes of girls’ marginalisation in education are addressed, there are inconsistencies across the portfolio and a number of important factors (for example health issues and genderbased violence) are not given sufficient attention.

Recommendation 2:
In its delivery plans and monitoring of programmes with objectives around girls’ education and marginalisation, DFID should introduce measures to ensure that this focus is not lost during implementation.

Problem statements:
• Many education programmes have performed poorly against their original ambitions for supporting girls’ education.
• There is a clear tendency for the focus on marginalised girls to be lost during programme implementation.
• The most common reasons impeding programme performance against expectations for girls’ education were:
• Lack of expertise of delivery partners in girls’ education.
• Girls’ education objectives being deprioritised because of competing priorities within the programmes.
• DFID’s lack of control/influence over a focus on girls in government education programmes that DFID helps to fund.
• Challenges in implementing programmes in difficult operating environments.
• Poor design of programme components.

Recommendation 3:
DFID should specify how to approach value for money analysis when targeting marginalised groups and harder-to-reach groups, emphasising equity as well as cost-effectiveness.

Problem statements:
• DFID has a sound general framework for assessing value for money in education programmes, but this does not explicitly address equity for marginalised groups and the commitment to leaving no one behind.
• The common emphasis on cost-efficiency within DFID’s value for money analysis may work against a focus on the most marginalised.
• There is limited evidence on the relative cost-effectiveness of different delivery channels to help achieve better value for money in DFID programming for marginalised girls.
• Learning from the Girls’ Education Challenge is not being fully utilised to inform the overall value for money framework for education.

Annex 1 Detail of scoring

Question 1: Relevance  Green/Amber

DFID has made a substantial commitment to promoting the education of girls and to tackling the range of barriers that lead to girls being marginalised in education. It has invested in generating evidence and learning on how to tackle marginalisation. There are programmes in most DFID priority countries that prioritise girls’ education and include targeted interventions. The innovative, centrally managed, multi-country Girls’ Education Challenge is the largest ever donor programme aimed at the most marginalised girls. However, DFID needs to strengthen its strategy for addressing the causes of marginalisation, both globally and in particular countries. The different strands of its work in this area – including country programmes, the Girls’ Education Challenge, the Global Partnership for Education and relevant work in other sectors such as health – are not sufficiently joined up. We rated performance on relevance as green-amber due to the significant investment in girls’ education programming, while recognising that greater strategic coherence and coordination will be needed to achieve transformative impact.

Question 2: Effectiveness Amber/Red

DFID met its overall results target for supporting girls in primary and lower secondary school, although this target relates to all girls and not the most marginalised. Many DFID country plans and programmes have performed poorly against their original objectives for tackling the marginalisation of girls in education, and there is clear evidence that the focus on marginalised girls is often lost during programme implementation. We identified a number of factors that were causing this, including competing priorities, a lack of expertise among implementers and political constraints in partner countries. DFID has reported strong performance of the Girls’ Education Challenge, although it is yet to deliver on its overall ambition for improving girls’ learning and has struggled with sustainability. We rated performance on effectiveness as amber-red due to the frequent loss of focus on girls and a pattern of poor performance against expectations for girls in education programming.

Question 3: Value for money Amber/Red

DFID has a framework for assessing the value for money of its education programmes, which is widely used for project management. However, this framework has not been adapted to include equity objectives and is therefore difficult to apply to programming for hard-to-reach groups. DFID has made some progress towards collecting the data required for better value for money analysis in education programming, but remains stronger at measuring economy and aspects of efficiency, rather than effectiveness. We have rated DFID’s performance amber-red on value for money to reflect the fact that its framework has not been adjusted to reflect its commitment to marginalised girls and leaving no one behind.

Overall Score – Amber/Red

Overall, we have awarded DFID an amber-red score. DFID has made a strong commitment to tackling girls’ education and has a good range of programming directed towards marginalised girls. However, we find that its various activities do not add up to a coherent strategic approach to tackling marginalisation. We have identified a series of factors that are causing DFID programmes to lose their focus on marginalised girls through the implementation process. In light of its promise to marginalised girls’ education and the “leave no one behind” pledge, this is a serious shortcoming. Furthermore, DFID has not yet adjusted its approach to value for money to reflect this commitment.

Annex 2 Methodology

Our methodology was structured to facilitate assessment of DFID’s strategic approach to supporting basic education for marginalised girls (covering expenditure, strategies, guidance and contribution to the evidence base) and to examine delivery (through assessment at the country and programme levels). There were three main components to the methodology: (1) a literature review, (2) a strategic review and (3)
country assessments.

1. Our literature review covered recent published material in academic journals, books and the publications of international organisations. It synthesised analysis of the barriers to education for marginalised girls across different country contexts and experience in attempting to overcome them. This provided a reference point for assessing the relevance and effectiveness of DFID’s programmes.

2. We carried out a strategic review of DFID’s overall approach to supporting basic education for marginalised girls. We examined the appropriateness and capacity of DFID’s strategic approach to supporting interventions that aim to benefit marginalised girls. This included the quality of guidance for those responsible for programming and use of research and evaluation outputs to support continual improvement. Specific aspects of this review included:

• Review of DFID strategy documents and guidance, including value for money guidance.
• Mapping of the level of focus on marginalised girls’ education in DFID country operational plans over the review period.
• High-level mapping of all bilateral education programmes in scope, including their level of focus on girls’ education, what girl-focused interventions they included, and whether they targeted particular groups.
• An overall assessment of the Girls’ Education Challenge (GEC), through document review and interviews with relevant DFID staff and staff from the consortium delivering the programme.
• Consultation with a range of key stakeholders in the UK, including interviews with DFID staff and external stakeholders from academia, civil society and other donor organisations, and a workshop with practitioners.

3. We carried out country assessments of DFID’s relevant activity in six countries, purposively selected to cover a range of country contexts (including situations of fragility and different degrees of gender inequality in education) and types of programming (ensuring that the countries selected had a good range of girl-focused country programmes, GEC projects and Global Partnership for Education activity). These assessments included detailed programme reviews of girl-focused centrally managed and country basic education programmes in these countries, and fewer in-depth reviews of other country basic education programmes (see Figure 5 below for a full list of the programmes reviewed). For four of these countries (Lebanon, Malawi, Pakistan [Khyber Pakhtunkhwa province] and South Sudan), these assessments were conducted remotely. For the other two countries (Kenya and Tanzania), we conducted country visits. As part of our assessment of the GEC, we carried out detailed programme reviews of four GEC projects in Kenya and Tanzania, selected to cover examples of projects from all three of the funding windows.

This approach reflects the importance of understanding the aspects of context that influence marginalisation and which may affect the development, implementation and performance of programmes. This approach provides a basis for detailed examination of the ways in which DFID’s education programming supports marginalised girls’ education, looking across the country education portfolio as a whole. This focus on the country level allows for an assessment of the coherence of DFID’s efforts on supporting marginalised girls’ education.

Country assessments were guided by the use of a standard Country Assessment Framework specifying the information that should be collected and providing standard criteria for evaluative judgements.

This framework covered:

• The relevance of the bilateral basic education portfolio for supporting marginalised girls’ education in the particular country context.
• The effectiveness of programming for supporting marginalised girls’ education, and constraints on and success factors for effectiveness.
• The strength of programme approaches to maximising value for money

Programme assessments involved examining programme documents and conducting structured interviews with key people involved with the programme, including (as appropriate):

• The responsible DFID adviser and/or programme manager.
• The responsible manager of the implementing agency.

Beyond interviews conducted as part of programme reviews, country assessments were informed by:

• Structured telephone interviews with DFID Education Advisors, as well as a limited number of key stakeholders in those countries, including government officials and staff from other donor organisations and civil society.
• Review of relevant documentation including education data and academic and donor reports.
• In the case of visited countries, face-to-face interviews with a broader range of stakeholders were conducted, including further DFID country office staff and staff from implementing agencies working at different levels. In these countries we also visited programme sites. Site visits informed our assessment of relevance, effectiveness and of aspects of value for money including sustainability. These site visits also allowed us to gain a better understanding of the context in which programmes are being implemented.

Collection of information from different sources on DFID’s support to girls’ education allowed triangulation of findings, including through seeking out critical voices.