Assessing DFID’s results in improving Maternal Health

ICAI score

DFID programmes have expanded access to family planning and some maternal health services, but a renewed effort is required to reach young women and girls and to generate lasting impacts on quality of care and maternal health outcomes.

In 2010, DFID set itself a global target of saving 50,000 women’s lives during pregnancy and childbirth by 2015. DFID significantly scaled up its efforts on family planning, through global advocacy and its own programming. It helped extend access to modern family planning methods to millions of women and girls but encountered challenges with ensuring reliable supplies of contraceptives at health facilities. DFID has expanded access to some relevant health services, but in the face of severe shortages of skilled personnel, equipment and supplies, it has struggled to raise service quality to the extent needed to improve maternal health outcomes. Overall, we find that DFID’s portfolio was not well balanced across family planning, health services and other interventions, so as to maximise medium- to longterm impact. We were unable to confirm DFID’s global results claim on saving maternal lives, owing to shortcomings in the way it estimated the impacts of its programmes.

DFID’s strategy for 2011 to 2015 prioritised the poorest 40% of women and the 15- to 19-year-old age group. We found a geographical focus on the poorest areas and most programmes worked to improve the accessibility and affordability of services. However, few programmes included specific measures to reach young, poor or marginalised women or monitored their impact on different population groups. They also gave little attention to discrimination and abuse of women in health facilities.

DFID’s maternal health programmes have had a limited focus on strengthening national health infrastructure and institutions to ensure sustainable improvements in maternal health. Faced with governance problems and difficult operating environments, DFID has often chosen to work with non-state providers, and for some family planning programmes the approach has risked displacing public sector provision. The new international goal of achieving universal health coverage is weakly reflected in DFID’s ongoing programes. DFID has been a strong champion of reproductive rights at the international level but has done less to support local efforts to effect legal, policy and cultural change in partner countries.

Individual question scores
Question 1 Impact: How well has DFID maximised the impact of its programming on maternal health?
Question 2 Equity: How well do DFID’s maternal health and family planning programmes reflect its commitments to equity and inclusion?
Question 3 Sustainability: Are DFID’s maternal health results likely to prove sustainable?

Executive summary

Every year, more than 300,000 women die from complications related to pregnancy and childbirth, with 99% of these deaths occurring in developing countries and more than half in sub-Saharan Africa. In the developing world, such complications are the number one killer of girls and women aged between 15 and 19. Poor maternal health also has serious consequences for the survival of babies, while a mother’s death has devastating effects on her children’s health, wellbeing and life prospects. Most maternal deaths, injuries and illnesses could be prevented by providing women and girls with appropriate, good quality sexual, reproductive and maternal health services.

During the Millennium Development Goals period, important progress was made to improve maternal health: the global maternal mortality ratio declined by 44%, from 385 maternal deaths in every 100,000 live births in 1990 to 216 in 2015. However, this fell short of the target to achieve a 75% reduction, and there was considerable variation across countries. Some of the poorest and most conflict-affected countries, particularly in sub-Saharan Africa, failed to make any substantial progress. Alongside the long-term challenges of providing access to quality health services and family planning to poor and remote communities, progress was held back by women’s unequal status, the prevalence of violence against women and girls and social attitudes to contraception, abortion and sex outside marriage. The Sustainable Development Goals now include the even more ambitious target of reducing maternal deaths by a further two thirds by 2030.

Improving maternal health has been a long-standing objective for the UK aid programme. In 2010, when progress towards the global target was lagging, DFID pledged to intensify its efforts. It published a Results Framework on reproductive, maternal and newborn health, setting targets for DFID’s contribution to family planning, safe delivery and maternity services, with a headline goal of saving 50,000 women’s lives during pregnancy and childbirth by 2015. Priority was to be given to the poorest 40% of women and the 15- to 19-year-old age group, who are particularly at risk. Over the four financial years covered by the Results Framework, DFID spent around £4.6 billion on programmes in health and other sectors (such as water, sanitation and hygiene) that contributed to its results on reproductive, maternal, neonatal and child health. Within this, £1.3 billion was focused more closely on maternal health, including programmes on family planning, reproductive health care and maternal and neonatal health. By the end of the period, DFID announced that it had achieved more than double its targets on safe deliveries and maternal lives saved.

This review looks at how well DFID maximised the medium- and long-term impact of its investments in maternal health programming over the 2011-15 Results Framework period. It covers both its family planning work (which prevents maternal deaths primarily by reducing unwanted pregnancies) and its efforts to strengthen maternal health services. We explore the robustness of DFID’s results data and the equity and sustainability of its investments. The review assesses a sample of global and multi-country programmes, and country-specific programmes in Malawi and the Democratic Republic of the Congo (DRC). These focus countries were selected because of their high maternal mortality rates, as well as their broad DFID programme portfolios and significant contributions to DFID’s global results claims. While the focus is on results achieved between 2011 and 2015, we also consider whether learning from that period has been applied to more recent programmes.

How well has DFID maximised the impact of its programming on maternal health?

DFID’s 2010 Results Framework set out a clear strategy for improving maternal health, based on global evidence of ‘what works’. Recognising that maternal health is influenced by a wide range of factors, it set out a well-balanced strategy for achieving long-term results through family planning, strengthening national health systems, empowering women and girls and addressing the social and economic barriers that prevent poor women from accessing health services.

Over the review period, DFID took on family planning as a departmental priority. It championed family planning and reproductive rights at the international level. It also launched a number of new programmes at the global and country levels, as part of a commitment to doubling UK aid for family planning to £516 million by 2020. Family planning contributes to achieving a range of objectives, including reducing poverty and empowering women. It also saves maternal lives by preventing unwanted pregnancies and allowing women to better space their pregnancies, both of which reduce the risks associated with pregnancy, abortion and childbirth.

DFID’s family planning programmes have expanded the availability of sexual and reproductive health services for women. The cross-country Prevention of Maternal Deaths from Unwanted Pregnancy programme, for example, provided 4.4 million family planning services in 2015 (such as providing contraceptive implants), preventing an estimated 1.7 million unintended pregnancies and 6,644 maternal deaths. DFID worked with the UN agency for family planning, the UN Population Fund (UNFPA), to boost the global supply of contraceptives. Programmes in both the DRC and Malawi reported helping large numbers of women to take up modern family planning methods. Globally, by the end of the results period, DFID reported that it had extended access to family planning to nearly 10 million additional women and girls.

However, there were considerable challenges involved in ensuring a regular supply of contraceptives to health clinics in developing countries. During our country visits, we heard that many women and girls were still unable to obtain their chosen family planning method locally. DFID also invested in improving maternal health services over the review period. Its partnership with UNFPA paid for medicines that are essential to maternal health, but again experienced acute problems with ensuring consistent availability in health facilities. DFID’s global Making It Happen programme used expert volunteers from the UK to train health workers in developing countries in obstetric and neonatal care, achieving mixed results across countries. The Evidence for Action to Reduce Maternal and Neonatal Mortality programme helped to make service providers more accountable to local communities, although DFID support was scaled back and then discontinued.

In our case study countries, DFID also invested in strengthening basic health services, including some that are important for improving maternal health, such as services that help prevent malaria in pregnant women and provide access to skilled birth attendance. However, while DFID programmes have expanded access to these services, they are yet to make a significant difference to the quality of services offered to women. In Malawi, for example, more women are giving birth in health facilities, but severe shortages of beds, trained personnel, equipment and supplies mean that women remain at risk of dying from basic obstetric complications. Within DFID’s health programmes, progress on improving emergency obstetric and neonatal care has been well short of targets. In Malawi, by 2016 15% of health facilities were able to provide basic emergency care, against a target of 30%. In the DRC, despite a slow start, DFID programmes enabled 143 facilities to provide some elements of emergency care in 2017, although progress still remains below target. Overall, shortfalls in progress on improving the quality of maternal health services meant that reductions in maternal mortality were significantly below what they could have been, given the level of investment.

While DFID’s Results Framework set out a strong case for a multidimensional approach to improving maternal health, we find that the portfolio has not followed this approach to the extent required to deliver medium- to long-term reductions in maternal mortality. There has been intensive focus on family planning, which has helped prevent deaths as well as provide other benefits. However, most maternal deaths result from intentional pregnancies, rather than accidental ones, and are therefore not prevented by access to contraception. Global figures suggest that satisfying all of the unmet need for contraception in developing countries would reduce maternal mortality rates by no more than 25%. We find that DFID did not pursue the strengthening of health systems to provide quality maternal care with the same intensity as it did for family planning, nor did it do enough to address the barriers that the poorest women face in accessing health services.

DFID reported that, over the review period, its programmes saved 103,000 maternal lives, well exceeding its 50,000 target. In the absence of robust health statistics in its partner countries, this result was determined through a mathematical model that assumed each type of DFID-funded intervention translated into lives saved at a given ratio. The use of such modelling is common practice for health programming.

We tested the robustness of this global figure by comparing it with health statistics in our case study countries and the results achieved by individual programmes in our sample. From our programme assessments, we were not able to confirm the results claim. In Malawi, we found that the total for maternal lives saved calculated through the model was based on assumptions that did not match the reality of DFID’s programmes, and was too high given other data on reductions in maternal mortality over this period. In the DRC, DFID claimed to have saved a significant number of lives by increasing the number of births in health facilities, even though its programme-level data from that period shows that very few health facilities were in a position to provide even a basic level of emergency obstetric care. As a result, we are not able to confirm that DFID’s global results claim is robust.

Despite a genuine commitment by DFID to improving maternal health and a visible effort to expand access to family planning, it is our assessment that the portfolio has not been as impactful as it could have been. This is principally due to shortcomings in the quality of interventions and the lack of a sustained focus on improving the quality of maternal care services. This merits an amber-red score. It is important to recognise that our findings are specific to the 2011-15 review period and involve a retrospective examination of programme achievements. DFID told us that some of its ongoing health systems strengthening work has since refocused on delivering quality maternal health care, and that more recently designed programmes have sought to reflect learning from this period.

How well do DFID’s maternal health and family planning programmes reflect its commitments to equity and inclusion?

DFID’s Results Framework, as well as its programme documents, prioritised reaching the poorest 40% of women and young women aged 15 to 19 with both maternal health and family planning interventions. The programmes in our sample included measures to make contraceptive supplies and maternal health services more affordable, although our visits suggested that some facilities still charge additional fees. Many programmes were also working to extend services into the poorest and most remote areas of DFID’s partner countries.

However, few of the programmes that we reviewed included additional measures to address the difficult task of reaching poor, young and marginalised women, who may face a range of practical, financial and social barriers preventing them from accessing services. DFID support helped some community groups to identify young women at risk in Malawi, but on the whole the programmes we assessed paid insufficient attention to issues of discrimination, abuse and disrespectful treatment of poor and young women at maternal health facilities. The key informants we spoke to stressed the need for context-specific solutions to these challenges. We saw limited evidence that DFID’s implementing partners had engaged with local communities to understand the barriers and how they can be addressed. However, DFID’s primary health care programme in the DRC has worked with police and communities to address sexual violence (which, among other things, can result in unwanted pregnancy), and we were encouraged to see that DFID’s new sexual and reproductive health programme in Malawi has used innovative methods to inform its design such as radio phone-ins to capture the views of young people.

DFID did not monitor whether its programmes were reaching the target group of young women aged 15 to 19. Very few programmes disaggregated their results, making it impossible to determine the impact of DFID programming on poor, young or otherwise hard-to-reach women and girls. Several programmes reported challenges in developing strategies to reach young people, and DFID’s own analysis suggests that this is common across the portfolio. DFID did not report on its Results Framework target of reaching 1 million young women aged 15 to 19 with family planning by 2015.

DFID’s international influencing activities regarding marginalised, poor and young women and girls’ reproductive health and rights have been very positive, helping to secure new commitments of funding and action and providing a counterbalance to more conservative voices on women’s rights and equality. However, DFID has been less successful in translating its considerable international influence into longer-term efforts to change harmful and discriminatory laws, policies and cultural practices at national and community levels in partner countries.

We have awarded an amber-red score for equity and inclusion, due to DFID’s limited focus on reaching the poorest, youngest and most vulnerable women through its programming or monitoring this during the period from 2011 to 2015 covered by its Results Framework.

Are DFID’s maternal health results likely to prove sustainable?

We explored the likelihood that the results of DFID’s maternal health work during the Results Framework period would be sustained beyond the life of its programming. A key element of achieving sustainability is to work with partner countries to build up their public health systems so as to create the institutional foundations for longterm results. We found that the centrally managed programmes in our sample prioritised short-term results (including the Millennium Development Goals), sometimes working in parallel to government services and without a clear strategy for how initiatives would be sustained beyond the period of external funding.

The DRC and Malawi are challenging environments for building sustainable health systems, owing to deep-seated governance challenges and, in the case of the DRC, continuing insecurity that hampers service delivery. In both countries, the public health system depends upon non-state providers – including non-governmental organisations, faith-based providers and the private sector – to deliver health services in some areas. DFID has often chosen to scale up service delivery through these non-state channels, which can be a more reliable and cost-effective way of achieving results in the short term. However, this needs to be accompanied by clearer strategies for integrating non-state providers into the public health system over the long term and securing sustainable finance. Furthermore, we encountered some examples of DFID family planning programmes supporting non-state provision in ways that risked displacing public provision.

DFID has demonstrated international leadership in promoting maternal health and has been a vocal champion of family planning and safe abortion, willing to take a clear stand on the often controversial issue of reproductive rights. DFID has succeeded in sustaining global progress on these issues. However, in our two focus countries, despite being a major donor to the health sector, DFID has not taken a similar lead in supporting advocacy for policy and institutional changes at national and local levels.

There is a positive trajectory in DFID’s use of learning to improve and adapt its maternal health programming. However, we found that ongoing maternal health programmes (active after the Results Framework period) only weakly reflect key components from the Sustainable Development Goals, such as providing universal health coverage and ‘leaving no one behind’. We also found little evidence to date that DFID’s non-health sector programming (for example nutrition, water and sanitation) had recognised the potential for investments that could support maternal health outcomes.

DFID’s global advocacy in sustaining international progress on reproductive health and rights has been strong. However, we have awarded an amber-red score for sustainability due to an emphasis on short-term impact goals, limited sustainability strategies and potential displacement of some public sector family planning provision.

Recommendations

We offer the following recommendations to help DFID improve the impact of its future work on maternal health:

Recommendation 1

As part of its commitment to the Sustainable Development Goals, DFID should develop a long-term approach to improving maternal health, planning through to 2030 in focus countries with high maternal mortality. These plans should focus on improved quality and continuity of care, cross-sectoral interventions and efforts to empower women and girls.

Recommendation 2

DFID should clarify its approach to health systems strengthening, prioritising improvements in the availability and accessibility of good quality, respectful care for women and their babies.

Recommendation 3

DFID should directly monitor the impact of its sexual, reproductive and maternal health services programmes on adolescents and the poorest women. This means including design features in programmes that target adolescents and the poorest women, monitoring whether they are effective and adjusting course where they are not.

Recommendation 4

When using models to generate outcome data, DFID should test its assumptions and triangulate its results claims using other quantitative and qualitative data.

Recommendation 5

As part of its commitment to the Sustainable Development Goal data revolution, DFID should prioritise and invest in international and country-level efforts to gather data on the quality of maternal health services and outcomes, including disaggregated data relating to key target groups.

Introduction

Improving maternal health is a pressing global development challenge. The World Health Organization estimates that 830 women die every day from factors related to pregnancy and childbirth – 303,000 women worldwide in 2015 alone. 99% of these deaths are in developing countries, with more than half in sub-Saharan Africa. The majority relate to complications during pregnancy, abortion or delivery, such as haemorrhage, infection and high blood pressure. With access to safe and effective sexual, reproductive and maternal health care services, many of these conditions can be treated or prevented. However, such complications remain common in many countries in the developing world, and they are the number one killer of girls aged 15 to 19 worldwide. Poor maternal health is linked directly to poor neonatal health – including preterm births, stunting, stillbirths and neonatal deaths – and a mother’s death has a serious impact on the health, wellbeing and life prospects of her children.

The Millennium Development Goals aimed to reduce maternal mortality by three quarters worldwide by 2015, from a 1990 baseline of 385 maternal deaths per 100,000 live births. Progress was slower than anticipated: by 2015, the global figure had declined by 44%. While short of the Millennium Development Goal target, this was a significant achievement considering the many obstacles to improving maternal health – including weaknesses in health care provision and social issues such as gender discrimination, attitudes to contraception, abortion and sex outside marriage, and violence against women and girls. Other contextual factors such as poor transport and communications infrastructure present significant barriers to improving women’s access to maternal health services.

The global figures mask considerable variation between different countries and regions. Although sub-Saharan Africa saw a reduction in maternal mortality of 49% between 1990 and 2015, its maternal mortality ratio still lags far behind the rest of the world (see Figure 1). In sub-Saharan Africa, a woman’s lifetime risk of dying during pregnancy and childbirth at the end of the Millennium Development Goals period was 1 in 36, compared to 1 in 3,700 across the developed world. Differences within sub-Saharan Africa were also large: Rwanda and Cape Verde were among a few countries worldwide to exceed the Millennium Development Goal target. However, all countries with a maternal mortality ratio of more than 500 maternal deaths per 100,000 live births in 2015 were in sub-Saharan Africa.

In 2015, the Millennium Development Goals were superseded by the Sustainable Development Goals, which set even more stretching and specific objectives for improving maternal health by 2030 (see Box 1).

Figure 1: Maternal mortality ratios globally and by region from 1990 to 2015

Source: Global Health Observatory (GHO) Data – Maternal Mortality, World Health Organization.

In December 2010, as part of a global drive to accelerate progress towards the Millennium Development Goals, DFID published its Framework for Results for improving reproductive, maternal and newborn health in the developing world (the Results Framework). This set ambitious targets for DFID’s maternal health programming through to 2015, including saving 50,000 maternal lives and supporting 2 million safe deliveries.

Because maternal health is a well-established area of programming for UK aid, where DFID has set and reported against ambitious results targets, we opted to conduct an impact review (see Box 2). The purpose is to examine the impact and sustainability of the UK’s contribution to international goals for maternal health, as well as to assess the validity of DFID’s results claims. We look at programming to improve health services for women and to expand family planning, which prevents maternal deaths by reducing unwanted pregnancies. (Family planning programmes have other important objectives in addition to reducing maternal mortality, around sexual and reproductive health and rights. Our focus here is solely on their contribution to maternal health.) The review assesses a sample of centrally managed programmes and country-level programmes in two countries: Malawi and the Democratic Republic of the Congo (DRC). We acknowledge that these two case study countries may not be representative of DFID’s global maternal health work, although they are likely to raise issues that are relevant across the portfolio.

The review is primarily concerned with the period 2011-15, framed by the global results targets that DFID set in its 2010 Results Framework. Our main findings relate to DFID’s achievements over this period, although we take into account that results from programming in this period may not become apparent until later. We also consider the application of learning from this period to DFID’s ongoing maternal health programming, and we look at more recent aspects of DFID’s influencing role in key international arenas, including interactions with the lead United Nations agency in this area, the UN Population Fund (UNFPA).

This is the first external review of DFID’s maternal health programme since an International Development Committee report in 2008, which urged DFID to put more emphasis on addressing the shortage of midwives and improving women’s access to emergency obstetric care. A 2014 report by the International Development Committee stressed the importance of strengthening health systems in order to provide better health services to women during pregnancy and childbirth.

Table 1: Our review questions

Methodology

Our methodology was designed to allow an assessment of DFID’s contribution to improving maternal health outcomes against the global consensus about what works. The methodology also helped us assess the robustness of DFID’s numerical results claims at the end of the Results Framework period, by examining the data and models used by the department to measure the impact of particular interventions on maternal health and mortality. Our methodology had four components:

• Literature review: we undertook a rapid review of published literature, including reports, journal articles and systematic reviews. This helped us to identify areas of consensus and disagreement, note gaps in the knowledge or evidence base and assess different perspectives on data quality and modelling approaches as well as future challenges.
• Strategic review: we reviewed over 100 DFID policy, strategy and guidance documents, conducted 34 interviews with global key informants and DFID policy staff and held a roundtable discussion with eight UK-based experts.
• Country programme reviews: we looked at a sample of programmes relevant to maternal health in Malawi and the Democratic Republic of the Congo (DRC) that were operational during 2011-15 (and in some cases beyond). We undertook case study visits to both countries, enabling us to interview DFID staff, contractors, government partners and other stakeholders. Through field visits, we were able to explore programme benefits and challenges with implementing partners, health care providers and communities. The two country case studies were chosen because they made a significant contribution to DFID’s overall results claims during the Results Framework period, offered a range of DFID programming related to maternal health, including family planning, and provided country contexts that presented several challenges for effective programming (see more on the country case studies in paragraphs 2.2 and 2.3).
• Centrally managed programme reviews: we identified 15 centrally managed DFID programmes in maternal health, reproductive health and family planning, which were implemented between 2011 and 2015 (and in some cases continuing beyond that). We sampled eight programmes that accounted for more than 90% of centrally managed programme expenditure since 2010-11, and which used wide-ranging partnership and delivery mechanisms (multilateral fund, contractor, non-governmental organisation, academic institution). We assessed them through document review and key informant interviews. We also explored the impact of some of these programmes in Malawi and the DRC.

The literature review was conducted at an early stage to inform the strategic, country programme and centrally managed programme reviews. As shown in Figure 2, data from all four components was triangulated to generate robust findings.

Of the 24 countries where DFID has family planning and maternal health programmes, we chose to visit Malawi and the DRC. Both countries fell well short of achieving the Millennium Development Goal target to reduce maternal deaths, and both have maternal mortality ratios among the highest in the world (see Figure 5 later in the report). Nevertheless, analysis commissioned by DFID indicated that its programmes in these countries made significant contributions towards meeting the department’s targets. Both are priority countries for UK aid and have DFID programming portfolios encompassing several areas related to maternal health, such as health systems strengthening, family planning, malaria control, nutrition, water, sanitation and hygiene, and efforts to improve the rights of women and girls. DFID’s programmes in Malawi are more focused on family planning, while its DRC portfolio emphasises access to primary health care services. This allowed us to explore potential differences in impact from these two approaches.

Figure 2: Components of the methodology

Both Malawi and the DRC are challenging operating contexts where it is difficult to achieve results, for various reasons (see Box 7). However, their ongoing high rates of maternal mortality, and the obstacles in the way of reducing them, are representative of the countries in sub-Saharan Africa where accelerated progress will be needed to achieve the Sustainable Development Goals and where UK aid is increasingly focused. While our findings draw mainly on these two case study countries, together with a sample of centrally managed programmes, they are likely to be relevant to understanding the impact of DFID’s programmes across its maternal health portfolio. However, we acknowledge that DFID’s maternal health programmes in some other countries may have performed better during the Results Framework period.

Background

What is involved in improving maternal health?

Women and girls die during or after pregnancy and childbirth from various complications, many of which can be prevented or treated if good quality health care is available. High blood pressure (preeclampsia and eclampsia), severe bleeding, infections, and complications from unsafe abortions to end unwanted pregnancies together account for around three quarters of all maternal deaths. Women also die due to complications arising from infectious diseases during pregnancy, such as malaria and AIDS.

For every woman who dies during or after childbirth, there are many more who suffer illness or disability due to complications arising from pregnancy or labour (known as ‘maternal morbidity’). Some of these instances of maternal morbidity can be relatively easily managed, but others are more serious. For example, some women sustain significant injuries or tears during childbirth that can only be addressed through surgery.

Levels of maternal health risk vary considerably, not just between countries but within them: in particular, poorer women and adolescents have higher risks of maternal death than other groups. Poorer women tend to receive lower quality maternal health services, including abusive and disrespectful care by health workers. They often live and work in conditions that increase the risk of pregnancy- and birth-related complications.

Complications related to pregnancy, labour and unsafe abortion are the number one cause of death among girls aged 15 to 19 globally. Adolescent mothers – those aged between 10 and 19 – face higher risks from high blood pressure and systemic infection than women in their 20s, and their babies are more likely to be born underweight or too early or to suffer serious neonatal conditions. Often adolescent mothers experience strong pressure from family and community to have many children spaced close together, further increasing the risks for both mother and baby. Many adolescents struggle to access contraception and advice, and girls and young women are also at greater risk of coerced sex and sexual violence. It is estimated that half of all pregnancies among girls aged 15 to 19 in developing countries are unintended.

There is a good degree of international consensus about what works to improve maternal health, reflected in the literature review, key informant interviews and expert roundtable conducted for this ICAI review. Strengthening health systems is fundamental to improving the effectiveness and quality of the care and other services provided to women and girls. This means ensuring facilities are developed and maintained, staff are trained and motivated, and equipment and other supplies are readily 12 available, so that women and girls can access good quality maternal health care when they need it. The WHO has developed a framework for health systems strengthening, shown in Figure 3, highlighting the components that need to be addressed. Progressing this in poor, fragile or conflict-affected countries is very challenging and requires long-term investment and technical support.

Figure 3: The WHO Health Systems Framework

Source: Everybody’s Business: strengthening health systems to improve health outcomes – WHO’s framework for action, WHO, 2007.

Preventing unwanted pregnancies through contraception or family planning is also recognised as an important and cost-effective way to improve maternal health, and many experts also flag the need for access to safe abortion. This is challenging and cannot be tackled through improved services alone. In countries where women and girls have few rights and opportunities to make their own reproductive choices, maternal health improvements also depend on changes to laws, policies and often deep-seated cultural practices. This calls for interventions that are adapted to the political and cultural context.

Not all maternal health service provision needs to take place within formal health facilities. Lives can be saved and illness prevented through community-based interventions, such as health education and promotion, provision of basic health and nutrition services and onward referral of women and girls to facilities by community health workers. Community health workers have been shown to be more effective in communities that are already informed and mobilised. Improving access to clean water, sanitation and hygiene in households, communities and health facilities also helps to reduce the risks of infection and disease for mothers and newborns.

An integrated set of programmes and interventions along the continuum of care is widely seen as key to achieving significant changes in maternal health outcomes. The continuum of care takes place across two dimensions, shown in Figure 4:

• Time: connecting and integrating service delivery and care for mothers and newborns from pre-pregnancy to delivery, the immediate postnatal period and the first weeks of a baby’s life.
• Place: connecting care giving and service provision at home, at the community level and in health facilities.

Figure 4: The WHO’s continuum of care for maternal, newborn and child health

Source: Reproductive, maternal, neonatal and child health continuum of care, fact sheet, The Partnership for Maternal, Newborn & Child Health, September 2011. Adapted from Make every mother and child count, WHO, 2005.

Linking services in this way helps women to get the care they need for themselves and their babies. For example, a mother may get a vital post-birth check while also receiving support with breastfeeding her baby. Services are also more efficient if provided this way, with a wider range of interventions supported. Importantly, if care for mother and baby is integrated, it can reduce the risk of stillbirth and support improvements in both maternal and neonatal health outcomes.

Uneven progress towards global targets for maternal health

Progress towards the maternal health targets from the Millennium Development Goals was slowest in parts of sub-Saharan Africa. Our two case study countries, the Democratic Republic of the Congo (DRC) and Malawi, were both assessed by the UN as having made “no progress” in reducing maternal mortality between 1990 and 2015. This term refers to countries where the reduction in the maternal mortality ratio was too low to suggest there had been any systemic change.

Figure 5: Maternal health from 2010 to 2015

While the steps involved in improving maternal health are well understood, their implementation in low-income countries and fragile states continues to be extremely challenging. Through our literature review and stakeholder consultations, a number of recurrent issues were highlighted:

• A lack of skilled health workers, including midwives and doctors trained to diagnose and treat conditions like pre-eclampsia and to deal with complicated deliveries and haemorrhaging.
• The security of the supply chain for drugs (for instance to treat high blood pressure or infections), family planning commodities (such as condoms, pills and injectables) and other supplies, which are frequently unavailable in health facilities.
• Poor quality of care and a lack of integration of services – enabling facilities to provide a range of services, with onward referral where necessary – particularly across the maternal-neonatal continuum (see Figure 4 above).
• Discrimination against and mistreatment of some pregnant women and girls within communities and health facilities (which can discourage girls and women from seeking care and increase the risks when they do).
• Barriers of distance and costs that prevent women from poor communities accessing health facilities, and the need to strengthen community health systems so that the right services can be delivered closer to women’s homes.

Data availability and quality also emerged as key barriers to progress. The lack of basic registration systems (births, deaths and marriages) and data on cause of death, as well as poor quality data from national health systems, make it difficult to monitor progress in improving maternal health. Data on maternal morbidity is even more limited, and the WHO is leading global efforts to improve the identification and measurement of maternal morbidity cases.

Since reliable data is hard to obtain, developing countries use surveys and modelling to estimate how many women are dying and why. Until recently, national demographic surveys collected data on maternal mortality by asking women whether their sisters had died during pregnancy or childbirth. The figures produced are subject to large confidence intervals, and the UN draws on a much wider range of data to generate country estimates every five years. The modelling and estimation process is overseen by a UN inter-agency group, and we use this data in this report.

DFID’s efforts to improve maternal health

Maternal health has always been a central concern for DFID. Reducing maternal mortality was highlighted as a focus area in the first DFID White Paper on international development, published in 1997 (the year DFID was established as a department), as was the need to improve basic health care and access to reproductive health services.

DFID published its first maternal health strategy in 2004. It stressed the importance of evidence-based intervention, advocacy, knowledge generation and addressing wider social and economic issues. DFID was an advocate for the Millennium Development Goal target on universal access to reproductive health, which was added in 2005. In a 2007 progress report, DFID noted that AIDS had become the leading cause of maternal death in parts of sub-Saharan Africa. It also flagged the need for long-term investment in health systems and human resources, and for increased efforts to improve sexual and reproductive health services.

During the Results Framework period (2011-15), the UK was a major global donor on health. In 2010, DFID accounted for 15% of worldwide donor expenditure on reproductive, maternal, newborn and child health across 27 high-priority countries. In Malawi the DFID share was 23% and in the DRC 8.7%. DFID’s health portfolio peaked at 24% of total UK aid in 2012, although this had fallen to 15% by 2015.

In 2010, G8 members agreed at the Muskoka Summit to intensify their efforts to meet the Millennium Development Goal on maternal health.34 They pledged to mobilise $5 billion of additional funding over five years, “with the objective of generating a greater collective effort by bilateral and multilateral donors, developing countries and other stakeholders” to accelerate progress on maternal, newborn and child health. They emphasised the need to strengthen developing country health systems to deliver “key interventions along the continuum of care”.

In December 2010, DFID published its Framework for Results for improving reproductive, maternal and newborn health in the developing world (the Results Framework). This set out what the UK would do to deliver on the G8 commitments and accelerate progress towards international goals. The Results Framework included a set of overarching targets for DFID’s work to improve reproductive, maternal and neonatal health. These included the headline target to save 50,000 women’s lives during pregnancy and childbirth by 2015, which represents approximately 4% of all maternal deaths that would have been anticipated between 2011 and 2015.

DFID’s target setting was informed by both the G8 commitments and the 2010 Bilateral Aid Review process, where DFID country offices were asked to estimate the impact of their ongoing and planned programmes on maternal death and to calculate how many lives DFID programming might save as a share of national forecasts.

DFID reported annually on its cumulative progress towards its targets. At the end of the Results Framework period, as more detail from country programmes emerged, DFID estimates were recalibrated and final results claims for 2011-15 were reported in the 2016 DFID Annual Report. The report announced that DFID had achieved more than double its targets for maternal lives saved, at 103,000, and for safe deliveries (using the presence of a skilled birth attendant as a proxy), at 5.6 million. It reported coming very close to achieving its family planning target, with 9.93 million additional users. The trajectory of progress against the three key targets related to maternal health can be seen in Table 2. The target for newborn lives saved is included for context, given the importance of the maternal-neonatal continuum of care.

Table 2: DFID’s reported progress against its Results Framework targets

In its Results Framework, DFID had committed to doubling its expenditure on women’s and children’s health between 2008 and 2012 and to sustaining this through to 2015. There is no way of separately identifying DFID expenditure on maternal health alone. However, using a G8 methodology that weights expenditure across different sectors according to their contribution to improving women’s and children’s health outcomes, we can estimate DFID’s level of investment in improving this broader set of outcomes. Figure 6 shows trends in relevant DFID expenditure across the four financial years covered by the Results Framework, 2011-12 through to 2014-15, and the two subsequent years.

DFID spent more than £4.6 billion on programming and global initiatives related to reproductive, maternal, neonatal and child health during the Results Framework period covering the financial years 2011-12 to 2014-15. Within this, £1.3 billion was spent on programmes more specifically related to family planning, reproductive health care and maternal and neonatal health. Figure 6 shows that several areas of expenditure grew dramatically during the Results Framework period and then dropped back to previous levels. Spending on family planning increased sharply initially and has been sustained.

Figure 6: DFID’s global expenditure on women’s and children’s health, 2011-17

Note: DFID has subsequently used the internationally-agreed FP2020 methodology (a variation of the G8 methodology) for reporting on family planning spending, which results in the figures above being lower than DFID’s more recent reporting against Family Planning Summit spending commitments.

Since 2015, DFID has reaffirmed its commitment to maternal health, with a further increased focus on family planning. The 2016 DFID Bilateral Development Review pledged to “help to end preventable maternal and child death, including among the poorest, most marginalised and hardest to reach”. The current Single Departmental Plan does not have numerical targets, but states that DFID will be “a global leader on sexual and reproductive health and rights, and boost our support for family planning for all who want it”. DFID now reports results for family planning rather than for maternal health more broadly.

The programming covered by this review

For this review, we assessed eight centrally managed programmes with combined budgets of £750 million (see Box 6 or Annex 1 for full details), which were operational during the Results Framework period. The programmes purchased health and family planning supplies, provided reproductive health services, trained health staff in obstetric care and promoted evidence and innovation for improved health outcomes.

We also explored DFID’s work during the Results Framework period to improve maternal and reproductive health in Malawi and the DRC (see Box 7 for more on the country contexts and portfolios or Annex 1 for full details). We reviewed nine programmes across the two countries. These programmes have combined lifetime budgets of £625 million, covering health sector support, family planning, water and sanitation, nutrition and girls’ empowerment.

Findings

Impact: How well has DFID maximised the impact of its programming on maternal health?

DFID developed a comprehensive strategy for achieving its maternal health objectives

DFID’s 2010 Results Framework set out a comprehensive strategy for improving maternal health, reflecting a broad international consensus on what works. 12 evidence papers were commissioned to inform the strategy and guide implementation, which were relevant and informative. The Results Framework pledged that DFID would align its portfolio behind four objectives:

1. Empowering women and girls to make healthy choices, including through access to family planning and other empowerment initiatives.
2. Removing barriers that prevent access to quality services, including financial barriers, through equitable health financing mechanisms and social assistance, effective referral systems, tackling stigma and discrimination and helping women in crisis situations.
3. Expanding the supply of quality services for women and newborns, including through stronger national health systems and community-level services, a renewed focus on the quality of care and strengthening both public and non-state services.
4. Enhancing accountability for results, including by improving data, mobilising communities to hold health service providers to account and making the international architecture for health support more transparent.

DFID’s proposed approach identified the need for action across a broad front including family planning, health systems strengthening and addressing the wider social and economic barriers that prevent poor women from accessing health services. It emphasised the importance of context-specific solutions. It stressed issues that had historically been neglected, such as adolescent sexual and reproductive health, women’s and girls’ empowerment and stillbirths and neonatal death. It also noted the importance of accurate data and putting in place the right accountability mechanisms at local, national and international levels.

While the DFID strategy was well formulated, mechanisms for translating it into programming were largely absent. DFID never developed an implementation plan for the Results Framework. Most of the programmes that were active during the 2011-15 period had been planned, and in some cases designed, before the Framework was developed. DFID Malawi mapped its programme portfolio against the Framework initially, to identify gaps, but we saw no evidence that the mix of programmes was adjusted in response. Only one of the centrally managed programme business cases we examined drew on the Framework to justify its proposed interventions.

Despite commitments by the then secretary of state, implementation of the Results Framework was not monitored. The Framework included proposed indicators in a monitoring and evaluation annex. However, the programme logframes we assessed included few if any of these indicators. Most did not disaggregate outcomes for the Results Framework’s two priority groups – the poorest 40% and young women aged 15 to 19 – even though this was recommended in the Framework. As a result, the results areas set out in the Framework were not monitored at either programme or global level. This was noted in a 2013 mid-term review of the Results Framework, which recommended a more systematic tracking system, similar to the one DFID had set up for its parallel results framework for malaria. However, the recommendation was never actioned.

Over the results period, DFID significantly increased its focus on family planning

Providing women with access to family planning is an important part of improving maternal health. It helps women to manage the number of pregnancies they have and to increase the amount of time between pregnancies, both of which lower the risks of illness or death from pregnancy and childbirth. It also serves a range of other objectives. It helps to lower fertility rates, which DFID’s business cases argue is important for economic growth. It also helps to strengthen reproductive rights and enhance gender equality – both important objectives for DFID. However, our interest in this review is on the contribution of family planning efforts to improving maternal health.

Early in the review period, DFID decided to scale up its efforts on family planning, both in its programming and in its global advocacy. A new ministerial team arrived in 2010 with a keen interest in family planning. DFID partnered with the Bill & Melinda Gates Foundation to host the first London Summit on Family Planning in July 2012, where new commitments were made by a wide range of governments, international agencies and other organisations. At that event, the UK made an additional financial commitment of £516 million through to 2020 for family planning activities, doubling DFID’s annual expenditure in this area compared to that at the beginning of the Results Framework period.

In 2012-13, DFID launched several new programmes with a strong focus on family planning, including:

• A £356 million contribution to UNFPA for the purchase of family planning commodities, such as contraceptive pills and implants.
• A £75 million contribution to the short-term Reproductive, Maternal, Neonatal and Child Health Trust Fund, managed by UNICEF and UNFPA, designed to increase access to 13 lifesaving commodities (such as contraceptive implants, emergency contraceptives and oxytocin to prevent post-partum haemorrhage) and to strengthen related services (for example by providing job aids and checklists for health workers).
• An expansion of the Prevention of Maternal Deaths from Unwanted Pregnancy programme, which supported sexual and reproductive health services, policy and advocacy across 14 countries in Asia and Africa, with an increased budget of £139 million and greater focus on family planning (having previously had a stronger emphasis on safe abortion and post-abortion care).
• Within the case study countries for this review, DFID agreed a new investment of £27 million to scale up family planning programming in Malawi.

DFID programmes have expanded access to family planning but ensuring a regular supply of commodities remains a challenge

DFID’s significant investments in family planning have played an important role in expanding access to sexual and reproductive health services for women. By the end of the Results Framework period, DFID reported that it had enabled almost 10 million additional women to access modern family planning methods. The Prevention of Maternal Deaths from Unwanted Pregnancy programme, for example, met or exceeded all its targets for service provision (its outputs), particularly for family planning. In the first year of the programme, more than 2.4 million services (such as contraceptive advice or the insertion of contraceptive implants) were provided, against a target of just over 1.4 million. In the fifth year, the programme provided over 4.4 million family planning services. The programme also reported its impact on public health (its outcomes). It modelled these outcomes based on its own output data. As a result, DFID also registered all outcome targets as exceeded. For example, in its fifth year, the programme estimated it had averted 6,644 maternal deaths against a target of 5,483 and prevented 1,678,731 unintended pregnancies against a target of 1,524,867.

DFID support to UNFPA – and to UNFPA’s global supplies programme – has been particularly important, especially since the United States government stopped funding UNFPA in 2017. The UNFPA Supplies programme purchases family planning and other commodities for sexual, reproductive and maternal health services and provides related capacity building (for example support to improve forecasting and procurement systems). In 2015 – the last year covered by the Results Framework – the UK contribution accounted for 72% of programme funding, although by 2017 the UK share had dropped to 42%. The programme has successfully advocated for 20 of its 46 partner countries to offer women a choice of at least three family planning methods through their public health systems. It has also succeeded in boosting demand for family planning. However, the partner countries have struggled to ensure that the necessary supplies are available at health clinics. In 2015, only seven partner countries were able to confirm a regular supply of contraceptives in at least 60% of their facilities. Globally, supply chain issues have proved a significant barrier to expanding family planning services in developing countries.

At the country level, DFID succeeded in expanding access to family planning services in both Malawi and the Democratic Republic of the Congo (DRC) during the Results Framework period, but with mixed performance against its targets:

• In Malawi, DFID supported two family planning programmes during the review period. The first programme (2009-15, £11.8 million) reported that it enabled 271,601 additional women to use modern family planning methods (21,509 aged 15 to 19 and 250,092 over 20, against targets of 32,633 and 433,554 respectively). The second programme (2011-18, £27.3 million) reported reaching 12,758 additional women aged 15 to 19 and 124,450 over 20 by the end of 2015.53
• In the DRC, DFID’s Access to Health Care programme exceeded its goals for provision of family planning services during the Results Framework period, reporting 143% delivery against its March 2016 target for new adopters of modern family planning methods. More recently, the programme has expanded into community-based delivery of family planning services, to improve access and uptake.

Although access to family planning services has definitely increased, it can be hard to determine exactly how many ‘additional’ or ‘new’ women are actually accessing modern family planning methods as a result of these programmes. For example, at the health centres we visited in the DRC and Malawi, we saw or heard about several challenges including poor record keeping, all clients being recorded as ‘new’, and clients being double-counted and given a new record if their name is misspelled or if they move to a new provider, even though they are already using family planning. Such issues appeared to be fairly common. For the Malawi Family Planning Programme, an electronic registration system was introduced to reduce over-reporting.

DFID has made significant investments in supply chain management and used a 2017 London Summit on Family Planning to galvanise international action on the issue. However, ensuring a reliable supply of commodities in health facilities remains a difficult challenge. In the DRC, key informants and communities told us that essential supplies, such as condoms and pills, were often unavailable at the local facilities supported by DFID. This does not necessarily reflect overall stock levels within the programme, but rather systemic weaknesses in forecasting, ordering, warehousing and distribution. In practice, it means that many women and girls are still unable to access their chosen family planning method when they need it.

DFID’s maternal health programming has expanded service coverage but made limited progress on improving service quality

Over the review period, DFID continued to work on strengthening maternal health services. In our sample, there were three centrally managed programmes working on aspects of maternal health care:

• Making It Happen used expert volunteers from the UK to train health workers in obstetric and neonatal care. It worked under the direction of national governments that chose where and how the programme would operate, with mixed progress across countries.
• Evidence for Action to Reduce Maternal and Neonatal Mortality focused on strengthening local accountability for maternal health care and outcomes (see Box 9). However, the programme was scaled back significantly during the review period, which according to stakeholders reflected a shift in DFID’s priorities away from this area.
• The UNFPA Supplies programme covered the supply of some medicines that are essential for maternal health, such as magnesium sulphate to treat pre-eclampsia and eclampsia, in addition to the family planning commodities noted earlier. However, the supply chain challenges were even more severe than for family planning. By the end of the Results Framework period, none of the 46 countries supported by the programme were able to confirm the availability of key maternal health medicines in all relevant facilities (against a 2015 target of 10 out of 46 countries). Without such supplies, health workers cannot manage many of the risks or complications that women experience during pregnancy and childbirth.

In the DRC and Malawi, DFID’s investments in strengthening health systems helped to increase access to basic health services, including some that are essential for maternal health. In the DRC, from a low base and in the face of a very challenging operating environment, both the Access to Health Care and Support to Malaria Control programmes reported impressive coverage of services such as intermittent preventative treatment for malaria during pregnancy, bed net distribution and skilled birth attendance. In Malawi, DFID’s Health Sector Support Programme has helped to prevent a serious decline in basic services, following a decision by DFID and other donors to discontinue budget support for the health sector after a major corruption scandal (see Box 7). One element of the programme pays for maternal and neonatal health services provided by faith-based organisations under contract from the Ministry of Health, which includes services in remote areas. Most of the programme, however, sustains day-to-day operations rather than expanding services.

Expanding service coverage (numbers of women reached) is necessary but not sufficient for improving maternal health outcomes. It needs to be accompanied by measures to ensure that the services are of sufficient quality to make a difference. In both countries widespread problems remain with service quality and with reaching high-risk groups. For example, a woman giving birth in an official health facility is generally counted in the statistics as having had a ‘skilled birth attendant’ or a ‘safe delivery’. In Malawi, however, a legal requirement for all births to take place in health facilities has placed huge pressure on hospitals and health services. A cap on public sector recruitment has contributed to severe shortages of medical personnel, with the result that not all births in facilities are attended by a nurse, doctor or midwife. High-risk cases or those needing emergency care can be missed. Even where skilled personnel are present, they can lack the equipment or supplies needed to support safe delivery. During our case study visit to Malawi, it was clear that maternity wards were under considerable strain, with large numbers of pregnant women queuing outside for a bed. As a result, the increase in health facility-based delivery has not necessarily translated into improved maternal health outcomes. The limited data that is available from facilities across the country indicates that many women are still dying from basic obstetric complications, such as severe bleeding or infection.

DFID support for improving emergency obstetric and neonatal care has generally been provided through wider health sector programmes. The Malawi Health Sector Support Programme has made slow progress in this area, with only 15% of facilities able to provide basic emergency care against a target of 30% (in 2016-17; no figures were available for 2015-16). The DFID DRC Access to Health Care programme, which supports health systems strengthening in five provinces, has worked to build health facility-level capacity to provide emergency care. It got off to a slow start during the review period. By March 2016, only one out of a planned 146 DFID-supported health facilities was able to provide core elements of basic emergency obstetric and neonatal care, in the face of severe challenges with infrastructure, personnel and supplies that were evident during our visit. Progress has since picked up: in 2017, according to DFID’s Annual Review, 143 facilities were able to offer the basic level of emergency care, against a target of 177, although there is as yet no data on whether and how outcomes for women may have improved.

Not all improvements in coverage are adequately captured by programme logframes. The DFID DRC Access to Health Care programme has made significant progress in providing surgery to women with injuries sustained during childbirth (obstetric fistula), a consequence of the lack of emergency obstetric care. This intervention is provided by mobile surgical teams who also train local staff in the procedure. This work on obstetric fistula is not captured in the programme logframe, but is being tracked by programme implementers, with 2,848 women having undergone successful surgery to date.

We conclude that DFID’s maternal health work in our case study countries, and more broadly through its centrally managed programmes, has helped to expand the coverage of some aspects of maternal health care. However, in the face of challenging national contexts, the programmes have not yet succeeded in improving access to quality health services on a scale that would make a significant difference to pregnant women and their newborn babies.

The scaling up of DFID’s family planning efforts has resulted in an unbalanced approach to improving maternal health

The Results Framework made a clear case for an approach to maternal health that balanced a range of different interventions – improving access to quality health care and family planning, addressing barriers to the uptake of services, empowering women and girls and addressing environmental factors, such as nutrition and access to water and sanitation – in order to work towards the medium-term objective of reducing maternal mortality. It stressed that there is “no single fix for saving the lives of women and babies during pregnancy and childbirth, [which is] one important reason why progress has been slow”.

Several DFID staff noted that the recent emphasis on family planning had generated significant short-term results, but at the expense of a more balanced approach to building up health systems. The emphasis on family planning may also have been reinforced by the good returns on investment offered by family planning and their relatively low unit costs, which were cited in DFID business cases.

While family planning is a vital, cost-effective maternal health intervention, there are recognised limits to the contribution it can make to reducing maternal mortality. The majority of maternal deaths result from intentional pregnancies, rather than unintended ones, and are therefore not prevented by access to contraception. Recent analysis suggests that satisfying 100% of the unmet need for contraception in developing countries would reduce maternal mortality by no more than 25%. Evidence from key informants, including through the expert roundtable conducted for this review, indicates that as maternal mortality rates fall, it becomes harder to secure further reductions through family planning interventions that offer relatively ‘quick wins’. Continued progress calls for long-term investments in health system quality and access.

Bringing down the numbers of unintended pregnancies will make an important difference. But to save lives, women also need quality care during pregnancy, birth and in the critical first days after birth, including emergency care. Changes are needed in both the supply and in tackling the wider barriers to the uptake of services.

Results Framework, DFID, 2010, p. 14.

While DFID’s contribution on family planning has been important, its portfolio has not reflected the multidimensional approach set out in the Results Framework. In the case study countries, there has been expansion in relevant health services, but these efforts have not been anchored in convincing medium-term strategies for strengthening national health systems, improving the quality of care or overcoming the barriers that poor women face in accessing services. We therefore conclude that DFID has not maximised the potential contribution of its programming to global objectives to reduce maternal mortality in the medium- and long-term.

Unrealistic assumptions behind modelled outcomes throw into question some of DFID’s results claims

Most development agencies, including DFID, estimate maternal health results through modelling, rather than measuring them directly, due to the lack of public health data in developing countries. DFID assumes that the provision of a certain intervention (for example family planning services or ‘skilled birth attendance’) will translate into maternal lives saved at a given ratio. Outcome-level results are therefore generated mathematically from output-level results: if DFID programmes deliver their planned interventions, the desired outcomes are presumed to follow.

To measure progress during the Results Framework period, DFID adopted a modelling approach – the Lives Saved Tool (LiST) – that could estimate ‘maternal lives saved’ from a wide range of interventions, including in health, family planning and other sectors, such as water, sanitation and hygiene. The tool and how DFID uses it are described in Annex 2. The first LiST analysis of DFID’s maternal health portfolio in 2011 indicated that DFID would be unlikely to reach its target of saving 50,000 women’s lives without including the lives saved by avoiding unintended pregnancies through family planning. This conclusion was noted in the mid-term review of the Results Framework.

Like all models, LiST is based on assumptions. We found that some of these assumptions were not realistic for the programmes in our case study countries. For example, the model assumes that the same number of lives are saved through family planning whichever women are using the services, rather than taking into account different factors, such as age. Many programmes measure their results in terms of ‘couple-years of protection’ provided. Using this measure, the greatest results are achieved by offering long-acting and permanent methods of contraception, which are also cost-effective to deliver. Data from DFID programme monitoring and from local key informants suggests that the majority of users of DFID-supported family planning services in Malawi during the review period were women aged 25 or older. Many of these older women already had several children and opted for sterilisation. Indeed, DFID programmes delivered by the non-governmental organisation Banja La Mtsogolo saw more women opt for sterilisation than for contraceptive implants each year until 2015. Sterilisation saves lives among older women, as it enables them to limit their family size and avoid the health risks from further pregnancy and childbirth, which generally increase after the age of 30. However, the impact on maternal mortality rates at the population level is less than if the programmes were helping younger women to have fewer children and better spaced births – a key objective of DFID’s Results Framework. As a result, it seems likely that DFID’s modelled results for 2011- 15 overestimated the impact of its family planning programmes on maternal mortality in Malawi (see Box 8 for more detail on this).

Based on its LiST modelling, DFID estimated that its programmes had saved 103,000 maternal lives during the Results Framework period 2011-15, exceeding its 50,000 target. This results claim was published in DFID’s Annual Report for 2015-16, but unpublished internal analysis subsequently revised that total downward to 80,100 when better data from Bangladesh and Ethiopia became available.

In its 2015-16 Annual Report, DFID indicated that final estimates would be published in 2017, but no such update was provided. Around 75% of the total estimated ‘maternal lives saved’ by DFID bilateral programmes related to family planning interventions (see Figure 7 below). The country-level results estimates generated by the LiST model were very different to the original forecasts from the Bilateral Aid Review process, which had been used to set the targets (see para. 3.19 above). For example, LiST estimated that DFID programmes had saved 10,100 maternal lives in Malawi, whereas the original forecast was 387; the LiST estimate for the DRC was 4,800 against a forecast of 500.

Figure 7: Breakdown of estimated maternal lives saved, 2011-15

Source: Lives Saved Analysis for the Department of International Development (DFID) Year 5: Lives Saved 2011-2015, Friberg, I., Johns Hopkins Bloomberg School of Public Health & Norwegian Institute of Public Health, 2017.

In our case study countries, we were unable to reconcile the outcomes modelled through the LiST tool with the more detailed results data generated by individual programmes. In Malawi, the claim of 10,100 maternal lives saved between 2011 and 2015 is very likely too large, for the reasons set out in Box 8. In the DRC, DFID reports that it saved 4,800 maternal lives between 2011 and 2015, mainly by improving the management of labour and delivery and through improvements in emergency obstetric care – the strongest reported performance from such interventions of any country programme. The LiST modelling suggests most of these results were achieved after 2013, when the £185 million Access to Health Care programme began, and derive from the increase in skilled birth attendance enabled through that programme. LiST uses regional data on the causes of maternal death and default assumptions around how many women can access potentially life-saving emergency care: for example, in a country with 80% of women delivering their babies in a health facility (such as the DRC), LiST assumes a quality of care that would enable 48% of these women to access comprehensive emergency care, 12% to access basic emergency care and 20% of women to access only basic services. These proportions do not match current reality for women in the DRC. The DFID Access to Health Care programme reported that only one of its supported facilities was able to provide a basic level of emergency obstetric care by 2015 (as noted in paragraph 4.16). Progress with the construction and rehabilitation of health facilities also lagged behind schedule during the period covered by the results claim. We have therefore not been able to corroborate the DRC share of DFID’s global results claim for 2011-15 using the more detailed results data generated by the relevant programme.

DFID missed opportunities to improve the availability and quality of data and to increase the accountability of service providers

The reliance on modelled results is linked to poor data availability and quality in the case study countries, making it difficult to accurately track improvements (or other trends) in maternal health outcomes. Long-term investment in data collection and management information systems is needed to address the issue.

We find that DFID has made some direct investments in improving maternal health data. Evidence for Action was the only centrally managed programme we assessed with a strong focus on this. However, in Malawi, there had been little planning for the ongoing collection and use of data as that programme drew to a close. Alongside this, the DFID Malawi Health Sector Support Programme supported district planning and management functions, including data improvement, supervision visits, data verification and improving the link between data and response. It has also begun to support the rollout of the Maternal Death Surveillance and Response approach, which is part of a global initiative by the World Health Organization (WHO) and other partners, including DFID, to generate better data on the extent and causes of maternal mortality. More information about these programmes and how they complement each other in Malawi is set out in Box 9.

DFID has made significant investments in health management information systems in the DRC through its Access to Health Care and Support to Malaria Control programmes. DFID-supported health facilities are also reporting maternal deaths in accordance with national policy,76 although they have not yet implemented full Maternal Death Surveillance and Response systems. The Access to Health Care programme has supported the development of community-based committees, which use scorecards to assess facilities and strengthen local accountability for improvement. This qualitative feedback could have been used by DFID to monitor changes in the quality of services in the facilities it supports, but we did not see evidence of this.

DFID has made some significant investments in strengthening civil society networks and coalitions on issues related to maternal health, such as those supported through the Prevention of Maternal Deaths from Unwanted Pregnancy programme and in the DRC through the girls’ empowerment programme, La Pépinière. However, we saw little evidence that local civil society networks were being drawn on to inform DFID’s thinking or to provide qualitative data about outcomes for women and girls.

Conclusions on impact

In its 2010 Results Framework, DFID set out a clear and evidence-based strategy for improving maternal health, with a global target of saving 50,000 women’s lives during pregnancy and childbirth by 2015. The strategy emphasised the importance of a balance of interventions, including family planning, improving the supply of quality health services and tackling the barriers that prevent women and girls from accessing them.

Over the review period, there was a significant expansion of DFID’s efforts on family planning, helping to redress past neglect of this area in the maternal health field. DFID has been an effective global champion of family planning and reproductive rights. Its programmes successfully increased access to family planning, although (alongside other agencies) DFID has found it challenging to ensure a reliable supply of contraceptives to local health facilities.

DFID also made significant investments in health care interventions, helping to expand access to basic services for pregnant women, although this did not receive the same intensity of effort as family planning. DFID has expanded access to a number of relevant services, but has not necessarily improved their quality on a large enough scale to make a significant difference to maternal health.

We find that the intensive focus on family planning, while valuable in its own right, has left the wider maternal health portfolio without a balanced approach across the different interventions that are needed to achieve significant reductions in maternal mortality over the medium- to longterm. Furthermore, we identified a number of shortcomings in the quality of maternal health and family planning services offered. This makes it likely that DFID has not maximised the impact of its programming.

We were unable to reconcile DFID’s global results claims on maternal lives saved with country data and programme-level results in our sample. In the absence of reliable health data, the models that DFID used to estimate its results rested on assumptions that were not entirely appropriate in those country contexts and produced some numbers that appear too high for the period in question. Furthermore, we find that DFID has not yet done enough to fill data gaps, to allow for better results measurement and greater accountability.

We acknowledge DFID’s genuine commitment to improving maternal health and particularly its ambitious commitment to expanding family planning. But we also find that over the review period the impact of DFID’s work on maternal health outcomes was less than it should have been and, at the programme level, we were unable to verify its reported results on maternal lives saved. We have therefore awarded DFID an amber-red score for impact.

Our findings are specific to our review period of 2011 to 2015. We note the likelihood that some of DFID’s investments over this period – in particular, its contributions to the long-term process of strengthening health systems and to the uptake of long-acting and permanent methods of contraception – will have continued to generate results after 2015. We also note that DFID has taken steps since 2015 to address some of the shortcomings identified here, including intensifying its support to improve emergency obstetric care in the DRC.

Equity: How well do DFID’s maternal health and family planning programmes reflect its commitments to equity and inclusion?

DFID policies prioritised reaching poor and young women, but this focus was not sustained through programming

In its Results Framework, DFID prioritised reaching the poorest 40% of women, as well as young women aged 15 to 19, through its programming related to maternal health. For younger women, the focus was on expanding access to contraception, both to improve outcomes for them individually and to slow population growth. Within its overall target to enable at least 10 million more women to access modern family planning methods, DFID aimed to help “up to 1 million young women aged 15-19 to access family planning with action for adolescent girls that enable[s] them to delay their first pregnancy.”

DFID staff told us that these priorities have expanded more recently to include other hard-to-reach or marginalised groups, such as women with disabilities and those in areas affected by conflict or natural disasters. After the adoption of the Sustainable Development Goals in 2015, DFID generalised this commitment to equity and inclusion through its pledge to “leave no one behind”.

During implementation, most DFID programmes in our sample worked to improve the accessibility and affordability of reproductive and maternal health services, potentially helping poorer women and girls to access health care. All but one of the centrally managed programmes had a strong focus on this. For example, at the policy level, the Evidence for Action programme worked to improve health budget transparency and resource allocation and the Reproductive, Maternal, Neonatal and Child Health Trust Fund included work to make the prices of contraceptive supplies more affordable. In the DRC, DFID’s implementing partners for the current Access to Health Care programme agreed low fees for antenatal care and delivery services with communities, although it was not always able to ensure that health facilities adhered to these fees. During our case study visit, we were given variable information about fees from facility staff and from local people in the communities they serve and we heard some women were being charged additional fees for antenatal care.

Apart from this general emphasis on affordability and accessibility, few of the programmes in our sample reflected DFID’s policy priority to target poor and young women. Only three of the eight centrally managed programmes we assessed had explicitly prioritised poor and young – or otherwise marginalised – women and girls in their original programme documentation:

• The ongoing UNFPA Supplies programme explicitly targeted the poorest and marginalised women and girls, with a strong focus on reaching women with sexual and reproductive health commodities in areas affected by conflict or natural disasters and with a logframe indicator dedicated to tracking this.
• The Prevention of Maternal Deaths from Unwanted Pregnancy programme identified “poor women and adolescent girls” as priorities from the outset and tendered for an implementing partner to develop a fuller programme design (and then deliver it) on this basis.
• The Safe Abortion Action Fund identified “hard-to-reach and marginalised women” as a priority, and this was reflected as a key criterion used by the fund’s secretariat and technical reference body to decide which projects to support.

At country level, DFID programmes often focused on geographical areas with high levels of poverty. In Malawi, both the centrally managed Evidence for Action programme and the in-country Malawi Health Sector Support Programme focused on districts with poor and rural populations, where health infrastructure is weak. In the DRC, the DFID office had identified priority provinces for all its programming. The Access to Health Care programme covered five of the country’s 26 provinces. In two, DFID was the only significant bilateral donor to the health sector, and two were also newly affected by conflict over the course of the programme.

A small number of programme business cases also set out specific mechanisms for reaching priority groups. The business case for the Prevention of Maternal Deaths from Unwanted Pregnancy programme identified mechanisms for targeting policy, advocacy and service delivery towards poor women and remote communities. During implementation, the programme collected socio-economic data about its clients and piloted the targeting of services in poorer areas in Kenya and Burkina Faso. At country level, DFID Malawi’s family planning programmes identified outreach services as the key mechanism for reaching poor women and remote communities. Data from the implementing partner’s client information system, and captured in the programme logframe, suggests large numbers of rural women have been reached through this approach. In most cases, however, the business cases of programmes in our sample failed to specify how they would reach young, poor and marginalised women.

In our interviews and at the expert roundtable, key informants stressed the need for flexible programme design, to respond to context-specific social and cultural issues and find ways to overcome the barriers that prevent women and girls from accessing maternal health care. In our case study countries we saw limited evidence that DFID’s programme implementers had taken the time to engage with local stakeholders – such as communities, traditional and religious leaders and local civil society organisations – to understand these barriers. They were also not working with adolescent girls to understand their needs and behaviours, despite them being a key target group. The Evidence for Action programme, through its MamaYe campaign, was an exception, but it operated at a relatively small scale and its positive approaches had not been systematised before the DFID programme was closed (see Box 9). More promisingly, DFID Malawi has undertaken wider consultation to inform the design of its new sexual and reproductive health programme, which will have a component focused on youth services. This consultation used new methods such as radio phone-ins and captured views from young people, as well as from traditional and religious leaders.

Concerns about discrimination against pregnant women and girls based on age, marital status and socio-economic status were raised in both the DRC and Malawi. These forms of discrimination can lead to some women being neglected or even verbally or physically abused within health facilities and in their communities, denying them quality of care and putting them and their babies at greater risk. This issue was also highlighted in the 2014 ICAI review of DFID’s contribution to the reduction of child mortality in Kenya. Our literature review, document review and key informant interviews all identified respectful care as central to improving the overall quality of maternal and neonatal health care. However, issues of respect and discrimination were not explicitly addressed by any of the DFID programmes we examined.

DFID did not track its progress in reaching poor and young women

The Results Framework included two results indicators that needed to be disaggregated in order to identify whether DFID was reaching its target groups:

• The number of babies delivered with ‘skilled attendance’ was to be tracked not just overall, but also specifically for the poorest 40%.
• The change in the contraceptive prevalence rate was to be tracked for all women of reproductive age and specifically for the poorest 40% and for young women aged 15 to 19.

However, disaggregated results were not collected systematically across DFID’s programme portfolio, contrary to the monitoring and evaluation requirements set out in the Results Framework. Nor did DFID disaggregate its higher-level estimates of maternal lives saved when it reported its progress against the targets in the Results Framework. The aim of reaching 1 million young women aged 15 to 19 with family planning was not reported on at all.

The 2013 mid-term review of the Results Framework found significant weaknesses in DFID’s tracking of progress against all targets, apart from the targets related to increasing aggregate expenditure. It noted that most programme-level data was not disaggregated, so it was hard to tell if services or interventions were reaching the poorest, young or vulnerable, including those in conflict- or disaster-affected areas.

Of the centrally managed programmes we assessed, only UNFPA Supplies had an equity-related indicator in its original logframe. This tracked the number of women and girls reached in settings affected by conflict or natural disasters. We found evidence that this indicator was monitored and led to changes in intervention design. DFID opted not to set targets for reaching the poorest or young women aged 15 to 19 for its Prevention of Maternal Deaths from Unwanted Pregnancy programme, to “avoid perverse incentives” and also due to data availability and quality issues. However, DFID became more concerned about outcomes for young women over the course of the programme, which led it to start tracking results for “high impact” users and to add a general progress indicator for young women – albeit at a very late stage of implementation.

DFID attempted to track progress in reaching poor and young women through some of its country programmes. In Malawi, family planning programme logframes tracked the contraceptive prevalence rate for all women, rural women and the poorest 40% and, from 2013, specifically for young women aged 15 to 19 and 20 to 24, in line with the Results Framework. Data from national surveys was used and results exceeded the expectations set out in the logframes in the two years for which data was available. The logframes for the Malawi Health Sector Support Programme and for both of the DRC Access to Health Care programmes (2008-13 and 2012-18) monitored skilled birth attendance rates (the proportion of births where a skilled professional was present) – another area targeted by the Results Framework, including for the poorest 40% specifically – but they did not disaggregate any outcome data.

DFID had planned to undertake a final evaluation of the Results Framework that would assess whether programming was reaching the poorest women. However, it opted not to proceed with the evaluation when the experience of the mid-term review and a subsequent assessment suggested it would not be technically feasible to do so. DFID instead commissioned an internal “Lessons Learned” paper, which focused on identifying lessons to inform future programming, policy and research.

Over the Results Framework period, DFID had limited success in reaching adolescent girls

We found that DFID’s family planning interventions in both Malawi and the DRC have struggled to reach young people. Both the mid-term review of the Results Framework and the “Lessons Learned” paper referred to above suggest that this is common across DFID country programmes. The key findings of the “Lessons Learned” paper that relate to young women are set out in Box 10.

We found that the Prevention of Maternal Deaths from Unwanted Pregnancy programme had mixed success in reaching young people across the different countries supported by the programme. For example, during our case study visit, the programme delivery partner in the DRC reported success in reaching young people through social media, educational institutions and female outreach workers who visit villages and camps. Equity reports developed by the programme indicate that bespoke adolescent-friendly services have been developed in some other countries, but at a high cost. Overall, however, the 2016 DFID Annual Review of the programme judged that there was considerable room for improvement in understanding and reaching young people.

In Malawi, the importance of including younger adolescents aged 10 to 14 in family planning interventions was specifically highlighted by key informants to this review, as some girls are exposed to early sexual activity despite a recent increase in the legal age of marriage to 18. DFID’s Results Framework had identified the 15 to 19 age group as a key target but not those aged 10 to 14.

DFID’s Results Framework acknowledged the importance of working with men and boys, for their benefit as well as that of women and girls, to influence social norms and behaviours and to enable all people to make informed choices about their sexual and reproductive health. In Malawi, organisations working with young people stressed the need to encourage condom use by young men to protect young women against HIV and other sexually transmitted infections, as well as unwanted pregnancy, while also encouraging young women to choose a method of contraception that they control, such as an implant or the pill. However, there was very little reference to adolescent boys in any of the centrally managed or country-level programme documentation we assessed for this review, although the 2016 Annual Review of the Malawi Family Planning Programme flagged the need to include men and boys in future programming. DFID policy and country office staff acknowledged in discussion that new approaches will be needed to better serve young people, and that any future work on adolescent sexual and reproductive health needs to include boys as well as girls.

Young women and girls who are pregnant face additional risks. Younger mothers may experience complications during pregnancy and delivery due to their physical immaturity or inability to access care, and they may also suffer a lack of support from their families and communities. Key informants in the DRC, including young women engaged in the Girl-Led Research Unit supported by DFID’s La Pépinière programme, cited the neglect of and discrimination against unmarried women and girls by health workers as a key threat to their health and wellbeing. As noted in paragraph 4.47, this may escalate to verbal or even physical abuse by health workers, presenting clear risks to both maternal and newborn health. All these factors can contribute to a greater risk of death or illness for teenage mothers than for those in their 20s. In the programmes we assessed for this review, DFID was not giving enough attention to these issues, although some facility-level health advisory committees in Malawi (supported through technical assistance under the DFID Malawi Health Sector Support Programme) were working with communities to identify young women at risk.

DFID’s global influencing and programme activity has included a strong focus on the rights of women and girls

DFID plays an important advocacy role internationally on issues related to maternal health. This has included speaking out about harmful practices, advocating for safe abortion and related women’s rights and defending a UNFPA mandate with a strong focus on women’s sexual and reproductive health and rights. In 2012 and 2017 the UK hosted global summits on family planning, which were particularly effective in advancing this agenda and securing new commitments of funding and action from donor and developing country governments, foundations, civil society and the private sector. Within the global family planning initiative FP2020, DFID has pushed for a focus on rights, empowerment and equity. Among key informants to this review, DFID was widely perceived as a strong global advocate for the rights of women and girls, and as an essential counterbalance to the more conservative or regressive voices heard in international arenas.

This rights-based approach has translated into some of DFID’s centrally managed programmes. For example, the Prevention of Maternal Deaths from Unwanted Pregnancy programme combined service delivery with advocacy on sexual and reproductive health and rights and it supported legal and policy reforms through the provision of technical assistance to local organisations. The Safe Abortion Action Fund (to which DFID contributed through its Reducing Maternal Mortality programme) prioritised awarding grants to initiatives aimed at redressing gender inequalities and power relations that affect women’s sexual and reproductive rights.

DFID should have a stronger focus on the empowerment of women and girls, and advancement of their rights, in the communities it supports

While its international advocacy activities have been considerable, DFID has not taken advantage of the opportunities created by its country programming to advance the rights and empowerment of women and girls or work proactively with traditional and religious leaders on social norms, harmful practices and other issues that relate to maternal health. We saw missed opportunities where DFID could have done this by building on existing community engagement activities, such as its work with community health committees through its Access to Health Care programme in the DRC and through the MamaYe campaign in Malawi and other countries. The DFID DRC Access to Health Care programme did engage in some other innovative ways, for example by promoting girls’ education and empowerment through community film screenings and by working with local police on issues related to sexual violence.

We did not assess DFID programmes aimed at tackling violence against women and girls, but these could also contribute to maternal health objectives – for example by targeting the reduction of the number of unwanted pregnancies due to rape or by addressing harmful cultural practices such as the forced sexual initiation of pubescent girls. An ICAI learning review in 2016 on violence against women and girls recommended that DFID scale up its positive work in this area by integrating it into programming across a range of sectors for transformative impact.

DFID DRC’s La Pépinière programme generated useful evidence about girls’ empowerment. La Pépinière originally focused on economic empowerment but evolved to include a focus on girls’ and young women’s rights and social empowerment. It established a Girl-Led Research Unit, which helped girls and young women undertake research in communities in the capital, Kinshasa. This included a study of sexual behaviour and practices, which provided important insights into the realities faced by young women in the city. We saw little evidence that resources generated by La Pépinière had yet been used to foster a stronger focus on women and girls aged 15 to 19 in DFID’s health programmes in the DRC, although the final evidence papers from the programme and the sexual behaviour study were only published in 2017. We welcome the fact that DFID DRC is now using these findings, as well as the gender audit of the Access to Health Care programme (which was conducted through La Pépinière), to shape new health programmes.

Conclusions on equity

DFID’s policy and programme documents prioritise reaching the poorest 40% of women, and young women aged 15 to 19, through maternal health and family planning interventions. Some effort has been made to improve access to services for these groups but, as a whole, the programmes we reviewed did not prioritise poor, young and marginalised women beyond having a geographical focus on poorer, often rural, communities. Discrimination and other factors that put poor and young women at higher risk of ill health and death during pregnancy and childbirth were given insufficient attention.

Only a few programmes identified specific mechanisms or set targets for reaching the poorest 40% or young women aged 15 to 19. Furthermore, very few programmes disaggregated their results, making it impossible to determine the impact of DFID programming on poor, young or otherwise hard-to-reach women and girls.

DFID did not monitor whether it reached adolescent girls (or boys) with its family planning and maternal health programming during the Results Framework period. However, at the 2017 London Family Planning Summit, DFID committed to the collection, reporting and use of age-disaggregated data for its own sexual and reproductive health programmes, and advocated for others to do the same.90 DFID has acknowledged the need to do more, and to do things differently, to reach married and unmarried adolescents of all ages – including 10- to 14-year-olds, who were not targeted in the 2011-15 Results Framework or associated programming.

DFID has been a strong advocate for women’s and girls’ rights internationally. However, it could do more to reinforce this at community level in focus countries, including through working more proactively with civil society and traditional leaders.

As a result of DFID’s limited translation of its commitment to reaching the poorest, youngest and most vulnerable women into effective programming strategies and monitoring arrangements, we have awarded an amber-red score for equity.

Sustainability: Are DFID’s maternal health results likely to prove sustainable?

There has been long-standing DFID investment in health infrastructure and capacity building, but the benefits for maternal health have been slow to materialise

Sustainability – or the likelihood that results will persist beyond the life of the programme – is a principle of good aid practice and one of the standard criteria used in programme evaluations. In contexts such as the DRC and Malawi, which are very poor countries with deep-seated governance challenges, we would not necessarily expect DFID’s efforts to expand maternal health services to be financially sustainable in the short term. However, we are concerned with whether DFID has a viable approach to building national health system infrastructure and supporting policy change that will, in due course, provide a basis for ensuring that good quality maternal health services are sustainable and accessible to all women and girls.

The DRC and Malawi are challenging environments for the development of health infrastructure and for sustainable capacity building. Both countries have large rural and hard-to-reach populations, and in the DRC continuing insecurity hampers service delivery. In both countries, the public health system depends upon non-state providers – including non-governmental organisations, faith-based providers and the private sector – to deliver health services in some areas. DFID has often chosen to scale up service delivery through these non-state channels, which can be a more reliable and cost-effective way of achieving results in the short term.

The Malawi Health Sector Support Programme has enabled DFID to continue to support the public health system following the decision to end direct support to the health sector budget. It has also given DFID a new platform for engaging on health policy and capacity building, with some areas of focus on maternal and newborn health (see paragraphs 4.14 and 4.30 for instance). The programme contributes to a Health Sector Joint Fund – a pooled funding mechanism outside government financial systems that supports delivery of the government’s Health Sector Strategic Plan. However, this fund has not been used strategically to date. For example, it has been used to cover utility bills rather than to invest in the development of health infrastructure.

DFID has also given long-standing support to Malawi’s Central Medical Stores Trust, a public trust that took a decade to set up and formally replaced the government’s central medical stores in 2011. This long-term investment appears to have contributed to sustainable improvements in the national supply chain. UNFPA and UNICEF now use the trust’s systems to store and distribute many family planning and nutrition commodities, and other agencies are expected to do so once a new logistics management information system has been installed. DFID’s work on supply chain management in Malawi has not had a particularly strong focus on maternal health supplies, however, and data from another centrally managed programme indicated that the trust’s central warehouse had only two of 12 key maternal health commodities in stock in 2015-16. Key informants acknowledged ongoing challenges but stressed that long-term investment in infrastructure and rigorous performance management systems were helping to ensure a trajectory of improvement.

In the DRC, DFID has invested in the development of health infrastructure through its Access to Health Care programmes (see Box 12). The work to construct and rehabilitate health facilities lagged behind schedule during the period covered by the DFID Results Framework and was assessed as performing “substantially below” expectations in DFID’s 2016 Annual Review of the programme. Progress has remained slow since. Furthermore, transport and communications infrastructure in the DRC is very poor. This presents a significant barrier to women and girls accessing sexual, reproductive and maternal health services, and is more difficult for DFID programming to address.

DFID has supported some important areas of institutional capacity building (as in the case of Malawi’s supply chain). Where investments in infrastructure have given specific attention to the provision of maternal health care, the benefits for women are more apparent, as in the design of clinics in the DRC. However, overall progress in health infrastructure development and institutional change has been slow, and sustained investment will be needed if benefits for maternal health are to materialise. While working with non-state providers is necessary, the state needs to have a clear commissioning or regulatory function, to make sure that basic service standards are met. Building stronger links between the state and non-state providers, alongside the development of longer-term domestic financing mechanisms, is therefore part of the route to sustainability.

There is a risk of some DFID family planning programmes undermining public health system capacity to deliver these services

DFID programmes supporting the delivery of family planning services in challenging contexts face a dilemma as to whether to work with and through the national health system or to set up parallel services – for example, through non-governmental organisations or commercial providers. The former is challenging and prone to setbacks, but likely to produce more sustainable outcomes over the longer term. The latter may offer a quicker and more reliable route to expanding services, but it is less likely to continue beyond the life of the programme. Provision through non-governmental organisations or forprofit providers may be the right choice in some circumstances – for example in conflict- or disaster-affected areas – but we would expect to see this approach linking with longer-term investments in health systems strengthening, particularly related to infrastructure and human resources, to create a route to eventual sustainability. We are concerned that DFID may have prioritised cost-effective delivery in the short term over longer-term sustainability in some instances.

The centrally managed programme Prevention of Maternal Deaths from Unwanted Pregnancy promoted comprehensive abortion care, post-abortion care and family planning services. While it provided some training and mentorship for national health workers, it delivered most of its services through its own facilities and outreach mechanisms, operating in parallel with national health systems. Late in the programme, the lead implementing partner also developed a method of delivering services via the public health system. In Ghana, this proved to be more cost-effective than private sector outreach, and it gave the programme an entry point to work with government on the inclusion of family planning services under the National Health Insurance Scheme, suggesting that this approach to service delivery could have been usefully pursued more widely.

DFID Malawi’s family planning programmes also used private sector or non-governmental organisation outreach rather than working directly with the public sector. The business case for the Malawi Family Planning Programme concluded that it would be marginally more cost-effective to work with the private sector. This programme and its predecessor, Support to Banja La Mtsogolo, undoubtedly contributed to expanding access to free family planning services over many years. However, in addition to “tented outreach” (mobile facilities) in more remote areas, the Banja La Mtsogolo outreach teams often delivered services at public health centres. This enabled them to provide some training to public sector staff in the provision of long-acting and permanent methods of contraception. Client data was reported through the government’s health information systems as well as being captured by Banja La Mtsogolo. Key informants expressed concern that this type of outreach may have reduced the incentive for public sector staff to offer family planning services themselves, preferring instead to rely on Banja La Mtsogolo’s visits. There is therefore a risk of undermining local health systems and their capacity to deliver over the longer term, despite DFID’s investments in training. The business case for the Malawi Family Planning Programme anticipated that exit strategies would be developed for each facility and district, with gradual handover to government providers supported through mentoring. We found no evidence that this approach had been implemented, with both the programme logframe and DFID annual reviews recording no action in this area.

We also noted some potential problems regarding the accessibility and quality of care provided through private sector or non-governmental organisation outreach models. In Malawi, while initial family planning services were provided for free through outreach, not all clients were able to have intrauterine devices or contraceptive implants removed this way: we were told that some clients must go to a Banja La Mtsogolo clinic and pay for removal (programme data suggests this affects around 30% of women). DFID had supported a voucher scheme for removal and follow-up but this came to an end in 2015.

DFID maternal health programmes demonstrated limited consideration for the sustainability of their impact after their completion

We found that only two out of the eight centrally managed programmes we reviewed had planned to any extent for sustainability after their completion. The Prevention of Maternal Deaths from Unwanted Pregnancy programme developed an exit strategy quite late into the programme’s seven-year life span. The Safe Abortion Action Fund supported grantee capacity building and required them to develop a sustainability plan as part of the grant application process. It also paid attention to the continuation of service delivery at the end of each project’s grant. Several projects supported by the Safe Abortion Action Fund have transferred services to local health partners or have entered into collaborations to ensure service continuity. As UK support looked set to end, DFID encouraged the fund to pursue these sustainability measures.

Several programmes initiated during the Results Framework period were specifically designed to give a short-term boost to progress towards the Millennium Development Goal targets. However, it is not clear from programme completion reports or evaluations that such efforts had any lasting impact. For example, the £75 million Reproductive, Maternal, Neonatal and Child Health Trust Fund disbursed all of its funding within six months. With no exit or sustainability plans and a lack of follow-on funding, the Trust Fund became a gap-filler rather than a catalyst for change. An evaluation of the Trust Fund could not confirm that any significant or lasting improvement in access to key commodities or services had been achieved in supported countries. It concluded that sustained investment was required for lasting change.

The Making It Happen programme engaged UK volunteers to provide intensive training to existing health workers and boost their skills and capability, particularly in emergency obstetric and neonatal care. This contributed to short-term improvements in the quality of care and outcomes for women and babies in the 830 facilities supported across 11 countries. However, engaging effectively with national health institutions proved challenging and the programme’s links to national training and accreditation systems were limited. The programme continues in South Africa with WHO funding, and is self-funded in Kenya and India (where it has been rolled out nationwide), but other countries have not been able to sustain the same level of activity now the DFID programme has ended.

Programmes that piloted new approaches did not have a clear strategy for sustaining successes. The Evidence for Action programme had mixed success in sustaining impact. Its MamaYe messaging appeared to have some durability, as we witnessed in Malawi where community representatives spoke passionately about the importance of valuing motherhood and improving the quality of maternal health care. However, local data – generated initially with the programme’s support – is not yet having a sustained impact on district-level decision-making. Local key informants in Malawi also expressed concern that gains would be eroded over time, as funding for MamaYe has stopped. In Ghana the programme reported good success in data use by central government, including Parliament, and in Ethiopia the programme was embedded in the WHO locally, which allowed for close working with government. The World Bank’s Health Results Innovation Trust Fund, which DFID supported, tested results-based financing approaches and generated an evidence base, but little consideration was given to sustaining any of these pilot approaches. However, a new Global Financing Facility is focused on helping governments to mobilise increased domestic, donor and private sector funding in support of the health and nutrition of women, children and adolescents (and also led by the World Bank). It is drawing on some of the work supported by the Health Results Innovation Trust Fund and over time its impact may be more evident.

Sustainability is very challenging in difficult operating environments such as the DRC. Most DFID DRC programmes we assessed had experienced interruptions to their implementation. Where programming ended at short notice, options were rapidly developed for handover to others (such as multilateral funders). DFID had not designed its programmes around different scenarios, with mechanisms for switching delivery options if required. However, the Access to Health Care programme did develop a continuity plan that helped maintain delivery of some services when conflict flared in one province and this enabled the programme to adapt by using humanitarian funds.

There is little evidence of attention to potentially reinforcing investments for maternal health beyond the health sector

DFID’s maternal health objectives do not appear to be routinely factored into programmes outside of the health sector that could have an impact on maternal health outcomes. We found several examples of programmes that could have been usefully linked to maternal health objectives. For instance:

• Initiatives focused on women’s and girls’ rights and empowerment, such as the centrally managed programme Toward Ending Female Genital Mutilation/Cutting in Africa and Beyond (see Box 11 above) and La Pépinière in the DRC, considered sexual and reproductive health issues but made only limited links to maternal health.
• DFID DRC’s water, sanitation and hygiene programme used gender-sensitive, disaggregated indicators to monitor its outputs and outcomes, but did not explore the links to maternal health.
• DFID Malawi’s Scaling Up Nutrition programme referred to nutrition for pregnant and lactating women at impact level, but its programmatic focus was on children under five and all its outcome and output indicators reflected this.

Conversely, DFID DRC’s Access to Health Care programme – which focused on improving reproductive, maternal, neonatal and child health at impact level – has attempted to deliver a very broad range of interventions beyond the health sector, including support for homestead agriculture and smokeless cookstoves. However, implementation was slow during the Results Framework period and has remained slow since. The 2016 and 2017 annual reviews of the programme identified several water and sanitation indicators as being significantly off target. The programme has engaged in innovative work with community police to facilitate the referral of victims of sexual violence for care and treatment, to reduce the risk of HIV infection, other illnesses or lasting injury. However, outcomes in this area also lagged well behind expectations during the Results Framework period and subsequently.

Overall, DFID has not yet maximised the potential of action beyond the health sector to improve the enabling environment for maternal health. The cross-sectoral links were recognised in the DFID Results Framework and its monitoring and evaluation framework. However, we saw limited evidence of cross-sectoral collaboration or programming, beyond a recent emphasis on girls’ education and its potential to support girls’ sexual and reproductive rights and choices.

DFID is learning and adapting its programmes, but does not use evaluation consistently to sustain improvement

We found that the response of centrally managed programmes to evaluations was variable. The Health Results Innovation Trust Fund and Evidence for Action were both examples of programmes that responded positively to evaluation findings, adapting their approaches. For example, Evidence for Action underwent dramatic changes following its mid-term review, which helped refine the way it held decision-makers accountable for maternal health outcomes, based on what was needed and effective in different settings. Some of the other six centrally managed programmes had commissioned evaluations, but these appeared limited in their learning potential as they focused on whether results had been achieved rather than why.

Most centrally managed programmes demonstrated broader learning and adaptation during the course of the programme cycle. For example, based on the experience of the first phase of Making it Happen, the second phase was able to set up and deliver the programme more quickly, which in turn enabled it to scale up appropriately in different contexts. For instance in Kenya, to adapt to high staff turnover, whole teams were trained in emergency obstetric and neonatal care rather than selected individuals. In Zimbabwe, to counteract staff shortages, the programme worked closely with the government to pass legislation enabling the shifting of tasks from doctors to midwives.

Although less significant, the implementing partners for the Prevention of Maternal Deaths from Unwanted Pregnancy programme adapted their delivery channels over the course of the programme to reach poorer women by making greater use of socio-economic data. Lessons on reaching young people were also documented. Key informants mentioned that this has informed the design of DFID’s new Women’s Integrated Sexual Health programme. The inception phase of this significant new regional programme will focus on absorbing the lessons from Prevention of Maternal Deaths from Unwanted Pregnancy and factoring these into delivery. The business case for the programme demonstrates learning from the Results Framework period, with plans to engage men and boys, roll out technologies that are popular with young people (such as social media-based advisory services, injectable contraceptives and medical abortion), disaggregate data on service delivery and access and make use of client feedback.

The Health Results Innovation Trust Fund was the only significant example we identified within our programme sample of learning on “last mile delivery” (reaching communities with health care, particularly in the most remote areas) during the Millennium Development Goal period. The knowledge and learning generated has informed the newer Global Financing Facility, particularly on community-based service delivery and taking services into remote and hard-to-reach areas.

In the DRC, DFID health systems strengthening programming has been adapted in response to learning. Early programmes removed health facility user fees and provided salary subsidies. This was a high-cost approach, which led to women and their families travelling long distances to access DFID-supported facilities rather than those closer to home. The current Access to Health Care programme responded to this learning and realigned DFID’s programming with that of other donors: it agreed low fees with communities, including for maternal health service packages (antenatal care and delivery). DFID DRC has commissioned evaluators for this programme, although the earlier Access to Health Care programme was not evaluated; nor were the Support to Malaria Control and La Pépinière programmes.

In Malawi, family planning outreach approaches have been adapted over the course of two programmes delivered through Banja La Mtsogolo. There is now greater use of “tented outreach” in remote areas. Clinics that were set up under social franchising arrangements – private sector providers using shared branding and standards – have been scaled back due to underperformance. Programming has been slower to respond to recommendations from an evaluation conducted in 2014, which concluded that Banja La Mtsogolo put all its “creative energy into reaching the under 20s”. DFID Malawi’s newest family planning programme starting in 2018 looks set to have a stronger focus on this age group (see paragraph 4.46).

DFID commissioned a “Lessons Learned” paper (see paragraph 4.53 and Box 10) to assist with cross-portfolio learning. Such reviews are particularly important given DFID’s increasing reliance on its implementing partners to determine what works after they have won the tender to deliver a programme. Several of the programmes we assessed for this review were tendered on the basis of a concept note with potential suppliers asked to develop more detailed programme designs. This can be sensible where DFID’s own knowledge is limited, but it may also limit DFID’s ability to apply its own learning and may not necessarily stimulate innovation by suppliers, particularly where the pool of those with relevant expertise is small, as for the provision of safe abortion.

DFID has not yet adapted its programming to the Sustainable Development Goals’ commitment to providing universal health coverage

The Sustainable Development Goals include a commitment to providing universal health coverage. While this review focused on the 2011 to 2015 period, we would have expected to see signs of DFID adapting its ongoing maternal health programmes (those still operational after 2015) to reflect this commitment. The Health Results Innovation Trust Fund most clearly reflected universal health coverage objectives and elements, with a strong focus on reaching remote communities; although it has not framed its activities in this way, it is well positioned to contribute to the goal of universal health coverage. But overall we saw no evidence that ongoing centrally managed programmes have aligned with this agenda. We would expect DFID to have taken stock of its ongoing and planned maternal health programmes, particularly those with a stronger focus on health systems strengthening, in light of the global commitment to universal health coverage.

In both the DRC and Malawi, the governments’ national health strategies focus explicitly on efforts to achieve universal health coverage. As yet, however, DFID country-level programming has no overt focus on ensuring universal health coverage in line with these national strategies. We acknowledge, though, that the concept is particularly problematic in the DRC, given the government’s inadequate investment in health services and the consequent reliance on user fees as a financing mechanism.

DFID is a strong global advocate in the effort to sustain progress on sensitive issues around reproductive health and rights

DFID has a strong international profile on sensitive issues related to maternal health. It is seen by key stakeholders as an important defender of safe abortion. It is also a key partner for UNFPA, the United Nations’ lead agency on reproductive health and rights. The UK is UNFPA’s most important donor, through both core and programme-specific funding, and DFID works closely with UNFPA to expand global access to family planning supplies.

We found more variable collaboration with UNFPA at the country level. In Malawi DFID had engaged well with UNFPA on issues related to commodity security; in the DRC this was less apparent. In both countries, however, local UNFPA representatives acknowledged the importance of DFID’s financial contribution. DFID’s UN and Commonwealth Department currently gathers information from DFID country offices to feed into the UK’s annual strategic dialogues with UNFPA headquarters, but there is scope for greater interaction between DFID policy and country staff on both UN reform and policy issues related to maternal health.

DFID has coordinated effectively with the relatively small pool of development partners working on maternal health and family planning. Internationally, DFID’s relationship with the Bill & Melinda Gates Foundation is particularly strong, with key informants to this review noting that the two organisations influence each other and have collaborated effectively to advance the family planning agenda. Country-level engagement in donor groups has enabled DFID to coordinate and align with others – for example, to ensure a balanced geographical coverage of primary health care services in the DRC.

DFID’s engagement in international partnerships to ensure sustainable progress on maternal health is less consistent. Key informants commented that DFID’s international work appears under-resourced and its staff thinly stretched, with inconsistent attendance at international meetings. This limits the UK’s influence, and can mean that an important rights advocate is missing from global policy discussions. DFID has played a lead role in the international family planning initiative FP2020 and in the global Partnership for Maternal, Newborn and Child Health. However, the contrast between DFID’s past leadership and its relative lack of presence within current global initiatives focused on health systems strengthening, coupled with the ongoing delay in publishing the planned DFID policy on health systems strengthening, has given rise to a perception that DFID is no longer as committed to this agenda. In discussion, DFID staff told us that this was a misperception by some external stakeholders.

DFID has also sought to influence the evolution of the new Global Financing Facility (see paragraph 4.82), drawing on the experience of the Health Results Innovation Trust Fund. This influence is more likely to be sustained following DFID’s £30 million commitment to the Global Financing Facility in 2017.

DFID supported several research programmes related to maternal health during the Results Framework period. We did not assess these directly. However, several key informants drew attention to this knowledge base and questioned whether DFID was using it as effectively as it could to push for international action on key issues, particularly where evidence could help bridge disagreement around sensitive agendas like safe abortion or female genital mutilation and cutting. The centrally managed programme Toward Ending Female Genital Mutilation/Cutting in Africa and Beyond had a research component designed to build the evidence base, the outputs from which may be helpful in overcoming disagreements and advocating for change. Key informants also suggested that DFID should forge new coalitions with countries that are beginning to show leadership on reproductive health and rights, in order to sustain progress on maternal health globally. Other suggestions from key informants are set out in Box 13 above.

Ongoing effort is needed to sustain legal and policy changes related to maternal health in DFID’s focus countries

In Malawi, the DFID programmes we reviewed did not appear to have engaged directly on policy or legal issues related to harmful practices, such as forced sexual initiation, which remains prevalent in Malawi and is associated with unintended pregnancy and sexually transmitted infections. However, a new DFID Malawi programme focused on violence against women and girls – an area of DFID programming that has demonstrated some positive impact (see paragraph 4.62) – may offer opportunities to address such issues in the future. Since 2015, DFID has funded a centrally managed programme tackling child marriage, which has reportedly supported legal reform on this issue in Malawi and elsewhere (although this programme started as the Results Framework period drew to a close and it was not in our sample for review). In 2015, Malawi raised the legal minimum age for marriage nationally from 15 to 18, and in 2017 the government amended the Constitution to reflect and reinforce this. However, since traditional leaders in Malawi have the ability to make by-laws that have a strong influence over local norms and practices, work at community level is key in addition to national policy development and legislative change.

In the DRC, DFID has had some success in engaging with the Ministry of Health in order to influence policy. The Access to Health Care programme supported work with the government (and the WHO) on norms, standards and guidelines focused on sexual, reproductive and maternal health. It also supported technical assistance to the Ministry of Health on budgeting and resource allocation. The programme’s implementing partners appear to have developed good relationships with the Ministry of Health as well as with provincial and health zone management teams, which has afforded them some influence over both policy and practice (see Box 12 above). The Support to Malaria Control programme also influenced government practice, for example on bed net distribution via schools to reach children and women of reproductive age. However, DFID has not yet engaged with the government on the restrictive legal environment related to women’s sexual and reproductive health and rights in the DRC.

Conclusions on sustainability

We found that DFID’s maternal health programming during the Results Framework period had a limited focus on the long-term development of health system infrastructure and institutions, with several centrally managed interventions designed for short-term impact to boost the Millennium Development Goals. The health systems strengthening programmes we assessed paid variable attention to maternal health, and improvements in facilities and supply chain infrastructure have been slow to materialise. Most programmes did not have convincing sustainability plans or exit strategies. While we appreciate the difficult contexts for work in the DRC and Malawi and acknowledge that a fully successful exit strategy is not attainable where states chronically underfund health services, we would have expected greater attention to sustainability from DFID.

Many of the programmes in our sample opted to deliver in parallel to government facilities through non-governmental organisations and faith-based providers. This is often a more cost-effective way to expand services in the short term and can be a necessary strategy in difficult governance environments such as the DRC and Malawi. However, as DFID attempts to rapidly expand family planning services through non-governmental organisation and private sector outreach, care must be taken not to displace public sector provision.

DFID is recognised as an influential voice internationally and a key development assistance partner in the area of reproductive health and rights. It is a strong supporter of UNFPA specifically. DFID has demonstrated particular leadership on family planning and safe abortion. It could use this leadership status more proactively to effect sustainable improvements for women globally and within focus countries. Policy, legal and cultural changes will take many years, and require long-term investments to sustain progress. DFID’s country programmes could draw more effectively on its global investments and centrally managed programmes to this end.

DFID has begun to adapt its maternal health programming in response to learning, and the design of some new programmes suggests that lessons from the Results Framework period are being applied. However, ongoing programmes have yet to reflect key concepts from the Sustainable Development Goals, such as providing universal health coverage and ‘leaving no one behind’. We found little evidence to date of attention to maternal health in DFID’s non-health sector programming. There is room for DFID to reassess its programming within the context of the Sustainable Development Goals, to develop longer-term plans for investment across sectors to facilitate transformative change for women and girls, and to plan more effective transition or exit strategies where DFID’s investment will wind down.

We have awarded an amber-red score for sustainability due to the short-term impact goals and limited exit strategies we found among our programme sample, as well as a lack of cross-sectoral effort to improve the enabling environment for maternal health. However, we acknowledge the importance of DFID’s global advocacy in sustaining international progress on reproductive health and rights, which on its own would score more positively.

Conclusions & recommendations

Conclusions

DFID has been an important voice at the global level for women’s and girls’ reproductive health and rights and has significantly improved access to family planning. It has also helped to expand access to maternal health services and to put in place some of the building blocks for more robust national health systems.

However, we find that DFID’s portfolio fell short of its potential to reduce maternal mortality during the period covered by its Results Framework. As family planning was being scaled up, the portfolio reduced its intensity of focus on maternal health care services and the approach became less balanced than that set out in the Results Framework. It did not have the level of focus on improving the quality and integration of health services that would be required to make pregnancy and childbirth safer. Nor did it do enough to tackle the barriers facing poor and young women in accessing health services.

While the lack of solid data on maternal health outcomes makes it necessary for DFID to use modelling to estimate its results, we found that its modelling for Malawi and the DRC rested on assumptions that did not hold true in those contexts, resulting in inflated estimates. DFID did not give enough priority to measuring the adequacy of health services or the causes of maternal mortality in order to support improved care, better targeting of interventions and greater accountability of service providers.

DFID’s maternal health policy prioritised reaching poor, young, marginalised and hard-to-reach women, but we found that this focus was not successfully sustained through the design and implementation of its programmes, and that DFID has not monitored whether it is reaching its target groups.

While sustainability is a difficult challenge in contexts like the DRC and Malawi, we found that DFID’s preference for funding family planning services through non-state providers may have displaced some public sector provision. We also noted a lack of convincing exit strategies for the majority of programmes in our sample.

DFID has been a strong advocate for women’s and girls’ rights and empowerment and has played an important global leadership role. However, it has not always translated this role into effective engagement at country level to improve maternal health policies and to challenge regressive cultural and social practices that undermine the reproductive health and rights of women and girls.

For these reasons, DFID merits an amber-red score for the impact of its work to improve maternal health, despite a few areas of stronger performance and some evidence that learning from the Results Framework period is now being applied.

We offer the following recommendations to help DFID position its maternal health programming to contribute strongly to the Sustainable Development Goal targets for 2030.

Recommendations

Recommendation 1

As part of its commitment to the Sustainable Development Goals, DFID should develop a long-term approach to improving maternal health, planning through to 2030 in focus countries with high maternal mortality. These plans should focus on improved quality and continuity of care, cross-sectoral interventions and efforts to empower women and girls.

Problem statements

• Reaching the Sustainable Development Goal target to reduce the maternal mortality ratio to 70 maternal deaths in 100,000 live births will require sustained investment by DFID and others, and a more comprehensive approach, through to 2030.
• Long-term support to the development of health and water, sanitation and hygiene (WASH) infrastructure, human resources, transport and communications will be needed, particularly in the poorest and most fragile settings with persistently high rates of maternal death.
• Until now, many of DFID’s programmes have focused on shorter-term goals, particularly in relation to family planning. Very few have developed compelling exit strategies.
• Change to policy, legal and social norms takes time and is only possible through ongoing engagement with decision-makers and influencers, including at local level.

Recommendation 2

DFID should clarify its approach to health systems strengthening, prioritising improvements in the availability and accessibility of good quality, respectful care for women and their babies.

Problem statements

• DFID programmes have had a limited focus on improving the quality of maternal health care, despite this being a core pillar of the 2011-15 Results Framework.
• Most maternal deaths relate to complications during pregnancy and delivery, which can be addressed by health workers skilled in emergency obstetric and neonatal care, but these skills remain in short supply in many of DFID’s focus countries.
• Discrimination, neglect and abuse further erode the quality of care received by poor and young women. The need for respectful care was not a focus of the DFID health sector programmes that we assessed.
• Challenges include a shortage of health workers and a difficult working environment. Midwifery is undervalued in many countries. Professional training generally focuses on skills more than behaviours and attitudes.
• Community health systems could play an important role in improving access to services – particularly for poor, marginalised and young women and girls – but DFID is yet to explore the opportunities fully.

Recommendation 3

DFID should directly monitor the impact of its sexual, reproductive and maternal health services programmes on adolescents and the poorest women. This means including design features in programmes that target adolescents and the poorest women, monitoring whether they are effective and adjusting course where they are not.

Problem statements

• DFID did not track whether or not it reached young women aged 15 to 19, the poorest 40% or otherwise marginalised groups during the Results Framework period.
• DFID experienced challenges in reaching adolescents with sexual and reproductive health services and learned lessons that can be applied in the next phase of programming. This includes the need to reach girls and boys aged 10 to 14, as well as those aged 15 to 19 who were prioritised in the Results Framework.

Recommendation 4

When using models to generate outcome data, DFID should test its assumptions and triangulate its results claims using other quantitative and qualitative data.

Problem statements

• DFID programme-level outcome data and its estimates of ‘maternal lives saved’ through its programming from 2011 to 2015 were based on modelling and rested on assumptions that were not robust.
• Additional qualitative outcome data is needed to improve programme performance and to triangulate quantitative data, including impact estimates. DFID could make much more use of logframe indicators assessing quality of care, feedback from local accountability bodies and data from client surveys and community scorecards.

Recommendation 5

As part of its commitment to the Sustainable Development Goal data revolution, DFID should prioritise and invest in international and country-level efforts to gather data on the quality of maternal health services and outcomes, including disaggregated data relating to key target groups.

Problem statements

• Data availability and quality are poor in many developing countries.
• The lack of basic registration systems (births, deaths and marriages) and data on cause of death, as well as poor quality data in health management information systems, make it difficult to effectively monitor progress on improving maternal health. This is one reason why DFID and others rely on modelling to assess the impact of their programmes.
• Investment in facility-level systems, such as Maternal Death Surveillance and Response, and in the collection and input of data into health management information systems could improve quality of care locally, as well as generating valuable insights when data is aggregated.
• DFID made some innovative investments in improving such data during the Results Framework period, but many initiatives do not yet operate at scale and the use of data for decision-making is not yet institutionalised.
• There is a need for data disaggregation to track progress in reaching different groups and to understand how maternal health outcomes vary.

Annexes

Annex 1 DFID programmes assessed for this review

Table 3: Centrally managed multi-country and multilateral programmes assessed

Table 4: DFID DRC programmes assessed

Table 5: DFID Malawi programmes assessed

Annex 2 DFID’s use of results modelling for maternal health

In 2010, DFID committed through its Results Framework to save 50,000 maternal lives and 250,000 newborn lives by 2015. DFID chose to use the Lives Saved Tool (LiST) to estimate the number of maternal, neonatal and child lives saved through its country programming during this period.

How LiST works

LiST is mathematical modelling software, which was developed by experts involved in the Lancet Child Survival series. It was further improved by an expert team coordinated by the Johns Hopkins Bloomberg School of Public Health. LiST estimates levels of mortality from different causes – for example, women dying from severe bleeding during or after childbirth, or newborns dying from infections – based on changes in intervention coverage and the prevalence of relevant risk factors. As certain interventions are scaled up, mortality reductions or ‘lives saved’ are attributed proportionally to changes in intervention coverage.

How DFID has used LiST

Annual LiST analyses were conducted using programme data from DFID country offices to generate estimates of lives saved for each year covered by the Results Framework. For each country, programme data was aggregated into health intervention coverage estimates. These coverage estimates were then used to produce paired projections of lives saved in two scenarios: ‘with DFID interventions’ and ‘without DFID interventions’ (to serve as a counterfactual). The difference between the two scenarios was assumed to be the estimated lives saved due to DFID programming. The estimates of maternal lives saved were disaggregated in LiST reports, to show the relative impact of DFID family planning and maternal health care interventions. The process is summarised in Figure 8.

Figure 8: Sequence for LiST-based analysis of lives saved by DFID programming on maternal, newborn and child health (MNCH) and family planning (FP)

Headline results from LiST analysis – global, DRC and Malawi

Using LiST, it was estimated that there were 66,300 maternal lives saved due to DFID country programming between 2011 and 2015. In addition, the Prevention of Maternal Deaths from Unwanted Pregnancy (PMDUP) programme used another tool, Impact 2, to estimate the number of maternal lives saved through its interventions. Further analysis was undertaken to ensure there was no double-counting of PMDUP and country programme impacts, and the conclusion was that 13,800 further lives had been saved from PMDUP interventions. Although attempts were made, data on maternal lives saved through the programmes Evidence for Action and the Reproductive, Maternal, Neonatal and Child Health Trust Fund was not added in to the final LiST analysis for 2011-15. Key informants from both programmes struggled with the LiST methodology and did not consider the resulting estimates to be robust.

The total estimate was therefore 80,100 maternal lives saved. Of these, more than 79% were due to family planning interventions – that is 63,700 lives (49,900 from bilateral programming and 13,800 from PMDUP) – with 16,400 maternal lives saved due to maternal care interventions. In Malawi, the estimated maternal lives saved were 10,100, of which 9,700 were related to family planning and 400 to maternal care interventions. In the DRC, the total maternal lives saved were estimated at 4,800. Rounded estimates showed 500 were related to family planning and 4,200 to maternal care interventions.

Weaknesses in the LiST methodology

The literature review and key informant interviews highlighted several weaknesses in the LiST methodology. Using intervention coverage to model lives saved does not account for the quality or accessibility of the intervention and may therefore overestimate its impact. LiST does not account for variations in the strength of different health systems.

The counterfactual in the paired projections (‘with’ and ‘without’ DFID intervention) assumes there are no changes to the status quo in the ‘without DFID intervention’ scenario. This may not be realistic. Similarly, using DFID’s share of total funding to attribute outcomes to DFID assumes that this support has supplemented other funding or activities working towards the same outcomes, rather than displacing them.

Key informants noted that LiST produces better estimates of lives saved among children under five and, to a lesser extent, newborns, than among mothers. LiST estimates of lives saved cannot be disaggregated by age, so the relative impact of interventions cannot be ascertained.

Where input data on the Contraceptive Prevalence Rate (CPR) was not available, the LiST analyst working with DFID used intervention data on couple-years of protection as a proxy for the CPR. This approach can result in frontloading of ‘lives saved’ as the longer-term benefits of permanent and long-acting contraceptives are factored into the estimates when coverage is reported. This was the case for DFID programming in Bangladesh, Malawi and Nigeria, for example.

Non-comparability with other methodologies

The estimates of lives saved produced using different models differ widely, even when they draw on the same input data. For example, LiST estimates of maternal lives saved in Malawi were double those of another modelling tool, Impact 2. A DFID-supported ‘harmonisation group’, which brought together specialists in different modelling approaches, found several inconsistencies across the various tools, the most critical being the differences in projected pregnancies within a given population in the absence of family planning interventions. Harmonisation of approaches has brought estimates closer together. Track20, the measurement project for the FP2020 initiative, has drawn on this work to minimise the variation in estimates.