The effects of DFID’s cash transfer programmes on poverty and vulnerability

ICAI Score

Good achievement on poverty reduction impact, value for money and learning, with scope for improvement on building national cash transfer systems

DFID’s support for cash transfers helps to alleviate poverty and vulnerability for the poorest households, in accordance with the “leave no one behind” commitment. Over the 2011-2015 period, DFID exceeded its target of reaching six million people with cash transfers. Independent evaluations show that the programmes have consistently delivered on their core objective of increasing incomes and consumption levels for the poorest households, with modest but positive effects on savings, asset accumulation and debt reduction. Evidence points to more variable results in relation to education, nutrition, health and the empowerment of women – areas where cash transfers may need to be combined with other interventions to improve results. DFID has made a significant contribution to promoting the use of cash transfers in national social protection systems in its partner countries. However, in the face of shortcomings in its approach to financial and technical assistance, it is not making enough progress in overcoming weaknesses in the national programmes it supports. We find that DFID’s cash transfer programme presents a strong value for money case, given its proven ability to deliver results for the poorest. DFID has made progress on building a focus on cost-efficiency into programme management, although its practice could be more consistent. It has made a strong contribution to the global evidence base on cash transfer programming, and has used evidence and learning well to strengthen its results. There is scope for DFID to increase its level of ambition for the cash transfer portfolio by improving results in individual programmes and helping to scale up promising national programmes.

Executive Summary

Cash transfers play an increasingly important role in the fight against global poverty. In 2014, developing countries provided cash transfers to 718 million people. Their primary purpose is to alleviate extreme poverty by supplementing the income of the poorest households, enabling them to increase their consumption of food and other basic items. They can also promote other benefits, including increased use of education and health services and empowerment of women. Cash transfers are an important element of national social protection systems.

In this review, we explore the impact of DFID’s cash transfer programmes on poverty reduction[1] over the period 2011 to 2015. There have been 28 such programmes, the majority of which support national cash transfer schemes and involve both direct funding for cash transfers and technical and financial support for system building. These programmes all aimed to mitigate extreme poverty and improve nutrition, as well as to achieve a range of other programme-specific objectives. Under its global results framework for 2011-15, DFID committed to reaching at least six million people with cash transfers. Over this period, DFID spent an average of £201 million per year – around 2% of its total expenditure – on cash transfers.

Cash transfers are a form of development assistance that lends itself to rigorous impact assessment, and there is a substantial body of global and DFID-specific evidence available on its effectiveness. Their importance and the available body of evidence mean that this is an appropriate area for an impact review, to examine DFID’s reported results and their significance for the intended beneficiaries.

We conducted desk reviews of 18 of the 28 programmes, and detailed case studies of programming in two countries: Bangladesh and Rwanda. We addressed three broad areas:

i. The impact of DFID-supported cash transfers on poverty and vulnerability.
ii. DFID’s contribution to the development of sustainable, nationally owned cash transfer systems.
iii. The value for money of DFID’s cash transfer programming.

DFID’s cash transfers have succeeded in their core objective of raising income and consumption levels, but show more variable results against secondary objectives

In its 2015 Annual Report, DFID reported that between 2011 and 2015 it had reached a ‘peak’ of 9.3 million people with cash transfers, against a target of six million[2]. We examined this claim thoroughly. We found some inclusion errors in the data that led to over-reporting of around 475,000 people – all from one programme. These errors were corrected when brought to DFID’s attention. For the rest of the programme portfolio, we were able to verify that DFID had exceeded its reach target.

DFID’s cash transfer programmes have succeeded in their primary purpose of alleviating extreme poverty. Independent evaluations have confirmed that DFID programmes have consistently helped to improve household incomes and boost consumption levels of food and other basic items, with no evidence of increases in unhealthy consumption choices (eg alcohol or gambling). The results data is robust and matches what would be expected from the literature.

DFID’s support for cash transfers has also brought a range of additional benefits to beneficiary households. The evidence suggests a modest but positive impact on savings, asset accumulation and debt reduction. This in turn helps to make poor households more resilient to external shocks (eg adverse weather or
unexpected health bills).

It is widely recognised that cash transfers are generally not sufficient on their own to lift the poorest households permanently out of poverty. DFID has been experimenting with a “poverty graduation” model that combines cash transfers with other interventions, such as asset grants, training and support for income generation. Its pilot programmes with a Bangladeshi NGO (BRAC) have delivered impressive results, although at a higher unit cost than a pure cash transfer programme. It is not yet known whether this model is replicable outside Bangladesh, or could be delivered by government, but DFID is experimenting with versions of the model in a number of other countries.

Most of DFID’s cash transfer programmes include secondary objectives in areas such as education, health and nutrition, and empowering women. In these areas, the evidence of impact is more mixed and, in a few cases, less than would be predicted from the global evidence base.

• Education: While many DFID-funded programmes include objectives around improving the education of children from poor households, the results have been uneven. Programmes in Pakistan and Zambia have made a positive contribution to school attendance, and one programme in Ethiopia has recorded an improvement in learning outcomes. A number of other programmes, however, showed no or very modest impact, and in Zimbabwe the programme had both positive and negative effects.

• Health and nutrition: The literature suggests a mixed but mostly positive relationship between cash transfers and improvements in health and nutrition outcomes. Independent evaluations of DFID funded programmes show a wider range of results, including some cases where no positive effect was observed. In several cases, DFID has recognised where programmes are underachieving in this area and is introducing corrective measures.

• Women’s empowerment: DFID programme design documents emphasise the objective of empowering women, and several programmes make payments directly to women. Unlike other areas, however, the focus on women’s empowerment has not been backed by rigorous measurement of results. There are encouraging signs of progress in some programmes, such as improved status for women within their households and communities and increased sexual autonomy, but the evidence is not strong enough to support a clear conclusion. We also found that DFID was not explicitly monitoring risks to women beneficiaries, such as increased domestic abuse.

Overall, we have awarded DFID’s portfolio a green-amber score for impact on poverty and vulnerability. DFID has achieved and exceeded its global reach targets. Across the portfolio, its core objective of alleviating income poverty and increasing consumption for the poorest and most vulnerable is being consistently achieved, with modest but positive impacts on building resilience to shocks. Impact on secondary objectives is more variable, with some positive results in areas such as school attendance (although not everywhere) and less positive results in health and nutrition. There is therefore scope for DFID to continue improving the impact from its cash transfer programming.

DFID has committed to strengthening national cash transfer systems but lacks a strategic approach

Beyond funding specific cash transfer programmes, DFID is also seeking to expand the coverage, quality and sustainability of national cash transfer systems. DFID has worked towards this objective by supporting pilot initiatives, as well as through funding, policy dialogue and capacity strengthening support. Over the review period, there has been clear progress in expanding national cash transfer programmes across DFID’s partner countries. In some cases, this is accompanied by increased domestic funding. While other factors are also at play, it is likely that DFID has made an important contribution to this result.

Wherever possible, DFID chooses to fund cash transfers through national programmes. This means tolerating weaknesses in the design and delivery of programmes over the short term in order to try to strengthen them over time. In most cases, DFID takes a long-term approach to systems development, sharing international and local evidence but generally choosing not to challenge partner countries on strongly held positions. While this approach is welcomed by national stakeholders, there are risks that it can lead to a lack of ambition and urgency in addressing core challenges. We observed ongoing challenges across the portfolio in the key areas of targeting, transfer size and timeliness. While recognising the positive achievements against core programme objectives, these challenges pose limits for effectiveness and sustainability. DFID is aware of these issues and, in some instances, has made good progress in addressing them, but overall progress is uneven.

DFID reports a range of practical results from its technical assistance programmes. However, its approach varies markedly across programmes, with no clearly stated rationale. We heard concerns from both DFID staff and implementers that technical assistance programmes can become drawn too far into short-term problem solving (“firefighting”) at the expense of a more strategic approach. We found that oversight and monitoring of technical assistance was not adequate, and that there have been no independent reviews or evaluations of results in this area.

There is also no clear strategy underlying the size or conditions of DFID’s financial contributions. With the exception of the Pakistan and Uganda programmes, DFID does not use performance triggers to drive reforms and it lacks a clear strategy for promoting financial sustainability.

We have therefore awarded DFID an amber-red for its system-building efforts. While DFID has good relations with its national counterparts and has helped to increase country ownership of cash transfer programmes, it lacks a systematic approach to both financial and technical assistance, and does not adequately monitor and assess the results of its system-building efforts.

DFID’s cash transfer programming offers a good value for money case, and there may be a value for money case for scaling up funding towards national coverage

We find that DFID’s support for cash transfers meets many of the criteria for value for money. It is consistently delivering on its core objective of alleviating extreme poverty and reducing vulnerability. It is an effective means of reaching the poorest and most vulnerable, in accordance with DFID’s commitment to “leaving no one behind”. There are short-term trade-offs involved in funding through national systems, and DFID should ensure that its technical assistance is sufficiently focused on improving financial sustainability. However, our analysis suggests that, where the core issues of targeting, timeliness and transfer size are being addressed, there may be a value for money case for scaling up funding towards national coverage.

Following a challenge from the Public Accounts Committee in 2012, DFID has made an effort to strengthen the focus on value for money in the management of its cash transfer portfolio. It has developed guidance and tools for undertaking value for money assessments at programme level. We saw some good use of value for money analysis in identifying the variables with the greatest impact on cost-efficiency and programme effectiveness (eg targeting, payment size and financial management). What was less clear was how these assessments were being used to inform broader, portfolio-level management decisions about funding for cash transfers.

DFID recognises evidence and learning as “cross-cutting enablers” of value for money. DFID has made an important contribution to building up evidence on what works in cash transfer programming. It has commissioned syntheses of existing literature to identify gaps and policy-relevant lessons. During the review period, it managed a centrally commissioned research portfolio of over £35 million that has been innovative in both themes and methodology.

DFID has also demonstrated a willingness to learn from international evidence and from its own programmes. It has an active community of practice that disseminates research findings and shares lessons. We saw good examples of real-time learning within and across programmes, and of the use of evaluative findings to inform new designs.

We have rated the portfolio as green-amber on value for money, owing to its demonstrated impact and strong learning orientation. There remains scope for DFID to continue strengthening the relationship between value for money assessments and management decision-making.

Conclusion and recommendations

Overall, DFID’s cash transfer portfolio merits a green-amber score. The portfolio has demonstrated its capacity to achieve impact in its core objective of alleviating extreme poverty. DFID has also made an important contribution to encouraging the spread of national protection systems. Notwithstanding the complexities of strengthening national systems and financial sustainability for social protection, there may be scope to achieve even greater value for money by taking successful programmes to scale.

The following recommendations are designed to help DFID to improve the impact and value for money of its cash transfers programming, in pursuit of its “leave no one behind” commitment.

Recommendation 1

DFID should consider options for scaling up contributions to cash transfer programmes where there is evidence of national government commitment to improving value for money, expanding coverage and ensuring future financial sustainability.

Recommendation 2

DFID should be clearer about the level and type of impact it is aiming for in each of its cash transfer programmes, and ensure that these are adequately reflected in programme designs and monitoring arrangements.

Recommendation 3

DFID should do more to follow through on its commitment to empowering women through cash transfers by strengthening its monitoring of both results and risks, and using this data to inform innovations in programming.

Recommendation 4

DFID should take a more strategic approach to technical assistance on national cash transfer systems, with more attention to prioritisation, sequencing, monitoring and oversight.

Introduction

In 2014, cash transfers to poor households reached an estimated 718 million people across the developing world.[3] Their primary purpose is to alleviate extreme poverty by supplementing the incomes of poor households and increasing their consumption of food and other basic items. Cash transfers can also help to empower women, improve school attendance and promote better nutrition and health.[4] Many developing country governments are now using cash transfer programmes as part of national social protection systems (see Box 2). At the Addis Conference on Financing for Development in July 2015, the global development community pledged to provide “sustainable and nationally appropriate social protection systems and measures for all”.[5] This commitment was repeated as target 1.3 of the Global Goals[6] and is a key element of the “leave no one behind” pledge.

“Implement nationally appropriate social protection systems and measures for all, including floors, and by 2030 achieve substantial coverage of the poor and the vulnerable.”

Target 1.3, Global Goals.

In this review, we explore what development impact DFID has achieved through its support for cash transfer programmes. DFID’s spending on cash transfers for poverty mitigation has increased as experience and evidence have grown, from £4 million in 2003 to an annual average of £201 million over the review period (2011-2015), reaching approximately 2% of DFID’s total expenditure.[7] Most of this was direct financing for cash transfers, but there was also a significant element of technical and financial assistance for the development of national cash transfer systems. In this review, we are interested both in the direct impact of DFID’s funding for cash transfers and in its contribution to building sustainable national cash transfer systems.

This report reviews DFID’s use of cash transfers for mitigating poverty and vulnerability. We have not covered the use of cash transfers for humanitarian assistance, which raises different issues, but we hope to come back to this in a future review. Our scope is DFID’s cash transfer portfolio over the four-year period covered by DFID’s Results Framework 2011.[8]

This portfolio was chosen for an impact review because it is mature enough to have generated a substantial amount of results data. There is also a large body of evidence-based literature on the topic of cash transfers.

Methodology

There are four main methodological elements to our review.

i. Our literature review served two purposes. First, we reviewed the available evidence on the effects of cash transfers as a yardstick against which to measure the impact of DFID programmes. Second, we compared DFID’s research choices and contributions with the worldwide body of cash transfer literature to assess the extent to which DFID has contributed to filling relevant evidence gaps.

ii. We carried out key stakeholder interviews with 36 DFID staff about the portfolio as a whole and about specific programmes. We also interviewed 12 academics and peers from other organisations to gain stakeholders’ views on DFID’s cash transfer approaches and choices. We sought out known “critical voices”, to help us challenge DFID’s thinking.

iii. We selected a sample of 18 programmes (out of a total of 28 cash transfer programmes identified by DFID) for desk review. A list of the programmes is included in Annex 2 (Table A1). For each programme we reviewed the available programme documents, including business cases, logframes, baseline studies, diagnostic studies, annual reviews, project completion reports and independent evaluation reports.

iv. We carried out country case studies of DFID cash transfer programming in two countries: Bangladesh and Rwanda. Each involved a visit by the team. The countries were selected for having a sizeable portfolio with a history of programming over several years, covering both technical assistance and financial support.

The methodology is explained in more detail in Annex 3, and in full in our Approach Paper, which is available on the ICAI website. Both our methodology and this report were independently peer reviewed.

Background

DFID is thought to be the largest funder of cash transfer programmes among bilateral donors,[9] with annual expenditure ranging between £170 million and almost £300 million over the past five years (see Figure 1). Most of its cash transfer programmes fall within its social protection portfolio, contributing to the “poverty, vulnerability, nutrition and hunger” pillar of DFID’s results framework.[10] These programmes provide supplementary income to the poorest and most vulnerable households in order to mitigate extreme poverty and ensure that beneficiaries are able to meet minimum consumption levels of food and other basic items. They thereby aim to have positive effects on nutrition and health. Individual programmes may also aim to provide additional benefits, such as improving school attendance or empowering women.

In its results framework, DFID committed to supporting at least six million people with cash transfers.[11] To be “supported” means different things in different programmes. It may involve the household receiving a small monthly payment, often set by reference to the number of people in the household and the cost of buying basic foodstuffs (see Box 5 on payment sizes). It can also mean payment for regular or occasional labour on public works programmes. Section A5 in Annex 2 provides a more
detailed description of one particular programme, in Rwanda.

In most instances, DFID works with and through national governments in order to contribute to the development of national cash transfer systems. Most DFID programmes offer a combination of financial support for cash transfers and financial and technical assistance for system development.

Within the broad goals of extreme poverty mitigation and consumption protection, DFID’s cash transfer portfolio is diverse.

• Programme budgets range from £1 million to £300 million, and their duration from four to ten years, with repeat programming common.
•  DFID may be the sole funder (as is currently the case for a World Bank administered programme in the Sahel), the main funder (Uganda, Zimbabwe) or one of many funders (Ethiopia).
• The programmes in Myanmar and Nigeria are delivered outside national governments, through NGO partners. Most other programmes are co-financed with government, with national contributions ranging from very little (Uganda, Zimbabwe) to most of the funding (Kenya, Pakistan, Rwanda).
• Seven programmes consist only of technical assistance. The other 21 provide a combination of technical assistance and direct funding for cash transfers (there are no cases of funding without technical assistance).
• Cash transfers are mainly unconditional. In Ethiopia, Myanmar, Nepal, Rwanda, Tanzania and Yemen, some or all of the cash transfers are provided in exchange for labour on public works. Programmes in Pakistan and Tanzania have education-related conditional components (for example the Pakistan programme provides a monthly payment of £1.80 for each child who attends school regularly).
• Some of these programmes are designed to reach the poorest in the community (though effective targeting is one of the challenges). Other programmes target a particular category of citizen, such as pregnant women in Nigeria or elderly people in Uganda.

Details of the programmes in our sample are included in Tables A1 and A2 in Annex 2.

DFID also has a substantial research portfolio on cash transfers. Since August 2009, it has centrally commissioned research contracts totalling over £35 million, many of which are ongoing. Its research partners include UK universities (Oxford, Manchester, Sussex), research institutes (the Overseas Development Institute, the Institute of Development Studies, 3ie, RAND Europe) and multilateral agencies (the World Bank, the Food and Agriculture Organization). Many of its country programmes also include research components.

According to key stakeholders in DFID, the department’s interest in cash transfers emerged around 2005-06. The aftermath of the 2007-08 “Triple F” crisis, when the global financial crisis was combined with sharp rises in food and fuel prices, reinforced this interest, as DFID identified that cash transfer systems in a few South American countries had shielded the poorest households from the worst effects. A view emerged within DFID that low-income countries in Africa and Asia could also develop such systems, and that DFID could support them to do so. The 2009 White Paper, Eliminating World Poverty, announced an intention to extend social protection to 50 million people in 20 countries.[13] This commitment was subsequently replaced with the target of reaching six million people.

In 2011, a DFID literature review noted that cash transfers were already “one of the more thoroughly researched forms of development intervention”. It concluded that “there is convincing evidence from a number of countries that cash transfers can reduce inequality and the depth or severity of poverty”, as well as “contribute directly or indirectly to a wider range of development outcomes”.[14]

This enthusiasm was mirrored in a February 2012 Public Accounts Committee report on “transferring cash and assets to the poor”. The report confirmed that transfer programmes “are effective in targeting aid, and ensuring the money goes directly to the poorest and most vulnerable people. There is strong evidence of short-term benefits for recipients of transfers, for example better nutrition and greater access to health and education services.” The Committee expressed its surprise “that the use of transfer programmes has not increased more in light of the evidence of positive outcomes [and that] the Department only plans to support transfer programmes in 17 of its 28 priority countries”.[15]

“The Global Goals for Sustainable Development offer a historic opportunity to eradicate extreme poverty and ensure no one is left behind. To realise this opportunity we will prioritise the interests of the world’s most vulnerable and disadvantaged people; the poorest of the poor and those people who are most excluded and at risk of violence and discrimination.”

Leaving no one behind: Our promise, DFID, November 2015.

DFID has nonetheless declined to set spending targets on cash transfers, arguing that the decision on whether to provide cash transfers, and in what amount, should be left to each country programme. It noted that: “[i]t is important the Department does not move ahead of local political and practical reality in seeking to support transfer programmes” and that “country offices are in the best place to judge which programmes offer the best value for money in achieving objectives”.[16]

The impact of DFID’s cash transfers on poverty and vulnerability

In this section, we look first at whether DFID has met the cash transfers target in its results framework. We then explore impact across our sample of 18 programmes. DFID-funded cash transfer programmes have a range of objectives (see Table A4 in Annex 2 for a DFID-funded cash transfer programmes have a range of objectives (see Table A4 in Annex 2 for a sample of logframe targets). All of the programmes are designed to alleviate extreme poverty and boost consumption, and some plan to improve household resilience to shocks. Most cash transfer programmes also include additional or secondary objectives in areas such as education, health and nutrition, and women’s empowerment (see Table 2). Not all programmes specify logframe targets in all these areas and some of the impact evaluations reveal results in areas that were not specified as logframe targets. To capture both planned and unplanned results, we looked at the evidence across five major areas:

i. Income poverty and consumption
ii. Savings, assets and resilience
iii. Education
iv. Health and nutrition
v. Women’s empowerment.

DFID provided us with an overview of 28 cash transfer and social protection programmes. Seven of them provided only technical assistance and 21 offered a combination of funding and technical support. The logframes and business cases of these 21 programmes covered a range of aims and objectives:

Table 2: DFID’s logframe targets and business case objectives

DFID’s reach target is a means to an end

Under its results framework, DFID set itself the target of supporting at least six million people with cash transfers between 2011 and 2015, measured as the total of the peak annual reach of each programme. This is known as a reach indicator: it shows the scale and coverage of DFID’s programming, but not the effects of cash transfers on beneficiaries. The indicator does not include the impact of DFID’s technical assistance on expanding national cash transfer systems.

While these are real limitations, the reach target remains relevant for three reasons. First, it builds on a large body of evidence showing a causal link between the provision of cash transfers and poverty mitigation. Second, DFID is a key advocate for the expansion of cash transfer coverage in Africa and Asia, and the reach target helps to signal this commitment. Finally, its inclusion in the results framework provided a clear signal to DFID country offices during the 2011 Bilateral Aid Review that cash transfers were a priority.

Cash transfers reached more people than DFID aimed for, but fewer than it reported

In its 2015 Annual Report, DFID reported that between 2011 and 2015 it had reached a ‘peak’ of 9.3 million people with cash transfers, including 4.9 million women and girls.[17] According to DFID data, most individual programmes met or exceeded their targets, with the exception of programmes in Bangladesh (after the correction of a reporting error, covered below), Zimbabwe (with an underachievement in 2015 only) and the Sahel.

We reviewed this data (see Box 7). We found that, with the exception of DFID Bangladesh, all country offices had followed the approved methodology and had mechanisms in place to check data provided by their implementers. Where the data was incomplete, DFID used explicit assumptions to calculate its total.[18]

We did find a few inaccuracies. The only material one is that, in Bangladesh, various errors associated with one specific programme resulted in an over-claim of almost half a million people (corrected after we brought it to DFID’s attention). Notwithstanding this error, we conclude that there is reliable evidence that DFID has achieved and exceeded its target of six million people, even though the total is somewhat less than DFID reported.

There are two main reasons why the six million target was exceeded. First, the original target was lower than the sum total of the 2011 country targets, which added up to 7.2 million people. Second, while a few of the programmes planned at that time never materialised (including in Sudan and the Democratic Republic of Congo), two new, larger Pakistan programmes began during the review period, adding more than three million people to the total reached.

DFID’s cash transfers have succeeded in supplementing incomes and increasing consumption

The primary purpose of DFID’s cash transfers is to alleviate extreme poverty for the duration of the transfer (which may be short-term or open-ended) by increasing income and consumption levels. Independent evaluations have confirmed that DFID programmes have, in almost all cases, succeeded in achieving this objective. This finding is consistent, whichever definition of poverty is used.[19]

The evaluations have consistently found that cash transfers increase consumption in beneficiary households.[20] There is no evidence of an increase in unhealthy consumption choices (eg increased spending on alcohol or gambling) and in some cases there is evidence of a decrease. The most impressive finding is from Kenya, where beneficiary households spend significantly more of their overall income (not just the cash transfers) on food, health and clothing, and significantly less on alcohol and tobacco.

These are very positive findings, suggesting that cash transfers are proving to be an effective means of alleviating severe poverty.

Cash transfers have modestly increased longer-term income and resilience

In addition to improving consumption levels, increasing household income through cash transfers can bring a range of additional benefits. According to the literature, cash transfers may help to increase and diversify household income by enabling beneficiaries to acquire productive assets and take calculated risks in their business ventures (which is more likely if cash transfers are regular and predictable). In addition, cash transfers have been shown to boost savings, reduce debt and change the ways in which beneficiaries behave. All of this, as well as the assets built or maintained by public works programmes, potentially contribute to households becoming more resilient to shocks (see Boxes 9 and 10).

While potentially complex to measure, these effects have been confirmed in a number of DFID’s cash transfer programmes. A common result is a reduction of indebtedness and an increase in creditworthiness – two factors that affect households’ ability to deal with crises. Such effects were reported in DFID-funded programmes in Bangladesh, Ethiopia, Nepal, Rwanda, Uganda, Zambia and Zimbabwe.

We also found a number of other effects that were specific to individual programmes.

• In Zambia, recipient households were found to be more resilient to shocks and external fluctuations in income than they had been in the past, due to various factors including reduced debt, increased investment in assets, reduced reliance on casual labour and livelihood diversification (such as increased livestock, more production of agricultural output for markets and increased non-farm enterprise).
• In Zimbabwe, an independent impact evaluation concluded that, after only 12 months, the programme had led to increased agricultural assets and livestock, diversified income sources and reduced indebtedness. As a result, there was a reduction in vulnerability to shocks, particularly among smaller households.
• Reviews in Rwanda and Uganda tracked data on savings and productive assets. Both reported an increase in livestock, though probably not to the point that the gains would survive a single household shock. A study in Uganda found that transfers were helping recipients to purchase other types of productive assets.
• Conversely, a study in Pakistan reported that recipients expressed an increased desire to save, but that an actual increase in savings occurred in only a single province (Khyber Pakhtunkhwa).

While these findings illustrate the potential for cash transfers to go beyond boosting consumption, the reported impacts are relatively modest and, in many cases, amount to resilience against only a single shock. DFID’s annual reviews and evaluation reports confirm that cash transfer programmes do not have a lasting impact on income-earning opportunities and resilience, unless accompanied by other interventions.[21] This is probably true everywhere, but especially in African countries, where the state of rural economies makes it particularly challenging to diversify income.

A more ambitious “poverty graduation” model in Bangladesh is achieving results by combining cash transfers with other forms of support

The biggest gains in both income and resilience are achieved by the DFID-funded programmes in Bangladesh. These programmes combine relatively small cash transfer components with other interventions, such as asset grants (eg livestock) and supporting products and services (eg animal vaccinations, food supplements, entrepreneurial and other training, and hygiene awareness raising).

DFID was a supporter of the Bangladeshi NGO BRAC when it first piloted this model in 2002. The approach has developed but not fundamentally changed since then, and there is strong evidence that it has enabled a large majority of beneficiaries to achieve substantial improvements in their socioeconomic status. The chances of these mixed interventions achieving results that continue after households have exited the programme are much stronger than for pure cash transfers. Furthermore, the results have shown an ability to survive climate shocks.

It is not yet known whether this model is replicable outside Bangladesh (only a few initial studies exist),[22] whether it can be delivered by government (as opposed to an NGO) or how it performs in times of economic downturn. Moreover, these combined interventions are more costly than standard cash transfers. However, the results are impressive enough for DFID to be supporting a pilot adapted from this model in Rwanda, while other pilots in Pakistan and Kenya are under development.

Despite significant success in Ethiopia and Pakistan, the effects on education have been generally modest

There is strong evidence that DFID’s cash transfer programmes have been successful in their core objectives of relieving income poverty, increasing consumption and, in some instances, providing a modest boost to resilience. However, most cash transfer programmes also include additional objectives in areas such as education, health, nutrition and empowering women. In these areas, the evidence of impact is more mixed and, in some cases, less than would be predicted from the literature.

Nine of the 18 programmes in our sample mentioned educational goals in their business cases. The programmes in Pakistan (see Box 11) and Ethiopia have achieved considerable success, but in the other programmes the impact has been modest, absent or, in one or two cases, negative. Overall, the impact of DFID’s programming on enrolment and attendance (although not necessarily learning) is below what the wider evidence suggests can be achieved through cash transfers. It is likely that weaknesses in programme design and implementation (see the sections on cash transfer targeting, timeliness and size) have held back results in these areas.

In seeking to understand this mixed performance on education, we found no particular correlation between the presence of explicit education-related logframe targets and a programme’s education related outcomes. Schooling is not among the Ethiopia programme’s logframe targets, but transfers have helped children from poor families to attend schools and have improved educational attainment and progress. In one region – Tigray – children in recipient households outperformed the children of better-off, non-recipient households. Conversely, the Kenya programme does have education-related logframe targets, but the results have been limited to a minor enrolment increase among children living a long way from school. In Uganda, a hope that the transfer would support schooling has not materialised: the programme has not increased household expenditure on education and has had no impact on children’s attendance or attainment at either primary or secondary level.

We also came across an example of negative impact. In Zimbabwe, poor government coordination across programmes meant that cash transfer recipients were discouraged from participating in the Basic Education Assistance Module (BEAM), another government programme that provides targeted resources for children to attend school. An external evaluation found a 6% decline in BEAM participation among DFID-funded programme participants, which offset other positive effects on education (including a seven percentage point increase in the probability of school progression for children of primary school age in small households). An external assessment suggested that there may also have been a negative impact from a public works programme in Nepal. The researchers noted that, while the programme does not employ children below the age of 16, it may nonetheless have increased school absenteeism as a result of children taking on extra responsibilities of caring for younger siblings while their parents are participating in the programme.

Where programmes did increase school attendance, this did not necessarily result in improved learning outcomes. In Zambia, one of the two grant types resulted in large impacts on enrolment of both primary- and secondary-age children, but there was no impact on educational outcomes for primary school age girls.[23] The wider evidence suggests that cash transfers on their own rarely secure a positive impact on learning, due in part to problems with the quality of national education systems. DFID has other programmes in most of its partner countries that aim to strengthen education systems.

Evidence of impact on health and nutrition is uneven across and within programmes

All of DFID’s cash transfer programmes (not just those in our sample) include objectives around health and nutrition, and 15 of them incorporate health and nutrition indicators in their logframes. This reflects empirical evidence from the literature that cash transfers can promote both greater use of health services and more dietary diversity, although impacts on child wasting and stunting are generally weaker.

Notwithstanding this strong focus on health and nutrition, evidence of impact in these areas is uneven in our sample programmes (see Table A3 in Annex 2). Where effects have been achieved, they are sometimes smaller than what the literature suggests is possible. In some cases, DFID recognises that its health and nutrition objectives are not being achieved; in Ethiopia and Bangladesh, DFID is supporting experiments combining cash transfers and other types of health and nutrition-related support in order to improve results.

The available evidence suggests that several of the programmes DFID is supporting are currently not optimised for maximum impact on nutrition and health, and that improvements in design and implementation – including timeliness and transfer size – could strengthen results in this area.[25] These challenges and the manner in which DFID is supporting governments to overcome them are discussed in the section on DFID’s work to strengthen national cash transfer systems.

Women’s empowerment is central to programme design, but impacts are not fully clear

DFID’s programmes place a strong focus on women’s empowerment in their business cases. In some cases, women are the sole recipients of cash transfers (eg Nigeria, Bangladesh). Other programmes take care to ensure that eligible women have access to the cash transfers. For example, the Ethiopian public works programme uses client cards that feature the names and photographs of both husband and wife, and includes a range of measures to ensure women have access to the income. In contrast, the Myanmar public works programme allowed only one member of each household to participate, resulting in a preponderance of men.

The literature suggests that cash payments to women can strengthen their decision-making power within the household and their choices about fertility. However, there is mixed evidence about the impact of cash transfers on domestic abuse, with both positive and negative results recorded depending on context and programme design.[25]

Despite the risk of causing harm under certain circumstances suggested by the literature, we found that none of the programmes in our sample were monitoring levels of domestic abuse. This is a surprising omission. While government counterparts may sometimes be reluctant to monitor domestic violence, owing to political sensitivities, DFID could invest in other monitoring or review mechanisms to ensure that programmes cause no harm in this respect.

DFID does monitor other empowerment indicators across its programmes. However, measuring empowerment is a challenging undertaking and we found DFID’s approach to be relatively weak. Only a few programmes were monitoring a basket of indicators capable of generating a robust picture of empowerment (see Box 12), despite the fact that DFID has been a pioneer of research on the impact of cash transfers on women’s empowerment.[26] The lack of robust programme monitoring in this area is a notable gap, suggesting that DFID is not putting its own research into practice or doing enough to build up practical experience on what works in this critical area.

The limited evidence that is available points to mixed results from DFID’s programmes. In Zambia, which set a logframe target on women’s decision-making, there was no measurable progress. Programmes in Kenya and Zimbabwe impacted positively on adolescent girls’ sexual health (see Box 13), even though this was not an explicit target for either programme.

DFID’s cash transfer portfolio is achieving its primary objectives, but with scope for improvement in other areas

Overall, we award DFID’s cash transfer portfolio a green-amber score for impact on poverty and vulnerability. The core objective of DFID’s cash transfer programmes is to alleviate income poverty and increase consumption for the poorest and most vulnerable, for the duration of the transfer. There is strong evidence that these results are being delivered consistently across the portfolio. Moreover, in some programmes the increases in income are in turn making beneficiary households more resilient to external fluctuations in their incomes, due mainly to increased creditworthiness and reduced debt. DFID has also achieved promising results from a more ambitious model of poverty graduation and is exploring how this could be replicated. These are strong results that, in our view, make a good case for DFID’s continuing investment in cash transfers.

There is no room for complacency, however. We found that DFID programmes are not always clear about what additional results to target, beyond the core objective of poverty alleviation, and therefore may not be optimised to deliver such results. Impact on school attendance has been below DFID’s targets, and we identified a few instances of zero or even negative impact. Evidence of impact on health and nutrition is uneven, both within and between programmes, with a pattern of results that DFID is sometimes unable to explain. Although women’s empowerment is a central concern in DFID’s business cases, its monitoring is not strong enough to generate robust results, and we are concerned that DFID has not done enough to monitor for unintended negative results such as increased domestic abuse. We are encouraged that DFID is beginning to explore how to combine cash transfers with other interventions, so as to maximise results in these areas.

DFID’s impact on the development of sustainable national cash transfer systems

DFID has successfully encouraged the introduction and expansion of national cash transfer systems in low income countries

When describing the cash transfer portfolio, DFID staff place more emphasis on the long-term aim of increasing the coverage, quality and sustainability of national cash transfer systems than on the immediate effects of DFID’s funding on today’s cash transfer recipients. The underlying objective is to extend coverage of cash transfers, particularly across sub-Saharan Africa, so that they become part of a standard toolkit in the fight against extreme poverty and vulnerability.

According to stakeholders in DFID, there are two dynamics that may, in the long run, support this goal. First, as national cash transfer systems become established, expand and demonstrate their effectiveness, public expectations and political commitment will grow and become mutually reinforcing. Second, the spread of national cash transfer systems across Africa would create a positive “neighbourhood effect” where national governments are under increasing pressure to emulate the actions of their neighbours.

Over the review period, government-operated cash transfer programmes in sub-Saharan Africa have indeed shown a rapid growth path, both in number and coverage. The number of countries with unconditional cash transfe programmes increased from 21 in 2010 to 40 in 2014 (out of 48),[28] reaching some 50 million people.[29] Within our sample, the programmes in Uganda, Rwanda and Zambia had expanded particularly fast.

In part, this expansion was underpinned by domestic political support that already existed at the start of the review period. Eight of DFID’s partner countries had signed the 2006 Livingstone Call for Action – an intergovernmental document calling for reliable long-term funding for social protection from both national budgets and donors.[30] The African Union had also adopted a policy framework on social protection.[31] In parallel, the World Bank has helped national governments to adopt or expand social safety nets.[32]

Building on this initial support, DFID contributed to the expansion in cash transfers through evidence based policy advocacy, financial contributions and technical assistance on a range of issues (Table A4 in Annex 2 lists some of the diverse objectives of technical assistance). DFID also funded pilot programmes, either through government or non-government partners, in order to demonstrate the potential benefits of cash transfers, create a body of context-specific evidence and stimulate public demand.

It is obvious that DFID funding has directly enabled national governments to improve and extend their coverage. It is more difficult to isolate the impact of other forms of DFID support, but it is reasonable to conclude that DFID has made a valuable contribution to building the commitment of its partner countries to developing national cash transfer systems for the poorest and most vulnerable.

African governments have increased their financial contributions, although sustainability is still some way off

One of the indicators of this increased commitment is national financial contributions and commitments (see Figure 2). In the last few years, the capacity and willingness of African governments to fund national cash transfer systems has changed dramatically. In the words of one DFID advisor, “governments have changed from the assumption that ‘we cannot afford this’ to the question of how much they would be able to contribute”.

This change has been facilitated by a decade of economic growth in Africa, buoyant commodity prices and better macroeconomic management, creating more budgetary space. However, it was by no means certain that these additional resources would be invested in social protection. DFID has provided seed funding for national systems and has actively advocated for national contributions. We find it plausible that this has helped to secure government funding in a number of cases.

DFID’s partner countries are, nevertheless, still a considerable distance away from providing sustainable national funding for their cash transfer systems (the same can be said for other public services and development programmes). Financial contributions from national governments are often irregular; in Rwanda and Uganda, pressure on national budgets has resulted in disbursements for cash transfers being below what was budgeted. Furthermore, existing programmes still fall well short of nationwide coverage, there are programme design problems and the amounts transferred to individual households are not always high enough to make a meaningful difference.

It is not yet clear that the political consensus is strong enough to sustain budgetary allocations through economic downturns and in the face of competing priorities. Recent discussions in the Ethiopian parliament illustrate this. The country has a long-standing, large-scale public works programme, but there are still voices arguing for a reprioritisation towards fuel subsidies (which are shown not to be effective in reducing poverty). This underscores the challenges that the government of Ethiopia faces in delivering on its objective of full domestic financing of the programme within the next decade.

While financial sustainability may be a long-term goal, it is nonetheless notable that DFID has not yet begun to formulate an overall strategy for achieving it. There is no explicit rationale for the size and duration of its financial contributions, which vary considerably. Nor does DFID explicitly seek to incentivise increased national contributions via its own funding (though it does encourage increased national contributions through other means). As national programmes begin to be consolidated, financial sustainability will become an increasingly important consideration.

DFID’s choice to work through national government systems makes sense, but there are some short-term disadvantages

There are both advantages and costs to working through national systems. When working with NGO partners, DFID has the significant short-term advantage of direct influence over programme design and implementation. In its NGO-implemented programme in Nigeria, for example, DFID can be confident that the cash transfers offered to pregnant and lactating women are soundly targeted, with a meaningful transfer size (£14 per household per month) and a good record on timeliness. On the other hand, being donor-financed and NGO-implemented, the programme has limited geographic coverage, rates poorly for sustainability and does not contribute to the goal of strengthening national cash transfer systems.

As only national governments are in a position to develop sustainable cash transfer systems with nationwide coverage, DFID chooses to work with and through national systems whenever conditions allow. As a result, while it tries to influence the design and delivery of programmes through policy dialogue and technical assistance, it must ultimately accept the partner country’s right to decide. In the short term, this entails trade-offs in several important areas – particularly around the efficiency of targeting, transfer size and the reliability of transfer payments – all of which can hamper the size and depth of programme impacts. Given DFID’s ultimate goal of building sustainable national cash transfer systems, we regard this as a legitimate trade-off, provided that DFID is doing everything reasonably possible to strengthen these programmes over time.

DFID takes a gradual and evidence-based approach to policy dialogue

In most instances, DFID supports the development of national systems over an extended period – either through long-term programmes (eg ten years in Zambia and eight years in Pakistan) or a sequence of shorter ones. During these engagements, DFID contributes to the development of national policies and systems but does not insist on a single programme model. Generally it avoids directly confronting counterparts on strongly held positions. Instead it chooses to tolerate shortcomings in the national systems it funds while aiming for incremental improvements over time. As one DFID advisor put it, “we are supportive, show our value, and then see if we are able to open other doors”.

DFID’s engagement is often informed by political economy analysis, which helps advisors to negotiate complex political terrain. In appropriate cases, DFID makes good use of evidence in its policy dialogue. It synthesises evidence from cash transfer programming around the world and shares it with national counterparts. It also invests in domestic research and in documenting local success stories, where it believes this would be more persuasive. It should be noted, however, that evidence-based advice is not always taken, and we have found examples where governments are resistant to evidence that challenges their design choices or calls into question the effectiveness of their programmes.

During our visit to Rwanda, we found evidence that DFID’s policy advocacy had significantly influenced counterpart attitudes and beliefs. Our documentary analysis and feedback from a number of Rwandan senior civil servants, DFID staff and technical advisors led us to conclude that DFID has verifiably helped to advance the government’s thinking about social protection. There is a clear progression of the technical assistance programme, moving from basic advocacy work on core design issues to more detailed advice on technical delivery challenges. For example, it is now agreed that people living with disabilities are an important target group; the assistance now focuses on how best to identify and target them. DFID has also influenced adjustments to some of the key social protection principles. These include ring-fencing (ie protecting against budget cuts) unconditional transfers and introducing a system of budgeting across regions based on need rather than pre-determined percentages.

Across the sample, DFID has made useful contributions to the development of national policies and strategies. Elements of the National Social Security Strategy in Bangladesh can be traced back to DFID’s engagement, as can the government of Myanmar’s decision to start a 1,000-day maternity cash grants system. Similar examples can be found in Kenya, Pakistan, Rwanda, Uganda, Zambia and Zimbabwe.

DFID are keenly aware of the need to protect against opportunities for fraud and leakage within the cash transfer programmes they support. We also noted, however, that DFID may end up avoiding politically difficult but important areas of engagement relating to targeting errors in their policy and advocacy work with national governments. We did note that on occasion, political sensitivities can mean DFID avoiding tackling targeting issues directly. In Bangladesh, for example, DFID’s technical assistance programme started out with a relatively lengthy diagnostic exercise rather than engaging directly with problems of targeting – both inclusion and exclusion errors – that were already well known and evidenced (and which were a key reason for DFID providing technical rather than financial assistance to the programmes in the first place).

It also chose not to address the systemic but politically sensitive problem of the fragmentation of the government’s many social protection programmes. While the Bangladesh case shows that DFID’s patient approach to policy dialogue helps to maintain good working relations with counterparts, this should not be at the cost of a lack of ambition or urgency in tackling the most pressing problems head on.

DFID’s technical assistance is focused on improving programme delivery

Cash transfers are a new undertaking for most of DFID’s partner countries. They often begin with limited capacity to design, implement and maintain a national cash transfer system. In the few countries with a history of cash transfer programmes, such as Bangladesh, these are fragmented and spread over multiple ministries, giving rise to an additional problem of overcoming vested interests.

Wherever it provides financial aid, DFID also provides technical assistance to strengthen national delivery mechanisms. DFID’s annual reviews identify a range of examples where its technical assistance has helped to translate government preferences into practical policies, systems and processes, with accompanying monitoring systems to promote adherence. Examples include the following:

• In Kenya, flexible technical assistance helped the government with its targeting and recertification processes (ie determining whether beneficiaries remain eligible). Other contributions related to a management information system and an electronic payment system.
• In Uganda, DFID supported the government in the development of a social protection policy framework. It helped to design and implement a consultation process that improved the quality of the policy and also helped to build government understanding and support for social protection at both national and local government levels.
• In Nepal, DFID helped social protection programmes with no history of cross-fertilisation to learn from each other.
• In Zambia, technical assistance was instrumental in enabling the programme to scale up from 32,000 recipient households in 2010 to 240,000 in 2016.
• A conditional cash transfer programme in Pakistan was mainly designed by DFID technical advisors before being handed over to government.

Across the portfolio, we identified technical weaknesses in three areas – targeting systems, timeliness of payments and transfer size – that can have a significant impact on effectiveness and value for money. While DFID is aware of and engaged on all of these issues, it is often required to tolerate shortcomings in one area in order to focus its efforts on another. We look at each of these areas in turn.

Targeting errors remain common across the portfolio

Targeting systems are basic to the design of cash transfer programmes. An effective means of ensuring that cash transfers reach the poorest and most vulnerable households is key to maximising impact. We found that targeting errors exist across the portfolio. They fall into three main types:

• Targeting that is not focused on the poorest and most vulnerable households. In Rwanda, the selection of recipients is based on the government’s Ubedehe system – a home-grown community cohesion instrument that correlates only weakly with poverty levels. Even where targeting is explicitly focused on the poorest and most vulnerable households, different concepts and measures of poverty and vulnerability can lead to very different selection outcomes. The Sahel programme provides a good illustration. In this programme, different tests were used to identify the poorest households and those most vulnerable to shocks. [33]These two groups should strongly overlap but, according to one key informant, the lists of eligible people had only a very minor overlap in some of the areas they surveyed.
• Inclusion errors. The inclusion of people who are not in fact eligible is all but inevitable in programmes that target on the basis of community selection or household income, as local power dynamics, misreporting and seasonal variations give rise to an unavoidable margin for error. Even programmes targeting specific groups may feature inclusion errors. A unique case was found in Nigeria where fake urine samples were reported to have initially led to the inclusion of women who were not actually pregnant. DFID successfully introduced random pregnancy testing to tackle the problem. The largest inclusion error we came across was in Pakistan, where the programme aims to reach the poorest 25% of the population but the World Bank found that a quarter of recipients fell well above that threshold. Rather than occurring from a failure to apply the targeting criteria properly, inclusion errors can also happen if the criteria themselves are inadequate. The child-focused programme in Zimbabwe used the criteria of labour constraints and food-poverty to identify households who may have been affected by the AIDS crisis and have children to care for. However, this resulted in 17% of the recipient households being included in spite of not having any children.[34]
• Exclusion errors. In Pakistan, the women who receive the transfer must hold a valid Computerised National Identity Card. This led to temporary exclusions, as it takes time to acquire this card. In the early implementation stages in Myanmar, the most remote and labour constrained households were effectively excluded from public works programmes, as were a range of other particularly vulnerable groups (eg single women with young children and other labour-constrained households, disabled people, households without local registration cards).

Not all targeting problems are equally problematic, nor do all require an immediate solution. In some cases, the effort required to fix the targeting would be disproportionate to the benefits. Sometimes the finer points of targeting assume a lower priority in the face of more pressing issues such as expanding national coverage.

We nonetheless found several cases (in Rwanda, Nepal and Zimbabwe) where, in our assessment, DFID should have made greater use of its influence to improve targeting and to monitor the results more systematically. In Pakistan, DFID and other key international stakeholders have shown that it is possible to persuade a national government to change its targeting paradigm even in the face of strong vested interests (see Box 14).

DFID has invested in improving the timeliness of cash transfers, but delays remain common

Across the sample, delayed transfers of funds to beneficiary households are the most common operational weakness (see Table 3). [35]Where the payments are not predictable, this reduces the likelihood of achieving the intended impact on consumption patterns. For this reason, many definitions of cash transfers incorporate the idea of regular and predictable payments.

DFID recognises the importance of timeliness and prioritises the issue in several countries. The country offices in Ethiopia, Nigeria, Rwanda and Zambia considered the issue important enough to include timeliness as a logframe target.

Nevertheless, the timeliness of payments is a recurrent problem in many of the national programmes. Gains are also sometimes temporary and can be reversed when new challenges appear. This is therefore an area requiring greater attention.

Table 3: Timeliness of payments across DFID-supported programmes

DFID does not have a clear view on the effects of transfer size on poverty outcomes

There is a wide variation in transfer sizes across the portfolio, and a range of rationales (see Table A2 in Annex 2). The size of the payment may be calculated based on the cost of certain food items, a percentage of the national poverty line or local labour costs in public works programmes. Affordability for national governments also plays a role and, given budgetary constraints, there is an obvious tradeoff between transfer size and coverage.

Transfers below a certain threshold are likely to be ineffective. Above that threshold, much depends on the country and context, but at household level and as long as the cash transfers do not provide a disincentive to work, more is likely to be better (see Box 15). Transfer size can also affect intrahousehold dynamics in unpredictable ways. Larger transfers may in some circumstances reduce abuse of women (due to a reduction in poverty-related stress and a better bargaining position for women), but in other circumstances increase their vulnerability to abuse (as a reassertion of male power). Balancing benefit and risk is not straightforward and requires careful context analysis and monitoring.

In the context of the “leave no one behind” commitment, it is particularly important to test and monitor the effects of transfer size on intra-household dynamics in certain sub-groups (eg for illiterate women or within marriages with a significant age difference) but DFID-funded programmes are not currently doing this.

DFID does not have a clear view on the effects of transfer size on poverty outcomes

In our case study countries, we saw very different DFID responses to the issue of transfer size. In Rwanda, evidence showed that unconditional transfers had more impact than public works, as participants were working too few days for the income to make a material difference. DFID then advocated successfully for an assured minimum number of annual work days. This is an example of the value of good monitoring and follow-up. In the urban poverty reduction programme in Bangladesh, however, we saw examples of education stipends amounting to just 60p per month, which were unlikely to have any impact on the girls who were their intended beneficiaries. Similarly, in another programme in Bangladesh (called “Economic Empowerment of the Poorest”), we observed that temporary pensions of £1.50 per person per month were too modest to make a significant difference to the lives of elderly people living in poverty. We saw no evidence that DFID had identified the problem or raised it with its programme partners (though DFID does advocate on precisely this issue in respect of cash transfer programmes run by the government of Bangladesh).

While there is no golden rule for transfer size, it remains an important determinant of the value for money of programmes. Our evidence indicates that DFID invests too little in the case-by-case monitoring and evaluation of these effects, and does not always recognise and act upon cash transfers that have dropped below the threshold needed to make a meaningful difference. DFID lacks a strategic approach to its technical assistance and its funding of national programmes

There is significant variation in the types of technical assistance offered by DFID for national cash transfer systems. Across the portfolio, DFID’s technical assistance includes funding for the development of national cash transfer systems, staff secondment, short-, medium- and long-term technical advisors, funding for governments to engage their own consultants and funding for project implementation units and coordination bodies. DFID staff also play an important role in most countries, actively engaging with government and other stakeholders on strategy, policy and technical issues.

There is no central guidance on capacity building. While DFID staff in each country are able to provide a rationale for their choices, we could find no clear explanation for variations in approach, budgets and timelines across programmes. While we acknowledge the need to respond to each national context, a more explicit strategy might help to keep DFID’s programmes focused on long-term goals.

There is also no clear strategy underlying the size or conditions of financial support. With the exception of the Pakistan and Uganda programmes, we did not see examples of performance triggers, where payments are linked to agreed reforms (common in DFID programmes in other sectors), or conditions designed to encourage governments to increase their budgetary contributions over time. We were not provided with any evidence, at country level or centrally, suggesting that DFID had articulated a pathway for achieving sustainable national systems.

The lack of a rationale for the volume of funding is significant. The national cash transfer programmes that DFID supports remain well short of achieving national coverage. If DFID believes that the programmes are successful, then increasing its financial support to enable them to scale up could represent a good investment, provided technical weaknesses in programme design and delivery could be addressed. The introduction of performance-based financing[37] might enable DFID to explore the viability of scaling up its assistance.

The balance between long-term capacity strengthening and short-term problem solving is not always clear or convincing

In the absence of a consistent approach, we found that DFID’s technical assistance was not always strategic in orientation. Because technical assistance is often provided in tandem with financial contributions, advisors can feel pressure to make sure that the flow of DFID funds through to the beneficiaries is not interrupted. While this is clearly legitimate, it can lead to advisors focusing on firefighting – that is, resolving immediate problems that threaten operations – even when the programme design suggests they should take a longer-term approach to building government capacity.

This concern was mentioned to us by technical advisors and DFID staff in a number of instances. While they acknowledged that a combination of short- and long-term focus is often required, they were not convinced that the balance was right. In Rwanda, we observed that DFID’s technical assistance has helped to resolve a range of short-term obstacles, but has struggled to achieve progress on some core issues such as establishing adequate baselines, building good administrative records and creating fiscal space for long-term funding.

Oversight and monitoring of technical assistance are not sufficient

Technical assistance is subject to a range of well-documented risks that can work against sustainable results, such as gap-filling and capacity substitution, generic rather than tailored programming and overreliance on technical solutions for problems that are not primarily technical. To mitigate these risks, technical assistance programmes should be carefully managed and monitored.

The evidence that we have seen suggests that this is not currently the case: programmes are often managed in a hands-off way by DFID staff, who do not always have the technical expertise to monitor the performance of technical advisors. Contact between technical advisors and the more specialised staff members in the DFID central team is infrequent.

A possible consequence of this is inconsistent advice to partners. DFID’s technical assistance is provided by implementing partners, which include both multilateral agencies and contractors. In a number of instances, DFID co-funds technical assistance with the World Bank. In Bangladesh, it cofunds with the World Bank, UNDP and the World Food Programme. According to key stakeholders, the social protection agendas of these agencies are not fully aligned with each other or with DFID. As a result, there are risks that DFID’s technical assistance is providing conflicting messages, or at least working without a clear consensus on the end goals and how to achieve them (for example one partner advocating for targeting on the basis of proxy means testing and another partner for community-based targeting).

We also found monitoring and evaluation of the results of technical assistance components to be inadequate. Across the portfolio, we found few explicit and coherent models of change and few meaningful baselines based on thorough capacity assessments. Not a single technical assistance contract has yet been independently evaluated. This makes it difficult to conclude that DFID is maximising the effectiveness of its technical assistance. This problem is likely to persist as the independent evaluations that are scheduled to take place in 2017 may not have the ingredients required to evaluate technical support: we have seen no evidence that DFID or the technical advisors are collecting the often fluid, time-bound data to make meaningful contribution analysis possible.

DFID’s system-building work has considerable scope for improvement

We have awarded DFID an amber-red score for its work on building sustainable national cash transfer systems.

We recognise that DFID has a long-term approach to building ownership of social protection that reflects a good understanding of national policy processes and political contexts. However, it lacks a strategic approach to technical assistance, does not have a consistent approach to improving the financial sustainability of national systems, and has not made consistent use of performance triggers to push forward reforms.

Evidence on the impact of DFID’s technical assistance for system building is fairly weak. We saw signs of progress in a number of countries in useful areas. However, DFID’s support has no coherent underlying model of change. The wide variations in approach seem to reflect the preferences of individual advisors rather than a clear strategy. Oversight is weak, monitoring systems are poorly developed and there have not yet been any independent evaluations. While capacity building is a challenging area, we conclude that there is substantial scope for DFID to strengthen its results.

Are cash transfers achieving value for money?

In this section, we assess the overall value for money case for cash transfers, based on the totality of evidence that we have reviewed. We then look at how well DFID integrated value for money analysis into its programme and portfolio management. We also look at how well DFID uses evidence and learning to improve impact over time.

DFID’s cash transfers demonstrate good value for money, with scope to provide additional funds to help partner countries scale up their cash transfer programmes towards national coverage

Looking across the full range of impact data collected for this review, we find that DFID’s cash transfer programming offers a strong value for money case. There is solid evidence that it delivers consistently on its core objective of alleviating extreme poverty and reducing vulnerability. Building national social safety nets is an important complement to DFID’s increased emphasis on promoting economic development. It is also consistent with the commitment to “leaving no one behind”. In November 2015, DFID pledged that “people who are furthest behind, who have least opportunity and who are the most excluded will be prioritised”.[38] Cash transfers are a proven method of helping the most vulnerable.

DFID has helped to promote the use of cash transfers in its partner countries and the expansion of national programmes. Recent evidence suggests that national contributions to cash transfer programmes are increasing across most of its partner countries.[39] There is also some evidence that programmes become more cost-efficient over time, as coverage expands and programmes mature.[40]

DFID’s choice to work through national government systems wherever possible necessarily entails a trade-off on value for money. While we found positive impact across the board, recurrent weaknesses in targeting, timeliness and transfer size show that there is scope to further improve value for money.
DFID must use its funding, policy dialogue and technical assistance as actively as possible to overcome these shortcomings in national programmes.

This analysis suggests that there may be a value for money case for providing additional funds to help partner countries scale up their cash transfer programmes towards national coverage – particularly where the additional funding could be used to help overcome programme weaknesses. For the time being, national programmes are still well short of national coverage, which means that there is room for DFID to increase its funding without displacing national expenditure. However, in due course, DFID will have to give greater attention to the question of how to achieve sustainable national financing.

A value for money focus is apparent in some aspects of programme management

In 2012, the Public Accounts Committee criticised DFID’s capacity to ensure value for money in cash transfer programming: [41]

In response, DFID developed a toolkit for achieving value for money in social transfer programmes that follows the “3E” model of economy, efficiency and effectiveness.[42] It aims to strengthen and promote consistency in value for money analysis, with a focus on cost drivers, cost-efficiency and cost-effectiveness. It also provides a set of benchmarks that can be used to compare the efficiency and effectiveness of programmes.

The influence of this toolkit is apparent in the value for money assessments that have been conducted across the portfolio (sometimes by the authors of the toolkit). The majority of the DFID advisors we interviewed reported that the toolkit had helped them to understand, adhere to and benefit from DFID’s value for money requirements. They identified two types of benefit.

First, they stated that value for money assessments focus their attention on the importance of minimising costs. In response to the Public Accounts Committee, DFID listed examples of measures that had been taken in particular programmes to reduce costs. We saw limited evidence that this was a major focus of effort, however.

The second benefit mentioned by DFID advisors is the use of value for money analysis to identify the variables with the greatest influence on the cost-effectiveness of their programmes, so as to address these in policy dialogue and technical assistance. We found a range of evidence that this is occurring. For example, in Rwanda this kind of analysis had led to technical assistance on targeting, financial management (to reduce payment delays), management information systems and accountability mechanisms.

Despite active support from central teams in DFID, the level of attention given to value for money issues still varies across country programmes. Some have conducted good quality value for money assessments and used them regularly to inform management choices (Kenya, Rwanda, Uganda and Zambia). Others have relatively weak assessments that are not regularly revisited (Bangladesh, Nepal and Pakistan).[43]

DFID collects value for money data but does not use it consistently to inform portfolio management

Across the portfolio, DFID measures efficiency indicators and in some instances tracks movement over time. However, it is not clear how this data informs its management decisions, and whether it enables DFID to drive up efficiency. Moreover, every value for money assessment in our programme sample reached the conclusion that the programme in question represented good value for money. Given the risk of optimism bias in such assessments, this suggests that value for money analysis is being used to justify existing choices rather than to inform active management.

DFID does not currently use cross-programme comparison of value for money indicators to inform portfolio management. The assumptions and measurement methods used across programmes are often too dissimilar to support a meaningful ranking. However, the available data could be used to identify outliers and assess whether remedial action is required. We have not seen evidence of this occurring,[44] and we found no examples of programmes being discontinued or put on improvement
plans because they demonstrated poor value for money.

DFID has made substantial investments in learning about what works

DFID’s Smart Guides on value for money recognise evidence and learning as important “cross-cutting enablers” of value for money. As described in Box 17, DFID has made an important contribution to building the evidence base on the effectiveness of cash transfers through a portfolio of innovative research. As well as large academic research projects, it has commissioned operational research to inform its programming. In particular, it has pioneered research into how the design parameters of cash transfer programmes affect results in different national contexts.

The DFID advisors we interviewed showed that they were familiar with this research, and were able to give examples of how it had shaped programme design. Despite this, we found that discussion of international research findings in business cases was limited.[47]

We also saw good evidence of DFID programmes “learning by doing” and adapting in response to evidence (see Box 18). Real-time monitoring is sometimes hampered by data gaps in national cash transfer systems, but with a few exceptions DFID’s investments in independent evaluations have been of good quality, and the findings have been used to strengthen performance. The business cases of forthcoming programmes in Bangladesh, Ethiopia, Kenya and other countries are all based on evaluative evidence from their predecessor programmes.

Cross-programme learning is facilitated by DFID’s small but active community of practice on cash transfers. Managed by the social protection team in policy division, the community of practice focuses on themes such as cash transfer financing, capacity building, accountability, value for money, gender, disability, shock-responsive social protection and nascent social protection systems in fragile contexts. It serves as a conduit for disseminating learning on these areas and for sharing lessons among country programmes, and we saw evidence of cross-programme learning shaping action. For example, experience from Kenya and Niger informed DFID’s Nigeria programme; Ugandan officials learned from experience in South Africa and Mozambique; and research in Zimbabwe was designed to complement similar research conducted in Kenya and South Africa.

Conclusions on value for money

Overall, we have given DFID’s cash transfer portfolio a green-amber score for value for money. It offers a good value for money case, based on consistent and well-demonstrated results in its core objectives and its good fit with DFID’s strategic objectives. DFID has done well at investing in evidence of what works, to inform its programming, and demonstrates good learning. It carries out value for money assessments across the cash transfer portfolio. However, it needs to get better at using value for money data to inform management decisions.

Conclusions & recommendations

Overall, we have given DFID’s cash transfer portfolio a green-amber score. It is delivering consistently well against its core objective of alleviating extreme poverty and reducing vulnerability. In secondary results areas, such as education, health, nutrition and women’s empowerment, its results are inconsistent and in some cases below what might be expected. There is a need for greater clarity about which results are being addressed in each programme, and programme designs and monitoring arrangements should be optimised to achieve those results.

DFID’s support to national cash transfer systems has helped its partner countries to introduce and consolidate national social safety net programmes. DFID’s policy advocacy is well informed by evidence and analysis, but its approach to technical assistance is inconsistent and its results are poorly evidenced. Across the portfolio, there are recurrent weaknesses in targeting, timeliness and payment size, which need to be more consistently and firmly addressed. DFID is yet to develop a clear strategy for using its funding to promote the development of sustainable national systems.

We find that DFID’s cash transfer programming presents a good value for money case, given its proven ability to deliver results for the poorest, consistent with the “leave no one behind” commitment. Given that the programmes DFID is supporting are still well short of national coverage, there is also a value for
money case for increasing funding during their scale-up phase, provided the performance issues can be addressed.

We offer a number of recommendations for how DFID could improve the impact and value for money of its cash transfer portfolio.

Recommendation 1: DFID should consider options for scaling up contributions to cash transfer programmes where there is evidence of national government commitment to improving value for money, expanding coverage and ensuring future financial sustainability.

Problem statements:

• The national cash transfer programmes that DFID supports are still well short of national coverage of the poorest and most vulnerable households.
• DFID at present lacks a clear rationale for the size of its financial contributions.
• Despite recurrent problems with targeting, timeliness and transfer size, DFID has not built performance triggers or similar mechanisms into its financial assistance.
• DFID lacks a strategy for promoting long-term financial sustainability.

Recommendation 2: DFID should be clearer about the level and type of impact it is aiming for in each of its cash transfer programmes, and ensure that these are adequately reflected in programme designs and monitoring arrangements.

Problem statements:

•  DFID is inconsistent in the way it sets the objectives and targets for its programmes, and there is weak correlation between the business case objectives, the logframe targets and the results actually achieved.
• DFID’s results in areas such as education, health and nutrition lag behind what the global evidence shows is achievable.
• DFID could do more to ensure that the design and delivery of individual programmes support the delivery of results. This may include a more prominent place for programmes that combine cash transfers with other interventions.

Recommendation 3: DFID should do more to follow through on its commitment to empowering women through cash transfers by strengthening its monitoring of both results and risks, and using this data to inform innovations in programming.

Problem statements:

• DFID’s strong focus on empowerment of women in business cases is often carried through into programme design and implementation, but poor monitoring means that the results are unclear, that negative unintended consequences may not be identified and that risks are not mitigated.
• The lack of close monitoring of the risks of harm to particularly vulnerable women beneficiaries in the form of increased domestic abuse is a problem across the portfolio.

Recommendation 4: DFID should take a more strategic approach to technical assistance on national cash transfer systems, with more attention to prioritisation, sequencing, monitoring and oversight.

Problem statements:

• DFID’s approach to technical assistance and capacity building varies substantially across countries and programmes, without a clear rationale.
• DFID’s technical assistance is not always based on holistic assessments of need and clear identification of priorities.
• There is a tendency for some technical assistance programmes to get drawn into addressing shortterm problems, at the expense of a sustainable focus on strategic challenges.
• DFID lacks a consistent focus on key determinants of programme effectiveness in areas such as targeting, timeliness and transfer size, where problems often remain unresolved for long periods.
• DFID’s oversight, monitoring and evaluation of its technical assistance programmes are inadequate.

Annex 1 Detail of scoring

Question 1: Impact on poverty and vulnerability  – Green/Amber

DFID’s support for cash transfers is an important part of its fight against poverty and vulnerability. Over the 2011-2015 period, DFID exceeded its target of supporting six million people with cash transfers. The portfolio has delivered well against its central objective of alleviating extreme poverty and increasing consumption levels for the poorest households, and has made a positive but modest contribution to increasing their resilience. In Bangladesh, DFID has helped to develop a more ambitious model for helping extremely poor households graduate from poverty, by combining cash transfers with other interventions. In secondary results areas, including education, nutrition, health and the empowerment of women, performance has been mixed and on occasion falls short of what the evidence suggests is achievable. We found that DFID programmes are not clear enough about what kinds of impact they wish to achieve. As a result, programme design and monitoring arrangements are not sufficiently results-led. Nonetheless, solid achievement against its core objective means that the portfolio merits a green-amber score for impact on poverty and vulnerability

Question 2: Building national cash transfer systems – Amber/Red

DFID has made an important contribution to encouraging and supporting its partner countries to establish and expand national cash transfer systems. It takes a long-term approach to policy dialogue that is both evidence-based and politically informed, but is at times risk-averse in its advocacy. Its technical assistance has delivered a range of useful results, but its approach is inconsistent without a clear rationale. We found recurrent problems across the portfolio in the core areas of targeting, transfer size and timeliness of payments that are not being consistently addressed. DFID has not made sufficient use of performance triggers in its financial assistance to drive progress. Its technical assistance is sometimes drawn into short-term problem solving at the expense of long-term strategic objectives. Oversight and monitoring of technical assistance is weak and has not been subject to independent evaluation. While we recognise the scale of the challenge in building sustainable national systems, we conclude that there is substantial scope for improvement. We have therefore awarded DFID an amber-red score for its work on building national cash transfer systems.

Question 3: Value for money – Green/Amber

We find that DFID’s cash transfer programming offers a strong value for money case, owing to the consistent results on alleviating poverty and vulnerability and its fit with DFID’s “leaving no one behind” commitment. However, there are short-term trade-offs involved in delivering through national systems, and recurrent weaknesses in targeting, timeliness and transfer size show that there is scope to further improve value for money. Our analysis suggests that there may be a value for money case for scaling up cash transfer programmes towards national coverage, provided that there is commitment to addressing these issues, improving value for money and increasing national funding. DFID has increased its focus on value for money in the management of its cash transfer portfolio over time, but its practice could be more consistent. We saw evidence of value for money assessments being used to inform programme design and technical assistance, but less evidence that it was driving cost savings. DFID has made a strong contribution to the global evidence base on what works, and has used research and learning to strengthen programme design. Overall, we have awarded the portfolio a green-amber score for value for money.

Overall Score

This adds up to a green-amber score for the cash transfer portfolio as a whole. Cash transfers have emerged as an important part of DFID’s fight against extreme poverty and its commitment to “leaving no one behind”. There is strong evidence that they are effective in their core objectives and offer a good value for money case. However, there is also room for improvement. DFID’s cash transfer portfolio is not yet achieving the full range of results that the evidence suggests is possible.

Annex 2 Further Data

Further data on our programme sample, transfer payment size, health and nutrition results across DFID programmes and examples of programme logframe outcome indicators is available in the PDF version of the report.

Annex 3 Methodology

Four methodological elements jointly provided the data and insights required to answer our three review questions. The figure below shows these four elements and the way in which they allow for an assessment of three levels of analysis: i) a review of a sample of cash transfer programmes, ii) a review of the cash transfer
portfolio as a whole, and iii) an assessment of the way in which this portfolio is embedded in higher DFID agendas.

The literature reviews served two purposes. First, synthesis reports produced at the start and end of the review period provided overall findings in relation to the effects of cash transfers. We used these findings to compare the effects of DFID-funded programmes. Second, we compared DFID’s research choices and
contributions with the worldwide body of cash transfer literature, to assess the extent to which DFID has contributed to filling relevant evidence gaps (we present the summary conclusions under the first review question and the full report as an appendix to this report).

DFID interviews and document reviews. Interviews with 36 DFID staff focused on either strategic portfolio issues or on one or more of 18 out of 28 programmes that DFID presented to us (see Box 19 for the methodological implications of our sampling choices). The documents we reviewed were a combination of higher-level policies (such as DFID’s policy paper on its “Leave No One Behind” promise) and programme documents (with a focus on business cases, logframes, baseline studies, political economy analyses, annual reviews, project completion reports and independent evaluation reports).

External interviews with a dozen academics and peers from other organisations helped us to gain outsiders’ views on DFID’s cash transfer approaches and choices. By selecting known “critical voices” we reduced the risk of us, the reviewers, getting unwittingly pulled into DFID-type thinking (such as the conviction that it is inherently important to work through and with national governments).

Country case studies. The selection criteria were that the country portfolio must be sizeable, include both technical assistance and financial support, and have been operational for at least a few years. Of the five countries that fulfilled these criteria, we selected Bangladesh and Rwanda because the programmes are so
different that they jointly cover a very wide spectrum of DFID’s cash transfer work (see Figure 4).

The methodology is explained in full in our Approach Paper, which is on the ICAI website. Both our methodology and this report were independently peer reviewed.

Limitations of our methodology

Some research is very thorough. Some other research does not stand up to scrutiny. In the case of the latter, we generally just ignored it (for example the section on education does not mention the poorly executed education assessments in Bangladesh’s UPPR and EEP). If there is a correlation between the quality of programmes and the quality of their assessments, this could mean that our evidence has a positive bias.

We looked at only a sample of DFID’s cash transfer portfolio. We sampled purposively to reflect the full spectrum of programme objectives, modalities, support, types and sizes, but the sample may nonetheless not be fully representative. This has been particularly problematic in cases where we report on exceptions. For example, we found that DFID Bangladesh had over-reported the number of cash transfer recipients by almost half a million people and we simply do not know whether or not this is the only case of overreporting.

It has been hard to gain real insight into the usefulness of DFID’s technical assistance. In social protection’s complex multi-stakeholder environment, assessing the results of DFID’s advocacy and influencing work is challenging. Evidence of DFID’s role in reaching trigger, tipping and turning points in a
government’s thinking and practice – which could potentially be among DFID’s biggest achievements – has not been captured in real time, and has therefore been lost or become unverifiable.

A fuller discussion of the methodological limitations is included in the review’s Approach Paper, available on the ICAI website.