When aid relationships change: DFID’s approach to managing exit and transition in its development partnerships 2

Purpose and scope of the review

What happens after DFID announces it will end bilateral aid to particular countries?¹ Is aid replaced with new types of assistance relevant to the country’s needs? Is the transition managed in a way that will protect past achievements and maintain a positive relationship with the country in question? This review looks at DFID’s approach to transitioning to new forms of financial and non-financial support. We will examine a number of recent and ongoing transitions and explore the implications for managing future transitions.

Since 2011, DFID has refocused its aid programme on 28 priority countries. This has involved terminating or significantly reducing bilateral aid to at least 15 countries. However, the UK continues to have important relationships with many of these countries, such as India and South Africa. Bilateral aid is often replaced by new forms of collaboration in areas of mutual interest or international concern, involving new partnerships with other UK government departments. With different UK objectives involved in each transition, there is no single pathway or process.²

This is a new area of the management of UK aid where there has been little evaluation or systematic lesson learning to date. This review is an opportunity to illuminate a relatively unknown, but increasingly important, part of the aid programme. The review is of public interest as it sheds light on the process and early consequences of reducing aid to particular countries, and the evolving nature of the UK’s relationships with some important countries, such as China and India. It will also enable us to explore how well DFID works with other UK government departments and balances ongoing development interests with other policy objectives.

Type of review

This is a performance review which explores the effectiveness of the transition process in partner countries. The review will explore the context for the decision to transition and how well the transition itself is planned and carried out. The emerging consequences of transition will also be examined. This review includes important learning elements, as the nature of the transition challenge is continually changing. It has elements of real-time assessment of an emerging area, offering a snapshot of efficiency and effectiveness with a view to informing future transitions.

Scope

This review examines DFID’s approach to transition from development partnerships based on bilateral aid to new forms of financial and non-financial partnerships delivering broader UK government objectives.³ We do not consider the original decision to transition away from bilateral aid, which, as a policy decision, is beyond ICAI’s remit. However, we do consider DFID’s approach to the transition and whether it is managed so as to maximise the benefits. These include achieving cost savings and promoting stronger bilateral relationships, and minimising the risks, such as loss of momentum on poverty reduction or a reduced ability to achieve UK government aims in-country.

Our review covers transitions initiated from the 2011 Bilateral Aid Review (BAR)⁵ onwards. In the BAR, DFID identified 16 countries for transition. India and South Africa were later added to the list. As there is no fixed definition of transition, there can be no definitive list of cases and we remain open to identifying additional cases during the course of the review.

The review examines the relevance of DFID’s approach to transition to each country’s development needs and to wider UK government strategic goals. It considers how effectively the approach was implemented. It also explores lesson learning on the transition process within DFID and across the UK government.

While the review focuses on DFID’s performance, rather than the performance of the UK government as a whole, it includes how well DFID engages with other government departments (OGD) in the UK and whether it promotes development objectives effectively throughout the transition process.

Background

The global footprint of UK aid is changing rapidly. In 2008-09, 140 countries received some form of UK bilateral aid,⁶ of which, 87 received support from DFID alone. Seventeen DFID bilateral country programmes closed in the five years following the 2011 Bilateral Aid Review. By 2015, DFID had prioritised just 28 countries.⁷ In 2015, financial aid to India ended, though its bilateral aid continues. Two long-standing UK bilateral aid programmes in South Africa and Vietnam also came to an end in 2015 and 2016 respectively. These live examples, together with recently closed programmes, provide an opportunity to explore DFID’s emerging post-aid relationships with countries with varying levels of income, poverty needs and regional profiles, and to examine the principles and processes through which DFID manages the different types of transition.

There are also a number of contextual factors that make this review timely. Patterns of global development finance are changing rapidly. Developing countries now have greater opportunity to fund their national development from domestic revenues and private financial flows, which have grown in importance relative to official development assistance (ODA).⁸ At the July 2015 International Conference on Financing for Development in Addis Ababa, it was recognised that the future role for ODA will increasingly be about leveraging other sources of development finance, rather than direct funding to services and investments.⁹

This means that the nature of the UK’s development partnerships is changing in many parts of the world. As the International Development Committee (IDC) has concluded, this calls for DFID to look ‘beyond aid’¹⁰ to other ways of influencing global development. Rather than ending its relationship with countries after transition, the UK government is increasingly trying to establish new forms of partnership, based upon areas of mutual interest. This will continue under the 2015 UK Aid Strategy,¹¹ which signalled a refocusing of the UK aid budget towards global challenges, such as mass migration, violent extremism, global health threats and climate change.

Theory of change

There is no explicit overarching theory of change governing transition. The process is driven by a wide range of objectives and interests. Some objectives are unique to each case, such as helping India to become a more effective development partner or working with Indonesia on climate change. The review will map how different outcomes are pursued in different types of transition.

Existing evidence

Although many bilateral donors have managed exit and transition from development partnerships in recent decades, there is limited literature on the subject. A 2008 Swedish evaluation found that “systemic and good monitoring of exit processes is extremely rare”.¹² DFID has published an internal note on the practicalities of exit, but there is no more recent or broader lesson-learning material available on the strategic aspects of exit, graduation, regionalisation or transition decision making, planning or implementation.¹³ There are internal papers from the Chief Economist’s Office which do not have policy standing, but which consider the technical criteria for deciding to exit or transition, and discuss the role of DFID in countries graduating to middle-income country (MIC) status. There are also analytical works which have relevance to the transition debate. However, there are no formal reviews, lesson-learning or policy directions available on transition itself. We are aware of an ongoing evaluation of exit by the Netherlands, and evaluations conducted by both DFID¹⁴ and the Government of Vietnam covering that particular transition.

This review will help close this information gap. It builds on our earlier analysis published in the Allocation Information Note¹⁵ (December 2015) and How DFID works with multilaterals to achieve impact¹⁶ (June 2015). It also builds on reviews by the IDC of DFID’s closure of its bilateral programmes in Burundi (2012), India (2011) and South Africa (2015).

Review criteria and questions

The review is built around the OECD-DAC evaluation criteria of relevance and effectiveness, with an additional question on evidence and learning. It will consider the following questions and sub-questions:

1. Relevance: is DFID’s approach to transition relevant?
   1. Are DFID’s objectives for transition clear and coherent?
   2. Are DFID’s choices about the approach to transition consistent with the goals of protecting development gains and reducing the risk of development reversals?²⁰
2. Effectiveness: how effectively is DFID transitioning to a post-bilateral aid relationship with partner countries?²¹
   1. How well has DFID managed its partnerships with country governments and other key stakeholders through the transition process?
   2. How well does DFID influence and work with other UK departments to ensure development considerations are embedded within wider UK government bilateral relationships?
   3. How does DFID ensure that it is delivering value for money in the context of such transitions?
3. Effectiveness – evidence and learning: how effectively is DFID capturing and applying learning to support its transition approaches?
   1. How effectively is DFID learning from current and previous transitions and applying this to its guidance, systems and practices?
   2. To what extent is DFID capturing and sharing learning across the government to determine the most strategic whole-of-government transition approaches, including around value for money?

The last section of the approach paper contains the full Review Framework, setting out the review questions and sub-questions alongside specification of which components of the methodology will generate answers to each question.

The available literature highlights a number of risks. It suggests that agencies are generally too inward-looking, focusing on their own exit management processes and missing opportunities to share experiences and learn from one another. Transition or exit decisions are often driven by domestic politics and budgets. This leads donors to act unilaterally, even though partner countries may actively seek new opportunities for trade, investment and technology transfer.¹⁷ The process often focuses too heavily on the financial capacity of partner countries as they graduate to MIC status, paying less attention to institutional capacity.

Exit/transition decisions taken unilaterally can catch the partner country unawares, potentially disturbing the chances for positive future collaboration. The need for clear transition plans and a gradual transition is a consistent theme.¹⁸ Slob and Jerven (2008)¹⁹ note that minimal attention is paid to institutional learning on how to manage aid exit and transformation. Donors need specific skills to manage a successful transition to new bilateral relationships, which are different from those required to wind up aid programmes.

Core ICAI issues

This review is closely linked to ICAI’s ‘Beyond Aid’ theme. It explores the UK’s role pre- and post-transition in-country on issues such as trade, migration, private sector development and global public goods including climate change.

ICAI has also identified certain core issues relating to DFID’s organisational capacity and its cross-cutting policy commitments. This review is primarily concerned with cross-government working, including DFID’s collaboration with other departments in delivering the UK Aid Strategy and other policy commitments. It also raises important questions on working with and through others, including the choice of delivery channels such as multilateral and vertical funds, central and regional programmes and development capital. Whilst not defined as a core ICAI issue, value for money is one of the other considerations explored below.

Overview of methodology

The review uses a range of qualitative methods, centred on a series of case studies of individual transitions. Consultations with key stakeholders in the UK and in each case study country will be used to capture how well the approach to transition reflected the strategic context and how well it was delivered. There are two main components to the methodology:

• A strategic review: this explores the larger strategic context for DFID’s approach to transition, as it has evolved over time. It includes a review of the literature, a review of DFID documentation on transition and key stakeholder interviews in the UK. Together, these elements of the review will allow the team to map different types of transition and their objectives, including how the objectives for transition have changed over time.
• Case studies of individual transitions: we will conduct seven individual case studies to identify the relevance and quality of implementation of the transition approach and generate lessons. In each case, we will generate an analytical narrative exploring the objectives and approach taken to transition. The seven case studies have been selected to cover a range of transition contexts to produce findings that can be more broadly used in helping to guide future transitions.

Component 1: Strategic review

A thorough assessment of DFID documentation will be conducted, covering policies, guidance and programme documents related to the different types of transition. This will include country-specific documents from the 18 countries that transitioned from bilateral aid between 2011 and 2016. We will also examine documents from other UK government departments, other donors and academic commentary. This documentation will be supplemented by analysis of published and unpublished data on ODA flows over time. Through the document review, we will:

• Explore the approach to transition across the DFID portfolio since the 2011 Bilateral Aid Review, including the policy drivers, decision points, phases and trends in aid disbursement, including value for money considerations.
• Elaborate a typology of transitions and transition objectives.
• So far as is possible, derive an implicit overarching theory of change for transition, drawing on DFID’s statements as to its objectives, approaches and desired results.

Component 2: Seven case studies of country transitions

We will conduct an assessment of the available literature to identify trends and good practices in transitioning from bilateral aid. We will collect evidence on other donor approaches to transition. Special attention will be given to identifying opinions from partner countries. The literature review will help us to:

• Frame the context of the review, including triangulating the timeline and typology of approaches to transition, bringing in the experiences of other donors and locating the topic in the changing international aid architecture and patterns of aid flows.
• Explore approaches to and views on transition from other UK government departments and other donors. We will pay particular attention to the existing evaluative evidence.
• Understand the needs of middle-income countries in the context of transition, both at process level and in terms of the desired end results.

We will conduct semi-structured interviews with a range of UK stakeholders to explore the review questions from different perspectives. This will include mapping the transition process and the engagement of the UK government, civil society organisations (CSOs) and others in the design and implementation of that process. We will seek to capture the range of UK government perspectives in different cases of transition, with other UK departments acting as primary interviewees for assessing lesson-learning and value for money. Stakeholder feedback will help us to understand the post-transition situation, including the ongoing role of DFID. To reduce the risk of bias in responses, we will triangulate feedback between the main stakeholder groups, including DFID and other UK departments, partner governments and other in-country stakeholders including other donors, CSOs, and the private sector. The review will ensure that critical voices are included for each topic of investigation.

We will conduct case studies of seven countries to review the experience of transition, its implementation and, where possible, its consequences. The country cases selected are China, India, Vietnam, Indonesia, Burundi, Cambodia and South Africa. The selection process is explained in the section on ‘sampling strategy’. These case studies have been selected to allow an assessment of varying transition approaches and country contexts. They cover both live transition cases (where institutional memory will be strong and many stakeholders will be available to discuss, but where it is likely to be too early to judge effectiveness), and completed transition experiences, where it will be possible to explore longer-term consequences (such as post-transition relationships and cooperation).

In each case, we will:

• Map the drivers, expectations and objectives for each transition.
• Assess the extent to which the transition objectives and approaches took into account partner country needs and interests as well as the UK’s comparative advantage.
• Assess DFID’s performance against its transition objectives.
• Construct a timeline of transition processes and actions.
• Explore whether transition decisions were informed by the need to protect development gains and reduce the risk of reversal.
• Map the changing patterns of ODA flows.
• Assess the degree to which value for money featured in DFID’s decisions and implementation processes.
• Assess how well DFID works with partner countries, national stakeholders and other UK departments during the transition process.
• Assess whether DFID learns and applies lessons about transition.

Each case study will include a documentary and literature review, together with key stakeholder interviews. For the desk-based case studies, the team will identify former DFID or government staff responsible for key aspects of the transition and interview them in London, or by telephone. Where possible, telephone interviews will also be held with a representative from the partner country government, from a local civil society group which has been involved in DFID’s transition, and from another donor in-country. For the two country-based case studies, we will conduct detailed consultations with DFID and other government departments represented in-country. We will interview key stakeholders in the partner country encompassing officials in government, donors, civil society groups and, depending on the local context, private sector bodies and academic representatives who have views on DFID’s approach to transition. We will explore their strategic interests in the UK development partnership, the extent to which they were consulted about transition processes and their experience with its implementation. As well as triangulating information received from DFID, we will identify critical views that can be used to challenge DFID on its choices.

The two country visits will each last up to ten days, during which we will conduct in-depth semi-structured interviews with particular stakeholder groups in the respective capitals, and conclude with stakeholder workshops with CSOs to triangulate initial findings. We will use a range of interview techniques to reduce the risk of bias. Stakeholder feedback will be kept confidential. We will develop a Country Approach Paper for each case study in advance of the visit, identifying the relevant questions from our review framework, linked to the classes of stakeholders we will interview.

The case studies have been selected to reflect a variety of transition experiences, based on the type of transition (exit, graduation, regionalisation, etc) and which themes of interest to the review were relevant (eg the UK’s relationships with emerging powers; global public goods issues such as climate change). The selection criteria are set out in full in the section on ‘sampling strategy’. Not all of the issues in our review framework will be pertinent to each case study. We will therefore adapt our methodology to explore different aspects of our review questions in each case. We will also ensure that our case study approach is flexible enough to enable us to identify and explore unanticipated difficulties arising in each country.

Our review questions ask us to explore how DFID ensures value for money during its transitions, and whether there has been learning across UK government departments on this issue. Given the variety in types of assistance across the case study countries, we do not propose to develop a specific value for money assessment methodology; rather, we will explore the extent to which DFID has taken account of value for money in its decision making.

Sampling strategy

The 2011 Bilateral Aid Review²² lists 16 countries where DFID ministers took the decision to end bilateral aid.²³ Transitions were subsequently announced for India in 2012²⁴ and South Africa in 2013,²⁵ both of which are still in the process of transitioning. This gives a total of 18 countries which transitioned from bilateral aid between 2011 and 2016.²⁶,²⁷

To select our case studies, countries were sampled purposively in two steps (see Table 1 for the detailed criteria).

First, we identified the most significant cases in the sample – that is, countries where there was sufficient aid before transition. We compared DFID’s ODA spend to each country’s GDP as well as the absolute level of DFID ODA each country received. This led to the exclusion of Angola, Bosnia and Herzegovina, Cameroon, Kosovo, Moldova, Niger, Russia and Serbia. Iraq was retained at this stage, despite a small DFID aid programme, because of substantial other UK government spend in-country. Lesotho and Gambia were retained because the UK footprint was substantial in relation to their GDP.

Second, countries were then classified according to their different types of transition, their relevance to the key themes of the review, and likely future transitions. At this stage, countries with a specific context that were considered unlikely to yield generalizable results were removed. Iraq was excluded on these grounds, as well as the fact that post-conflict countries are unlikely to feature substantially in DFID’s forthcoming transitions, given DFID’s commitment to spending half of its budget in fragile states. Burundi, however, also a fragile state, is included in the sample primarily because of the significance of DFID’s ODA in country – although absolute levels were low, the significance of DFID ODA to Burundi’s GDP was relatively high (representing 0.7% of Burundi’s GDP).

Table 1: Sampling criteria and rationale

Of the seven resultant significant countries, only China, India and South Africa were identified as suitable for country visits – primarily because there are still DFID offices in those places. One field visit should consider the consequences of transition and should therefore cover a country that has already transitioned. Only China meets this condition. The choice of India over South Africa for the second case study visit is primarily influenced by size, with India reflecting a much more significant DFID aid footprint (£191 million a year on average in India; £16 million a year on average for South Africa) and strategic significance (strength of links to the UK, importance as a regional power, trading links and diaspora ties).

As a result of these sampling processes, the following case studies were selected:

Remote (desk based) case studies: Vietnam, Indonesia, Burundi, Cambodia and South Africa.

Case studies with country visits: India and China.

Limitations to the methodology

There are a number of limitations to this methodology. Firstly, as this is a rapidly evolving area of practice, many of our case studies are of ongoing or very recent transitions. This limits our ability to explore the consequences of transition other than in a provisional way.

Secondly, transition is not a form of development assistance, and there are no clearly stated objectives against which we could measure performance. There are also no counterfactuals available for comparison. In strict terms, the ‘evaluability’ of the topic is therefore limited, a fact we took into consideration in our decision to undertake this review. To manage this, we will establish working definitions of the concepts involved and derive an implied theory of change from DFID’s written documentation, in order to inform our enquiry.

Thirdly, the small sample size, the limited number of stakeholders who are in a position to comment on each case and the purposive selection of cases all introduce a risk of bias. We have mitigated this by including seven country case studies. Our key stakeholders will be assigned to groups (eg DFID, other UK government, partner government, etc) and findings from interviews will be flagged as coming from particular stakeholder groups. Feedback from each group will be collated and triangulated with other groups and the literature.

Finally, the findings from the case studies will be specific to each country. By sampling across transition experiences over seven countries in total, findings will be triangulated across experience and some broader lessons might be determinable. However, care will have to be taken in generalising lessons for transition in other contexts.

Ethical considerations

ICAI reviews are undertaken with integrity and transparency. We are guided by ethical guidelines and codes of conduct of professional bodies such as OECD DAC.²⁸ When engaging with stakeholders we respect the principles of confidentiality and informed consent. When consulting participants or key stakeholders the review team ensures the research does not constitute harm. We aim to ensure a high degree of scrutiny of DFID’s performance while ensuring an awareness and appreciation of the operating context and challenges in which DFID operates. There is a possibility that consulting with government and civil society in-country may re-open conflicts that have been closed or raise expectations about future programming. We will mitigate this by consulting with UK offices (FCO and DFID) in-country to agree an appropriate form of words to use while interviewing in each specific case.

Research tools

The following research tools will be developed, to support the delivery of this methodology:

• Terms of Reference for the literature review.
• Operational Review Framework mapping questions to methods and approaches including guiding which sections are applicable to each case study.
• Country case study approach papers with country review framework and causality map alongside a high level stakeholder interview plan.
• A work programme for each country visit, identifying which stakeholders will be interviewed.

Quality assurance and peer review

This review will be carried out under the guidance of Lead Commissioner Francesca Del Mese. The Review Manager in ICAI is Miriam McCarthy.

This review will be peer reviewed externally at two points: an initial assessment of the methodology, which has been incorporated into this approach paper; and the draft report. The peer reviewer is Richard Manning, Fellow at the Blavatnik School of Government, University of Oxford. Richard served in DFID from 1965 to 2003. From 2003 to 2008, he was chair of the OECD DAC, and served as co-chair of the High-Level Forum on Aid Effectiveness in 2005 at which the Paris Declaration on Aid Effectiveness was adopted. He has held a number of other senior positions, including chair of the Board of the Institute of Development Studies from 2008 to 2014 and board member of BBC Media Action from 2009 to 2014.

Risk management

Timeline and deliverables

Review framework

Footnotes

1. Although bilateral aid may end, countries can continue to receive official development assistance through centrally managed programmes, multilateral channels and other government departments. India is a special case: financial aid ended in 2015, but bilateral aid spending continues. ↩
2. Closing Aid Programmes in Middle-Income Countries: A Big Opportunity, Runde, D.F., Center for Strategic and International Studies (CSIS) Critical Questions, 2011, link; CSIS Strategic Foreign Assistance Transitions: Enhancing US Trade and Cooperation Relations with Middle-Income Countries, A report of the CSIS project on prosperity and development, 2012, link. ↩
3. In the case of India, financial aid has ended but bilateral aid spending continues. ↩
4. Mobilising Resources for Sustainable Development, OECD Development Cooperation Report 2014, link. ↩
5. Bilateral Aid Review: Technical Report, DFID, 2011, link. ↩
6. This included 18 countries where DFID only provided pension payments, 35 countries in which the only bilateral aid was provided by OGDs and bodies, and 87 countries where DFID alone delivered aid. Bilateral Aid Review: Technical Report, DFID, 2011, link. ↩
7. Bilateral Aid Review: Technical Report, DFID, 2011, link. ↩
8. Combining Finance and Policies to Implement a Transformative Post-2015 Development Agenda, European Report on Development, European Union, 2015, link. ↩
9. Outcome Document of the Third International Conference on Financing for Development, The Addis Ababa Action Agenda of the Third International Conference, 13-16 July 2015, link. ↩
10. The Future of UK Development Co-operation: Phase 2: Beyond Aid, International Development Committee, Tenth Report of Session 2014-15, February 2015, link. ↩
11. UK aid: tackling global challenges in the national interest, November 2015, link. ↩
12. Managing Aid Exit and Transformation: Lessons from Botswana, Eritrea, India, Malawi and South Africa, Synthesis Report, Slob, A. and Morten Jerven, A., 2008, link. ↩
13. Country Exit Guidance, internal How to Note, DFID, 2011. ↩
14. Evaluation of the Long-term Development Cooperation Between the UK and Vietnam, DFID, 2016, link. ↩
15. IDC inquiry into DFID’s allocation of resources, ICAI, December 2015, link. ↩
16. How DFID Works with Multilateral Agencies to Achieve Impact, ICAI, June 2015, link. ↩
17. The Transition from “The Developing World” to “A Developing World”, Kharas, H., Kapuscinski Development Lectures, Brookings, 2015 link. ↩
18. The Swedish Exit from Vietnam: Leaving painfully or normalizing bilateral relations? Project Report, Fride 2010, link. ↩
19. Managing Aid Exit and Transformation: Lessons from Botswana, Eritrea, India, Malawi and South Africa, Synthesis Report, Slob, A. and Morten Jerven, A., 2008, link. ↩
20. This review will not assess the decision to transition itself, which as a policy decision, is beyond ICAI’s remit. ↩
21. Or post-financial aid relationship in the case of India. ↩
22. Bilateral Aid Review: Technical Report, DFID, 2011, link. ↩
23. Angola, Bosnia and Herzegovina, Burundi, Cameroon, Cambodia, China, Gambia, Indonesia, Iraq, Kosovo, Lesotho, Moldova, Niger, Russia, Serbia and Vietnam. ↩
24. India: Greening announces new development relationship, The Rt Hon Justine Greening MP, Speech, 9 November 2012, link. ↩
25. UK to end direct financial support to South Africa, The Rt Hon Justine Greening MP, Speech, 30 April 2013, link. ↩
26. Note that India has transitioned from financial aid rather than bilateral aid. ↩
27. Statistics on International Development, DFID, various years. ↩
28. Quality Standards for Development Evaluation, DAC OECD, 2010, link. ↩