DFID’s work with multilaterals – Terms of Reference

1. Introduction

1.1 The Independent Commission for Aid Impact (ICAI) is the independent body responsible for scrutinising UK aid. We focus on maximising the effectiveness of the UK aid budget for intended beneficiaries and on delivering value for money for UK taxpayers. We carry out independent reviews of aid programmes and of issues affecting the delivery of UK aid. We publish transparent, impartial and objective reports to provide evidence and clear recommendations to support UK Government decision-making and to strengthen the accountability of the aid programme. Our reports are written to be accessible to a general readership and we use a simple ‘traffic light’ system to report our judgement on each programme or topic we review.

1.2 We have decided to undertake a thematic review of DFID’s work with multilateral agencies. This review has been requested by a number of DFID’s stakeholders, led by the International Development Committee (IDC). The main focus of this review will be to examine how well DFID is served by and leverages its relationships with multilateral agencies to achieve its objectives. We intend that this review will also provide useful guidance to the Multilateral Aid Review (MAR) process that DFID is planning for later in 2015.

1.3 These Terms of Reference outline the purpose and nature of the review and identify its main themes. A detailed methodology will be developed during inception.

2. Background

2.1 DFID spent over £6.3 billion through multilaterals in 2013-14. This was 63.9% of DFID’s total expenditure.

2.2 As Figure 1 shows, DFID provides at least 42.8% of its overall annual budget as core funding to multilateral agencies. Notably, 27.1% of this proportion is to the European Union and the World Bank. In addition to this core funding, DFID uses multilateral channels to fund specific activity at global and regional levels through trust funds and to deliver programmes in individual countries as part of DFID’s bilateral spending. Spending directly through multilateral organisations comprises 26% of DFID’s bilateral programme (£1,437 million, an increase from £1,075 million in 2012-13). In addition, DFID uses multilateral organisations to spend other bilateral funds such as humanitarian assistance and other financial aid. In total, £2,015 million of the bilateral programme is identified by DFID as being spent through the multilateral programme.

2.3 DFID provides funding through multilaterals, therefore, via several mechanisms:

a) it provides core funding to multilateral organisations through their headquarters, directly from DFID’s central budget;

b) it channels funding from DFID’s bilateral programmes for particular specified activities through multilateral agencies in places where DFID works (often termed multi-bi funding);

c) it finances particular trust funds which have a defined purpose and might be significantly under the control of partner governments or not (DFID’s funding for most of these trust funds is provided through DFID’s bilateral programme);² and

d) it funds specific global initiatives that use multilateral channels. This might include, for instance, funding for the global response to climate change, provided through the central International Climate Fund,³ or the vaccine alliance, GAVI.⁴ This is sometimes termed vertical funding.

2.4 ICAI has previously undertaken reports that:

directly assess DFID’s relationship with multilateral agencies at global and local scales (for example, World Bank,⁵ Asian Development Bank (ADB),⁶ EU,⁷ UNICEF,⁸ UNDP,⁹ UNRWA);¹⁰ and¹¹

assess multilateral delivery as part of other assessments (for example, DFID’s Support to the Health Sector in Zimbabwe,¹² ICAI Report on the UK Emergency Response in the Horn of Africa,¹³ DFID’s Contribution to the Reduction of Child Mortality in Kenya,¹⁴ Evaluation of DFID’s Bilateral Aid to Pakistan,¹⁵ DFID’s Water, Sanitation and Hygiene Programming in Sudan,¹⁶ DFID’s Health Programmes in Burma,¹⁷ DFID’s Education Programmes in Three East African Countries,¹⁸ DFID’s Education Programmes in Nigeria,¹⁹ DFID’s Support to Agricultural Research).^{20 21}

2.5 The IDC and others have consistently requested that we address DFID’s overall approach to multilateral agencies. We have not yet addressed the overall issue of DFID’s core funding of multilateral agencies as a theme, although it has arisen in some of our reports (for instance, our review of the World Bank²²).

2.6 ICAI wishes to step back and look at the overall manner in which DFID works with multilateral agencies and the extent to which this channel serves the needs of the department and provides impact for UK aid. It is also clear that a new agenda for global development is emerging and the old certainties of the roles of many multilateral players will come into question over the decade ahead. This will raise new questions with regard to the fitness for purpose of this channel, both at the core and bilateral levels.

3. Purpose of this review

3.1 The purpose of this review is to answer the question: ‘How well is DFID served by and how well does DFID leverage its engagement with multilateral organisations to achieve impact?’

Scope and reach

3.2 As the NAO has recently noted: ‘In 2011, the Committee of Public Accounts warned that the Department might increase its multilateral aid programme because it did not have the capacity to spend its increased budget through its own bilateral programmes.’²³ Multilaterals may not be fully efficient or appropriate as a mechanism in particular circumstances. ICAI recognises that (in specific cases) there may not be other options but wishes to raise the question of whether the current model of UK engagement with multilaterals is appropriate.

3.3 We wish to look, therefore, at the UK’s use of the multilateral system as a whole to spend official development assistance (ODA) through each of these mechanisms. We will do this by considering:

a) DFID’s engagement with the multilateral system as a whole; and

b) evidence from specific countries where UK aid is spent through multilaterals.

3.4 We will map and summarise DFID’s engagement with the multilateral system. We will also, through illustrative case studies, capture lessons on DFID’s interaction with specific agencies in particular locations. We will draw on where ICAI has reviewed DFID’s engagement with specific multilateral organisations in the past.²⁴ We will not, however, focus primarily on DFID’s engagement with individual multilateral agencies.

3.5 A key approach will be to seek to compare and contrast the control that DFID exerts over the different channels it uses. We will seek to identify how DFID manages its entire portfolio given the different characteristics of bilateral and multilateral channels. This will include comparing DFID’s control and management of its multilateral spending with the control and management it exerts over its purely bilateral programme. We will not, however, seek to answer in this review the question whether DFID achieves better impact or value for money through bilateral or multilateral channels. The relative apportionment of UK aid through bilateral and multilateral channels is a policy question and outside our mandate. We will, however, note whether and how DFID’s management of its multilateral portfolio varies in comparison with its bilateral programme and whether data exists comparing their relative effectiveness.

3.6 We are, therefore, looking at:

• the impact and value for money of the UK’s contributions (which primarily looks at current and past performance);
• how fit for purpose multilaterals are for the future development agenda (given that the old certainties around multilaterals are eroding); and
• the coherence of DFID’s overall portfolio of support through multilaterals (overall and in the specific countries we will consider).

3.7 ICAI is looking to inform:

a) DFID’s decisions on using multilaterals in comparison with other delivery routes, specifically what the strategy is for using multilaterals and to what extent this is appropriate given the UK’s overall objectives for aid;

b) DFID’s choices with regard to its multilateral portfolio and its fitness for purpose; and

c) how DFID ensures that the money given to multilaterals achieves what is intended (particularly given the size of the UK’s core contributions). This will include looking at how DFID manages the relationships with and performance of multilaterals and what it achieves in doing so. It will, therefore, include identifying what the approaches to measurement of effectiveness and accountability are.

4. Relationships to other initiatives and evaluations

4.1 This work will seek to assess critically and add additional insight to other multilateral assessments which are being undertaken. It will strive not to duplicate other analysis, such as DFID’s Multilateral Aid Review (MAR).²⁵ DFID plans to start MAR 2015 in the third quarter of the year and the final report is likely to be published in early 2016, We hope that our report will be a valuable input or adjunct to this. We will draw upon the MAR’s findings to date as a starting point and comment on the utility and usage of the MAR in DFID’s engagement with multilateral organisations. We will also draw upon the OECD DAC Multilateral Organisation Performance Assessment Network (MOPAN) assessments and analysis derived from them.²⁶

4.2 A key element of the review will be that it will seek to synthesise learning from across the aid world with regard to multilateral effectiveness. We will collect any joint evaluations and assessments that have taken place of DFID’s funding through multilaterals and synthesise ICAI’s findings on multilaterals made to date.

4.3 We will also draw upon the wealth of reporting from the IDC, in particular the IDC’s commentary on DFID’s annual reports and relevant IDC enquiries that relate to multilateral organisations. We will also draw on NAO’s reports such as its commentary on the MAR.²⁷ We will take account of other assessments, such as general analysis by the OECD DAC²⁸, analysis for organisations such as BOND²⁹ and commentary by key individuals to inform our findings.³⁰

4.4 DFID’s Internal Audit Department (IAD) will also conduct a review of DFID’s work with multilaterals during 2014-15. We are in communication with IAD on its work and where appropriate (and if it does not compromise our independence) will share information with them.

5. Analytical approach and methodology

Our approach

5.1 This review will be split into two main phases: first, a mapping and consultation phase, followed by a review phase.

Phase I (April-October 2014)

5.2 This phase will look at the “What?” and “Why?” of the UK’s engagement with multilaterals. It will include (among other things):

• mapping DFID’s contribution to multilaterals;
• identifying the UK’s and other countries’ shareholding in each of the agencies;
• identifying how DFID and multilateral agencies engage with each other;
• obtaining data from DFID (documents, data on funding through multilaterals); and
• talking to DFID HQ staff to identify current UK policy and practice.

5.3 We will also hold wider consultations, including:

• a series of focus groups and interviews with UK stakeholders outside DFID (e.g. the IDC, BOND, ODI and the OECD) on their view of the UK’s relationship with multilaterals;
• talking to multilateral agencies to identify what they see as the UK contribution and influence, as well as their own plans for change and development. This will include agency HQ meetings; and
• talking to other funders to identify the UK contribution and influence.

Phase II (January-May 2015)

5.4 This will focus on the “How”, looking to answer the questions in the assessment framework. This will require revisiting the multilateral agencies and DFID. It will look at management arrangements, how influence works in practice and how performance is managed. We will also make a clear comparison of DFID’s approach with that of other funders where possible. We will compile findings from all our evidence sources, including from country case studies. We will look to include both the leverage of multilaterals in countries where there is a strong DFID presence as well as places where there is significant UK funding being channelled through multilateral agencies but where there is no DFID office. This will allow ICAI to take a view of the comparative oversight of DFID’s spending through multilaterals in the two types of context. Case studies will identify the amount of expenditure that DFID channels through multilateral organisations in particular locations, the relative oversight and controls used for each mechanism and performance data. We will gather information from DFID and partner documentation, from DFID and partner staff in country and from third parties. The extent of beneficiary involvement in programme delivery will also be assessed.

Indicative Questions

5.5 The following uses ICAI’s standard assessment framework (objectives, delivery, impact and learning) to propose initial questions for our review.

Objectives

1. What is the UK rationale for whether and how much to spend through multilaterals?
2. Is there a clear and convincing plan (a ‘theory of change’) underlying DFID’s approach to achieving its objectives through the multilateral system and individual agencies, with evidence and assumptions, to show how the contributions will lead to the desired impact?
3. How do DFID/the multilaterals aim to achieve impact for the poor?
4. Do the multilateral agencies respond to intended beneficiary needs and to the context when they design programmes? Does DFID assess this?
5. Are the policy priorities, reform priorities and conditionalities that DFID associates with multilateral contributions well designed? Are there appropriate objectives and choices of partnerships, funding and delivery options?
6. Is the portfolio of DFID’s multilateral assistance comprehensive and fit for purpose?
7. Do DFID multilateral interventions complement its own bilateral programmes and the efforts of government and other aid providers to fill gaps and avoid duplication?

Delivery

1. How does DFID ensure that there is good governance at all levels of the multilateral system, with sound financial management and adequate measures to avoid corruption?
2. On what basis and for what purpose is DFID choosing to use multilaterals to deliver particular services in country?
3. Are the delivery systems for multilateral programmes operating in a cost-effective and efficient manner? How do different multilateral delivery channels compare with regard to value for money?
4. Does DFID have a clear view of costs and value for money throughout the delivery chain for multilateral agencies?
5. How does DFID seek to ensure that the international system as a whole is flexible enough to engage rapidly with emerging crises and changing context?

Impact

1. How does DFID ensure that there are appropriate arrangements in multilateral agencies for monitoring inputs, processes, outputs, results and impact?
2. Are the multilateral programmes delivering against their planned results? How do the different multilateral channels compare in this regard?
3. Is there sufficient line of sight to the poor?
4. Are the programmes delivering clear, significant, sustainable and timely benefits for the intended beneficiaries, including women and girls?
5. Is there evidence of lasting impact and sustained improvement and do the measurement systems of multilateral agencies allow this impact to be measured on an ongoing basis to drive decision–making?

Learning

1. Is DFID taking a coherent approach to learning from its work with multilaterals? Is there real-time monitoring that seeks to turn learning into action?
2. Is there transparency and accountability to intended beneficiaries, donors and UK taxpayers for multilateral programmes and is there learning from their feedback?
3. Do multilateral interventions demonstrate innovation, response to changing evidence, shared learning across countries and use of global best practice?
4. Are there gaps in the international system and is there anything currently not being done by individual agencies that should be undertaken?
5. Have lessons about the objectives, design and delivery of multilateral programmes been learned and shared effectively within the relevant multilateral and across different organisations, regions and sectors?
6. Does the staffing of multilaterals (taking into account turnover) provide institutional memory and value for money?

6. Timing and deliverables

6.1 The review will be overseen by Commissioners and implemented by a small team from ICAI’s consortium. The lead Commissioner will be Mark Foster. The review will take place from the second quarter of 2014 into the first quarter of 2015, with a final report available during the second quarter of 2015.

Footnotes

1. Most trust funds are multi-donor, hence they are regularly termed Multi-Donor Trust Funds (MDTFs). ‘An MDTF takes contributions from a variety of donors and administers them under a single governance structure with a goal to support development-related programmes through a predictable and reliable funding source. This support can be ‘on-budget’, meaning that it is provided directly to a recipient government in order to support its operations, or ‘off-budget’, in which the fund supports development programmes and objectives through funding to international organisations, non-governmental organisations (NGOs) and other entities.’ Barakat S., Rzeszut K. and Martin N., What is the track record of multi donor trust funds in improving aid effectiveness? EPPI-Centre University of London, May 2012, http://r4d.dfid.gov.uk/pdf/outputs/systematicreviews/Q48_Multidonor_trust_fund_2012Barakat(2)FINAL.pdf.
2. The UK’s International Climate Fund, ICAI, December 2014, http://icai.independent.gov.uk/wp-content/uploads/2014/12/ICAI-Report-International-Climate-Fund.pdf.
3. ICAI has considered GAVI (http://www.gavi.org) as part of Contribution to the Reduction of Child Mortality in Kenya, ICAI, March 2014.
4. The Effectiveness of DFID’s Engagement with the World Bank, ICAI, March 2012.
5. The Effectiveness of DFID’s Engagement with the Asian Development Bank, ICAI, July 2012.
6. DFID’s Oversight of the EU’s Aid to Low-Income Countries, ICAI, December 2012.
7. DFID’s work through UNICEF, ICAI, March 2013.
8. Evaluation of DFID’s Electoral Support through UNDP, ICAI, April 2012.
9. DFID’s Support for Palestine Refugees through UNRWA, ICAI, September 2013.
10. All these reports are available at http://icai.independent.gov.uk/.
11. Support to the Health Sector in Zimbabwe, ICAI, November 2011.
12. Report on the UK Emergency Response in the Horn of Africa, ICAI, September 2012.
13. Contribution to the Reduction of Child Mortality in Kenya, ICAI, March 2014.
14. Evaluation of DFID’s Bilateral Aid to Pakistan, ICAI, October 2012.
15. Water, Sanitation and Hygiene Programming in Sudan, ICAI, February 2013.
16. Health Programmes in Burma, ICAI, July 2013.
17. Education Programmes in Three East African Countries, ICAI, May 2012.
18. Department for International Development’s education programmes in Nigeria, ICAI, November 2012.
19. DFID’s Support for Agricultural Research, ICAI, October 2013.
20. All these reports are available at http://icai.independent.gov.uk/.
21. The Effectiveness of DFID’s Engagement with the World Bank, ICAI, March 2012, http://icai.independent.gov.uk/wp-content/uploads/2013/12/ICAI-World-Bank-Final-Report_P1-7.pdf.
22. See para 2.21-2.22, Managing the Official Development Assistance target, National Audit Office, January 2015, http://www.nao.org.uk/wp-content/uploads/2015/01/Managing-the-official-development-assistance-target.pdf; and Para 32, Department for International Development’s Performance in 2012-2013: the Departmental Annual Report 2013, International Development Committee, April 2014, http://www.publications.parliament.uk/pa/cm201314/cmselect/cmintdev/693/693.pdf.
23. For instance; The Effectiveness of DFID’s Engagement with the World Bank, ICAI, March 2012; Evaluation of DFID’s Electoral Support through UNDP, ICAI, April 2012; The Effectiveness of DFID’s Engagement with the Asian Development Bank, ICAI, July 2012;, DFID’s Support for Palestine Refugees through UNRWA, ICAI September 2013; and DFID’s Oversight of the EU’s Aid to Low-Income Countries, ICAI, December 2012. All these reports are available at http://icai.independent.gov.uk/.
24. See here for the 2013 update to the MAR, https://www.gov.uk/government/world-location-news/dfids-multilateral-aid-review-update-2013-interim-report.
25. See http://www.mopanonline.org.
26. The Multilateral Aid Review, Report by the Comptroller and Auditor General, NAO, September 2012, http://www.nao.org.uk/wp-content/uploads/2012/09/1213594.pdf.
27. For instance OECD 2012 DAC report on multilateral aid, http://www.oecd.org/dac/aid-architecture/DCD_DAC(2012)33_FINAL.pdf.
28. For instance BOND submission to Andrew Mitchell on the MAR, 2010, http://www.bond.org.uk/data/files/bond_letter_to_andrew_mitchell_re_dfid_multilateral_aid_review_13.08.10.pdf.
29. For instance Manning, R., The multilateral aid system: an assessment following the major replenishments of 2013, 2013, http://www.wider.unu.edu/publications/working-papers/2014/en_GB/wp2014-110/.