DFID’s work with multilaterals

1 Introduction

1.1 The Independent Commission for Aid Impact (ICAI) is the independent body responsible for scrutinising UK aid. We focus on maximising the effectiveness of the UK aid budget for intended beneficiaries and on delivering value for money for UK taxpayers. We carry out independent reviews of aid programmes and of issues affecting the delivery of UK aid. We publish transparent, impartial and objective reports to provide evidence and clear recommendations to support UK Government decision-making and to strengthen the accountability of the aid programme. Our reports are written to be accessible to a general readership and we use a simple ‘traffic light’ system to report our judgement on each programme or topic we review.

1.2 We have decided to undertake a thematic review of DFID’s work with multilateral agencies. This review has been requested by a number of DFID’s stakeholders, led by the International Development Committee (IDC). The main focus of this review will be to examine how well DFID is served by, and leverages its relationships with, multilateral agencies to achieve its objectives. We intend that this review will also provide useful guidance to the Multilateral Aid Review (MAR) process that DFID is planning for later in 2015.

1.3 Between June and October 2014, ICAI undertook a mapping and consultation exercise to scope the topic. This inception report is a product of that process. ICAI will look to publish the final report in June 2015.

2 Background

2.1 DFID spent over £6.3 billion through multilaterals in 2013-14. This was 63.9% of DFID’s total expenditure.

2.2 As Figure 1 shows, DFID provides at least 42.8% of its overall annual budget as cored funding to multilateral agencies’ central budgets (‘core funding’). Notably, 27.1% of this proportion is to the European Union and the World Bank. In addition to this core funding, DFID uses multilateral channels to fund specific activity at global and regional levels through trust funds and to deliver programmes in individual countries as part of DFID’s bilateral spending. Spending directly through multilateral organisations comprises 26% of DFID’s bilateral programme (£1,437m, an increase from £1,075m in 2012-13). In addition, DFID uses multilateral organisations to spend other bilateral funds such as humanitarian assistance and other financial aid. In total, £2,015 million of the bilateral programme is identified by DFID as being spent through the multilateral programme.

2.3 DFID provides funding through multilaterals, therefore, via several mechanisms:

a) it provides core funding to multilateral organisations through their headquarters, directly from DFID’s central budget;

b) it channels funding from DFID’s bilateral programmes for particular specified activities through multilateral agencies in places where DFID works (often termed multi-bi funding);

c) it finances particular trust funds which have a defined purpose and might be significantly under the control of partner governments or not (DFID’s funding for most of these trust funds is provided through DFID’s bilateral programme);³ and

d) it funds specific global initiatives that use multilateral channels. This might include, for instance, funding for the global response to climate change, provided through the central International Climate Fund,⁴ or the vaccine alliance, GAVI.⁵ This is sometimes termed vertical funding.

2.4 ICAI has previously undertaken reports that:

• directly assess DFID’s relationship with multilateral agencies at global and local scales (for example, World Bank,⁶ Asian Development Bank (ADB),⁷ EU,⁸ UNICEF,⁹ UNDP,¹⁰ UNRWA¹¹); and;
• assess multilateral delivery as part of other assessments (for example, DFID’s Support to the Health Sector in Zimbabwe,¹² ICAI Report on the UK Emergency Response in the Horn of Africa,¹³ DFID’s Contribution to the Reduction of Child Mortality in Kenya,¹⁴ Evaluation of DFID’s Bilateral Aid to Pakistan,¹⁵ DFID’s Water, Sanitation and Hygiene Programming in Sudan,¹⁶ DFID’s Health Programmes in Burma,¹⁷ DFID’s Education Programmes in Three East African Countries,¹⁸ DFID’s Education Programmes in Nigeria,¹⁹ DFID’s Support to Agricultural Research).²⁰²¹

2.5 The IDC and others have requested that we address DFID’s overall approach to multilateral agencies. We have not yet addressed the overall issue of DFID’s core funding of multilateral agencies as a theme, although it has arisen in some of our reports (for instance, our review of the World Bank²²).

2.6 We wish to step back and look at the overall manner in which DFID works with multilateral agencies and the extent to which this channel serves the needs of the department and provides impact for UK aid.

2.7 It is also clear that a new agenda for global development is emerging and the old certainties of the roles of many multilateral players will come in to question over the decade ahead. This will raise new questions with regard to the fitness for purpose of this channel, both at the core and bi-lateral levels. We wish to explore this readiness of the multilateral agencies and DFID’s relationship for these changes.

Some issues from ICAI reports

2.8 The following section summarises some of the key findings from our reports to date. It is an overview of the findings and indicates from which reports we have drawn our evidence.

DFID recognises that multilateral organisations can add value to its work due to their privileged relationships and mandate

2.9 We have seen how DFID often chooses to use a multilateral organisation as a delivery partner due to their status, global reach and relationships with other donors and governments. DFID also uses multilaterals as a mechanism to deliver aid where the security of the delivery chain is vital and where it is especially important to manage the risk of corruption. We have noted this in our reports on DFID’s work with UNDP,²³ UNICEF²⁴ and the World Bank.²⁵

2.10 We have also noted, however, that the mandate of UN agencies can be problematic. They have to maintain a relationship with partner governments. This means that they sometimes do not behave in as independent (or challenging) a way as DFID might wish them to.²⁶

DFID is able to influence the work of multilateral agencies

2.11 DFID is often effective at using its reputation and funding to push for improvements in performance and seeks to set specific agendas with its partner multilateral agencies. For example DFID’s 2011 Multilateral Aid Review²⁷ highlighted the need for the World Bank to address better the needs of girls and women through its International Development Association (IDA) funding. DFID successfully ensured that improved performance measures on gender were included when it negotiated the IDA16 replenishment. This included new targets for including gender, a World Development Report on gender equality and integration of gender issues into every Country Assistance Strategy.²⁸

2.12 However, DFID does not always fully utilise its potential for high level influence. For example in our report on DFID’s engagement with the European Union, we noted that DFID could exert greater influence by providing clearer guidance to its country offices. In country, DFID and EU are highly active in co-ordinating forums and groups but structures tended to be complex and cumbersome, especially where there are many donors. ICAI also identified a gap in utilising wider EU levers, such as trade policies.²⁹

2.13 We have become concerned that sometimes DFID’s greater reliance on multilateral partners risks those relationships now being considered ‘too big to fail’. DFID is a key funder of many, as well as being reliant on them for delivery of the UK’s priorities. This dependency might reduce DFID’s ability to challenge or question its partners’ behaviour or operations.

DFID’s choice of multilaterals as a delivery partner is not always evidence based

2.14 We have seen that DFID tends to trust multilaterals, especially once an individual organisation has scored positively in the Multilateral Aid Review. We have, however, noted that DFID does not always ensure due diligence in assessing a multilateral organisation’s capacity to deliver particular services in a specific country.³⁰³¹³² Multilateral organisations are regularly DFID’s partner of first choice but we have seen that DFID does not always formally consider alternatives, making it hard to ensure transparency and value for money. DFID could sometimes seek to use a more diverse set of delivery agencies, which would help to ensure both effectiveness and value for money. This would also help to prevent monopolies being created, as arguably happened with UNDP’s electoral support.³³ Similarly, in our report on DFID’s work on Education in Nigeria, we noted that UNICEF was reappointed to deliver the third phase of a programme without competition, although a contractor was appointed following open competition. ICAI assessed, however, that UNICEF’s prior performance was not enough to reappoint without a competitive process.³⁴

DFID’s management of multilaterals has often been too ‘light touch’

2.15 DFID often takes a lighter touch in managing multilateral partnerships and contracts than it does with other partners, either due to mandated protocols or through trusted relationships. With some multilateral partners this has been effective, for example with the World Bank where the focus of DFID’s engagement has been at the level of strategic influence rather than micromanagement. In our report on DFID’s engagement with the World Bank, we noted that ‘DFID provides effective oversight of the World Bank, both as a shareholder and as a donor. The nature of the relationship between the two organisations is that DFID does not micro-manage the Bank. In our view, DFID’s combination of ongoing engagement through the Executive Board, assurance of effectiveness of the Bank’s processes and risk-based country monitoring is a sensible approach.’³⁵ We also noted that DFID relies on the effectiveness of World Bank assurance processes, which it assessed to provide adequate control. DFID then uses a risk-based approach to monitor IDA performance, relying on information from a small sample of countries. Beyond these countries, there is limited central monitoring by DFID of projects at risk.³⁶

2.16 This light touch relationship with multilaterals can also, however, lead to instances of poor management of risk (which we found in our report on DFID’s work with UNICEF).³⁷ We found, for instance, that UNICEF’s ability to deliver was often (inappropriately) ‘taken on trust’.³⁸ We have heard of possible country level corruption in multilaterals and of appointments into multilaterals that have not been based on merit. We have also seen that DFID’s results framework has not been consistently applied. Our report on DFID’s engagement with UNICEF noted that DFID’s relationship with multilaterals can mean that DFID does not consistently engage in some country programmes at a level that would enhance risk or performance management.³⁹

Oversight of multilaterals within DFID has been (and remains) variable

2.17 We have noted in several reports that DFID does not have a clear picture of the portfolio of programmes it funds through multilaterals. For instance, our review of DFID’s programme of work with the World Bank noted that there was no one person within DFID responsible for overseeing the department’s portfolio of trust funds and ensuring strategic coherence.⁴⁰ Indeed, the detailed analysis of contributions to trust funds used during our review was prepared for DFID by the World Bank. Similarly we noted that there was insufficient oversight of major projects co-financed by DFID and the Asian Development Bank.⁴¹ Preparatory work for this inception report indicates that the overview of DFID’s funding through Trust Funds is still inadequate and DFID is still not capturing all funds channelled through these mechanisms. Similarly, DFID does not know what proportion of its core funding to multilaterals (47% of its total budget) is spent in its priority countries or in fragile states. The same is true at country level. DFID does not clearly have sight of all the activities it funds outside the specific bilateral funding channel.⁴² In Bangladesh, we found multiple channels of funding for climate change that were not being managed or overseen by the DFID country office, even though in some cases UK money was the bulk of that provided in some cases.⁴³

2.18 It is notable, however, that DFID is gradually shifting its relationship with multilaterals. For instance, we noted that, in its relationship with UNICEF, it is seeking to incorporate good practice from managing commercial contractors. DFID country offices have, (since January 2013) started undertaking organisational due diligence of financial and technical capacity of local UNICEF offices as standard practice.⁴⁴ We are aware of considerable scrutiny, in some cases, of DFID’s portfolio of work with specific multilaterals.

Bureaucracy and complexity as delivery partners

2.19 ICAI has found several instances where multilateral agencies’ cumbersome governance and decision-making processes were slowing down delivery. In our report on DFID’s Support for Palestine Refugees through UNRWA, we noted its governance ‘makes for a far less nimble body, constrained by having to please numerous stakeholders and weighed down by cumbersome processes which, in turn, slow down reform.’⁴⁵ We have noted that the IDC made a similar point about the EU: ‘there still seemed a long way to go to reduce bureaucracy on procurement.’⁴⁶ Similarly, our report on UNICEF made the point that ‘it has taken longer than anticipated to deliver programme activities to the villages, in part due to delays in UNICEF contracting with its delivery partners to provide services. Whilst this reflects the low capacity of many of the partners, we believe that UNICEF could find ways to speed up its contracting procedures without increasing fiduciary risk.’⁴⁷

2.20 At the same time, DFID needs to be clear on the outcomes and impacts it is getting from its aid spending through multilaterals. DFID has not always been able to do this. For example, because UNICEF has multiple donors and a lack of documentation attributed directly to DFID our review reported it to be hard to assess the exact impact outcome attributed to DFID aid.⁴⁸⁴⁹

DFID sometimes outsources the management of risk to multilateral organisations

2.21 DFID largely relies on multilateral agencies to manage the delivery chain directly. DFID passes the risks to the agency but in so doing reduces DFID’s visibility and ability to understand how well risks have been managed. In our report on the EU, we noted that DFID relied on the EU’s own processes for managing and monitoring aid.⁵⁰ In our report on DFID’s Humanitarian Response in the Horn of Africa, we noted that DFID did not always have a full picture of how well multilateral organisations apply their risk management systems and, therefore, cannot be certain that all key risks are appropriately managed.⁵¹ Similarly, we noted in our review of UNDP’s Electoral Support that UNDP could have been more proactive in its risk monitoring and that many problems/risks were not anticipated.⁵² Our review of DFID’s work through UNICEF made a related point, emphasising the need for DFID to engage more. We found that closer monitoring of progress and risks by DFID could have improved UNICEF’s performance.⁵³

DFID does not have a consistent view of multilateral agencies

2.22 Multilateral agencies play multiple roles, including working alongside DFID as partners at the global scale. DFID funds many multilaterals through core contributions as well as being a shareholder in them (for instance of IDA and of the African Development Bank). At the same time, in many cases DFID is choosing to channel bilateral funds through the same multilateral agencies which are acting as sub contracted delivery agents of DFID. We have identified in our reports that the institutional culture (in particular) of UN agencies makes them ‘less willing or able to change [their] ways of operating to meet DFID’s requirements’.⁵⁴ They do not see themselves as delivery agencies under contract, but partners with a role mandated by international treaty or agreement.⁵⁵ We noted in the case of our review of education in Nigeria: ‘UNICEF regards itself as a development partner rather than a sub-contractor and, as a UN agency, adopts a standard approach to working with different donors, which makes performance management and programme modifications more difficult. In contrast, Cambridge Education does regard itself as a sub-contractor and it has been responsive to feedback from DFID’.⁵⁶

DFID’s relationship with multilaterals is often constrained by factors outside its control

2.23 DFID’s relationship with multilateral bodies is often determined by protocols that govern UK relations with UN agencies, restricting DFID’s influence. Its approach to managing UN organisations is ‘partly shaped by the privileges, immunities and financial rules and regulations which apply to the UN’.⁵⁷ For instance, we note that in many cases agencies levy a flat 8% management charge on DFID funds that may hide the true cost of delivery through other intermediaries.⁵⁸ We have noted that a lack of financial transparency and independent oversight in UN funding makes it hard for DFID to assess fiduciary risk management or obtain a full picture of impact. In our report on water and sanitation in Sudan, we noted that this lack of transparency made it difficult to assess whether fiduciary risks were being managed appropriately.⁵⁹ We were particularly concerned that the special status of UN agencies exempted them from independent oversight.

Other issues

2.24 From the work that they have conducted, Commissioners have noted a series of further issues that they would like to investigate, such as concerns about the mandates and actions of multilaterals that can result in both overlapping activities and gaps in action.⁶⁰ Commissioners have observed that the turnover of multilateral agency staff can, like turnover of DFID staff, also undermine institutional learning and effectiveness.

3 Purpose of this review

3.1 The purpose of this review is to answer the question: how well is DFID served by, and how well does DFID leverage its engagement with, multilateral organisations to achieve impact?

Scope and reach

3.2 As the NAO has recently noted: ‘In 2011, the Committee of Public Accounts warned that the Department might increase its multilateral aid programme because it did not have the capacity to spend its increased budget through its own bilateral programmes’.⁶¹ Multilaterals may not be fully efficient or appropriate as a mechanism in particular circumstances. ICAI recognises that (in specific cases) there may not be other options but wishes to raise the question of whether the current model of UK engagement with multilaterals is appropriate.

3.3 We wish to look, therefore, at the UK’s use of the multilateral system as a whole to spend official development assistance (ODA) through each of these mechanisms. We will do this by considering:

3.4

• DFID’s engagement with the multilateral system as a whole;
• DFID’s engagement with specific key multilateral organisations at both the corporate level and in country; and
• evidence from specific countries where UK aid is spent through multilaterals.

3.5 We will map and summarise DFID’s engagement with the multilateral system. We will also, through illustrative case studies, capture lessons on DFID’s interaction with specific agencies in particular locations. We will draw on where ICAI has reviewed DFID’s engagement with specific multilateral organisations in the past.⁶² We will not, however, focus primarily on DFID’s engagement with individual multilateral agencies.

3.6 A key approach will be to seek to compare and contrast the control that DFID exerts over the different channels it uses. This will include comparing DFID’s control and management of its multilateral spending with the control and management it exerts over its purely bilateral programme. We will not, however, seek to answer in this review the question of whether DFID achieves better effectiveness and value for money through bilateral or multilateral channels.

3.7 We are, therefore, looking at:

• impact and value for money of the UK’s contributions (which primarily looks at current and past performance);
• the coherence of DFID’s overall portfolio of support through multilaterals (overall and in the specific countries we will consider); and
• how fit for purpose multilaterals are for the future development agenda (given that the old certainties around multilaterals are eroding).

3.8 ICAI is looking to inform:

a) DFID’s decisions on using multilaterals in comparison with other delivery routes, specifically what the strategy is for using multilaterals and to what extent this is appropriate given the UK’s overall objectives for aid;

b) DFID’s choices with regard to its multilateral portfolio and its fitness for purpose; and

c) how DFID ensures that the money given to multilaterals achieves what is intended (particularly given the size of the UK’s core contributions). This will include looking at how DFID manages the relationships with and performance of multilaterals and what it achieves in doing so. It will, therefore, include identifying what the approaches to measurement of effectiveness and accountability are.

3.9 This will be a thematic review that will consider the UK’s strategy towards the use of multilaterals (while not making specific policy recommendations).

4 Relationships to other initiatives and evaluations

4.1 This work will seek to assess critically and add additional insight to other multilateral assessments which are being undertaken. This work will seek to assess critically and add additional insight to other multilateral assessments which are being undertaken. It will strive not to duplicate other analysis, such as DFID’s Multilateral Aid Review (MAR).⁶³ DFID plans to start the MAR 2015 in the third quarter of the year. The final report is likely to be published in early 2016. We hope that our report will be a valuable input or adjunct to this. We will draw upon the MAR’s findings to date as a starting point and comment on the utility and usage of MAR in DFID’s engagement with multilateral organisations. We will also draw upon the Multilateral Organisation Performance Assessment Network (MOPAN) assessments and analysis derived from them.⁶⁴

4.2 A key element of the review will be that it will seek to synthesise learning from across the aid world with regard to multilateral effectiveness. We will collect any joint evaluations and assessments that have taken place of DFID’s funding through multilaterals and synthesise ICAI’s findings on multilaterals made to date.

4.3 We will also draw upon the wealth of reporting from the IDC, in particular the IDC’s commentary on DFID’s annual reports and relevant IDC enquiries that relate to multilateral organisations. We will also draw upon NAO’s reports, such as its commentary on the MAR.⁶⁵

4.4 We will take account of other assessments, such as general analysis by OECD DAC,⁶⁶ analysis for organisations such as BOND⁶⁷ and commentary by key individuals to inform our findings.⁶⁸

4.5 DFID’s Internal Audit Department (IAD) will also conduct a review of DFID’s work with multilaterals during 2014-15. We are in communication with IAD on its work and where appropriate (and if it does not compromise our independence) will share information with them.

5 Methodology

Our approach

5.1 This will be a thematic review. ICAI’s published 2014 work plan says of this review that ‘we will examine how well DFID manages its relationships with multilateral agencies with regard to a particular sector, such as infrastructure. We will look at DFID’s oversight of these multilaterals to establish whether their programmes in that sector are effective and delivering good value for money’. Commissioners discussed this review at their May 2014 Board meeting and agreed that a wider thematic view would complement our previous considerations of multilaterals. They also agreed that a longer, more deliberate process to develop the overall assessment framework for the review was appropriate and that a wider sectoral view was appropriate. We will now consider, therefore, infrastructure and health sectors.

5.2 The review thus has two phases:

1. an initial mapping and consultation phase to scope key issues and from that derive the key questions for the main review (completed); and
2. the main review phase to gather evidence.

Mapping and consultation phase

5.3 An initial mapping and consultation phase ran from April to October 2014. Its outputs were:

• An information dossier containing facts and figures on 47 multilateral agencies was prepared for Commissioners. This provides information on DFID spend, donor contributions and shareholding, geographic coverage, agency size (for example, staff and programme numbers) and a summary of findings from the MAR. An early version of this dossier helped Commissioners to identify a shortlist of 17 multilateral agencies / funding channels to provide the focal point for the consultation exercise. These agencies were selected by ICAI to represent the range of sectoral, thematic and geographical coverage of the multilaterals that DFID funds through:
  • African Development Fund (AfDF)
  • Asian Development Fund (AsDF)
  • Climate Investment Funds (CIFs)
  • European Development Fund (EDF)
  • Food and Agriculture Organisation (FAO)
  • GAVI Alliance
  • Global Environment Facility (GEF)
  • Global Fund for Aids, Tuberculosis and Malaria (GFATM)
  • International Committee of the Red Cross (ICRC)
  • International Development Association (IDA)
  • Islamic Development Bank (IsDB)
  • Private Infrastructure Development Group (PIDG)
  • United Nations Office for the Coordination of Humanitarian Affairs (UN OCHA)
  • United Nations Development Programme (UNDP)
  • United Nations High Commissioner for Refugees (UNHCR)
  • United Nations Children’s Fund (UNICEF)
  • World Food Programme (WFP)

  The dossier includes OECD DAC reporting (where available) on geographic and sector spends for the shortlisted multilateral agencies and funding channels.⁶⁹

• A comprehensive documents portfolio has been built for each of the 17 shortlisted multilateral agencies. A document request, submitted to DFID at the end of September 2014, asked for Annual Reviews, Project Completion Reports and any Portfolio Delivery Reviews from financial year 2011-12 to financial year 2014-15. As of October 2014 documentation has been received from DFID for 9 of the 17 shortlisted agencies (the rest is awaited) and this have been supplemented by reporting available on Development Tracker. DFID has also provided us with comprehensive Aries financial and reporting data for financial year 2009-10 to financial year 2014-15 inclusive. This is included in the information dossier.
• An initial literature review has identified and listed approximately 50 different reports, evaluations and think pieces that capture the different schools of thought around the multilateral aid system.
• The team has carried out consultations with 16 multilateral agencies, 25 DFID staff and 8 UK based thought leaders and NGOs working in the multilateral space. This has included meetings in New York, Washington, Paris and Geneva. The lead commissioner for this review, Mark Foster, also attended meetings in New York with UNDP and UNICEF. Questions during all of the interviews were kept strategic and high level.

5.4 Commissioners then met in October 2014 and, informed by the above information, chose a methodology for the full review phase and agreed the questions set out in the assessment framework. Both are set out below.

Review Phase from November 2014 to March 2015

5.5 Our review will be a strategic assessment of DFID’s engagement with multilaterals as a whole. It will thus be wide-ranging and thematic rather than focussing on project level detail. It will address DFID’s corporate approach to engaging with multilaterals in four ways: as recipients of funding from DFID centrally into their overall budgets (‘core funding’), as implementing agencies for bilateral funds (where DFID’s country budgets are channelled through multilateral organisations), as partners and managers of trust funds, and as partners in international development challenges (such as the establishment of international standards or addressing a particular issue, such as HIV/AIDS). We will, therefore, include a categorisation of the different ways DFID engages with different multilateral organisations. We will seek to identify how DFID manages its entire portfolio given the different characteristics of bilateral and multilateral channels.

5.6 This will be a mixed methods review, synthesising previous information (not least ICAI’s 38 reports to date). We will build on the information captured during the mapping and consultation phase. Four case studies will be undertaken (including two country visits) to deepen the insights gained from the broad assessment. Elements that will contribute during the review phase are set out below.

Synthesis of ICAI findings to date

5.7 We will extract all findings that relate to multilaterals from our 38 reports to date, synthesising them into themes.

Capture of DFID policy statements

5.8 We will augment the literature review by capturing all DFID’s public policy statements relating to its work with multilaterals to clarify DFID’s stated objectives for its work with multilaterals.

Capture of DFID performance data

5.9 We will synthesise DFID’s internal performance information (annual reviews and project completion reviews) for the 17 multilateral organisations in our sample, as well as for all relevant multilateral projects in our four case study countries (see below) from the last five years. We will also capture evaluations where possible. We will sample at least 20 reviews to allow us to have an indication of the quality of evidence provided (ensuring that these reviews cover the range of types of multilateral covered). We will also include DFID’s MAR data, captured during the mapping phase.

Capture of Multilateral Performance Data

5.10 We will review publicly available data on the impact of multilateral organisations, seeking to derive cost effectiveness comparisons between multilateral organisations; and if possible cost effectiveness comparisons between multilateral and bilateral donors. To do this we first undertook an assessment of data availability on the following sectors (in order of priority):

1. water and sanitation;
2. roads;
3. power;
4. health; and
5. education.

5.11 From available data, we then chose the sectors of infrastructure and health to be our priorities. We will (where possible) gather, aggregate and analyse this data to demonstrate as clearly as possible the impact of multilaterals in these chosen sectors and the unit costs of identified activities. This will be presented in a form that allows easy comparisons between the different institutions. We will seek to capture information for our case study countries first.

Semi structured interviews

5.12 Using the assessment framework we will gather information from key respondents from multilateral organisations (all 17 in our sample), other bilateral donors (at least five), DFID (key decision-makers) and other UK stakeholders (such as CSOs and policy influencers). We will also seek to engage with the FCO where appropriate, as well as with the IDC and the NAO.

Country Case Studies

5.13 We will undertake four country case studies. The criteria for selection were that Commissioners wished these to be representative of both Asia and Africa and to include countries with a DFID office and locations where DFID provides significant funding though multilaterals, but has no country office. This will specifically seek to compare the level of oversight of multilateral partners in countries where DFID has and does not have an office.⁷⁰ They should include multilateral programming that spans humanitarian/fragile states as well as mainstream development programmes. Ideally at least one country should be a fragile or conflict affected state and one should be a middle income country. These criteria were used to identify Uganda and Madagascar for the country visits and Burma and Vietnam for desk-based case studies.

5.14 We will modify the assessment framework for each case study country, drawing out key themes where appropriate. Prior to the fieldwork, we will develop additional assessment tools to guide our research, such as interview guides for particular groups of informants. On the basis of other components of the methodology, we will formulate a set of hypotheses on the strengths and weaknesses of DFID’s approach for testing through the case studies.

5.15 Each case study will, therefore, have individual terms of reference that will define the key questions to be considered (extracted from the assessment framework) and specify the approach to be taken. Each will draw out particular themes; in Uganda this will include how DFID chooses different delivery channels in a context where it has stopped all funding through government. In Vietnam (where DFID is closing its bilateral programme) it will include considering the changing relationships with the World Bank in an emerging middle income country with high prevalence of poverty. It will consider how DFID has made choices to support, in particular, infrastructure and health programming through multilateral partners. For Burma, we will consider how DFID has chosen to provide assistance through multilaterals in a context of instability and weak infrastructure but where there are multiple agencies. Madagascar will provide insight into the operation of multilateral agencies in a context where the UK only has minimal representation and where there is no on-going relationship at country level with DFID.

5.16 The case studies will involve the following activities:

• a brief review of literature on the country in question, including political economy analysis and any analysis of its key challenges and institutions;
• a review of the DFID country strategy and poverty diagnostics (where appropriate);
• a review of programme documents, including programme memoranda or business cases, diagnostic and analytical work generated by the programme, monitoring and evaluation frameworks, baseline reports, annual reviews, activity and financial reports and any external evaluations;
• briefings from and interviews with DFID country office or FCO staff;
• interviews with implementing multilateral partners;
• interviews with agencies and partners implementing programmes funded through multilaterals;
• interviews with national NGOs, think tanks, academics, journalists and other informed observers where appropriate; and
• site visits to areas where the programmes are active (including beneficiary consultations where appropriate) to identify how beneficiaries participate in the delivery of activities throughout the programme cycle.⁷¹

6 Roles and responsibilities

6.1 The team leader, who will manage this study on a day-to-day basis, will be the primary point of contact with DFID. The team will work closely with the Lead Commissioner, who will participate in the engagement with multilateral agencies and other stakeholders. KPMG will also provide oversight of this review under the overall leadership of the ICAI Project Director.

6.2 It is proposed that this review is undertaken by the following:

Team Leader (Agulhas)

The Team Leader is a Director of Agulhas Applied Knowledge. He specialises in aid effectiveness, governance and institutional development. He is on the core consortium team delivering ICAI’s reports. He has led ICAI reviews considering Bangladesh Climate Change, UNDP’s management of elections, DFID’s programmes of Health and Education in Bihar, its support for Rural Livelihoods in Western Odisha, Programme Partnership Arrangements (PPAs) and How DFID Learns.

Multilaterals expert (Agulhas)

He is an economist and senior manager with 25 years of development experience and a wide knowledge of the international development system. He has, since 2010, been a Senior Fellow with Agulhas. Prior to this, he spent five years as International Director of an international development organisation, with responsibility for over 1000 staff in fragile states. He previously worked worked for the UK’s Department for International Development. He was a DFID economic adviser working on India and Bangladesh and served as Head of Asia Regional Economic Policy Department and as Deputy Director, Asia. His final post was as Director, International with responsibility for the development aspects of the UK’s multilateral partnerships, including the World Bank, Regional Development Banks, EU and UN; and for trade, conflict and humanitarian work. He led ICAI’s team working on DFID’s contribution to the reduction of child mortality in Kenya. He will provide deep knowledge of DFID’s relationships with multilateral organisations and their working.

Team Member (Agulhas)

This Team Member is a Fellow of Agulhas. Prior to joining Agulhas, she worked within UNICEF UK and UNICEF HQ, leading on climate change and sustainability. She has a particular knowledge of UK civil society and non-governmental organisations and their relationship with international NGOs. She has contributed to the Synthesis Review of previous ICAI reports, in support of work on the annual review and on the synthesis of Impact findings in support of the Impact review. She will support the analysis of UK policy and provide key facilitation of UK agency consultations.

Team Member (Agulhas)

She is a Fellow with Agulhas Applied Knowledge and a former programme manager in the DFID Somalia team. She conducted much of the field research for ICAI’s Bangladesh climate change review, ICAI’s evaluation of DFID’s support for elections with UNDP and field research for the evaluation of DFID’s Pakistan programme. She also supported Agulhas’s work on PPAs and on learning for ICAI. She will provide analytical, research and administrative capability to the team.

Team Member (Agulhas)

With over 15 years of experience in development consulting, including policy advice, research and analysis, programme design and evaluation, he has worked for a variety of clients on diverse issues including governance and civil society programming and aid effectiveness. He is an authority in international law and human rights and has written widely on post-conflict reconstruction, state-building and the restitution of property. He will have overall management responsibility for all stages of the process including the country case studies and will ensure delivery of the outputs.

7 Expected outputs and time frame

8 Risks and mitigation

8.1 The following sets out the key risks and mitigating actions for this review:

9 How will this review make a difference?

9.1 ICAI has been subject to several requests, led by the IDC, to assess systematically DFID’s work with multilaterals. It is expected that this review will contribute to clarifying DFID’s decision making on the use of multilaterals, on the choice of funding mechanisms DFID uses. It will also help set the guidelines for DFID’s management approach to multilateral organisations. It will seek to influence the next MAR. It may also provide a reference point for a new Ministerial Team post May 2015.

Footnotes

1. IDA figure of £1,181.2 million includes £76.2 million spent through the Multilateral Debt Relief Initiative
2. https://www.gov.uk/government/publications/statistics-on-international-development-2013-to-2014-gpex-tables
3. DFID reports it currently funds at least 131 Trust Funds, mainly through bilateral channels. Most trust funds are multi-donor, hence they are regularly termed Multi-Donor Trust Funds (MDTF). ‘An MDTF takes contributions from a variety of donors and administers them under a single governance structure with a goal to support development-related programmes through a predictable and reliable funding source. This support can be ‘on-budget’, meaning that it is provided directly to a recipient government in order to support its operations, or ‘off-budget’, in which the fund supports development programmes and objectives through funding to international organisations, non-governmental organisations (NGOs) and other entities.’ Sultan Barakat S., Rzeszut K., and Martin N., What is the track record of multi-donor trust funds in improving aid effectiveness?, EPPI-Centre University of London, May 2012, http://r4d.dfid.gov.uk/pdf/outputs/systematicreviews/Q48_Multidonor_trust_fund_2012Barakat(2)FINAL.pdf.
4. The UK’s International Climate Fund, ICAI, December 2014.
5. ICAI has considered GAVI (http://www.gavi.org) as part of Contribution to the Reduction of Child Mortality in Kenya, ICAI, March 2014.
6. The Effectiveness of DFID’s Engagement with the World Bank, ICAI, March 2012.
7. The Effectiveness of DFID’s Engagement with the Asian Development Bank, ICAI, July 2012.
8. DFID’s Oversight of the EU’s Aid to Low-Income Countries, ICAI, December 2012.
9. DFID’s work through UNICEF, ICAI, March 2013.
10. Evaluation of DFID’s Electoral Support through UNDP, ICAI, April 2012.
11. DFID’s Support for Palestine Refugees through UNRWA, ICAI, September 2013.
12. Support to the Health Sector in Zimbabwe, ICAI, November 2011.
13. ICAI Report on the UK Emergency Response in the Horn of Africa, ICAI, September 2012.
14. Contribution to the Reduction of Child Mortality in Kenya, ICAI, March 2014.
15. Evaluation of DFID’s Bilateral Aid to Pakistan, ICAI, October 2012.
16. Water, Sanitation and Hygiene Programming in Sudan, ICAI, February 2013.
17. Health Programmes in Burma, ICAI, July 2013.
18. Education Programmes in Three East African Countries, ICAI, May 2012.
19. Department for International Development’s education programmes in Nigeria, ICAI, November 2012.
20. DFID’s Support for Agricultural Research, ICAI, October 2013.
21. All these reports are available at http://icai.independent.gov.uk/.
22. The Effectiveness of DFID’s Engagement with the World Bank, March 2012, http://icai.independent.gov.uk/wp-content/uploads/2013/12/ICAI-World-Bank-Final-Report_P1-7.pdf.
23. ‘UNDP is able to deliver electoral assistance even in very difficult environments. Its global mandate also helps to provide legitimacy to donor involvement in an area that touches closely on questions of national sovereignty’, Evaluation of DFID’s Electoral Support through UNDP, ICAI, April 2012, paragraph 2.19.
24. ‘Its proximity to governments and its global presence are major reasons why DFID chooses to partner with UNICEF’; ‘DFID relies heavily on UNICEF to take on the challenge and risk of working in difficult environments, often where few others will work’, DFID’s work through UNICEF, ICAI, March 2013, paragraphs 2.2 and 3.1.
25. World Bank Trust Funds enable donors to ‘pool resources and take advantage of the reach and expertise of the multilateral agency.’ The Effectiveness of DFID’s Engagement with the World Bank, ICAI, March 2012, paragraph 1.23.
26. Evaluation of DFID’s Electoral Support through UNDP, ICAI, April 2012.
27. Multilateral Aid Review, DFID, 2011, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/67583/multilateral_aid_review.pdf.
28. The Effectiveness of DFID’s Engagement with the World Bank, ICAI, March 2012.
29. DFID’s Oversight of the EU’s Aid to Low-Income Countries, ICAI, December 2012.
30. Evaluation of DFID’s Electoral Support through UNDP, ICAI, April 2012.
31. DFID’s work through UNICEF, ICAI, March 2013.
32. The Effectiveness of DFID’s Engagement with the Asian Development Bank, ICAI, July 2012.
33. Evaluation of DFID’s Electoral Support through UNDP, ICAI, April 2012.
34. DFID’s Education Programme in Nigeria, ICAI, November 2012.
35. The Effectiveness of DFID’s Engagement with the World Bank, ICAI, March 2012, Conclusions, page 21.
36. The Effectiveness of DFID’s Engagement with the World Bank, ICAI, March 2012.
37. DFID’s work through UNICEF, ICAI, March 2013.
38. DFID’s work through UNICEF, ICAI, March 2013.
39. DFID’s work through UNICEF, ICAI, March 2013.
40. The Effectiveness of DFID’s Engagement with the World Bank, ICAI, March 2012.
41. The Effectiveness of DFID’s Engagement with the Asian Development Bank, ICAI, July 2012.
42. DFID’s Oversight of the EU’s Aid to Low-Income Countries, ICAI, December 2012.
43. DFID’s Climate Change Programme in Bangladesh, ICAI, November 2011.
44. DFID’s work through UNICEF, ICAI, March 2013.
45. DFID’s Support for Palestine Refugees through UNRWA, ICAI, September 2013, page 9.
46. DFID’s Oversight of the EU’s Aid to Low-Income Countries, ICAI, December 2012, page 14.
47. DFID’s work through UNICEF, ICAI, March 2013, paragraph 2.38.
48. DFID’s work through UNICEF, ICAI, March 2013, page 8.
49. DFID has, notably, put much effort into trying to identify specific results over the last two years.
50. DFID’s Oversight of the EU’s Aid to Low-Income Countries, ICAI, December 2012, page 14.
51. DFID’s Humanitarian Emergency Response in the Horn of Africa, ICAI, September 2012, paragraph 2.49.
52. Evaluation of DFID’s Electoral Support through UNDP, ICAI, April 2012.
53. DFID’s work through UNICEF, ICAI, March 2013, paragraph 2.9.
54. DFID’s work through UNICEF, ICAI, March 2013.
55. DFID’s Education Programme in Nigeria, ICAI, November 2012.
56. DFID’s Education Programme in Nigeria, ICAI, November 2012, paragraph 2.68.
57. DFID’s work through UNICEF, ICAI, March 2013.
58. This has been observed by Commissioners during a number of reviews, see for example DFID’s work through UNICEF, ICAI, March 2013
59. DFID’s Water, Sanitation and Hygiene Programming in Sudan, ICAI, February 2013.
60. For instance, see Where is Everyone?: responding to emergencies in the most difficult places, Médecins Sans Frontières July 2014, http://www.msf.org.uk/sites/uk/files/msf-whereiseveryone_-def-lr_-_july.pdf.
61. See para 2.21-2.22, Managing the Official Development Assistance target, National Audit Office, 14 January 2015; Para 32, ‘Department for International Development’s Performance in 2012-2013: the Departmental Annual Report 2013, International Development Committee, 5 March 2014.
62. For instance, The Effectiveness of DFID’s Engagement with the World Bank, ICAI, March 2012; Evaluation of DFID’s Electoral Support through UNDP, ICAI, April 2012; The Effectiveness of DFID’s Engagement with the Asian Development Bank, ICAI, July 2012; DFID’s Support for Palestine Refugees through UNRWA, ICAI, September 2013; and DFID’s Oversight of the EU’s Aid to Low-Income Countries, ICAI, December 2012. All these reports are available at http://icai.independent.gov.uk/.
63. See here for the 2013 update to the MAR, https://www.gov.uk/government/world-location-news/dfids-multilateral-aid-review-update-2013-interim-report.
64. See http://www.mopanonline.org
65. NAO, The Multilateral Aid Review, Report by the Comptroller and Auditor General, September 2012, http://www.nao.org.uk/wp-content/uploads/2012/09/1213594.pdf.
66. For instance OECD 2012 DAC report on multilateral aid, http://www.oecd.org/dac/aid-architecture/DCD_DAC(2012)33_FINAL.pdf.
67. For instance BOND submission to Andrew Mitchell on the MAR, 2010, http://www.bond.org.uk/data/files/bond_letter_to_andrew_mitchell_re_dfid_multilateral_aid_review_13.08.10.pdf.
68. For instance Richard Manning, The multilateral aid system: an assessment following the major replenishments of 2013, 2013, http://www.wider.unu.edu/publications/working-papers/2014/en_GB/wp2014-110/.
69. The Dossier has been shared with DFID Internal Audit Department.
70. There have been criticisms of the MAR that DFID only assessed performance of multilaterals in countries where it has offices, see for instance http://www.simonmaxwell.eu/blog/dfids-mar-update-is-thorough-now-they-need-a-multilateral-policy.html.
71. Since this is primarily an assessment of DFID’s relationship with multilateral organisations the focus will be at the institutional, not project, level. We will not, therefore, be undertaking any specific beneficiary survey work for this review.