The Effectiveness of DFID’s Engagement with the World Bank

1. Introduction

1.1 The Independent Commission for Aid Impact (ICAI) is the independent body responsible for scrutinising UK aid. We focus on maximising the effectiveness of the UK aid budget for intended beneficiaries and on delivering value for money for UK taxpayers. We carry out independent reviews of aid programmes and of issues affecting the delivery of UK aid. We publish transparent, impartial and objective reports to provide evidence and clear recommendations to support UK Government decision-making and to strengthen the accountability of the aid programme. Our reports are written to be accessible to a general readership and we use a simple ‘traffic light’ system to report our judgement on each programme or topic we review.

1.2 The UK is a major contributor to the World Bank International Development Association (IDA) and a member of the World Bank. We have decided to conduct an evaluation of the effectiveness of the Department for International Development’s (DFID’s) engagement with the World Bank. These Terms of Reference outline the purpose and nature of the review and identify the main themes that it will investigate. A detailed methodology will be developed during an inception phase.

2. Background

2.1 The World Bank is the most important global international financial institution for concessional finance and the second largest channel (£979 million in 2010-11) of multilateral development assistance for the UK after the EU (£1.28 billion in 2010-11).1

2.2 The recent Multilateral Aid Review noted the following background information about IDA:2

  • IDA is one of the largest sources of concessional financing and technical assistance to low-income countries. It committed $14 billion in Official Development Assistance (ODA) in the Financial Year 2009 and disbursed $9 billion according to its Annual Report;
  • IDA closely aligns with DFID’s strategic priorities, is focussed on poverty reduction and the Millennium Development Goals (MDGs) and engages in all of DFID’s priority sectors and main aid modalities;
  • it is focussed on low-income countries (56% going to Africa and 30% to South Asia in 2009);
  • its comparative advantage is the breadth and quality of its technical knowledge and expertise and its global reach. This enables it to support developing countries around the world on a wide range of priorities in national development plans through both policy dialogue and financial assistance; and
  • the range and depth of its expertise is substantial, enabling it to play a convening role and ensuring better coherence across aid efforts.

2.3 The UK contribution to IDA was £880.5 million in 2010-11. IDA is, therefore, an extremely important item of expenditure for UK aid, equivalent to 28% of the total budget for bilateral aid that year (£3.117 billion). The IDA contribution is approximately equal in size to DFID’s top four country programmes combined (India, Ethiopia, Pakistan and Bangladesh).

2.4 DFID relies on the management and evaluation functions of multilateral institutions in order to assess the effectiveness of the spending that such institutions undertake on behalf of the UK taxpayer as the ultimate donor. DFID does not undertake its own independent evaluations of World Bank projects and programmes (although it does undertake reviews of institutional effectiveness).

2.5 The World Bank introduced a results measurement system in 2002.3 This tracks results at two levels – overall progress in the country (‘country effectiveness’) and the contribution of IDA towards this progress (‘agency effectiveness’). The World Bank Independent Evaluation Group’s (IEG’s) evaluations are a key input into the assessment of agency effectiveness. They provide an aggregate measure of the percentage of projects which are rated as ‘satisfactory’.

2.6 Following the Multilateral Aid Review, DFID has placed emphasis on the impact and effectiveness of its IDA contribution to the World Bank. In the most recent replenishment of funding for the World Bank (IDA 16), DFID stipulated that further reforms of the Bank’s operations are required. DFID’s reform objectives included: ‘accountability for results’ (stronger results framework and more appropriate procedures and instruments); ‘delivering for women and girls’ (stronger focus on girls and women in country strategies, operations and policy dialogue); ‘working in fragile contexts’ (improved performance, co-ordination and resourcing of the Bank in fragile and conflict-affected countries) and ‘partnership behaviour’ (more flexible instruments and reforms to procedures which facilitate stronger partnership working). The World Bank has developed a ‘results framework’ in which it will measure itself against IDA 16 objectives. In October 2012, the World Bank will review its own progress against that framework.

3. Purpose of this Review

3.1 To assess the effectiveness of DFID’s engagement with the World Bank and its impact on the Bank’s activities, in order to maximise value for money for the UK taxpayer. In particular, to assess how DFID is ensuring that the reforms agreed as part of its IDA 16 replenishment are being implemented. In examining DFID’s engagement across the Bank’s activities, the review will provide an assessment of the risk factors and issues that could be the focus of future ICAI studies.

4. Relationship to other evaluations/studies

4.1 DFID’s Multilateral Aid Review in March 20114 provides useful context for the current review. The report summarises a number of strengths of the World Bank including:

  • it has strong capacity in a range of sectors;
  • it invests significant resources in poor countries and is a critical part of the global aid system;
  • it can demonstrate good delivery against challenging development objectives; and
  • it is one of the top multilateral organisations for spending aid where it is needed most.

4.2 The review also found that the Bank’s evaluation system provided assurance on the quality of World Bank investments, other than in fragile states: ‘On the strength of the Bank’s results achievements as evidenced by its independent evaluation group and its robust country results frameworks, IDA’s performance in many countries is strong, but fragile states performance is not. However, IDA’s recent results measurement system is strong.’2

4.3 In 2011, the International Development Committee (IDC) published a report on the World Bank.5 It concluded: ‘We support the UK’s decision to provide a substantial contribution to the IDA[16] replenishment and also support the priorities for IDA16. By contributing 12% of the target for donor contribution (or 15.6% of actual donor pledges), DFID will remain a leading contributor to the main institution for assistance to low income countries. This commitment to IDA reflects the importance the UK attaches to the work of IDA and to international development generally. We share that commitment, but call upon DFID to continue to press for improvements in the way that the World Bank delivers assistance on the ground, especially in speeding up the procurement process and producing a greater impact on the MDGs.’ The IDC report also made recommendations on retaining sufficient influence within the Bank during planned changes to voting shares between donors and developing countries; prioritising the promotion of girls’ education; on the transition to low-carbon energy use; and on strengthening the independence of the IEG.

4.4 DFID is a member of the Multilateral Organisation Performance Assessment Network (MOPAN) which undertook a review of the World Bank in February 2010.6 This study noted the independence of the evaluation function as a strength. It also commended the World Bank for tracking the implementation of evaluation findings and for involving beneficiaries in the evaluation process.

4.5 In 2008, an Overseas Development Institute (ODI) study published for the Danish International Development Agency (DANIDA) looked at frameworks for measuring organisational effectiveness.7 This highlights the use of the Annual Review of Development Effectiveness by the World Bank. The study is useful in that it places evaluation within the context of tools which can be used to assess multilateral effectiveness.

4.6 The World Bank’s own evaluation tools will be an important resource for this study. Recent examples, demonstrating a range of synthesis, policy and country evaluations, include:

  • results and performance of the World Bank in 2011;8
  • evaluation directions for the World Bank’s Safeguard and Sustainability Policies;9 and
  • country programme evaluation of Timor-Leste.10

5. Analytical Approach

5.1 The analytical approach will enable the study to draw conclusions about DFID’s oversight of the World Bank based on the information which DFID obtains from the World Bank and other independent sources. The study will use an assessment of the implementation of the reforms agreed as part of the IDA 16 replenishment as a means of testing the effectiveness of DFID’s oversight. It will focus on DFID’s oversight of the overall governance and corporate level of the Bank rather than at a country and individual trust fund level. The study will also provide an overview of the Bank’s activities and an assessment of the risk factors and issues that could be the focus of future ICAI reviews.

5.2 In order to achieve this it will be important to assess different sources:

5.3 Internal World Bank Evidence
We will consider what internal evidence is available from World Bank staff and stakeholders using desk research and telephone interviews, while respecting DFID’s shareholder arrangements. This might include evidence from both Board and operational/management level, as well as from the IEG. It will include information from the World Bank’s Results Measurement System. The information received by DFID from the World Bank will be quality assessed.

5.4 Comparison with DFID’s and other government departments’ own procedures for assessing performance of other multilateral institutions
To make comparisons about the assurance that DFID has over World Bank expenditure, it will be important to benchmark against what DFID does for other multilateral institutions and what other government bodies do in other situations to measure the performance of international agencies involving pooled funding mechanisms.

5.5 Independent Evidence
Evidence will be used from organisations (such as MOPAN), think tanks, NGOs and intended beneficiaries which are independent of the World Bank.

5.6 The overview of the World Bank’s activities will include:

  • the way it is governed;
  • its geographical focus and network of offices;
  • the sector focus and expertise it applies;
  • the different models of delivery used, for example trust funds;
  • how it evaluates the impact and effectiveness of its work; and
  • how it reports back on its performance.

5.7 As part of our review, we will follow up the IDC’s 2010 report on the World Bank11 where our work is relevant to the report’s focus and recommendations.

6. Indicative Review Questions

6.1 This review will use as its basis the standard ICAI evaluation framework, which is focussed on four areas: objectives, delivery, impact and learning. The questions outlined below are based on those questions in our standard evaluation framework which are of particular interest in this review, as well as other pertinent questions we want to investigate. The full, finalised list of questions that we will consider in this review will be set out in the inception report.

6.2 Objectives

6.2.1 How does DFID at an operational level, management level and Board level obtain and assess the accuracy and sufficiency of information it receives about World Bank performance and spending? How effective is the co-ordination of this information-gathering?

6.2.2 How does DFID systemically use other sources and independent assessments of the World Bank to form its own assessment of the World Bank’s expenditure? What is the quality of those sources?

6.2.3 How does DFID use its role on the World Bank Board and its key relationships at a senior level to assess and if necessary challenge how its money is spent in order to achieve maximum value for money for the UK taxpayer?

6.2.4 How has DFID determined the reform objectives agreed as part of the IDA 16 replenishment contribution?

6.3 Delivery

6.3.1 How does DFID obtain its assurance that the World Bank allows for appropriate voice and participation by other stakeholders, including recipient governments and communities?

6.3.2 How does DFID use the World Bank’s Results Management System to track overall value for money and effectiveness? How does the World Bank use the results?

6.3.3 How effectively, in practice, does DFID make use of independent sources in its examination of World Bank expenditure?

6.3.4 How does DFID monitor the World Bank’s progress against the agreed IDA 16 replenishment reform objectives, for example improving performance in fragile states?

6.4 Impact

6.4.1 What evidence exists of active engagement and assessment by the UK at the World Bank Board level of the importance of value and impact of World Bank expenditure, including through the evaluation function?

6.4.2 To what extent is DFID able to know how the funding it gives to the World Bank is spent?

6.4.3 How does DFID measure the level of influence it exerts and what is its impact?

6.4.4 To what extent does DFID measure the impact of the agreed IDA 16 replenishment reforms?

6.5 Learning

6.5.1 What does DFID see as the major concerns to how its money is being spent by the World Bank and the obstacles to further improvement?

6.5.2 Have DFID’s reform objectives and actions as part of the IDA 16 replenishment influenced change and further reform discussions amongst the World Bank’s donors?

7. Methodology

7.1 The evaluation will have a number of elements:

  • a rapid review and synthesis of evidence available internationally on the role of the World Bank and its governance arrangements including the use of evaluation and results frameworks in the context of multilateral institutions;
  • evidence gathering through review of DFID’s institutional and performance framework for the World Bank and discussions with DFID staff, including with the UK Executive Director’s office and DFID’s World Bank team. This will include a focus on how DFID ensures that the World Bank reflects the voice of intended beneficiaries in its programming;
  • comparison with DFID’s and other government departments’ own procedures for assessing performance of other multilateral institutions and lessons from the oversight of other types of organisations such as global corporations;
  • evidence gathering through representatives of independent organisations who take an evidence based interest in World Bank issues; and third parties such as think tanks, NGOs and intended beneficiaries; and
  • follow up of relevant IDC recommendations.

8. Timing and Deliverables

8.1 The review will be overseen by Commissioners and implemented by a small team from ICAI’s consortium. DFID will facilitate information-gathering and meetings with World Bank staff as required.

8.2 The review will take place from November 2011 to January 2012, with a final report available in the first quarter of 2012.

Footnotes

  1. DFID Annual Report 2011, DFID, www.dfid.gov.uk/Documents/publications1/departmental-report/2011/Annual-report-2011-vol1.pdf.
  2. Multilateral Aid Review: Assessment of the International Development Association, DFID, February 2011, www.dfid.gov.uk/Documents/publications1/mar/IDA.pdf.
  3. Results Measurement System, World Bank, http://web.worldbank.org/WBSITE/EXTERNAL/EXTABOUTUS/IDA/0,,contentMDK:20189503~menuPK:2607492~pagePK:51236175~piPK:437394~theSitePK:73154,00.html.
  4. Multilateral Aid Review, DFID, March 2011, www.dfid.gov.uk/Documents/publications1/mar/multilateral_aid_review.pdf.
  5. International Development Committee Fourth report: The World Bank, The House of Commons, March 2011, www.publications.parliament.uk/pa/cm201011/cmselect/cmintdev/606/606.pdf.
  6. MOPAN Common Approach: The World Bank, MOPAN, February 2010, www.mopanonline.org/upload/documents/World_Bank_Final_February_19_issued.pdf.
  7. Assessing Multilateral Organisation Effectiveness, Ministry of Foreign Affairs of Denmark, March 2008, www.odi.org.uk/resources/download/1542.pdf.
  8. IEG Annual Report 2011: Results and Performance of the World Bank Group, IEG, 2011, http://ieg.worldbankgroup.org/content/dam/ieg/rap2011/rap2011_vol1.pdf.
  9. Evaluative Directions for the World Bank Group’s Safeguards and Sustainability Policies, IEG, 2011, http://ieg.worldbankgroup.org/content/dam/ieg/EvalBriefs/eb_safeguards.pdf.
  10. Timor Leste Country Programme Evaluation, 2000-2010, IEG, April 2011, http://ieg.worldbankgroup.org/content/dam/ieg/cpe/TimorLesteCPE.pdf.
  11. International Development Committee Fourth report: The World Bank, The House of Commons, March 2011, www.publications.parliament.uk/pa/cm201011/cmselect/cmintdev/606/606.pdf.