Evaluation of FCO’s Bilateral Aid Programme

1. Introduction

1.1 The Independent Commission for Aid Impact (ICAI) is the independent body responsible for scrutinising UK aid. We focus on maximising the effectiveness of the UK aid budget for intended beneficiaries and on delivering value for money for UK taxpayers. We carry out independent reviews of aid programmes and of issues affecting the delivery of UK aid. We publish transparent, impartial and objective reports to provide evidence and clear recommendations to support UK Government decision-making and to strengthen the accountability of the aid programme. Our reports are written to be accessible to a general readership and we use a simple ‘traffic light’ system to report our judgement on each programme or topic we review.

1.2 Under the current Spending Review (2011-12 to 2014-15), the Foreign and Commonwealth Office (FCO) has an obligation to spend at least £248 million in Official Development Assistance (ODA) each year. In 2011-12, it spent £271 million, representing just over 3% of all UK ODA. Its ODA-related activities include contributions to multilateral organisations, contributions to the British Council and the Westminster Foundation for Democracy, a series of strategic and bilateral programmes and the costs of providing diplomatic support to the UK aid programme.

1.3 In each of our annual work plans, we have included at least one evaluation looking at ODA administered by a department other than DFID. In this case, we have decided to conduct an evaluation of ODA managed by FCO. The evaluation will assess a sample of FCO’s ODA-funded programmes to determine whether they are strategic in nature, managed appropriately and achieving their intended impact.

1.4 A number of FCO spending programmes mix ODA-eligible and other funds. As our mandate is to scrutinise ODA, we will not directly evaluate the outcomes or impact of non-ODA funded activities but we may need to consider them as part of our review of programme strategy.

1.5 These Terms of Reference outline the purpose and nature of the evaluation and the main themes that it will investigate. A detailed methodology will be developed during an inception phase.

2. Background

2.1 FCO is one of a number of UK government departments, other than DFID, that spend ODA. A proportion of FCO expenditure each year is designated as ODA, in accordance with the international definition agreed by the Organisation for Economic Cooperation and Development (OECD).

2.2 FCO’s ODA falls into five categories, as follows:

1. UK contributions to multilateral organisations: assessed (mandatory) and voluntary contributions to four multilateral organisations: the United Nations (UN), the Organisation for Security and Cooperation in Europe (OSCE), the Commonwealth Foundation and the Commonwealth Small States Offices;
2. The British Council: the British Council is a registered charity and an executive non-departmental public body (NDPB). Each year, it receives a grant from the UK government though FCO, a proportion of which is ODA, which funds its activities in developing countries in the areas of English language, the arts and education and society. The British Council is also engaged in the delivery of DFID projects as a contractor, with its contract management separated from its grant-funded activities by a financial firewall;
3. Strategic programmes: FCO manages a series of strategic programmes at the international or regional level. In most of the programmes, a proportion of the expenditure is ODA-eligible. Funds are allocated to projects based on procedures that are specific to each programme;
4. Bilateral programmes: these are country-specific funds, usually managed by posts; and
5. ‘Aid-related frontline diplomacy’: this FCO term refers to the costs of work done by FCO in developing countries to support the aid programme, which the UK claims as ODA in accordance with OECD rules.

2.3 The British Council’s ODA allocation is agreed as part of the Spending Review process and the assessed contributions to multilateral organisations are fixed. Beyond this, the FCO Board and Ministers make baseline allocations to the different categories of expenditure through FCO’s Resource Allocation Round. These allocations are made by reference to the UK’s foreign policy priorities and to country and directorate business plans. FCO’s ODA spending in 2011-12 is shown in Figure 1 on page 3.

2.4 The Strategic Programmes are organised under three themes:

• security;
• prosperity; and
• diplomatic influence and values.

2.5 In the security area, the largest ODA programmes in 2011-12 are Counter-Terrorism (£10.9 million ODA spending) and a special fund for Afghanistan (£11.8 million). There is a Prosperity Fund including £17 million of ODA, while the diplomatic influence and values theme includes the Arab Partnership Participation Fund (£5 million), Chevening Scholarships (£13.9 million) and a Human Rights and Democracy Fund (£5 million).

2.6 The Prosperity Fund (£17 million ODA) supports projects on economic reform, clean energy and climate change. Examples include supporting a National Plan on Energy Efficiency in Brazil, capacity building for local government in Mexico on promoting low-carbon construction and transport development, an Investment Policy Review in Kazakhstan and a Commission of Integrity in Iraq to improve financial investigations and tackle corruption. The Arab Partnership Participation Fund (£5 million ODA) was created in February 2011 as a mechanism for responding to democratic uprisings in the Middle East (the ‘Arab Spring’). It is designed to promote political participation, public voice, improved governance and anti-corruption. Its activities include a voter education programme in Tunisia, support for parliamentary committees in Iraq and the ‘Young Arab Voices’ programme of youth debates. The Human Rights and Democracy Fund (£5 million ODA) is a global instrument. Examples of its support are prison reform in Tajikistan, improving human rights standards in the criminal justice system in Brazil and management of electoral disputes in Kenya. The Reuniting Europe Programme (£4 million ODA) focusses on the Balkans and Turkey, assisting EU accession candidate countries to meet the political criteria for accession. Each programme is controlled by its own programme board and may allocate funds globally, regionally or to particular countries.

2.7 Allocation to particular countries from the strategic and bilateral programmes follows the UK’s foreign policy priorities, such as promoting human rights, supporting international peace and security and promoting emerging markets, rather than prioritising the poorest countries. FCO, therefore, spends ODA in a wider range of countries than DFID. In recent years, FCO has been increasing its ODA expenditure in fragile and conflict-affected states, complementing DFID’s scale-up in this area. In 2011-12, nearly 35% of FCO ODA expenditure went to fragile states. Overall, approximately 25% of FCO ODA is budgeted for expenditure in low-income countries and 70% for middle-income countries (5% is non-country specific).

2.8 Bilateral programmes are under the authority of FCO Regional Directors, who generally allocate the bulk of funds for management by country posts. Funds from the bilateral programmes are usually spent in the form of small grants supporting any of the priorities in the country business plan.

2.9 Aid-related frontline diplomacy refers to the costs of FCO support to the UK aid programme, which is reported as ODA under OECD rules. Prior to 2011, this was calculated as a fixed percentage of FCO ODA programme expenditure. It is now calculated more accurately, based on an activity recording system. There are three elements to the methodology:

1. FCO staff are provided with written guidance as to which activities should be reported as ODA. In ODA-eligible countries, Heads of Missions identify ODA-eligible activities as part of their country business plans;
2. FCO staff are required to record time spent on aid-related activities through timesheets, which are collated on a quarterly basis. FCO uses a cost management model to identify the costs of the activities reported; and
3. in its annual ODA returns, the UK Government reports FCO aid-related activities as part of the administrative costs of the aid programme. These are reported separately from DFID’s administrative costs.

2.10 The figure for aid-related frontline diplomacy in 2011-12 is £71 million, about 10% of the costs of running FCO’s network of overseas posts.

2.11 The British Council was established in 1934. Administratively, it is a separate entity from FCO. It receives a core grant from FCO, for which it is accountable to Parliament. It also uses income earned from its commercial activities to support its charitable work. It has a permanent presence in 110 countries and projects in approximately ten others. It is organised into three business units devoted to English language, the arts and education and society. Funds are allocated through a combination of a top-down strategic planning process and bids from the country level. The country allocation reflects the relative importance of different countries to the UK, their significance to the British Council’s goals and the capacity of the British Council to make a difference.

3. Purpose of this review

3.1 To assess whether ODA programmes managed by FCO are strategic, managed appropriately and achieving their intended results.

4. Relationship to other reviews

4.1 There is little published information or review literature regarding FCO ODA programmes.

4.2 In 2011, FCO established a Policy Programme Evaluation Board to improve the results orientation of its programme expenditure. The Board includes an FCO Minister, FCO Directors, two Heads of Mission, DFID’s Head of Evaluation, two non-executive directors from industry and an NGO representative. It is supported by a cadre of internal evaluators and evaluation expertise from DFID and other departments. The Board carried out a high-level review of its programmes in May 2012. It has embarked on a set of evaluations of individual programmes over the next three years. It has also introduced a system of self-evaluation by programme teams. Evaluations have already been completed of the Reuniting Europe and the Human Rights and Democracy Programmes.

4.3 The British Council commissions independent evaluations of many of its projects from external contractors.

5. Analytical approach

5.1 Our evaluation will examine FCO strategic and bilateral Programmes in the Middle East and North Africa, with a focus on programming on human rights and democracy preceding and in response to the ‘Arab Spring’. We will also look at British Council programmes, with a focus on programming designed to encourage youth engagement and active citizenship.

5.2 The evaluation will focus on three themes.

1. Do programme management and funds allocation processes support the strategic objectives of the programmes?
2. Are activities managed so as to maximise effectiveness and value for money?
3. Are the programmes achieving sustainable results?

5.3 The evaluation will include a number of elements. We will look at a sample of strategic programmes in FCO’s ‘Diplomatic Influence and Values’ category and at the British Council’s ‘Active Citizens’ programme, to assess how well they support the UK’s policy objectives. This will include reviewing funding strategies and assessing the portfolio of activities in each programme.

5.4 We will examine the processes by which FCO identifies and manages activities within its strategic and bilateral programmes. This will include a review of the rules, procedures and guidance governing project identification, selection of partners, management of activities, monitoring, evaluation and learning. We will also examine the work of the Policy Programme Evaluation Board.

5.5 We will visit two countries in the region (provisionally identified as Egypt and Tunisia). In each country, we will examine a selection of activities from relevant FCO and British Council strategic programmes and the two bilateral programmes. This will involve discussions with FCO and British Council staff and project partners, reviews of project documentation and reports and, where appropriate, visits to project sites to view on-going or completed activities and consult with stakeholders. We will also assess how FCO uses its diplomatic efforts to support the goals of the strategic and bilateral programmes and related DFID regional programmes.

6. Indicative evaluation questions

6.1 A detailed methodology will be developed during the inception phase, setting out the evaluation questions and the methods to be used for answering them. Likely evaluation questions will include:

6.2 Objectives

   6.2.1 Does FCO have a strategic approach to allocating its resources, based on clear policies and objectives and logical theories of change?

   6.2.2 Does FCO have clear policies and strategic guidance on programming choices and activity selection?

   6.2.3 Does FCO complement other activities by the UK Government and other agencies and donors and avoid duplication?

   6.2.4 Are individual activities technically sound and based on clear and logical theories of change?

6.3 Delivery

   6.3.1 Are the choices of implementing partners appropriate?

   6.3.2 Does FCO have adequate approaches to the governance and financial management of its activities and are adequate steps being taken to avoid corruption?

   6.3.3 Is FCO spending helping to leverage resources from other UK and international sources?

   6.3.4 Are risks to the achievement of programme objectives identified and managed effectively?

   6.3.5 Are intended beneficiaries and other national stakeholders of FCO activities effectively engaged?

6.4 Impact

   6.4.1 Are the FCO programmes delivering sustainable results in support of their objectives?

   6.4.2 Is there long-term and sustainable impact from the programmes?

6.5 Learning

   6.5.1 Does FCO have appropriate arrangements for monitoring inputs, processes, outputs, results and impact from its activities and learning from what it finds?

   6.5.2 Is there evidence of innovation in FCO programmes and sharing of learning both internally and with external partners such as DFID?

7. Methodology

7.1 The evaluation methodology will comprise the following elements, across British Council and FCO strategic and bilateral programmes:

• a review of literature and evidence on the Arab Spring and democracy promotion in the Middle East and North Africa;
• a review of documentation, including strategies, policies, guidelines and reporting;
• a portfolio review of programmes in the MENA region, looking at spending patterns and types of activities and whether they indicate strategic use of resources; and
• case studies of two countries (provisionally identified as Egypt and Tunisia), each involving:
  • a review of activities undertaken through the British Council and FCO strategic and bilateral programmes, including a synthesis of activity reporting and results data;
  • an examination of management, delivery and monitoring arrangements for the programmes; and
  • a detailed review of a sample of activities, including consultations with project partners and (where appropriate) other stakeholders and a review of evidence on impact.

8. Timing and deliverables

8.1 The evaluation will be undertaken by a small team from ICAI’s consortium and overseen by the Commissioners. It will commence in November 2012, with a final report available during the second quarter of 2013.

Footnotes

¹ The Commonwealth Foundation is one of three Commonwealth intergovernmental organisations. It supports the work of 70 civil society organisations and professional and academic associations, focusing on cultural affairs.