ICAI-FCO-Inception-Report-FINAL-100113

1. Introduction

1.1 The Independent Commission for Aid Impact (ICAI) is the independent body responsible for scrutinising UK aid. We focus on maximising the effectiveness of the UK aid budget for intended beneficiaries and on delivering value for money for UK taxpayers. We carry out independent reviews of aid programmes and of issues affecting the delivery of UK aid. We publish transparent, impartial and objective reports to provide evidence and clear recommendations to support UK Government decision-making and to strengthen the accountability of the aid programme. Our reports are written to be accessible to a general readership and we use a simple ‘traffic light’ system to report our judgement on each programme or topic we review.

1.2 Under the current Spending Review (2011-12 to 2014-15), the Foreign and Commonwealth Office (FCO) has an obligation to spend at least £248 million in Official Development Assistance (ODA) each year. In 2011-12, it spent £271 million, representing just over 3% of all UK ODA. Its ODA-related activities include contributions to multilateral organisations, contributions to the British Council and the Westminster Foundation for Democracy, a series of strategic and bilateral programmes and the costs of providing diplomatic support to the UK aid programme.

1.3 We have decided to conduct an evaluation of ODA managed by FCO, including the British Council as an FCO grantee. The evaluation will assess a sample of FCO’s ODA-funded programmes to determine whether they are strategic in nature, managed appropriately and achieving their intended impact.

1.4 This Inception Report sets out the evaluation questions, methodology and work plan for the evaluation. It is, however, intended that the methodology and work plan be flexible enough to allow new questions and lines of inquiry to emerge over the course of the evaluation.

2. Background

2.1 The background to this review, including the amount and nature of ODA managed by FCO and the British Council, is as described in the Terms of Reference.¹

3. Purpose of this review

3.1 To assess whether ODA programmes managed by FCO and the British Council are strategic, managed appropriately and achieving their intended results.

4. Relationship to other reviews

4.1 There is little published information or review literature regarding FCO’s ODA programmes.

4.2 In 2011, FCO established a Policy Programme Evaluation Board to improve the results orientation of its programme expenditure.² The Board carried out a high-level review of FCO’s programmes in May 2012. It has embarked on a set of evaluations of particular strategic programmes over the next three years, supported by a system of self-evaluation by programme teams. Evaluations have already been completed of the Reuniting Europe and the Human Rights and Democracy Programmes.

4.3 It is usual to conduct internal evaluations of individual projects above a certain budgetary threshold.³ We are not aware of any specific reviews of FCO’s programme expenditure by either the Foreign Affairs Select Committee or the National Audit Office.

4.4 The British Council makes use of logical frameworks to set out its programme logic at corporate, programme and project levels. It commissions external evaluations of its programmes, supplemented by internal reviews and reporting. An external evaluation of its Active Citizens programme, which falls within our review sample, has recently been undertaken and shared with us. For the past eight years, it has conducted an Annual Impact Survey of its work globally, in which it surveys some 5,000 people involved in its programmes 6-24 months after their participation, to assess their individual learning, the impact on their organisation and the development of linkages with the UK. It also carries out Customer Satisfaction and Net Promoter surveys, to collect feedback from participants in its programmes.

4.5 We conclude, therefore, that there is no duplication between our evaluation and other reviews. We will make use of existing reviews as part of our evidence base, where appropriate.

5. Methodology

Analytical approach

5.1 Our evaluation will focus on three themes:

do programme management and funds allocation processes support the strategic objectives of the programmes;
are activities managed so as to maximise effectiveness and value for money; and
are the programmes achieving sustainable results?

5.2 Our evaluation will review FCO and British Council ODA-funded activities, including spending programmes and aid-related frontline diplomacy, in the Middle East and North Africa (MENA). Our thematic focus will be on activities preceding and in response to the ‘Arab Spring’ revolutions, including on human rights, citizen engagement, democratic development and social and economic exclusion, particularly youth unemployment.

5.3 In the case of FCO, this will include looking at strategic programmes in the ‘Diplomatic Influence and Values’ and ‘Prosperity’ categories, together with related bilateral programming and diplomacy. In the case of the British Council, the focus will be on spending programmes falling under the ‘Education and Society’ business unit, particularly those with a conflict reduction and youth focus.

5.4 We will assess whether funds allocation, programme design and the selection and implementation of activities adequately support the overarching policy objectives of each organisation. This will include a review of internal procedures and guidance and an assessment of financial and project management capacity. We will examine whether the relevant systems are adequate to deliver the intended results and provide value for money.

5.5 We will conduct a portfolio review of relevant FCO and British Council activities in the MENA region, to determine whether the pattern of spending and project selection adequately supports the relevant strategic objectives.

5.6 We will visit two countries in the region, Egypt and Tunisia. In each country, we will choose a number of FCO and British Council projects for detailed review. This will involve discussions with FCO and British Council staff and project partners, reviews of project documentation and reports and – where appropriate – visits to project sites to view ongoing or completed activities and consult with stakeholders. We will also assess how FCO uses its diplomatic efforts to support the goals of the strategic and bilateral programmes and related DFID regional programmes.

5.7 Most of the FCO projects within the scope of this review are designed to support processes of democratic transition in the MENA region. While they may provide financial or other support to particular groups of people, such as parliamentarians or journalists, their intended beneficiaries are the population at large. In most cases, the assistance will not be visible to the general public. It will therefore not always be possible to collect feedback from the intended beneficiaries. We will, however, seek feedback from project participants and other national stakeholders able to offer an informed view on the relevance and effectiveness of the assistance.

Evaluation framework

5.8 The evaluation framework for this review is set out in the table below. It is based on the standard ICAI guiding criteria and evaluation framework, which cover four areas: objectives, delivery, impact and learning. The questions which are highlighted in bold are those from our Terms of Reference (ToR) on which we will focus in particular. These have been adapted to include specific reference to the British Council as well as to FCO.

Relevant ICAI evaluation framework question	Evaluation question	Criteria for assessment	Sources of evidence
(1) Objectives: what is the programme trying to achieve?
Does the programme have clear, relevant and realistic objectives that focus on the desired impact? (1.1)	Do FCO and the British Council have a strategic approach to allocating their resources, based on clear policies and objectives and logical theories of change? (ToR 6.2.1)	Clear organisational objectives, mandates and strategies Resource allocation criteria and decision-making processes that support established priorities Clarity on the role and comparative advantage of each organisation	Policies, strategies, guidance material Interviews with senior management Consultations with external stakeholders
Is there a clear and convincing plan, with evidence and assumptions, to show how the programme will work? (1.2)	Do FCO and the British Council have clear policies and strategic guidance on programming choices and activity selection? (ToR 6.2.2)	Clear goals and objectives for programmes, based on a sound programme theory Adequacy of criteria for activity selection Adequacy of guidance on project design Evidence of options appraisals and a clear plan showing how individual activities will work	Policies, programme strategies, guidance material Interviews with senior management, programme managers and staff Portfolio review of programme activities Consultations with external stakeholders, including UK think-tanks and grant-making bodies with similar aims Case studies
Does the programme complement the efforts of government and other aid providers and avoid duplication? (1.3)	Do FCO and the British Council complement other activities by the UK Government and other agencies and donors and avoid duplication? (ToR 6.2.3)	Clarity on the role and comparative advantage of each organisation Clarity on the role of ODA-funded activities and the relationship to non-ODA activities Appropriateness of guidelines and practices for classifying expenditure as ODA Existence of robust UK Government country business plans Complementarity of activities with DFID regional programmes Quality of partnerships and complementarity of programming with other donors and organisations active in similar areas	Interviews with senior management and programme managers Interviews with FCO and British Council country programme boards and staff and implementing partners in the case study countries Interviews with other UK Government departments Guidance material and programme documentation Interviews with relevant international agencies (e.g. Open Society Institute) and donors Case studies Interviews with government representatives and informed national observers
Are the programme's objectives appropriate to the political, economic, social and environmental context? (1.4)	Are individual activities technically sound, relevant to the needs of beneficiaries and based on clear and logical theories of change? (based on ToR 6.2.4)	Quality of regional, country, stakeholder and thematic analysis Activities are tailored to identified opportunities and threats Activities tailored to needs of project participants and intended beneficiaries Balance of portfolio, by: geographical coverage; length of engagement; type of activity; and choice of delivery partner Quality of approaches taken against international learning and good practice	Guidance material Research and analysis Programme strategies Activity lists Interviews with senior management and programme managers Interviews with intended beneficiaries, project participants and other national stakeholders
(2) Delivery: is the delivery chain designed and managed so as to be fit for purpose?
Is the choice of funding and delivery options appropriate? (2.1)	Are the choices of implementing partners appropriate? (ToR 6.3.1)	Adequacy of partner selection criteria and capacity assessment processes Quality and appropriateness of implementing partners	Guidance material, assessment criteria and templates Interviews with FCO and British Council country programme boards and staff and implementing partners in the case study countries Review of project documentation Case studies
Does programme design and roll-out take into account the needs of the intended beneficiaries? (2.2)	Are intended beneficiaries and other national stakeholders of FCO and British Council activities effectively engaged? (ToR 6.3.5)	Extent to which stakeholder analysis informs design and delivery of country/regional programmes and individual projects Consultation with intended beneficiaries and other national stakeholders Effective identification of and support to new civil society actors	Interviews with intended beneficiaries, project participants and other national stakeholders Stakeholder analysis Interviews with FCO and British Council country programme boards and staff and implementing partners in the case study countries Case studies Programme documentation and analysis
Is there good governance at all levels, with sound financial management and adequate steps being taken to avoid corruption? (2.3)	Do FCO and the British Council have adequate approaches to the governance and financial management of their activities and are adequate steps being taken to avoid corruption? (ToR 6.3.2)	Effective project cycle management Effective challenge and accountability around activity design and resource allocation Robust partner selection criteria and capacity assessment Strong oversight of implementing partners, including reporting requirements Specific anti-corruption measures	Interviews with FCO and British Council country programme boards and staff and implementing partners in the case study countries Guidance material Review of project documents
Are resources being leveraged so as to work best with others and maximise impact? (2.4)	Is FCO and British Council spending helping to leverage resources from other UK and international sources? (ToR 6.3.3)	Effective advocacy to improve the quality of engagement of other international partners, including the European Union, the G8 Deauville Partnership and International Financial Institutions Quality of partnerships with other international organisations and actors Extent of collaboration and challenge between UK government departments Adequate strategies for scaling up pilots and ensuring sustainability of results Level of co-financing of activities	Interviews with FCO and British Council country programme boards and staff and implementing partners in the case study countries FCO and British Council programming guidelines Review of programme and project documents Interviews with third parties in country, including other donors and international organisations
Do managers ensure the efficiency and effectiveness of the delivery chain? (2.5)	Do managers ensure the efficiency and effectiveness of the delivery chain?	Rigorous processes for selecting delivery partners Consideration of the relative cost and cost-effectiveness of different delivery options Sufficient oversight of delivery and cost-effectiveness Use of consistent definition of costs, including administrative costs Effective management and oversight of individual projects Regular reporting on expenditure and results	Interviews with FCO and British Council country programme boards and staff and implementing partners in the case study countries Activity proposals and budgets Financial reports
Is there a clear view of costs throughout the delivery chain? (2.6)	Is there a clear view of costs throughout the delivery chain?
Are risks to the achievement of the objectives identified and managed effectively? (2.7)	Are risks to the achievement of programme objectives identified and managed effectively? (ToR 6.3.4)	Adequacy of guidance on risk management Effective risk assessment Programme and project designs incorporate risk analysis Active approaches to managing risk	Risk assessments Programme and project designs Risk management tools Interviews with programme and country teams Reviews of individual projects in our case study countries
Is the programme delivering against its agreed objectives? (2.8)	Are the programmes delivering against their agreed objectives?	Evidence of activities delivering against objectives Active measures taken to correct failing projects and ensure continuous improvement	Interviews with FCO and British Council country programme boards and staff and implementing partners in the case study countries Interviews with intended beneficiaries, project participants and other national stakeholders Review of project reports Case studies
Are appropriate amendments to objectives made to take account of changing circumstances? (2.9)	Are FCO and the British Council flexible and responsive to changes in circumstances?	Appropriate balance between long- and short-term objectives Speed with which funding can be mobilised or reallocated in response to changing situations Funds set aside for responding to crises or opportunities Evidence of individual activities being adapted to take into account of changing circumstances	Interviews with FCO and British Council country programme boards and staff and implementing partners in the case study countries Review of project reports Case studies
(3) Impact: what is the impact on intended beneficiaries?
Is the programme delivering clear, significant and timely benefits for the intended beneficiaries? (3.1)	Are FCO and British Council programmes delivering sustainable results in support of their objectives? (ToR 6.4.1) Are the programmes delivering clear, significant and timely benefits for the intended beneficiaries?	Evidence of delivery of project outputs and outcomes Evidence of policy change, institutional reform, capacity development or changes in knowledge, attitudes and practices Evidence of ongoing ownership by national stakeholders Availability of follow-up funding for sustainability	Programme and project reports, monitoring returns and evaluations Interviews with FCO and British Council country programme boards and staff and implementing partners in the case study countries Discussions with peer organisations Interviews with intended beneficiaries, project participants and other national stakeholders
Is the programme working holistically alongside other programmes? (3.2)	Do the results of the programmes complement those of other agencies and donors?	Quality of partnerships Fit between FCO/British Council programmes and approaches and those of peer organisations	Interviews with FCO and British Council country programme boards and staff and implementing partners in the case study countries Interviews with other donors Review of project reports Case studies
Is there a long-term and sustainable impact from the programme? (3.3)	Is there long-term and sustainable impact from the programmes? (ToR 6.4.2)	Monitoring and evaluation indicate sustainable impact Results convincingly attributed to project and activities Adequate measures taken to ensure ownership and sustainability	Interviews with FCO and British Council country programme boards and staff and implementing partners in the case study countries Review of project reports, monitoring returns and evaluation reports Case studies Interviews with intended beneficiaries, project participants and other national stakeholders
Is there an appropriate exit strategy involving effective transfer of ownership of the programme? (3.4)	Do FCO and the British Council take appropriate measures to ensure the sustainability of results?	Activities designed so as to support new voices for reform and catalyse change processes Evidence of ongoing ownership of reform processes by national stakeholders Availability of follow-up funding	Interviews with FCO and British Council country programme boards and staff and implementing partners in the case study countries Review of project reports, monitoring returns and evaluation reports Case studies Interviews with intended beneficiaries, project participants and other national stakeholders
Is there transparency and accountability to intended beneficiaries, donors and UK taxpayers? (3.5)	Are FCO and the British Council operating with an appropriate level of transparency and accountability to national authorities, intended beneficiaries and UK taxpayers?	Policies on release of information Publication of spending data, activities and results Adequate accountability relationships and reporting to the UK Parliament	Interviews with senior management, programme managers FCO and British Council corporate reporting Interviews with UK stakeholders Interviews with national stakeholders in the case study countries, including (where appropriate) government officials and civil society representatives
(4) Learning: what works best and what needs improvement?
Are there appropriate arrangements for monitoring inputs, processes, outputs, results and impact? (4.1)	Do FCO and the British Council have appropriate arrangements for monitoring inputs, processes, outputs, results and impact from their activities and learning from what they find? (ToR 6.5.1)	Activities designed with clear intended results, to support evaluability and learning Adequacy of programme and project reporting and monitoring Adequate use of independent evaluation Use of lessons learnt to inform strategies, corporate guidance and future programming decisions	Corporate guidance on results management Interviews with FCO and British Council country programme boards and staff and implementing partners in the case study countries Review of programme and project logical frameworks Monitoring and evaluation reports
Is there evidence of innovation and use of global best practice? (4.2)	Is there evidence of innovation in FCO and British Council programmes and sharing of learning both internally and with external partners such as DFID? (ToR 6.5.2)	International best practice assessed and incorporated into programme and project design Adequate process for identifying lessons from successes and failures Lessons disseminated internally and shared with partners	Interviews with FCO and British Council country programme boards and staff and implementing partners in the case study countries Interviews with DFID and peer organisations engaged in similar activities Review of project reports, monitoring returns and evaluation reports Case studies
Is there anything currently not being done in respect of the programme that should be undertaken? (4.3)	Is there anything currently not being done in respect of the programmes that should be undertaken?	Good practice or learning from peer organisations not taken up by FCO or British Council Opportunities and entry points identified in country analytical work not taken forward in programmes	Literature Interviews with partner and peer organisations Regional and programme strategies
Have lessons about the objectives, design and delivery of the programme been learned and shared effectively? (4.4)	[Already covered in questions 6.5.1 and 6.5.2]

5.9 The evaluation methodology will comprise the following elements:

Literature review

5.10 We will carry out a brief review of literature and evidence on the Arab Spring and democracy promotion in the Middle East and North Africa. Given the breadth of this topic, we will focus on literature from the most influential think tanks and research institutes and map the main schools of thought on the risks and opportunities posed by the Arab Spring.

Consultations with peers and stakeholders

5.11 We will identify a range of UK and international stakeholders for consultation on the strategies and approaches to ODA management and expenditure taken by FCO and the British Council, both in general and in the MENA region. This is likely to include other UK departments (e.g. DFID, Ministry of Defence), other diplomatic services active in similar fields (e.g. Sweden), private foundations with similar aims (e.g. the Open Society Foundation and the Anna Lindh Foundation), research institutes and think-tanks (e.g. Chatham House) and UK academics.

5.12 As well as providing an informed critique of their strategies and approaches, these consultations will enable us to assess whether FCO and the British Council are linked in to a wider policy community on MENA and the Arab Spring response and whether ideas and lessons are being shared amongst peer organisations. It will also enable us to assess whether FCO and the British Council are sufficiently transparent with and accountable for their use of UK ODA.

Review of financial and project management systems

5.13 We will review FCO and British Council financial and project management systems. This will involve reviewing their documentation, including corporate strategies, policies and guidelines, programme strategies, financial and programme rules and procedures, financial and activity reports, monitoring and evaluation reports and corporate reporting. It will also involve interviews with senior management and London-based staff. Where appropriate, we will draw comparisons between FCO and British Council systems and those used by DFID.

Portfolio reviews of the MENA region

5.14 We will conduct portfolio reviews of FCO strategic programmes in the MENA region under the ‘Prosperity’ and ‘Diplomatic Influence and Values’ categories and British Council spending programmes falling under its ‘Education and Society’ business unit, particularly those with a conflict reduction and youth focus. The portfolio reviews will focus on spending patterns and types of activities and whether they indicate strategic use of resources.

Review of partnerships and influencing

5.15 As an example of aid-related frontline diplomacy, we will assess how effectively FCO builds partnerships with other international actors and influences their programming choices and approaches in responding to the Arab Spring. We will look in particular at FCO’s partnerships with the European Union, the G8 and the Deauville Partnership.⁴ We will assess the level of complementarity between diplomatic engagement and programme expenditure. We will also examine how FCO measures the impact of its aid-related diplomacy.

Case studies

5.16 We will carry out country visits of one week each to Egypt and Tunisia. In each country, we will review a sample of relevant projects from FCO’s strategic and bilateral programmes and related diplomacy. We will assess how well FCO bilateral spending supports the Country Business Plan in each country. We will also examine a selection of British Council activities under its ‘Education and Society’ business unit, focusing on activities related to conflict reduction and youth skills and empowerment.

5.17 The projects for detailed review will be chosen prior to the country visits, by reference to materiality of expenditure, representativeness of the MENA portfolio, phase of implementation and feasibility of access to project sites and intended beneficiaries. The sample should include some completed projects, for which we will assess sustainable impact.

5.18 For projects in our sample, we will review project design, the quality of implementation, sustainable impact and learning. We will review project design documents and financial and activity reports and results data. We will interview the responsible FCO and British Council project managers, implementing partners and project partners, project participants and intended beneficiaries and, as far as appropriate, national stakeholders and informed observers.

5.19 We will identify and interview other individuals who can offer informed assessments of the strategies and approaches taken by FCO and the British Council, including government officials, civil society representatives, academic experts, journalists and international partners.

6. Roles and responsibilities

6.1 KPMG will oversee and support the evaluation under the overall leadership of the ICAI Project Director. It is proposed that the evaluation be undertaken by a core team of three, together with a researcher for the literature review and a peer reviewer familiar with FCO as an organisation. While lead responsibility for answering sections of the framework is indicated below, all members will contribute to research and analysis as required.

Team member	Role
Team leader	Team Leader: design and management of the evaluation; strategic analysis; country visits; delivery of draft evaluation report
Team member 1	Principal Consultant: interviews and stakeholder consultations; portfolio reviews; country visits; strategic and programming assessment; support for drafting and analysis
Team member 2	Consultant: review of programme and financial management; financial analysis; mapping of spending patterns; fiduciary risk assessments; country visits; support for drafting and analysis
Team member 3	Peer Reviewer and Advisor: provide the team with knowledge of FCO's processes and how strategy and programming decisions are made
Team member 4	Researcher: literature review

Team leader (Agulhas)
With over 15 years in policy analysis, he has worked for a variety of clients on a range of high-level policy issues including implementation of the Paris Declaration, aid effectiveness and fragile states. He is an authority in international law and human rights and has written widely on post-conflict reconstruction, state-building and the restitution of property. He has more recently specialised in aid effectiveness and governance processes at all levels, including policy development, programme design and evaluation. He will have overall management responsibility and ensure delivery of the outputs. He will also lead the Tunisia and Egypt case study teams.

Team member 1 (Agulhas)
He is an economist and senior manager with over 25 years of development experience and a wide knowledge of the international development system. He has been a Principal Consultant with Agulhas Applied Knowledge since 2010. Prior to this, he spent five years as International Director of Tearfund, including responsibility for over 1,000 staff in fragile states. Between 1990 and 2005, he worked for the UK’s Department for International Development. His final post, from 2002 to 2005, was as Director, International with responsibility for the development aspects of the UK’s multilateral partnerships, including the World Bank, Regional Development Banks, EU and UN; and for trade, conflict and humanitarian work. Previously, he was a DFID economic adviser working on India and Bangladesh and served as Head of Asia Regional Economic Policy Department and as Deputy Director, Asia.

Team member 2 (KPMG)
He works in KPMG’s International Development Assistance Services group in the UK and has a range of professional experience in research, analysis, project management and policy advisory services in a wide range of organisations in the government, diplomatic and international development sectors. He has undertaken both qualitative and quantitative research and analysis and has experience working in large and complex organisations. He has previously worked for the European Bank for Reconstruction and Development, the BBC, the US Senate and the US Department of State. He will participate in the Egypt and Tunisia case studies and will also conduct a detailed review of resource allocation, project cycle management and financial expenditure mapping to programme objectives.

Team member 3 (Independent)
She has extensive experience as a practitioner, helping governments, civil society and businesses to be more strategic, prepared for the future and effective global citizens in an interconnected and uncertain world. She has previously worked for the Cabinet Office and FCO in strategy development roles. She is the peer reviewer for this evaluation.

Team member 4 (Agulhas)
With over 35 years’ international experience of the UK aid programme, often on secondment to Britain’s Diplomatic Service, he served for 20 years in politically and often physically challenging environments. He has managed short- and long-term projects on international development and disaster relief and has delivered a wide range of consultancy assignments on policy and strategic change. He offers extensive knowledge and analytical skills regarding political analysis in volatile environments. He will conduct the literature review for this evaluation and will provide the team with advice on British Council operations and developments.

7. Management and reporting

7.1 We will produce a first draft report for review by the ICAI Secretariat and Commissioners by 26 March 2013, with time for subsequent revision and review prior to completion and sign off in June 2013.

8. Partner liaison

8.1 This evaluation will require close liaison with FCO and the British Council in London and in post in Egypt and Tunisia. The Commissioners, ICAI Secretariat and evaluation team met with both organisations on 27 September 2012 to receive initial briefings and discuss the scope of the review. Since sharing the draft ToRs, the evaluation team has conducted further meetings with both organisations to outline the evaluation methodology and agree processes for accessing documents and interviewees.

9. Expected outputs and time frame

9.1 The main deliverables will be:

the main report, in the standard ICAI format; and
additional data annexes, as required.

9.2 The main deliverables will be:

Phase	Timetable
Planning Preliminary consultations Planning and methodology Finalising inception	15 November 2012 - 14 December 2012
London-based research Literature review Interviews with FCO and British Council senior management and programme teams Interviews with other UK stakeholders Review of policies, strategies and guidance Analysis of governance and management arrangements Mapping of MENA portfolios	19 November 2012 - 18 January 2013
Field research Tunisia case study Egypt case study	w/c 21 January 2013 w/c 28 January 2013
Analysis and write-up Roundtable with Commissioners Further analysis and first draft Report quality assurance and review by Secretariat and Commissioners Report to DFID for fact checking Report finalisation	27 February 2013 By 26 March 2013 w/c 1 April to w/c 13 May 2013 20 – 31 May 2013 w/c 10 June 2013

10. Risks and mitigation

10.1 The following sets out the key risks and mitigating actions for this evaluation.

Risk	Level of risk	Specific Issues	Mitigation
Inability to access information, due to confidentiality surrounding FCO programmes	Low	FCO is the key instrument of UK foreign and security policy and operates with a higher degree of confidentiality than DFID. Some FCO programme documents are classified.	Design evaluation and select case studies and programmes so as to enable robust evaluation without the need for access to higher-level documents. This includes selecting programmes from the 'Prosperity' and 'Diplomatic Influence and Values' categories, rather than 'Security'. The core team all hold a minimum of SC-level Security clearance. Appropriate arrangements have been put in place to safeguard protectively marked documents.
Security risks associated with case studies in conflict zones	Medium	The case studies will include Tunisia and Egypt, with the latter carrying relatively high security risks.	While formal duty of care rests with the contractor, we will discuss with FCO posts what logistical and security support they can offer the team during the country visits.
Lack of impact data makes impact assessment impossible	Medium	The nature of the activities under review makes them difficult to evaluate. Evaluations methodologies for assessing diplomacy and influence are not well developed, making robust assessment challenging. Many of the activities are designed to contribute to complex outcomes in highly dynamic environments, making attribution difficult.	The evaluation team will review available methodologies for evaluating influence and use these to refine the evaluation methodology as necessary. The team will examine a range of projects across the two case studies, to obtain sufficient results data. If robust impact evaluation of FCO activities does not prove possible, the evaluation will make this clear and make recommendations for improving results management in the future.
Political sensitivities restrict access to intended beneficiaries and national stakeholders	Medium	In some instances, UK support for the activities in question will be undisclosed. FCO may advise against interviews with government officials and other national stakeholders as potentially harmful for the projects in question.	The evaluation team will be guided by FCO in deciding what consultations are appropriate in each country. In the event that key stakeholders cannot be accessed, every effort will be made to identify alternatives to receive informed and balanced feedback.

11. How this ICAI review will make a difference

11.1 FCO-managed ODA is an aspect of the UK aid programme on which there is very limited public information or independent scrutiny. Although it is not large in financial terms compared to DFID’s expenditure, it is designed to be highly strategic. It provides diplomatic support to the rest of the aid programme. It operates in a wide range of countries in which there is no bilateral UK aid programme. It also serves as the UK’s primary response instrument to key international events. This ICAI review will help to introduce this aspect of the UK aid programme to the public, promoting transparency and accountability.

11.2 This review will also examine the UK’s ODA response to the Arab Spring, which is a subject of considerable public interest and strategic significance.

11.3 Both FCO and the British Council have welcomed our decision to review their ODA-funded activities, seeing it as an opportunity to benefit from critical appraisal and to ensure that they are taking the necessary measures to ensure effectiveness and value for money.

Footnotes

Terms of reference: Evaluation of FCO’s Bilateral Aid Programme, ICAI, http://icai.independent.gov.uk/wp-content/uploads/2011/11/Evaluation-of-FCOs-Bilateral-Aid-Programme.pdf.
The Board includes an FCO Minister, FCO Directors, two Heads of Mission, DFID’s Head of Evaluation, two non-executive directors from industry and an NGO representative. It is supported by a cadre of internal evaluators and evaluation expertise from DFID and other departments.
Projects with a budget of over £100,000 are evaluated by a member of the programme team from outside the country post in question, while projects over £500,000 are evaluated by someone from outside the programme, wherever resources permit.
The May 211 G8 Summit in Deauville, France, led to the formation of the Deauville Partnership with Arab Countries in Transition. It includes the G8 countries, Egypt, Jordan, Libya, Kuwait, Morocco, Qatar, Saudi Arabia, Tunisia, Turkey and the United Arab Emirates, together with a range of multinational organisations including the African Development Bank, the European Investment Bank, the International Monetary Fund, the Islamic Development Bank and the World Bank. See http://www.state.gov/e/eb/ecosum/2012g8/deauville/index.htm.