UK aid in a conflict-affected country: Reducing conflict and fragility in Somalia

1. Purpose, scope and rationale

The purpose of this review is to explore how effective UK aid has been at tackling conflict and reducing fragility in Somalia. Covering both DFID bilateral programming and the Conflict, Stability and Security Fund (CSSF),¹ it will examine the evolution of the strategies governing aid interventions and the effectiveness of programming since 2009.²

The review is timely in view of the UK government’s commitments under the 2015 National Security Strategy³ and Aid Strategy⁴ to spending 50% of DFID’s budget in fragile states and to using the aid programme to tackle the root causes of global insecurity. Building on an earlier ICAI review of the scaling up of UK aid in fragile states,⁵ it will consider whether the UK government has an effective approach to addressing conflict and fragility.

This is a performance review, which means it will probe whether the design and delivery of programmes are effective and maximise value for money. This will include assessing whether effective mechanisms are in place for delivering aid in an insecure environment.

2. Background

Somalia is one of the world’s most fragile states,⁶ and has been the recipient of UK assistance for over two decades. It is a priority country for the UK National Security Council (NSC).⁷ DFID’s programming in Somalia has the aim of ‘promoting longer-term stability and of transforming the lives of poor Somalis.’⁸ According to DFID’s wider peacebuilding and state-building framework,⁹ economic development and service delivery are key elements in tackling conflict and fragility, alongside stabilisation and governance interventions.¹⁰ According to DFID’s Country Operational Plan (2011-2016), its programming focuses on four areas: i) governance and peacebuilding; ii) economic development; iii) human development (health); and iv) humanitarian assistance and community resilience.¹¹

In total, the UK accounts for approximately one fifth of the total Official Development Assistance (ODA) flows to Somalia.¹³ The profile of UK aid in Somalia has evolved over the past five years. The total ODA spent in Somalia between 2009 and 2014 is just over £484 million, of which the Foreign and Commonwealth Office (FCO) and the Conflict Pool each spent £19 million, with DFID spending the remaining £446 million.¹⁴ DFID’s spend in humanitarian assistance increased from £30 million in 2010-11 to £39 million in 2015-16. However, the share of its expenditure spent on humanitarian assistance has declined from 65% to 30% during the same period.¹⁵ Governance and economic programmes now have a more prominent place in the portfolio, at 31% and 26% respectively. Under the human development pillar, DFID Somalia focuses on the health sector.¹⁶

DFID relies on a range of delivery channels to implement its interventions in Somalia. These include large multi-donor funds managed by the World Bank and United Nations agencies and, since 2013, the Somali New Deal Compact, which provides an overarching framework for international engagement in Somalia.¹⁷ Private contractors implement about a quarter of DFID’s programme budget.¹⁸ DFID has developed remote management approaches to help overcome its lack of direct access in Somalia, including dedicated third-party monitoring programmes.

While DFID remains by far the largest spender of UK aid in Somalia, programming by the Conflict Pool and its successor, the CSSF, has increased in recent years, reaching 5% of UK ODA to Somalia in 2014.¹⁹ The CSSF is a cross-government fund under the authority of the National Security Council. It is managed by a Joint Programme Hub that draws on expertise from across government. Along with DFID, it supports the implementation of National Security Council country strategies. In Somalia, under the strategic direction of the interdepartmental Whitehall Africa Group, the CSSF funds programmes in a number of thematic areas of interest to this review, including the rule of law, the security sector, stabilisation, reconciliation, state formation and elections, and human rights. The CSSF contains both ODA and non-ODA funding; our review will be limited to ODA-funded activities.

3. Review questions

This performance review is built around the evaluation criteria of relevance, effectiveness and efficiency.²⁰ It will address the following questions and sub-questions:

Table 1: Our Review Questions

4. Methodology

Given security constraints on research in Somalia, the review will make extensive use of DFID and CSSF programme data, but will include a number of elements of triangulation. The methodology will involve four main components. The following figure shows how the components interact and the levels at which they apply.

Component 1 – Strategic Review: To assess the relevance of UK aid, we will examine the evolution of UK strategies and objectives in Somalia and the extent to which they have offered a coherent framework for UK aid. To assess consistency, we will map the aid portfolio against those objectives. This will include a balance between strengthening the central institutions of the state, and peacebuilding at local and community levels. The strategies and approaches taken will be compared to available evidence on the drivers of conflict and fragility in Somalia and what works in addressing them. We will also explore coherence and coordination across the UK government, including by mapping coordination processes and conducting a number of small case studies on how they have operated in specific instances. We will gather relevant information through the following: i) a desk review and qualitative assessment of UK strategies, policies and results frameworks for conflict-affected states more broadly and Somalia in particular; ii) a mapping of programming and spending; iii) a review of interdepartmental coordination mechanisms; iv) a review of learning processes; and v) consultations with key government, academic, non-governmental organisations (NGOs), and programme implementation stakeholders in London. This will include at least two stakeholder workshops in London.

Component 2 – Literature Review: A review of literature on the country context and its recent evolution will help us to assess the relevance and effectiveness of UK aid. We will review evidence from the literature to determine positive and negative impacts of external assistance over the review period and learning on what works. We will collect data on conflict and fragility in Somalia, drawing on international indices. This data will inform our contribution analysis by enabling us to contextualise and identify rival explanations for claimed results. Furthermore, it will enable us to assess the adequacy of the UK government’s investments in research and data collection. The review will cover published and unpublished academic and official literature and commentary, including appraisals by DFID and other development actors of what works and lessons learned for programming in conflict affected contexts.

Component 3 – Programme Desk Reviews: We have selected a cross-section of programmes for desk review, in order to assess their relevance, effectiveness and approach to value for money. Programme reviews will be undertaken at two different levels of detail. We will carry out detailed reviews of programmes with explicit objectives of tackling conflict and fragility. We will conduct more limited, rapid reviews of programmes with primary objectives in other areas (eg health or livelihoods), but with potential indirect impact on conflict and fragility. For these programmes, our assessment will be limited to collecting evidence of their impact on conflict and fragility, rather than their performance against other development objectives.

Component 4 – Country Visit: We will conduct a three-week visit to Kenya and Somalia. This will enable detailed consultations with DFID and the FCO staff working on Somalia in Nairobi and Addis Ababa, and interviews with key stakeholders from implementing partners and other development partners based in Kenya. If security conditions permit, we will visit both Mogadishu and Hargeisa to meet with Somali counterparts and view implementation arrangements. The country visit will enable us to triangulate and deepen the analysis in the programme reviews, through key informant interviews and feedback from counterparts. It will also enable detailed consultations with the responsible UK departments to ascertain their strategies and approaches, their delivery arrangements and their approaches to value for money.

5. Sampling approach

We have chosen to review a sample of 25 UK aid programmes in Somalia. The sample includes all of DFID’s major governance and security programmes, and a purposive selection of other programmes based on their potential contribution to addressing conflict and fragility. Our sample includes 12 DFID bilateral programmes active in Somalia during the review period, including one centrally-managed and 11 country-led programmes. Excluding centrally-managed programmes, this constitutes almost 90% of DFID programming by expenditure.²¹ Over a quarter of those programmes – by expenditure – will be assessed through detailed reviews and the remainder through more rapid reviews. The Conflict Pool/CSSF portfolio of programmes differs from DFID’s, featuring a higher number of programmes, usually considerably smaller than DFID’s and often of relatively short term duration. We have made a purposive selection of 13 Conflict Pool and CSSF programmes, based on their thematic relevance to the review questions. They constitute approximately 40% of the portfolio of programmes included in our sampling framework (by expenditure).²²

6. Limitations to the methodology

Results data: In assessing the effectiveness of UK aid programmes in Somalia, we will rely primarily on monitoring and evaluation data generated by the programmes themselves. We will manage the resulting risk of bias by triangulating in a number of ways. We will assess the quality of DFID’s independent monitoring data, including by checking source data coherence on a sample basis. We will interview counterparts and other development partners on programme effectiveness, and we will consult programme implementers at the working level on how implementation challenges have been addressed. This will enable us to reach some conclusions about the accuracy of DFID’s results data. However, in the event that the data is inaccurate, we will have limited capacity to reach independent conclusions about programme effectiveness. Consequently, we will highlight any data gaps and areas where it is not possible to draw robust conclusions about programme effectiveness.

Attribution: The dynamics of conflict and instability in Somalia are complex, with many different actors and local variations. Attributing impact on levels of conflict and fragility to UK aid is likely to be challenging. We will base our assessment on a comparison of the UK’s claims with evidence from stakeholder feedback and evidence on what works. We will assess progress against the intended outcomes of UK aid and test alternative causal hypotheses for positive and negative movements in key indicators. We will seek to assess whether the UK’s claims regarding its contribution are credible.

7. Risk management

8. Quality assurance

The review will be carried out under the guidance of ICAI Lead Commissioner Richard Gledhill, with support from the ICAI Secretariat. The review will be subject to quality assurance by the Service Provider consortium. Both the methodology and the final report will be peer reviewed by Alice Hills, Professor of Conflict Studies in the School of Government and International Affairs of the University of Durham. Professor Hills is an academic expert on security and development, counter-insurgency and security governance in fragile states. She has published extensively on state-building and security institutions in Somalia.

9. Timing and deliverables

Subject to measures needed to mitigate the risks outlined above, the study will be executed within nine months starting from mid August 2016.

Footnotes

1. The review will include programming by the CSSF’s predecessor, the Conflict Pool.
2. 2009 has been selected because it allows the review to track the UK’s approach prior to the decision to establish a distinct, strategic DFID Somalia operation and because it is the earliest date for which DFID is able to supply the comprehensive programme and management data needed for a performance review.
3. National Security Strategy and Strategic Defence and Security Review, HM Government, November 2015, link.
4. UK aid: tackling global challenges in the national interest, 2015, p. 3, link.
5. Assessing the Impact of the scale-up of DFID’s Support to fragile states, ICAI, February 2015, link.
6. For example, Somalia was ranked as the most fragile state in the Foreign Policy Fragile states index in 2016, link. It was ranked second in 2014 and 2015.
7. Somalia Operational Plan, DFID, 2014, p. 5, link.
8. Somalia Operational Plan, DFID, 2014, p. 5, link.
9. Building Peaceful States and Societies – DFID practice paper, DFID, 2010, p. 6, link.
10. DFID is currently developing a new stability framework to better take into account global factors impacting fragility, the role of inclusive economic development and review the assumption that services delivery leads automatically to stability: Building Stability Framework, DFID, draft August 2016, p. 5, unpublished.
11. Somalia Operational Plan, DFID, 2014, p. 6, link.
12. 2016-17 figures are indicative. The actual resource allocation round is £119m for this year.
13. OECD ODA Data, 2014, link.
14. According to latest ODA data available (2015 ODA data are not available yet). UK ODA Data, 2009-2014.
15. UK ODA Data, 2009-2014.
16. UK aid funds the education sector in Somalia through the Global Partnership for Education and the centrally-managed Girls’ Education Challenge.
17. The Somali Compact, 2013, link.
18. Based on data provided by DFID.
19. UK ODA data, 2014.
20. Based on the OECD DAC Evaluation criteria. See OECD DAC 1991, Principles for Evaluation of Development Assistance, link.
21. Our sampling frame does not include programmes in inception or at an earlier stage.
22. Our sampling frame included a total of 95 UK aid programmes active in Somalia. We excluded from our sampling frame programmes still in inception, centrally-managed programmes with less than 5% or £100,000 expenditure in Somalia, Conflict Pool/CSSF projects with above 90% non-ODA spend, as well as Conflict Pool/CSSF projects that are limited to funding staff positions or secondments.