The Conflict, Stability and Security Fund’s aid spending

ICAI score

The CSSF is flexible and responsive to the priorities of the National Security Council, but variable programme quality, weak results management and insufficient learning undermine its contribution to building peace, stability and security.

The CSSF is a flexible and responsive instrument for supporting the implementation of National Security Council strategies. Its programmes are well informed on conflict dynamics, and its programming is able to adapt and stay relevant in volatile contexts. However, the country and regional portfolios we have seen often lack a clear logic connecting the activities they support to the objective of promoting sustainable peace, stability and security.

Results management practices are inadequate, given the scale of the funding. As a result, there is little reliable data on whether projects are achieving their intended results or delivering value for money. While we saw good quality programming in some areas, most of the programmes we reviewed showed design or implementation flaws, and at times the approach was contrary to the available evidence of what works. The CSSF does, however, have some good practices on gender and conflict sensitivity.

The CSSF has introduced some useful learning processes but does not give enough attention to capturing or sharing learning from its own experience. Given its size, the CSSF’s efforts to fill global evidence gaps are modest.

Individual question scores
Question 1 Relevance: How relevant and strategic is the CSSF’s response to particular conflicts and crises?
Question 2 Effectiveness: How effective is the CSSF at addressing conflict, instability and insecurity and at promoting sustainable peace, stability and security?
Question 3 Learning: How well is the CSSF learning what works in tackling conflict, instability and insecurity?

Executive summary

Conflict is both a major constraint on development and poverty reduction, and a source of threats to the UK and its interests. The intensification of destructive conflict in a number of countries and regions since 2010 is widening the poverty gap between conflict-affected and other developing countries. As the number of people around the world living in extreme poverty falls, an increasing proportion of the world’s poorest are living in conflict-affected places.

Over the past decade, the UK government has significantly increased its aid to fragile and conflict-affected countries and regions. The Conflict, Stability and Security Fund (‘the CSSF’ or ‘the Fund’) is one of its main instruments for tackling conflict and instability. Accessible to a range of government departments, it combines Official Development Assistance (ODA) and non-ODA funding to support the implementation of National Security Council (NSC) strategies.

The CSSF was established in April 2015, as the latest in a series of cross-government conflict prevention funds that stretch back to 2001. Its initial annual budget of £1 billion is set to rise to £1.3 billion by 2020-21. This review is only concerned with the CSSF’s aid programming expenditure. Therefore, in its first two years of operation, programming within the scope of this review amounted to £353 and £460 million (or 34% and 41% of the CSSF’s total spending). Our findings, comments and recommendations relate only to this aid programming.

We assess whether the CSSF takes a relevant and evidence-based approach to spending ODA on addressing conflict, instability and insecurity, whether its programming is likely to be effective, and how well it is learning from the available evidence and its own experience. We conducted case studies of four CSSF country portfolios (Colombia, Iraq, Jordan and Pakistan) and two regional portfolios (the Caucasus and the Sahel), with visits to Colombia, Jordan and Mali. As CSSF aid programming is very diverse, our findings are specific to those case studies and may not be representative of the Fund’s wider portfolio.

Relevance: How relevant and strategic is the CSSF’s response to particular conflicts and crises?

The CSSF is tasked with supporting the implementation of the NSC’s country, regional and thematic strategies. The older NSC strategies we looked at were too broad to provide a strong strategic framework for CSSF programming, but more recent iterations show greater focus. In five of our case studies, the CSSF portfolios were aligned closely with the NSC strategies. In the sixth, Colombia, the strategy predated the November 2016 peace accord and was no longer used to guide the Fund’s work at the time of this review.

The CSSF’s aid programming is also aligned with the UK aid strategy. It has helped to deliver on the government’s commitment to increase funding to address conflict and instability, with a focus on tackling threats to UK national interests. The CSSF contributes to two of the aid strategy’s four strategic objectives: strengthening global peace, security and governance, and strengthening resilience and crisis response.

We found that the CSSF is well informed on ODA eligibility rules and has effective procedures to ensure that they are met.

The CSSF and its participating departments invest in good quality analysis of particular conflict situations. While the written analysis is not always kept up to date, we found CSSF staff to be well informed about conflict dynamics and current developments. We saw many examples of CSSF programming adapting in response to a changing context (for example following the Colombian peace agreement or the retreat of Daesh in Iraq) to maintain its relevance.

The CSSF is set up to be nimble and flexible. Its ability to react to changing contexts with quick, tactical interventions can be important in responding to conflict situations. However, the Fund is weaker at ensuring that the totality of its aid programming is greater than the sum of its parts and working towards the end goal of sustainable peace and stability. We found that the CSSF’s programme-level theories of change, which should describe how each intervention contributes to this end goal, were unconvincing and did not offer an adequate basis for managing programmes and measuring results. We recognise that strong theories of change are rare in the field of conflict-related programming, and that there is limited evidence available on how to use aid to promote sustainable peace. However, this has been the case since the UK government established its first conflict prevention funds in 2001, and we would have expected more progress in overcoming these challenges.

The CSSF’s coordination arrangements are much improved compared to those of its predecessor, the Conflict Pool, with better communication, mutual support and collaboration between participating departments, both at the central level and within UK diplomatic posts. We saw examples of CSSF programmes and UK diplomatic initiatives pursuing shared objectives.

Wherever possible, the CSSF aligns itself closely with the conflict management and peacebuilding agendas of partner country governments (although some of its work promoting media plurality and civil society support is appropriately done without engaging the national government). The Fund uses its aid programming to help build relationships with national actors and to influence national policies. The CSSF is also an active participant in many multilateral and multi-donor conflict management efforts, contributing ODA funding to obtain a ‘seat at the table’ in international initiatives, and using its technical expertise to help shape them.

These are strong positive findings on relevance, meriting a green-amber score. Our main reservation on the relevance of the CSSF’s aid programming is the absence of a clear rationale for how it contributes to the UK’s objective of promoting sustainable peace, stability and security.

Effectiveness: How effective is the CSSF at addressing conflict, instability and insecurity and at promoting sustainable peace, stability and security?

It is difficult to measure the results of programmes that tackle conflict, instability and insecurity. Progress tends to be slow and uneven, attribution of results to external interventions is problematic, and the volatile context makes it difficult to sustain activities for long enough to assess the results. In recognition of this, we looked at whether the CSSF is designing, implementing and monitoring its programmes in a results-focused way, based on generic good programming principles and on the available evidence on what does and does not work in conflict-related aid programming.

We found that the CSSF has inadequate results management processes, with basic information on what the programmes had achieved either missing or incomplete in almost all the programmes we reviewed. Programme design documents often fail to distinguish accurately between activities (such as training security forces), outputs (the skills that are acquired) and intended outcomes (people using their new skills in useful ways). Many of the programmes we reviewed lacked plausible indicators, baselines, targets or milestones, and therefore had no way of assessing whether they were achieving their intended results. This was particularly the case for programmes made up of multiple, loosely connected activities.

Inadequate theories of change and results frameworks were mirrored, across our sample, in weak monitoring and evaluation arrangements. The CSSF is aware of these shortcomings and is taking a range of steps to address them, but these are at an early stage. The lack of meaningful results data means that neither the CSSF nor external reviewers such as ourselves can ascertain whether CSSF investments are effective and achieving good value for money. It also makes it difficult to detect any unintended harmful outcomes in unpredictable conflict settings.

In the absence of effective results management, we found considerable variation in the quality of the CSSF aid programming that we assessed, and many instances of poor design and delivery. Across our six case studies, we found an over-reliance on training, which contradicts the available evidence on what works. Training improves the work of government agencies only if skills gaps are the main constraint on their performance, rather than political barriers or conflicting objectives. In the examples we saw, some of the training was of uncertain relevance, and the monitoring of impact was limited to post-training satisfaction surveys, which do not measure changes in institutional performance.

We were impressed by the CSSF’s work on strategic communications and media, aimed at strengthening mutual understanding and counteracting extremist or polarising messaging in divided communities. This was one of the first areas where the CSSF had produced guidance on what works. We also found examples of processes to monitor the quality of outputs and measure outcomes. This area of work shows the value of investing more in learning and results measurement.

The CSSF sometimes designs projects as platforms for diplomatic engagement to further conflict reduction or peacebuilding, and we saw a few examples of successful influencing outcomes. However, the CSSF does not routinely articulate its influencing objectives or measure and report on its achievements. This makes it difficult to judge whether the level of investment in influencing represents value for money. We are also concerned that aid programmes designed to secure influence are not maximising the direct return on their ODA investment. While diplomatic and development goals can be consistent with one another, a grey area emerges when ODA-funded projects serve unstated diplomatic objectives, over which there is limited reporting and accountability.

Across our sample, monitoring and evaluation arrangements were not strong enough to support meaningful value for money assessments. The CSSF has been slow to adopt value for money tools and practices at programme level, and has recognised this in internal reports. The Fund needs stronger portfolio management tools to help it improve value for money at country and regional portfolio levels. We also found that some of the CSSF’s contracting practices – such as short timeframes for projects and disproportionately heavy reporting requirements – may restrict value for money by raising management costs and encouraging a focus on short-term, easy-to-measure outputs.

We found that the CSSF is conscientious in the area of conflict sensitivity – analysing the potential impact of an intervention on the conflict to maximise positive effects and minimise unintended negative consequences. In Iraq, for example, the Fund has worked with implementing partners to incorporate conflict sensitivity indicators into results frameworks and procurement processes, and has supported the conflict sensitivity capacity of its main implementer.

The ‘do no harm’ principle raises particular challenges in respect of working with national government agencies with poor human rights records. UK government policy on security and justice assistance requires that all programmes planning to work with national security agencies perform human rights risk assessments and build risk mitigation measures into their activities. We found that these assessments were not always done in a timely way or were not of adequate quality. In our sample, the assessments always led to decisions to proceed with the planned activities, without any design modifications, suggesting that they may not be an effective control mechanism. We were also unconvinced by mitigation measures, which were sometimes limited to training security services in human rights and humanitarian law.

In keeping with the 2014 International Development (Gender Equality) Act, as well as with research findings on what works in peacebuilding programming, the CSSF has a strong focus on gender sensitivity and furthering the role of women in peace and security initiatives. Gender issues feature in its templates for programme design and reviews. The Fund conducts training sessions on gender issues and has gender experts to advise country teams. However, across our sample, the focus on gender was not always fully maintained during implementation.

Based on these findings, we have given the CSSF an amber-red score for the effectiveness of its aid programming. Inadequate results management has led to programming of varied quality and unverifiable results. Despite recent efforts to improve results management, the CSSF’s performance is below the level we would expect to see for a large fund that builds on extensive prior UK government experience in spending ODA to address conflict and instability. While our findings are not necessarily representative of the entire portfolio, they do suggest that weaknesses in programme management in the CSSF may be compromising effectiveness and value for money.

Learning: How well is the CSSF learning what works in tackling conflict, instability and insecurity?

There is a growing body of research on what causes and perpetuates violent conflict. However, the literature on how to use aid to reduce conflict and instability is much less advanced. It provides some evidence on approaches that have not succeeded, but not much guidance on what works.

Against this background, we find that the CSSF draws on academic literature for its conflict analysis, but not so much to guide policy and programming choices. Efforts to develop and communicate guidance material for some of its major areas of programming, including security and justice, countering violent extremism and serious organised crime, are underway but at an early stage. Moreover, the shift of responsibility for security and justice programming from the Department for International Development (DFID) to the CSSF in 2015 seems to have been accompanied by a loss of knowledge and expertise.

The CSSF has introduced some internal learning processes, including regular programme reviews and regional and global learning events. These reviews and events often focused on programme management issues, which was helpful for participating departments that are new to spending ODA. However, given our findings on the CSSF’s programme management weaknesses, we are not convinced that the efforts were commensurate with the need.

Implementing partners were not often part of the CSSF’s learning processes. Annual reviews were not always shared with implementers, there was no practice of bringing together implementers active in similar areas to share experiences, and the CSSF did not generally share knowledge with external partners. While we appreciate that confidentiality and security requirements may preclude the sharing of some information, we find that the approach is overly cautious and that the lack of transparency inhibits learning. There are practical solutions available: in Iraq, the CSSF team made an unclassified version of its conflict assessment available to partners.

As one of the largest ODA funds of its type in the world, the CSSF could make a much larger contribution to global evidence on what works.

We have given the CSSF an amber-red score due to its inadequate procedures for capturing, using and disseminating evidence and knowledge beyond individual projects and country portfolios.

Recommendations

Recommendation 1

The CSSF should introduce country or regional plans specifying how its portfolios of aid programmes and influencing efforts will contribute to achieving NSC objectives, the intermediate outcomes that the portfolios will achieve, and the assumptions that need to hold for this to happen.

Recommendation 2

Where the CSSF projects are intended to support diplomatic access and influence, the influencing objectives should be explicit and progress reported so that the value for money of the investment can be confirmed.

Recommendation 3

Programmes should demonstrate more clearly and carefully how they identify, manage and mitigate risks of doing harm.

Recommendation 4

The CSSF should address gaps in its results management and in its assessment of value for money of existing programmes as soon as possible. Meanwhile, it must ensure that all new programming includes adequate results management and measures to assess value for money.

Recommendation 5

The CSSF should create conditions that allow for the evaluation of a larger part of its portfolio. Independent reviews and evaluations of the Fund’s work should be undertaken where it is possible to do so.

Recommendation 6

The CSSF should synthesise the evidence on what works in its most important programme areas, both from its own experience and from the literature, and share this with participating departments and implementing partners in the form of thematic guidance.

Introduction

Poverty and violent conflict are closely linked. Poverty and slow, unequal economic growth increase the risk of conflict, while conflicts often make countries poorer. Countries that have previously experienced violent conflict have a greatly increased risk of renewed conflict. This can lead countries to become caught in a ‘conflict trap’ – a cycle of impoverishment and violence from which it is difficult to escape.

After a decade in which the world seemed to become more peaceful, the past seven years have seen a rapid rise in major civil wars with disastrous humanitarian consequences, particularly in the Middle East, the Sahel and the Horn of Africa. By the end of 2016, there were 65.6 million refugees and internally displaced people globally, with 84% of them living in developing countries. The total number of displaced people has increased substantially since 2010, due mainly to a few large-scale conflicts: in 2016, 32% of the global refugee population were Syrians, while refugees from Afghanistan, Somalia, South Sudan and Sudan together made up another 32%.

The intensification of conflict in certain countries and regions is driving a growing gap between conflict-affected and other developing countries. The World Bank estimates that, by 2030, nearly half of all people living in extreme poverty will live in conflict-affected places.

The Conflict, Stability and Security Fund (‘the CSSF’ or ‘the Fund’) is a UK  government instrument for addressing issues of conflict and instability around the world (see Figure 3). Established in April 2015, it combines Official Development Assistance (ODA) and non-ODA funding into a resource that supports the implementation of National Security Council (NSC) strategies. These strategies guide efforts to build stability and to counter terrorism and serious organised crime in countries and regions that are important to UK security interests. The CSSF is accessible to UK government departments that contribute to the NSC objectives of addressing conflict, instability and insecurity abroad. Its annual budget is projected to grow from £1 billion in its first year of operations (2015-16) to £1.3 billion in 2020-21.

The CSSF is the latest in a series of UK cross-departmental conflict prevention funds blending ODA and non-ODA funding. The first two – the African Conflict Prevention Pool and the Global Conflict Prevention Pool – were established in 2001. In 2007-08, they were merged into the Conflict Prevention Pool, which was renamed the Conflict Pool in 2009 and then replaced by the CSSF in 2015.

In 2012, ICAI published a review of the CSSF’s predecessor fund. It concluded that the fund “has proved effective at identifying and supporting worthwhile conflict prevention initiatives and has delivered some useful, if localised, results. It has, nonetheless, struggled to demonstrate strategic impact: it lacks a clear strategic framework and robust funding model… and it has little capacity for measuring results.”

This report comes three years after the launch of the CSSF. Although the Fund is relatively new, we have chosen to conduct a performance review rather than a learning review (see Box 2), since the UK government has had extensive prior experience with multi-departmental blended ODA and non-ODA funding mechanisms focused on conflict, stability and security.

The review assesses whether the CSSF has adopted a relevant and evidence-based approach to addressing conflict, instability and insecurity, and whether this approach is likely to be effective. We also review the extent to which the CSSF is learning from internal and external evidence and experience, and how it uses and shares this evidence and experience to contribute to the global body of knowledge on using aid to help prevent, reduce and resolve violent conflict. Our review questions and sub-questions are set out in Table 1.

Our assessments are limited to the CSSF’s aid programming, and do not cover its non-ODA and non-programming expenditures. We did not include the ODA component of the UK’s international peacekeeping contributions, since this is paid as a percentage of overall costs and is not linked to specific programming. Programming within the scope of this review amounted to £353 million (34% of total CSSF expenditure) in 2015-16 and £460 million (41%) in 2016-17.

This review is based on six case studies of country and regional portfolios. We did not assess the Fund’s governance or management arrangements, except when these impacted on the performance of our sample portfolios. This was to avoid overlap with the work of the UK Parliament’s Joint Committee on the National Security Strategy, and with the National Security Capability Review of the CSSF, the Prosperity Fund and the Empowerment Fund that took place in parallel with our review.

Table 1: Our review questions

Methodology

Our methodology was designed to assess how well the CSSF responds to conflict, instability and insecurity. The core of our methodology is a set of six case studies: the country portfolios of Colombia, Iraq, Jordan and Pakistan, and the regional portfolios of the Caucasus and the Sahel (see Figure 1). We selected these portfolios following consultation with the CSSF. They reflect the CSSF’s diversity in terms of geographical presence, budgets, sectors of work, delivery channels, length of programming history, and the nature of the instability or threat. Together, these portfolios represent approximately 12% of the CSSF’s total programmable ODA expenditure in the two financial years 2015-16 and 2016-17.

From these six portfolios, we sampled 34 projects in 17 programmes. A list of the programmes we sampled is included in Annex 2. Of these 34 projects, 26 were led by the Foreign Office (FCO), five by DFID and three by the National Crime Agency (NCA). This is roughly proportionate to their share of CSSF spending (72% FCO, 20% DFID and 8% other departments).

We assessed the relevance and effectiveness of each of the six portfolios against NSC goals, ODA eligibility criteria and the UK’s wider ODA objectives, and against the context in each country and region. We also assessed the way these portfolios had been informed by and had contributed to global and internal learning. To do this:

• We conducted 126 interviews and 39 group discussions, reviewed CSSF and external documentation, and visited project sites in Colombia, Jordan and Mali.
• We compared our findings with CSSF-relevant findings of related ICAI reviews, to help determine whether findings from our sampled projects may have wider applicability to the CSSF portfolio.
• We compared the CSSF’s approaches against evidence about what does and does not work in using aid to address conflict, instability and insecurity. To do this, we conducted literature reviews on the CSSF’s key fields of work (including reviews that were specific to the three countries we visited) and conducted ten interviews with external specialists (included in the 126 interviews mentioned above).

The methodology is explained in full in our approach paper, which is available on the ICAI website. Both our methodology and this report were independently peer reviewed. Limitations to the methodology are summarised in Box 3.

Figure 1: Our methodology

Background

Aid programming by itself is unlikely to resolve conflicts, instability and insecurity

In the period between 2002 and 2014, global ODA flows to fragile contexts (countries or regions within countries deemed by the international community to be particularly susceptible to crisis and conflict) nearly doubled. However, these ODA flows have mostly responded to symptoms of crises, in the form of humanitarian support (as conflicts cause 80% of global humanitarian needs) and post-conflict reconstruction. Work on conflict prevention and reduction has received far less support.

Moreover, there is no clear consensus on how to use aid funds to reduce conflict risk. Conflicts have complex causes. Over the long term, aid programming may contribute to reducing poverty and inequality and to strengthening the capacity of states to deliver public services and infrastructure, thereby increasing their legitimacy and reducing the risk of conflict. However, aid programming is unlikely to resolve current conflicts by itself (and in some circumstances aid may even contribute to conflict). Aid programming may help create conditions that are conducive to a stable and just peace, but resolving conflicts ultimately requires political solutions at the national or international level.

The UK has increased its aid to fragile contexts

Since the UK government established its first conflict prevention fund in 2001, it has gradually increased the share of its aid going to conflict-affected and fragile regions. In February 2011, DFID wrote that conflict and instability are “amongst the most stubborn barriers to successful development” and the UK government committed to “direct at least 30% of the UK’s aid to fragile and conflict-affected states” by 2015. This led to an increase from 22% (£1.8 billion) in 2011-12 to 30% (£3.4 billion) in 2014-15 (see Figure 2 for related data).

In the 2015 UK aid strategy, the government moved from its 2011 pledge to a new approach that included commitments to “allocate 50% of all DFID’s spending to fragile states and regions” and to “an expanded cross-government Conflict, Stability and Security Fund”. The CSSF commitment translated into a budget of £460 million for the CSSF’s conflict-related aid programming in 2016-17, compared to £114 million in the first year of the Conflict Pool (2008-09).

Figure 2: UK ODA spend and CSSF budget

The CSSF includes a specific UK national interest focus

The UK government justifies its growing conflict-related expenditure both by its importance to achieving development and poverty reduction and by reference to UK national interests. The UK interest in global peace and stability and the need to deal with specific identified threats such as terrorism, organised crime and irregular migration are emphasised in the UK aid strategy and the Strategic Defence and Security Review.

The government’s security strategies for particular countries and regions are determined by the NSC. The CSSF helps to achieve these strategies. In the words of the former National Security Adviser, Sir Mark Lyall Grant, “the whole purpose of the Fund, and the sole purpose of the Fund, is to help deliver the overall strategic effect set by the National Security Council”. Other work that helps deliver the NSC strategies includes defence and diplomatic efforts, as well as the work of other cross-government funds.

The focus on the UK’s security interests is accompanied by an ambition that the CSSF take a “more strategic approach to [the UK’s] work in conflict-affected states” than its predecessor funds did. This involves creating “a closer link between the NSC’s strategic decision-making and CSSF action on the ground”. It also involves bringing together all the UK departments active in this area, including departments with expertise on international and transnational security issues such as counter-terrorism and organised crime.

The CSSF’s portfolio and governance arrangements

The CSSF is the largest of the UK’s cross-government ODA funds. Its total annual budgets are listed in Figure 2. In 2015-16 and 2016-17, the ODA programmable portions of CSSF expenditures were 34% and 41% (£353 million and £460 million) respectively.

In 2016-17, the CSSF funded 97 programmes supporting the implementation of NSC strategies across 70 countries (see Figure 3). Over half of its expenditure was in the ten largest country portfolios, which include our case study portfolios of Jordan, Iraq and Pakistan. Its programming started as single-year programming in its first year of operations, but is now often multi-annual (though always with annual break clauses). However, the Fund is also designed with the flexibility to reallocate funding during a financial year. For example, in 2016-17, £22 million was released for additional work on irregular migration. The CSSF has access to an additional £50 million in ODA funds from the ODA crisis reserve, to fund short (less than 12 months) and urgent interventions.

Figure 3: CSSF-supported countries and regions and their levels of peace

All UK government departments involved in the delivery of NSC strategies can apply for CSSF funding. In 2016, the FCO spent 72% of all CSSF ODA funds. DFID spent 20%. The Ministry of Defence, the NCA and the Home Office spent about 2.6% each, and HM Revenue and Customs and the Crown Prosecution Service together spent less than 0.5%. Departments manage the funds they are allocated according to their own rules and procedures.

The CSSF’s thematic areas are listed in Figure 4. The CSSF uses three main delivery channels (see Figure 5):

• direct delivery by the participating departments
• multilateral organisations and multi-donor trust funds
• contracts with commercial companies and non-governmental organisations,  as well as grants to non-governmental organisations and local civil society organisations.

Figure 4: CSSF spending by thematic area

Figure 5: Delivery mechanism for programmable funds in 2015-16

Includes ODA and non-ODA programmable regional and thematic funds. Excludes ring-fenced activities including peacekeeping.

The NSC sits at the top of the CSSF governance structure. It is responsible for overseeing all issues related to national security, intelligence coordination and defence strategy. It sets the overall strategic direction for the Fund, oversees its activities and agrees its annual budget. Under the NSC, there are nine regional and cross-regional CSSF boards that include senior representatives from the participating departments. These boards decide on the best approach to implementing the applicable NSC strategies, allocate funding to particular countries and approve individual programmes. Local boards, located within UK embassies and consulates, propose programmes for approval by the regional boards and oversee delivery by country teams.

The Joint Programme Hub, an inter-departmental team based in the FCO, provides overall coordination, offers technical and programme management support, issues written guidance, runs a technical assistance network and organises learning events.

The CSSF also funds the Stabilisation Unit, a centre of expertise that supports the work of the participating departments. It has a roster of Deployable Civilian Experts available to support UK and multilateral missions to conflict-affected countries.

Altogether, the CSSF network – including in-country teams, London-based staff, the Stabilisation Unit and other UK government officials with a part-time CSSF involvement – comprises 360 staff and consultants.

Figure 6: Overview of case study countries and regions

Findings

vThis section presents the findings of our review. The findings are specific to our case study countries and programme sample, and may therefore be indicative rather than representative of the CSSF portfolio as a whole.

Relevance: How relevant and strategic is the CSSF’s response to particular conflicts and crises?

CSSF portfolios help deliver NSC strategies, but early iterations of these strategies provided limited guidance for programming

In each country where it operates, the CSSF’s portfolios support the implementation of NSC country or regional strategies. These strategies are relatively recent innovations – the earliest ones for our case study countries and regions date back only to 2015. This first generation of strategies did not provide strong strategic frameworks for programming, and in some cases strategies were written to reflect existing Conflict Pool and CSSF activities.

We were told that the most recent iterations of NSC strategies have  been the product of more cross-departmental dialogue and in-country analysis. This has reportedly resulted in more focused and feasible strategies that provide a clearer rationale for new programming. This was the case for Jordan, which is the only country for which we have seen both the 2015 and 2017 versions of the strategy.

In five of our six case studies, the CSSF’s ODA-funded programming could potentially help to achieve at least some of the NSC objectives, and in most instances we were able to identify the link between individual project goals and one or more of the NSC sub-objectives. In the sixth case, Colombia, the NSC strategy preceded the 2016 Colombian peace agreement. It no longer guided programming at the
time of our country visit, which took place nearly a year after the peace agreement had been signed. We were informed that a new NSC strategy was under development.

The CSSF is aligned with the UK aid strategy

The 2015 UK aid strategy announced a number of policy commitments that are of particular relevance to the CSSF:

• The commitment to increasing the CSSF budget has been met. There has also been an increase in CSSF expenditure, although the Fund’s ODA expenditure has been below planned spending throughout the review period.
• The commitment to using aid to address threats to UK national interests and tackle global insecurity is reflected in the geographical distribution of CSSF funding. Funding allocations to countries and regions reflect NSC assessments of their importance to the UK.
• The CSSF’s work contributes to two of the four strategic objectives of the UK aid strategy (see Box 4).

The CSSF complies with rules on ODA eligibility

DFID and the Joint Programme Hub have provided participating departments with training, written guidance and advice on how to identify the ODA-eligible components of their CSSF work. We found that procedures for ensuring ODA eligibility are sound and that nearly all the ODA-categorised programming in our case study countries is ODA-eligible.

Some activities are partially ODA-eligible, and recorded as such. For example, the Ministry of Defence provides CSSF-funded training to military counterparts. In such cases, only training on certain topics, such as international humanitarian law and the protection of civilians, qualifies as ODA (in accordance with new rules adopted by the OECD in February 2016). We found that appropriate assumptions were made as to what share of the budgets of such programmes was reported as ODA.

For all six case studies, the CSSF has produced or co-produced conflict, stability and security assessments

In 2011, the UK government introduced Joint Analyses of Conflict and Stability (JACS). These analyses are meant to “identify the situation-specific interventions that will be most likely to succeed in helping to prevent conflict and build stability”. The responsible departments used JACS to inform their Conflict Pool and CSSF programming in all case study countries and regions with the exception of Jordan, where, in the absence of an active conflict, the embassy used an alternative DFID tool (the Building Stability Framework) to inform the 2017 version of its NSC strategy and the CSSF portfolio.

These analyses have been useful for identifying conflict drivers and threats to stability. As cross-departmental initiatives, they have also sometimes helped to align the agendas of different departments to address conflict drivers and threats, thus contributing to the government’s goal of acting as ‘one HMG’.

In addition to these joint overall assessments, the CSSF and its projects make use of more tailored assessments. In our case studies, these assessments ranged from quick, internal options papers to a few substantial pieces of externally commissioned research. These assessments have helped identify fields of programming and have even, in some cases, led to well-argued decisions not to invest in a certain field. We have also seen some evidence of CSSF-funded assessments contributing to wider cross-government thinking on mitigating transnational threats.

Analytical work is not always updated when the context changes, but CSSF staff demonstrate up-to-date knowledge

The various assessments have provided the CSSF with insights into drivers of conflict and informed the CSSF’s initial programming, but some of these assessments are now outdated. At the time of our visits, the JACS for Mali was more than two years old, in a rapidly changing environment, and the JACS for Colombia was written in 2013, well before the November 2016 peace agreement was concluded.

However, we found CSSF staff to be up to date about recent developments and analyses in their respective countries (see Box 5). They often had limited access to project sites, but were well connected with FCO political staff, as well as with the host government and the wider international community. They also had access to a range of reports from multilateral and other bodies. As a result, they were able to reflect on conflict drivers, recent developments and the implications for CSSF programming.

Programming retains short-term relevance by adapting to changing realities, but the link to the goal of sustainable peace, security and stability is not always clear

A recent joint study by the World Bank and the United Nations looked at 20 case studies where violence was prevented from escalating or recurring due to early interventions. The research identified lessons from successful prevention efforts. One of these was that action had been taken swiftly to address critical and immediate risks.

With its ability to blend ODA and non-ODA spend and to act nimbly, the CSSF is equipped to help address such immediate risks, and we saw a few examples of this happening. More commonly, and in each of our case studies, we saw evidence of programming that was changed swiftly to seize opportunities, address risks and reflect broader progress – or regression – in pathways to peace, stability and security (such as the Colombian peace agreement, the retreat of Daesh in Iraq, and the
increasing instability in central Mali).

The CSSF’s flexibility to respond to changing contexts with quick, tactical interventions can be important in managing conflict situations. However, this is only one part of the CSSF’s ambitions. The Fund also aims to address the longer-term drivers of conflict, instability and insecurity. For CSSF programming to do this, it needs a clearer rationale for how it will contribute to sustainable peace, stability and security.

This rationale should be set out in theories of change. At a minimum, a theory of change must specify plausible links between desired end results (the what) and the means for achieving them (the how). It should also include key assumptions that must hold in order for the pathway from means to end to be realisable. These assumptions should include the identification of risks that could undermine progress. To retain relevance, particularly in volatile settings, theories of change should be revised regularly.

A theory of change must cover a coherent effort. In the case of the CSSF, this would often include one or more CSSF programmes (ODA and non-ODA), as well as diplomatic or defence engagements and possibly other UK and wider international efforts. At a minimum, a theory of change would cover the CSSF projects that together form a coherent programme.

There has been significant progress in programme coherence, compared to the Conflict Pool portfolios. Nevertheless, roughly half the programmes within our sample still consisted of disparate and poorly joined-up projects. Within the same programme, projects often focused on different issues in different areas, and implementers did not engage with each other (or even know of each other’s existence). Often their grouping into one programme was a form of window-dressing: adhering in form, but not substance, to the requirement of defragmenting programming portfolios. Even in the case of programmes that did seem to be internally coherent, we have not seen examples of theories of change that provide a plausible causal link with the end goal of peace, stability and security (see Box 6). Sometimes, other parts of programme documentation came closer to outlining the link between end results and the means of achieving them, and the various assumptions that must hold for the causal link to be plausible.

We recognise that good theories of change are still uncommon in the broad field of conflict-related programming. However, this is not a recent insight. In the years leading up to the UK establishing its first conflict prevention pool in 2001, the Utstein Study assessed 336 then-recent peacebuilding projects by four donor countries in 13 recipient countries. It concluded that most peacebuilding projects lack a connection to a broader strategy, and that links to broader strategies that did exist were often superficial and little more than pro forma. In the years since then, we would have expected the UK government to achieve significantly more progress in this field than we have seen.

The logic underpinning donor [peacebuilding] activities is often unclear. Numerous strategies and programmes are poorly designed with ill-defined objectives and a lack of clearly stated, tested (or testable) theories of change (ie the implicit or explicit understandings of how it is hoped that what is being done will contribute to peace).

Evaluating peacebuilding initiatives in settings of conflict and instability, OECD, November 2012, p. 33.

The CSSF ensures good cooperation across departments and cross-fertilisation with wider UK government efforts

The 2012 ICAI review of the Conflict Pool concluded that the inter-departmental nature of the fund meant that the transaction costs were high, and that “there seems to be… considerable scope for simplifying the structure”.

Even though the number of participating departments has increased, collaboration among them has improved since the Conflict Pool period. This is apparent at headquarters level, within embassies and, sometimes, regionally. The Joint Programme Hub and the local and regional CSSF boards provide a strong coordination function. Communications have improved, reporting is more complete and of better quality, and there is more mutual support and cross-fertilisation. Delays in contracting and financial transfers are now less common than they were in the CSSF’s first year.

The CSSF implements its work with wider UK programming in mind, and this helps to avoid gaps and overlaps. In Jordan, in anticipation of a renewed DFID presence, the CSSF piloted work for DFID to take over. In Pakistan’s Federally Administered Tribal Areas (FATA), the CSSF also developed a programme for eventual handover to DFID. Also in Pakistan, the CSSF decided not to enter the field of combatting drug trafficking because it would overlap with other UK government efforts. Instead, the CSSF focused on other areas, in part so that its political engagement could ‘oil the machinery’ of larger UK initiatives in Pakistan.

CSSF programming and UK diplomatic initiatives sometimes reinforce each other to achieve the CSSF’s ODA-eligible objectives. We saw evidence of this in Pakistan, as well as in parts of the Caucasus, Jordan and Iraq, and it was most obvious in Colombia. Here, the UK’s long-standing relations with the Colombian government have given embassy staff high-level access to, and insights into the work of, national counterparts. This helps to inform well-targeted CSSF interventions. The Colombian counterparts generally welcome this support, especially in areas where the UK has particular expertise (such as in strategic communication, homicide investigations and cross-government cooperation).

The CSSF aligns its efforts with the priorities of host governments and sometimes seeks to influence these priorities

Most of the CSSF programmes in our case study countries were aligned with the host governments’ conflict- and peace-related priorities. The exceptions were some of the civil society funding and work to promote media plurality, which was appropriately done without engaging closely with the national government.

Sometimes the alignment was the consequence of shared views on the drivers of conflict and how to address them. This was the case in Jordan, where the CSSF and the government of Jordan agreed on a short-term focus on refugee camp stability and on providing municipal infrastructure and service delivery to benefit both refugees and host communities.

In other instances, the agendas were less naturally aligned. In the field of gender equality, for example, the CSSF’s sense of importance and urgency was rarely matched by that of host governments. In such cases, the UK supported the government agendas while, in parallel, attempting to influence these agendas through a combination of CSSF programming and diplomatic efforts. In the examples we reviewed we were satisfied that the CSSF had found an appropriate balance between supporting the host government and attempting to influence its agenda.

The CSSF works with and seeks to influence wider international efforts

The CSSF generally opts for bilateral programmes where it believes the UK government has specific expertise to offer. In other instances, it contributes to joint international efforts (multi-donor trust funds, UN agencies and the World Bank). This has the advantage of reducing transaction costs and sharing risk. When selecting multilateral partners, the CSSF tends to follow the preferred choice of host governments. In Pakistan and Iraq, this is the United Nations Development Programme (UNDP), and in Colombia these are the United Nations Multi-Partner Trust Fund for the Post-Conflict projects and the International Organization for Migration for direct government support.

Much of the CSSF programming is embedded in wider international efforts. CSSF staff are aware of the work of other donors and of the other funding sources of the CSSF’s implementers. We saw only a few CSSF activities that duplicated other international efforts, and a good range of examples where coordination had avoided such duplication, sometimes at the UK’s instigation.

The CSSF also positions itself to influence wider international efforts. It often makes early contributions to multi-donor endeavours to ensure it has a say in decision-making, and then makes the minimum contribution required to retain a seat at the table.

The CSSF’s technical expertise helps shape multi-donor efforts. For example, CSSF staff participated in the UN trust fund project selection process in Colombia. This helped to ensure a sound approach to demining – a new area of work for the trust fund. In Iraq, the team helped shape the UNDP Transitional Justice Programme by funding an advisor and agreeing to be the programme’s first donor.

Conclusions on relevance

We find that the CSSF is aligned to the high-level objectives and policy choices of the UK aid strategy, and to NSC country and regional strategies. The latter were initially retrofitted to existing CSSF portfolios, but are becoming more effective frameworks to guide programming.

In our case study countries, the CSSF has produced or co-produced good quality conflict, stability and security assessments. While the written analysis is not always up to date, we found CSSF staff to be well informed. The country portfolios are guided by the available analysis, and able to respond in real time to changes in context and conflict dynamics.

The CSSF is helping to improve coordination across UK departments, which has improved significantly since the Conflict Pool era. It also coordinates well with host governments and with other international actors. The CSSF prioritises maintaining close relationships with partner governments, and is to some degree able to use those relationships to influence its partners and further UK priorities. It also provides useful technical input into multi-donor efforts.

Our main reservation on the relevance of CSSF programming is that its interventions are often reactive and tactical, without articulating how they will contribute to the longer-term goal of promoting sustainable peace, stability and security, and the assumptions which need to hold for this to happen. The CSSF has prioritised the ability to respond quickly and flexibly to NSC priorities and in-country risks and opportunities, leading to portfolios that can appear fragmented (though less so now than two years ago) and reactive, rather than focused on following a pathway towards the strategic goals identified by the NSC and the UK aid strategy. While quick, tactical interventions can be important in managing conflict situations, the scale of the CSSF’s budget suggests that they need to be balanced by a more concerted effort to address the underlying drivers and maintainers of conflict.

Based on these conclusions, we have awarded the CSSF a green-amber score for relevance.

Effectiveness: How effective is the CSSF at addressing conflict, instability and insecurity and at promoting sustainable peace, stability and security?

In this section, we set out our conclusions on the effectiveness of CSSF portfolios and programmes in our case study countries and regions. In making our assessment, we take into account that the results of programmes that tackle conflict, instability and insecurity are always difficult to measure. Progress tends to be slow and uneven, attribution of results to external interventions is problematic, and volatile contexts make it difficult to sustain activities for long enough to assess the results. We therefore look at whether the CSSF is designing, delivering and monitoring its programmes in a results-focused way, at whether its programming is well planned and implemented, and at the available evidence on the effectiveness of programming.

Results information is either missing or incomplete

CSSF programmes generate activity- and output-level data, but often little else. Programmes are subject to annual reviews, but in our case studies these reviews contained little robust information on whether the activities and outputs were contributing to the intended outcomes – and therefore to the achievement of NSC strategy objectives.

In part, this is the consequence of the CSSF’s inadequate theories of change. Within our sample, we have not seen a theory of change that could be used as intended for planning, monitoring and evaluation. Moreover, the CSSF’s programme design documents and results frameworks often do not distinguish accurately between activities (such as training people), outputs (the skills and competencies gained as a result) and intended outcomes (people using their new competencies in
ways that help address conflict, instability and insecurity).

Many of the results frameworks that we reviewed lacked plausible indicators, baselines, targets or milestones. As a result, even if they were clear about what they wanted to achieve, they offered no means of measuring their progress. In the examples we reviewed, it was not always clear which of a programme’s projects was responsible for which output, or contributed to which outcome, and poor version control of results frameworks meant that the CSSF and its implementers did not always work with the same set of outputs and indicators. Furthermore, where programming did not lend itself to results frameworks, the CSSF did not encourage its implementers to use alternative planning and monitoring tools.

Because of the inadequate theories of change and results frameworks, it is unsurprising that, across our sample, monitoring and evaluation arrangements were not strong enough to test whether the often implicit assumptions behind CSSF project designs were being borne out in practice. Most programmes were not collecting data on their achievements at outcome level. While there was often relatively close oversight by CSSF staff, implementing partners were being held to account for their delivery of planned activities and outputs, rather than their contribution to NSC country objectives.

Of our six case studies, only Pakistan had made use of external evaluation, despite the fact that thirdparty monitoring is increasingly recognised as good practice in security-constrained environments, where third parties with sufficient monitoring calibre and access exist. CSSF staff argued that the relatively small size of most projects meant that the cost of an external evaluation could not be justified (a reasonable argument at project level, but unconvincing across a fund the size of the CSSF).

In January 2017, the parliamentary Joint Committee on the National Security Strategy only had access to documentation in the public domain and concluded that it was therefore “unable to verify whether the CSSF’s programmes are delivering the NSC’s strategic objectives, let alone whether they are collectively having a strategic impact and therefore represent value for money for the taxpayer”. For this review, we also reviewed internal documentation, and we found that evidence required to verify the portfolio’s value for money does not exist. We find the CSSF to have an inadequate set of results management processes for a fund of this size. We note that the absence of results monitoring also means that the Fund does not know if it might cause unintended harm.

The Joint Programme Hub is aware of the Fund’s shortcomings in this area, and is taking action to address them (see Box 7).

Of the small number of project sites we visited, almost all showed signs of poor quality delivery

In the three countries we visited, we conducted a small number of site visits to consult with project implementers, beneficiaries and external observers. On these visits, we found that planned outputs had been delivered but that, in the face of project design and implementation flaws, many of the outputs were of low quality (see Box 8 for examples).

Our sample of site visits was small and not representative of the CSSF portfolio. However, the types of weaknesses we saw are common in projects that are implemented in volatile contexts where direct supervision of implementers is limited by access constraints. Our observations suggest that the CSSF’s current system of quality control over design and implementation decisions which, in our case studies, was largely based on implementer self-assessment, is inadequate.

Some CSSF activities compare poorly to good practice and evidence on what works

There are areas in which the CSSF follows approaches that research has found often to be ineffective. The most common and prominent examples relate to training and broader capacity building efforts. We are also concerned about some of the CSSF-funded ‘quick-impact’ infrastructure repairs.

The effectiveness of using training as a means to improving organisational performance usually depends on a range of other institutional factors changing at the same time. If poor organisational performance is a result of political constraints or conflicting priorities, rather than a lack of capacity, then training is unlikely to improve institutional performance. Even where capacity is a binding constraint, training may only be effective if the right organisational policies, structures and systems are in place to enable individuals to employ their new skills.

Training is a common activity for the CSSF in all six of our case studies. In some instances, it is a sensible part of a wider package of support. The CSSF’s work with community police in Jordan, for example, included not just training but also mentoring, new standard operating procedures, institutional reforms, infrastructure and equipment. In most cases in our sample, however, the decision to provide training was not based on needs assessments that suggested that capacity was the binding constraint or that the conditions for addressing capacity gaps were present. Monitoring of impact was limited to post-training satisfaction surveys, which are not a useful measure of changes in institutional performance.

During our visit to the EU Training Mission in Mali, we saw basic weaknesses that are likely to affect results. The mission had no control over the selection of trainees from the Malian armed forces and did not keep records of the participants’ identities or backgrounds. One key informant reported that some trainees had received the same training more than once, and parts of the training were inconsistent with Malian realities (for example by incorrectly assuming the participation of women in combat operations). The key performance measure was related to ‘the number of people trained’. As it is not known if the training strengthens the trainees’ and their employers’ performance, the number of people trained is not a meaningful indicator. While our findings are limited to our case studies, there is evidence that undue focus on training individuals is a problem across the CSSF portfolio.

Capacity building beyond the provision of discrete training courses requires longer-term investment in a range of interventions to improve institutional performance. We found that the CSSF timelines and annual cycles are poorly suited to the needs of such long-term capacity building. Even for multiyear projects, CSSF contracting and management processes encourage a focus on short-term results. Moreover, we saw a few cases where CSSF capacity building work was unlikely to be sustainable beyond the period of UK support: UK nationals paid by the CSSF played managerial rather than supporting or advisory roles and the host governments showed no sign of being willing to take over the cost of maintaining capacity.

Restoring infrastructure is a common activity in post-conflict settings, designed to provide a ‘peace dividend’ to the population and to restore confidence in national authorities. It features in three of our case study countries.

In a few key Jordanian municipalities, CSSF investments in infrastructure may have helped to reduce friction between Syrian refugees and their host communities. This was appropriate programming, even if there was insufficient attention, in the first few years, to maintenance and opportunities to create jobs through these investments.

For the CSSF-supported infrastructure work in Iraq, however, we found no evidence to back up the stated objective of supporting stabilisation. The implementing partner presents the investments as the work of the national government, in the hope of rebuilding public trust. However, there is no evidence that infrastructure investments are indeed increasing trust. Available evidence in relation to similar projects in Afghanistan reaches mixed conclusions. The Afghanistan experience also suggests that quick-impact projects such as these (the timeline for UNDP infrastructure projects in Iraq is only 90 days) in challenging operating contexts give rise to a high risk of corruption and elite capture, which could undermine public trust. Stabilisation Unit analysis from 2014 found that such quick-impact projects are very rarely a useful application of resources in stabilisation, and that the risks of misuse of funds, corruption and poor delivery are likely to outweigh the potential for positive outcomes. We saw limited evidence of the implementing partner mitigating against these risks. When we asked why the CSSF nonetheless continues to fund this work, the response was that the decision was partly political, and partly based on the expectation that the CSSF could positively influence the international approach through engagement. To date, the CSSF’s efforts have not caused an important shift in the implementer’s approach in relation to its rapid infrastructure work.

The work in Pakistan started in September 2016. The CSSF noted that the implementer paid insufficient attention to conflict drivers and dynamics, but did draw on lessons learned in Afghanistan to reduce some of the key risks. For example, the implementer’s approach was less hasty than had been the case in Afghanistan (and still is in Iraq), the budgets were smaller, and the project worked through community platforms to increase engagement between citizens and public authorities. However, evidence about the extent to which the project is achieving its stated objective of supporting stabilisation is still inconclusive.

The CSSF portfolio also includes potentially effective programming

The portfolios we have assessed include some programming with good prospects of success, designed by the many CSSF staff and implementing partners who are highly knowledgeable in their fields. For example, we found the work on strategic communications to be well designed and implemented in several of our case studies (see Box 9). This is a thematic area where the CSSF was quick to produce guidance on what works. It is also the area of programming where monitoring of the quality of outputs is the most advanced.

CSSF influencing efforts would benefit from more consistently explicit objectives and strategies, and from progress monitoring

Influencing host governments’ agendas and approaches is an important part of the CSSF’s work. In a few cases, we saw evidence suggesting that the CSSF managed to do so. In Colombia, for example, the National Crime Agency (NCA) saw a causal link between a stronger judiciary and a stronger peace, and encouraged parts of the judicial system to coordinate their efforts, using CSSF funding for technical support to facilitate that coordination. Elsewhere, we saw influencing efforts that seemed to have limited prospects of success. For example, some of the influencing work on gender equality, social security reform and countering violent extremism seemed insufficient to stand a chance of countering deeply entrenched views or interests in counterpart institutions.

Influencing is a difficult field of work, but a few important breakthroughs can be sufficient to justify a number of unsuccessful efforts. To reduce the high risks of failure, we would expect to see explicit influencing goals and strategies, a clear rationale for the level of investment, and at least some steps toward measuring what has been achieved. In our case study countries and regions, we did not see enough of this.

We were concerned to find cases where the CSSF had embarked on ODA projects that had limited prospects of success, because they believed the investment would provide the UK with diplomatic access to key actors in government. While diplomatic and development goals can converge, a grey area emerges when the stated objective of an ODA-funded project is actually serving an unstated diplomatic objective, over which there is limited reporting and accountability.

The CSSF’s value for money approach is underdeveloped

The CSSF introduced new guidance on value for money in the second half of 2017, and the Joint Programme Hub is now working to embed new practices across the Fund. Previous programme guidance referred the participating departments to DFID’s Smart Guide on value for money and its 3E and 4E frameworks:

• Economy: getting inputs of the appropriate quality at the right cost
• Efficiency: turning inputs into outputs as cost-effectively as possible
• Effectiveness: the extent to which the outputs are leading to the desired outcomes
• Equity (the additional fourth E): the degree to which the results are equitably distributed.

Although the CSSF has formally followed the 3E framework from the start of its operations, its annual programme reviews were not able to assess value for money meaningfully in practice. DFID’s Internal Audit Committee concluded that “ensuring the CSSF achieves [value for money] appears challenging, and would benefit from a more structured approach across efficiency, effectiveness and economy”. This is not surprising – applying value for money management approaches would require significant improvement in the CSSF’s ability to specify and measure its results. As a result, in the words of one CSSF staff member, progress on building the value for money approach is proceeding “in baby steps”. Pakistan was the only one of our six case studies where the CSSF had attempted value for money assessments of any part of its portfolio.

Some CSSF funding conditions may limit opportunities for value for money

In its first year, the CSSF used only single-year contracts. The Fund has now moved to multi-year contracts in many instances, but these include clauses allowing the CSSF to cancel them at the end of each year, so implementers feel they have only single-year certainty. In fact, and especially in the first year, delays in the release of funding often meant that implementers had less than a year to implement their projects.

In the case of commercial companies and non-governmental organisations, the CSSF imposes overly demanding contractual terms. It requires activity-based budgeting,51 imposes stringent expenditure restrictions related to salaries and equipment, and requires frequent and relatively heavy reporting. The rationale for these measures is to mitigate risk for the UK government. However, the drawbacks include a reporting burden that leads to a disproportionate focus on donor accountability, short time horizons for activities and strong incentives to focus reporting on successful activities and outputs, rather than on outcomes and on addressing more complex and longer-term challenges. In contexts where results take time to achieve, these funding terms may be contrary to maximising value for money.

The CSSF could improve its management of value for money at the country and regional portfolio level

The CSSF has established national and regional boards. In parallel with the Joint Programme Hub, these boards provide an oversight and challenge function that the Conflict Pool did not have. These boards have strengthened the coherence of country and regional portfolios in our case studies, but there are a few areas where stronger portfolio management tools could help these boards to improve value for money at country and regional portfolio level. Current shortcomings in portfolio management include:

• Insufficient use of risk and return analysis to assess the balance of interventions and to manage and monitor risk at a portfolio level – including by aligning the CSSF’s direct and third-party monitoring and supervision requirements to the level of risk of its projects.
• Unclear criteria for the selection of implementing partners, and in particular for choosing between delivery by a UK government agency or an external partner. This includes lack of clarity about the implementation requirements of each project (such as the value of building bilateral ties between counterpart agencies in the two countries, and the need for confidentiality).
• The absence of criteria for right-sizing activities in relation to their objectives and strategic significance. In the portfolios we assessed, it was rarely clear on what basis budgets were set (other than other donors’ choices, in the case of multi-donor contributions). The options appraisals were generally superficial and unconvincing. In particular, where the primary objective was influencing rather than delivery of results at scale, this objective was not explicitly (even if confidentially) expressed, making it hard to assess the level of investment required.

CSSF programmes are relatively sensitive to how their interventions affect conflict dynamics, and the CSSF also provides conflict sensitivity support to key implementers

Conflict sensitivity analysis is mandatory for all CSSF programmes. A conflict sensitivity analysis seeks to understand the effects that an intervention may have on a conflict, so as to maximise positive effects and minimise unintended negative consequences (see Box 10).

CSSF experts in this field who we spoke to were of the opinion that there was significant scope to strengthen such analyses. While this may be true, we note that the CSSF has substantial conflict advisory support and appears to be ahead of many other international stakeholders. Within our sample, this was obvious when comparing the conflict sensitivity of interventions designed by the CSSF with those prepared by other actors (for multi-donor initiatives). The CSSF tries to help the
partners it is working with to achieve higher standards.

Iraq provides a good example of its efforts to accomplish this. The CSSF team in Iraq has made available an unclassified version of the UK’s conflict analysis to other international stakeholders, and will soon be assessing all project options against a set of commonly agreed criteria, including whether they are consistent with the findings of the CSSF’s conflict analysis. Already, the team has embedded a range of processes and principles both in its decision-making and in its day-to-day work to ensure conflict sensitivity.

Conversely, it remains unclear how the CSSF’s key implementer in Iraq – UNDP – uses conflict analysis to guide its project design and implementation. Key elements of its work appear to be conflict-insensitive. Moreover, UNDP is mandated to work in support of government authorities only, which creates the potential for support that is biased in the choice of beneficiaries, projects and local implementing partners, and in procurement decisions, towards priorities that are not always conflict-sensitive or conflict-focused. The extent to which UNDP is mitigating these risks is unclear. The CSSF is attempting to address these shortcomings by funding and organising a conflict sensitivity mapping exercise for UNDP, and by funding the recruitment of additional advisory capacity and local conflict analyses.

The CSSF works with counterparts with contested human rights records, and does not always assess the risks in an adequate and timely fashion

As is often necessary in conflict environments, we found across our case studies that the CSSF is working with armed forces, police, paramilitary forces, prison services, the judiciary and criminal investigators. In another ICAI review, we also covered CSSF work with coastguards. Several of these counterparts are suspected of violating human rights. Working with such counterparts risks legitimising them and their actions, or even becoming complicit in violations. The CSSF considers these risks at some length in programme documentation, but has a high risk appetite. In our case studies we only came across one decision not to work with an institution because of its human rights record. This risk appetite is appropriate, within reason, because the likelihood of successfully addressing conflict without host government involvement is remote.

It is important that we work with a wide range of countries. This includes some countries where we have concerns about human rights. It is of fundamental importance that HMG work on security and justice overseas is based on British values, including human rights and democracy.

Boris Johnson, Foreign Secretary, in the foreword of Overseas security and justice guidance (OSJA), Human Rights Guidance, HM Government, 2017.

For security and justice work in particular, UK government policy requires two risk mitigation measures in cases where a project or programme is set to work with a counterpart that may do harm. First, the work must be designed with an explicit human rights element built into it. In a few cases, we found such elements to be unconvincing. This was particularly the case with training-based solutions (for instance human rights training for Malian security forces), which assumes that violations are the result of a knowledge deficit on the side of the people involved.

Second, the UK government is required to undertake a human rights assessment (known as an Overseas Security and Justice Assessment, OSJA) once a programme’s design is ready but before implementation begins. This is to ensure that the level of risk is adequately assessed and the mitigation measures are signed off at the appropriate level (including by ministers, for the highest-risk projects). Here, we found a number of problems with CSSF-funded programmes. Several OSJAs were produced after programming had commenced and some OSJAs were incomplete or of low quality (typically with a stronger analysis of the UK’s reputational risks than of the risk of CSSF support aggravating human rights violations) or had not been conducted at all. The human rights assessments within our sample always gave the green light for the proposed activity, without requiring any design modifications. This may be the consequence of a small sample but it raises a concern as to whether these assessments are an effective control mechanism.

Some programme designs pay due attention to gender issues and the role of women in peace and security, but this is not always sustained through implementation

As required by UK legislation and government policy, gender sensitivity and furthering the role of women in peace and security are high on the CSSF’s central agenda. We understand that all recent NSC strategies include gender-relevant strategic objectives, and the CSSF has chosen to follow the 2014 International Development (Gender Equality) Act in its ODA programming.

There is UK-wide and CSSF-specific guidance on the issue.60 Gender experts advise on country operations and the Stabilisation Unit organises gender training sessions. In 2016, the CSSF conducted a gender audit for every programme in sub-Saharan Africa. Gender issues feature prominently in the templates of reports and annual reviews, as well as in the CSSF’s first annual report. All CSSF programmes have a three-tier gender scoring system and concomitant targets to meet.

The gender focus starts in the programmes’ inception phase, and the various boards assess proposals accordingly. The rationale is that programmes that are based on a gender-sensitive conflict analysis are more likely to be effective in addressing the specific needs, capabilities and experiences of the whole society in question, including women, men, boys, girls and sexual and gender minorities. At the same time, the CSSF is not dogmatic and emphasises that “care must be taken to ensure that addressing gender is done in a conflict-sensitive manner that does not fuel or exacerbate tensions, gender-based violence and conflict”.

This central commitment to gender sensitivity and to furthering the role of women in peace and security is not yet consistently mirrored at programme level. The CSSF is aware that, across its global portfolio, many annual reviews pointed to the need to strengthen gender sensitivity approaches and to introduce indicators to track progress. This aligns with the findings from our six case studies, as well as from our Somalia review. From interviews we understood that, for some CSSF stakeholders, achieving gender equality means, for example, ensuring that women are included among police trainees, rather than addressing the security and justice needs of women, girls, men and boys. Moreover, a focus on gender issues at programme design is sometimes lost during implementation. We saw examples of this in Mali, Jordan, the Caucasus and in Pakistan.

Only a few of the programmes we have seen were explicitly and clearly gender-sensitive throughout the implementation stage (the recent work in Iraq and parts of the work in the Caucasus stood out). In some cases, even generating gender-disaggregated data poses a challenge. In part, the difficulties in keeping gender issues alive are caused by the fact that gender issues are a higher priority for the CSSF than for partner country counterparts.

The CSSF’s programming is not designed to reach the most marginalised and vulnerable groups

The CSSF is about conflict, instability and insecurity, and its focus is on helping to achieve NSC objectives. Most CSSF programming is not designed to reach the most marginalised and vulnerable populations. The CSSF does not generally focus on marginalisation as a potential driver of conflict, and some vulnerable groups – such as Syrian Palestinians who have been barred from entering Jordan for years – are blind spots for the international community at large. In the words of one respondent: “When it comes to stability… you do not necessarily target those most in need”.

Conclusions on effectiveness

The CSSF’s results management has improved over the CSSF’s first two years, but is still inadequate. Programmes are not designed with clear results frameworks and theories of change, and there is an insufficiently rigorous system of monitoring to support management, accountability and learning. This means that neither CSSF programme managers nor external reviewers such as ourselves have hard evidence as to whether projects are being effectively delivered and achieving their intended outcomes. Plans are promising but progress to date has been slow. The lack of hard evidence will not be resolved in the next two years as this would require the basic requirements for monitoring and evaluation to be in place already, which is not the case.

This is a very challenging area for results management, in which other development actors also struggle. Nonetheless, we find that the level of results management is well below what we would expect to see, given the size of the CSSF budget and the length of time that the CSSF and its predecessor funds have been operating.

We saw some examples of programming with good prospects of contributing to conflict reduction, including the Fund’s work on strategic communications in four of our six case studies. This excludes both Jordan (where there was no stratcom) and Pakistan (where there was stratcom, but it was of poor quality). However, in most other cases we found that programmes were based on implausible assumptions and sometimes contradicted good practice or the available evidence on what works. Capacity building is a major part of the CSSF portfolio, but was generally done poorly. There is a tendency to default to training, even where a lack of skills is unlikely to be the binding constraint on organisational performance. We also found that the CSSF was engaging in quick impact infrastructure work in Iraq, despite a body of evidence questioning whether such activities contribute to conflict reduction.

Given the weakness of its results management systems, the CSSF has made little progress on integrating value for money into programme management. Moreover, its contract management arrangements with the non-governmental organisation implementers in our case study countries and regions incentivise a short-term perspective from implementers and are not conducive to maximising value for money.

We find that the CSSF has strong practices around conflict sensitivity, and is making efforts to improve the conflict sensitivity of its multilateral partners, sometimes from a low base. It also has a strong approach to gender sensitivity, even if this is not always sustained through implementation. However, its implementation of UK government rules on human rights risk management is inconsistent.

Overall, we find this to be a disappointing result, meriting an amber-red score, with the clear red elements around results management practices and some of its approaches to programming.

Learning: How well is the CSSF learning what works in tackling conflict, instability and insecurity?

The global body of evidence provides more insight into drivers of conflict than on how to use aid programming to contribute to peace, stability and security

There is a growing body of research literature on why conflict occurs and how it is perpetuated. The literature is much less advanced on how to use aid to address conflict, instability and insecurity. Even the considerable research around the large-scale aid expenditure in Iraq and Afghanistan came to conclusions that are less clear about what works than about programming approaches that have verifiably failed to reduce conflict and insecurity, and about the risks of unintended consequences of aid in complex and volatile conflict settings. (In addition, it should also be noted that findings from these two countries may not be relevant in settings that have not seen large-scale external military interventions.)

This scarcity of knowledge on what works is reflected in global programming practice. In 2012, the OECD concluded that “the theories underpinning international support to peacebuilding, conflict prevention and statebuilding are weak. There is a lack of agreed upon, proven strategies for effectively working towards peace… Programme approaches are often contested.” This has not fundamentally changed in the years since then.

It is therefore not surprising to find that the CSSF draws on academic literature for its analytical work on drivers of conflict in particular settings, but makes less use of the literature to guide its programming choices. The Stabilisation Unit sometimes conducts quick and largely conflict-specific literature reviews in preparation for its own conflict analyses. These literature reviews may have helped to inform country and regional strategies and programming at some level, but the links between external research and programme design are difficult to trace and our CSSF respondents rarely referred to external research when explaining their choices.

The CSSF does not consistently seize opportunities to contribute to the global body of evidence

The CSSF’s programming – both its successes and failures – have contributed little to inform the research literature on the use of aid in fragile and conflict settings. None of our case studies led to peer-reviewed or other externally available publications on topics within the field of conflict, stability and security. We were made aware of other CSSF work, particularly by the Stabilisation Unit, informing and supporting research papers, journal articles and conference proceedings. However, these are modest contributions to global research efforts and evidence building, considering the size of the UK government’s ODA investment in the CSSF and its predecessor funds. This is a missed opportunity, considering the many evidence gaps in the research literature on what works and the wealth of practical evidence the CSSF’s large portfolio of diverse programming could contribute to fill these gaps.

By means of illustration, we found three examples of experimental programming that could potentially contribute to the public body of evidence if their effectiveness were adequately assessed:

• In the Caucasus, Action Against Hunger is implementing a project entitled ‘Building confidence through veterinary interventions in rural communities across divides’. The project employs a multi-ethnic team and regularly gets people from both sides of the divide together in neutral territories. It is likely that the project is increasing capacity to contain regional epidemics, which is a useful result in itself. In addition, cooperation among ethnic groups might have wider peacebuilding effects, and assessing this might add to the research body on the issue.
• The Colombia Serious Organised Crime programme is addressing bottlenecks that, if successfully tackled, could strengthen the performance of the Colombian criminal justice system. This, in turn, could conceivably make a meaningful contribution to the Colombian peace process by supporting the rule of law in areas newly liberated from insurgent groups. We were struck by the close integration of CSSF programming and UK diplomatic efforts in this area. The Joint Programme Hub has already committed to “look further at best practice on this point and how we can share it across the CSSF”.
• In Pakistan, it is likely that CSSF and other UK efforts have increased the proportion of women who registered to vote in elections. Whether this contributes to wider stability and security remains untested. Further analysis in this area would be a useful contribution to the body of knowledge on what works in the field of women, peace and security.

There has been learning on programme management

The CSSF carries out annual reviews for all its programmes. This is a step change in comparison with the Conflict Pool, and the CSSF staff we spoke to consider this a significant improvement. The national and regional boards are also playing an increasingly valuable challenge function. There are regional learning activities and (strictly internal) annual CSSF global learning events. Visits to and from countries are common, technical support is readily available and we saw some evidence of country staff learning from neighbouring countries and replicating concepts and tools.

The annual reviews and annual global learning events have contributed to substantial improvements in programme management, communication and reporting. Many respondents mentioned that the quality of communications from the Joint Programme Hub to the departments responsible for CSSF programming and to the in-country CSSF teams has improved. Departments with little knowledge of ODA requirements have received support and guidance on programme management and reporting principles. Results frameworks are still often inadequate, but they are better than they were at the start of the Fund’s operations. Given the low starting point of some of the participating departments, learning efforts in this area have not yet been sufficient, but the direction of travel is positive.

There has been insufficient thematic learning

The CSSF has done some work on thematic learning. The Stabilisation Unit offers a portfolio of thematic training options and has led a few thematic learning exercises, and we saw examples of sharing of learning between country or regional CSSF offices. The Stabilisation Unit has also developed a ‘What Works Series’ that includes papers on two thematic areas: (1) strategic communication in conflict and stabilisation interventions; and (2) principles and guidance to inform international policing assistance. Preceding the CSSF, the Stabilisation Unit produced a paper on ‘the UK government’s approach to stabilisation’. In addition, the CSSF’s responsible departments sometimes make use of thematic guidance produced by DFID.

However, there is no central policy guidance in the field of security and justice (the thematic area with the greatest funding). The development of guidance on transnational threats (such as serious organised crime and violent non-state actors) has only just started, with useful but early-stage documents circulated internally in the second half of 2017.

The UK aid strategy says that the CSSF “will include increased resource and expertise to tackle the drivers of violent conflict which threaten stability and development such as extremism, illegal migration and serious and organised crime”. In our review, we saw some evidence of this increased expertise, and there is a chance that this may lead to the development of stronger thematic guidance in the coming period. But security and justice is not a new field of work, and policy guidance could have been developed some time ago (see Box 11).

We recognise that there are challenges in extracting generic learning on conflict, security and stability programming, where results tend to be highly context-specific. However, we found that this difficulty is greatly compounded by the fact that the CSSF’s aid programming does not have clear lines of sight to the end results that the Fund’s activities are meant to work towards, and that it does not evaluate or otherwise adequately assess the results of its programming, or the programming of its predecessor, the Conflict Pool.

We conclude that it is insufficiently clear what thematic lessons the CSSF has learned from the UK’s many years of conflict-related programming (see Box 12) or how this learning is reflected in current policy guidance and programming.

Learning is not routinely shared with implementing partners

Implementing partners are not routinely part of the CSSF’s learning. Relevant portions of annual reviews are often not shared with implementing partners, even when recommendations are directly relevant to their work. CSSF programmes often have several implementing partners, but there are no mechanisms for them to routinely share their experiences and learn from each other. Indeed, we found that implementers did not always know which other implementers were part of the same CSSF programme. This does not matter in programmes where projects are grouped together for the sake of administrative convenience rather than programmatic coherence, but it poses a learning gap in cases where projects are intended to be complementary.

Sometimes, sensitivity and confidentiality are used as arguments for the decision not to share information and not to invite outsiders into learning events. Although this may be valid in specific circumstances, our evidence indicates that this justification is over-used. First, it is often possible to find ways to enable learning while respecting confidentiality, as the Iraq team illustrated by making an unclassified version of its conflict assessment available to peers. Second, we found that different people classified CSSF documents in different ways, with some restricting access as the default option, and others not.

Conclusions on learning

The CSSF has introduced a number of learning processes that its predecessor funds did not have, and this has strengthened elements of programme management. Because of the CSSF’s inadequate results management practices, these efforts have not yet managed to create sufficient thematic learning. Where programme-specific insights are gained, they are not routinely shared with implementing partners.

The CSSF does not make full use of the research literature to inform guidance on what works in thematic aid programming. Such guidance is insufficient in a few key areas of CSSF programming, including the themes of security and justice programming and countering terrorism and violent extremism.

The CSSF’s contribution to the body of research on how ODA can be used to promote stability and security is modest, considering the size of the Fund and the UK government’s extensive prior experience in conflict-related programming.

We have given the CSSF an amber-red score for its use of and contribution to learning in the field of conflict, peace and security. This score is based on the lack of prioritisation of the capturing, use and dissemination of evidence and knowledge beyond individual projects and country portfolios.

Conclusions & recommendations

Conclusions

Based on our review of four country and two regional portfolios, we have awarded the CSSF an overall amber-red score. While our conclusions are limited to the work that we have reviewed, our assessment raises a range of issues that are likely to be pertinent to the CSSF as a whole.

The CSSF rates relatively well on relevance to the immediate consequences of conflict and instability. It is flexible and responsive to the priorities set by the NSC. We found the CSSF to be well informed and up to date on conflict dynamics in its theatres of operations, and its programming showed an ability to keep pace with volatile contexts. Its work supports UK engagement with partner countries and international processes in key conflict settings around the world. It makes good use of the expertise available across the UK government. However, its portfolios remain quite fragmented (although less so now than in the first year of operations). It often lacks a causal logic from its reactive and tactical use of aid programming to its long-term strategic objectives of reducing conflict drivers and promoting sustainable peace in particular contexts.

We found that CSSF results management practices are inadequate. While this is a challenging area for conflict-related programming, we would have expected it to be given much higher priority, considering the size of the Fund and the long experience the UK government has in conflict-related ODA spending. As a result, there is little reliable data on whether CSSF projects are achieving their intended results or delivering value for money. The problem is not just one of demonstrating results: unless the CSSF clearly articulates what it is trying to achieve and how, and monitors progress towards its goals, it is unlikely to achieve results commensurate with the level of investment. On a few occasions, CSSF funding was used to gain access to key partners and processes, with inadequate attention paid to the direct results of the activities funded.

We saw promising work in some areas, such as strategic communications, but also programmes that were flawed in conception and poorly delivered. Some of the CSSF’s programming – particularly on capacity building – runs contrary to the available evidence. The Fund’s management practices are not conducive to optimising value for money, encouraging a short-term perspective from implementers. The CSSF does, however, have good practices on conflict sensitivity and the beginnings of a strong approach to gender sensitivity.

In relation to learning, the CSSF has introduced a number of innovations that are helping to promote learning on programme management, although progress has been slow and there is still a long way to go. It draws on the available research to inform its conflict analysis, but does not give sufficient priority to learning and producing guidance on programming. Strategic and policy guidance is missing in some key areas and is work in progress in others. The CSSF does not often share its experiences externally, which is a missed opportunity to contribute to filling global evidence gaps.

We have seen that the CSSF is aware of many of these shortcomings and is taking steps to address them. The following recommendations are intended to contribute and add urgency to those efforts.

Recommendations

Recommendation 1:

The CSSF should introduce country or regional plans specifying how its portfolios of aid programmes and influencing efforts will contribute to achieving NSC objectives, the intermediate outcomes that the portfolios will achieve, and the assumptions that need to hold for this to happen.

Problem statements

• The CSSF has good insights into the drivers of conflict, but less awareness of the programming approaches that can help to achieve sustainable peace, stability and security.
• It is beginning to develop some stronger theories of change at programme level, but they are not yet of the standard required for planning, monitoring or evaluation purposes.
• The CSSF does not clearly articulate how its various activities in each setting will work together to advance its strategic objectives.

Recommendation 2:

Where the CSSF projects are intended to support diplomatic access and influence, the influencing objectives should be explicit and progress reported so that the value for money of the investment can be confirmed.

Problem statements

• We have seen examples of CSSF projects whose real purpose appeared to be securing diplomatic access and influence, rather than the stated objectives.
• In our case studies, the CSSF did not report on influencing outcomes, making it impossible to assess whether the investment represented value for money.

Recommendation 3:

Programmes should demonstrate more clearly and carefully how they identify, manage and mitigate risks of doing harm.

Problem statements

• Human rights assessments were not always conducted prior to programme commencement (or indeed at all) and were not always up to date and of adequate quality.
• When working with counterparts that may do harm, mitigation measures were sometimes superficial.
• As human rights risk assessments are inconsistent and programme monitoring is often weak, we do not know if CSSF programming is causing harm.

Recommendation 4:

The CSSF should address gaps in its results management and in its assessment of value for money of existing programmes as soon as possible. Meanwhile, it must ensure that all new programming includes adequate results management and measures to assess value for money.

Problem statements

• The CSSF’s results management processes have been improving but are not strong enough for the volume of expenditure it manages.
• Neither the CSSF’s management nor external parties can at present judge whether the expenditure represents value for money.
• The CSSF has been slow to adopt value for money tools and practices at programme level; it needs stronger portfolio management tools to help its boards to improve value for money at country and regional portfolio level.
• The CSSF’s quality assurance of programme design and delivery is improving but is still, in the programming we have seen, inadequate.
• In cases of weak programme delivery (of which we saw many examples) the CSSF may be unaware of the failings because it does not have an adequate system for monitoring programmes in access-constrained contexts.
• Not all programming lends itself to results frameworks, but the CSSF does not encourage its implementers to use alternative planning and monitoring tools.

Recommendation 5:

The CSSF should create conditions that allow for the evaluation of a larger part of its portfolio. Independent reviews and evaluations of the Fund should be undertaken where it is possible to do so.

Problem statements

• Evaluations are important to support accountability and learning.
• The CSSF and its predecessor funds have not invested enough in evaluating programming, although there has been progress over the course of our review period.
• In many cases, the basic requirements for programme evaluation are not in place. An assessment of the extent to which CSSF portfolios can in fact be evaluated would help to identify early candidates for evaluation and priorities for further work.
• The evaluation of potential success stories is particularly important as this could add to the very limited body of knowledge on what works.

Recommendation 6:

The CSSF should synthesise the evidence on what works in its most important programme areas, both from its own experience and from the literature, and share it with the participating departments and implementing partners in the form of thematic guidance.

Problem statements

• The CSSF could have done more to generate and gain insights into what works and to develop thematic policy guidance in a few key areas of the CSSF’s work. No central policy guidance currently exists for security and justice work, and the development of guidance on transnational threats such as serious organised crime and violent non-state actors is at an early stage.
• Some CSSF programming is based on approaches that are likely to achieve sub-optimal results. We noticed this in the field of training and some of the quick infrastructure works in particular.
• Considering that the CSSF is one of the leading global aid instruments on conflict and security, the Fund’s contribution to the global body of evidence of what works is modest.
• Implementing partners are not routinely involved in its learning processes, and programme learning is not routinely shared with the CSSF’s implementers.
• Sensitivity and confidentiality may be valid arguments not to share learning, but our evidence indicates that this justification is over-used.

Annexes

Annex 1 Detail of scoring

Question 1: Relevance

How relevant and strategic is the CSSF’s response to particular conflicts and crises?

The case study portfolios are well aligned with the UK aid strategy and relevant NSC strategies. They are underpinned by adequate, if sometimes outdated, conflict assessments, and managed by well-informed CSSF staff that often have considerable conflict expertise. They also use this expertise to influence multilateral efforts and host government agendas. The portfolios are coordinated with other UK and wider international aid programming, and remain relevant by responding to changes in context and conflict dynamics in real time. These are the green elements of our green-amber score.

The amber element of the score is the CSSF’s lack of a clear rationale for how its activities contribute to the long-term goal of promoting sustainable peace, stability and security.

Question 2: Effectiveness

How effective is the CSSF at addressing conflict, instability and insecurity and at promoting sustainable peace, stability and security?

Because of inadequate results management, there is little hard evidence as to whether programming is achieving its intended outcomes, and limited data the CSSF could use to integrate value for money into programme management. While results management is challenging for any conflict programming, we nonetheless expected to see much stronger systems, given the size of the Fund and the length of time that the UK government has been operating conflict prevention funds. This is the red component of the score.

The lack of results evidence did not allow us to judge the effectiveness of programming, but we saw some examples of strong work, particularly on strategic communication. We also saw programmes that were based on implausible assumptions, that contradicted the available evidence on what works (particularly on capacity building), or where the quality of delivery was poor. The quality of the programming is therefore inconsistent.

The CSSF has strong practices around conflict sensitivity and works to improve the conflict sensitivity of its multilateral partners. It also has a strong approach to gender sensitivity, even if this is not always sustained through implementation. Its human rights risk management is inconsistent. Together, these form the amber part of the score.

Question 3: Learning

How well is the CSSF learning what works in tackling conflict, instability and insecurity?

The CSSF has introduced a number of learning processes that its predecessor funds did not have. This has led to improvements in elements of programme management. However, thematic learning remains limited.

There is a need to strengthen the body of research on how ODA can be used to promote stability and security. The CSSF’s contribution to this body of research is modest and not commensurate with the size of the Fund and the UK government’s accumulated experience in conflict-related programming. Sharing project-specific and wider lessons with implementers does not routinely happen.

Overall score

The CSSF is a flexible and responsive instrument for supporting the implementation of NSC strategies. Its programmes are well informed on conflict dynamics, and its programming is able to adapt and stay relevant in volatile contexts. It supports UK engagement with partner countries and international processes, and helps to mobilise the expertise available across the UK government. However, the country and regional portfolios we have seen often lack a clear causal logic from programming to the end objective of sustainable peace, stability and security.

The CSSF’s results management practices are inadequate, given the scale of the funding. As a result, there is little reliable data on whether its projects are achieving their intended results or delivering value for money, and whether they might be doing harm. We have seen examples where CSSF funding is used to gain access to partners and processes, rather than focused on their stated objectives. While we saw good quality programming in some areas, most of the programmes we reviewed showed signs of basic design errors, and some of the Fund’s work runs contrary to the available evidence of what works. Delivery is often poor, suggesting a lack of quality control in difficult security environments. There is an over-reliance on training, including in contexts where it is unlikely to be effective. Some of the contract management processes are not conducive to optimising value for money. The CSSF does, however, have some good practices on gender and conflict sensitivity.

The CSSF has introduced some useful learning processes. These have resulted in improvements in programme management, from a low base, but not in significantly stronger thematic programming. Efforts to develop strategic and policy guidance on programming in some of its key areas of work are at an early stage. It does not give enough attention to capturing learning from its own experience, and its contribution to addressing global evidence gaps is too modest for a sizeable fund that builds on experience gained through the Conflict Pool and its predecessor inter-departmental conflict prevention funds.

Annex 2 List of sampled programmes

For reasons of data sensitivity, two programmes and three projects are missing from this list.