Review of UK Development Assistance for Security and Justice a

Executive Summary

Security and justice (S&J) assistance, including support for policing, courts and community justice, is an increasingly important part of the UK aid portfolio. In 2013-14, it accounted for £95 million in expenditure, across DFID and the Conflict Pool. In this strategic review of the UK S&J assistance portfolio, we examined S&J programmes in 10 countries, including through visits to Malawi and Bangladesh. We looked mainly at DFID’s assistance, together with the question of coherence and coordination across the UK Government. We paid particular attention to whether the portfolio is addressing the needs of women and girls.

1 Introduction

Scope and purpose

1.1 Development assistance for security and justice (S&J) – namely, support for policing, judicial systems, community justice and related initiatives – is an increasingly important part of the UK aid programme. In recent years, the UK has funded substantial S&J programmes in 16 countries and smaller activities in a number of others. Commitments have increased in number and size and, with more of the UK’s bilateral aid devoted to fragile and conflict-affected states, this trend is likely to continue.

1.2 We decided, therefore, to conduct a thematic review of the UK S&J assistance portfolio. Our review covers policies and strategies, patterns in programme design, delivery arrangements and the generation of knowledge to inform programming. While we cannot quantify results across the portfolio as a whole, we assess impact through a series of case studies to assess which types of S&J assistance are delivering on their intended objectives.

1.3 While our focus is primarily on programmes funded by the Department for International Development (DFID), we have also looked at some Conflict Pool projects and at how various UK Government departments and agencies collaborate on the delivery of S&J assistance. Our scope is limited to activities that qualify as official development assistance (ODA). This excludes some aspects of UK S&J assistance, such as support for counter-terrorism and to the military.

1.4 As a cross-cutting theme for the review, we have chosen to look at how well the portfolio delivers results for women and girls. DFID has made an overall commitment to providing 10 million women and girls with improved access to S&J services.² We assess whether DFID’s programming reliably identifies the S&J needs of women and girls in particular contexts and whether it is able to overcome the challenges they face in accessing quality S&J services. We have not, however, limited our enquiry to programming that explicitly targets women and girls. Rather, we take the perspective of women and girls in assessing whether the portfolio as a whole is delivering meaningful changes to S&J services and outcomes. We have looked at DFID programming on violence against women and girls only in the S&J arena, not in other areas such as civil society support.

Methodology

1.5 Our methodology consisted of six main elements.

1. We commissioned a literature review on the challenges of delivering improved S&J for women and girls. It focussed on identifying the S&J needs of women and girls, on the entry points for S&J programming and on common obstacles to delivering improved S&J outcomes for women and girls. The literature review was carried out by staff of the Overseas Development Institute. This review also drew extensively on other literature, including empirical studies of what works in S&J programming (see Figure 1 on page 2 for some key issues emerging from the literature).
2. We carried out a strategic assessment of DFID’s overall approach to S&J assistance. This included:
   • reviewing relevant policies, strategies and guidance;
   • dialogue with the S&J Team in DFID’s Conflict, Humanitarian and Security Department (CHASE), which provided a series of briefing notes on different aspects of S&J assistance;
   • interviews with other DFID teams and the cross-departmental Stabilisation Unit;
   • interviews with UK development non-governmental organisations (NGOs) with DFID Programme Partnership Agreements that are active in the S&J area and a consultation meeting with other NGOs organised through the network organisation Bond;
   • consultations with UK-based academic experts;
   • consultations with companies and consultants involved in the design, delivery or review of UK S&J programmes;
   • consultation with UN NGOs active on security, justice and violence against women; and
   • a review of how DFID measures results across the portfolio.
3. We carried out desk reviews of a sample of eight current or recently completed DFID and Conflict Pool S&J programmes. The sample covered programmes in the Democratic Republic of Congo (DRC), Ethiopia, Libya, Nepal, Nigeria, Sierra Leone, Sri Lanka and Sudan. The sample was not random; rather, it was chosen to cover a cross-section of programming in terms of scope, duration, region, country context (post-conflict, fragile and other), implementing partner (companies, NGOs and multilateral agencies) and funding source (DFID and the Conflict Pool). For each programme, we reviewed programme design documents and related analytical work, results frameworks, annual reviews and any external assessments. Where possible, we also looked at prior programmes in that country. We conducted telephone interviews with the responsible DFID staff, implementing partners and, where possible, consultants or Stabilisation Unit advisers involved in design work or reviews. We identified recurring patterns in programme activities and in reported results.
4. We reviewed DFID’s approach to innovation and knowledge management. This included desk reviews of a range of programmes and initiatives managed by the CHASE S&J Team or DFID’s Research and Evidence Division, including on technical support, innovation, strategic partnerships and research. We assessed DFID’s approach to converting its knowledge pool into informed programming choices.
5. We assessed the level of co-ordination and coherence across the UK Government in the international S&J assistance field. We consulted with a number of sections in the Foreign and Commonwealth Office (FCO), the Stabilisation Unit, the Ministry of Defence, the Home Office, the Ministry of Justice, the Crown Prosecution Service and the National Crime Agency. We spoke to various stakeholders about the development of National Security Council country strategies and the design of the new Conflict, Stability and Security Fund (CSSF). We considered how UK security and other policy interests influence the approach to S&J assistance. We also looked at the experience of involving other UK Government departments and agencies in the delivery of S&J programmes.
6. Finally, we carried out detailed case studies of the full range of UK S&J programmes in Bangladesh and Malawi. Our original intention had been to visit Sierra Leone but this proved impossible due to the Ebola epidemic. We responded by making Sierra Leone a desk study and elevating Malawi from a desk study to a full case study. Bangladesh and Malawi have both received more than one generation of S&J programming, allowing us to examine the cumulative results of sustained DFID support. Along with many countries in which DFID works, they have acute problems of violence against women and girls, which is, in turn, a significant focus of DFID’s programming (see Figure 2 on page 5).

1.6 In two-week visits to each case-study country, we consulted with DFID, programme implementers and counterparts, interviewed a range of independent observers, visited project sites and consulted with intended beneficiaries.³ We reviewed the results of surveys and other monitoring tools used by the programmes. Our beneficiary consultations took the form of community meetings, focus groups and individual interviews, including taking case histories from individual users of services supported by UK programmes. The data collected was qualitative in nature. It enabled us to test the plausibility of results reported by the programmes we reviewed and to assess their relevance to the needs of beneficiary communities. Our findings on impact are drawn mainly from DFID’s own results data, as generated by the programmes, supplemented by our own observations and findings.

1.7 Altogether, through desk reviews and case studies, our review covered nine of the largest current DFID S&J programmes (see Figure 4 on page 6).

1.8 We have also drawn on the findings of other ICAI reports. In 2013, we reviewed DFID S&J programming in Nepal⁴ and in the Occupied Palestinian Territories.⁵ We examined a DFID justice programme in Nigeria as part of our second review on anti-corruption programming⁶ and a security sector and policing programming in DRC in our review of the scaling up of DFID assistance to fragile states.⁷ The findings of these reviews formed part of the desk review component of our work for this report.

1.9 The review was carried out by an international team of S&J specialists, supported by local experts in Bangladesh and Malawi.

Overview of DFID’s S&J portfolio

1.10 The UK’s major S&J assistance programmes are funded by DFID through the bilateral aid programme. In addition, the tri-departmental Conflict Pool funds a significant number of smaller S&J projects, oriented towards ensuring stability in fragile or conflict-affected countries. The FCO has a number of strategic programmes on S&J themes, including on human rights and counter-terrorism, while some of the UK Government’s domestic S&J agencies, including the Crown Prosecution Service and the National Crime Agency, have a range of international assistance activities.

1.11 DFID’s expenditure on S&J assistance is not separately identified in its management information system. As a result, there are no exact figures available on the amount that DFID spends. A reasonably accurate picture can be gained, however, by looking at two ‘input sector’ codes: ‘security sector management and reform’, which includes support for police; and ‘legal and judicial development’, which includes programmes working with the judicial system and community justice.¹³

1.12 The data shows that the portfolio has grown from around £10 million in annual expenditure in 2000-01 to £53 million today (see Figure 3 on page 6). Expenditure on legal and judicial development was stable or on a slightly declining trend from 2007-08 until 2013-14, when it increased again. Expenditure on security sector management and reform (including policing) has risen substantially over the past five years. In 2013-14, S&J accounted for 7% of DFID’s total expenditure on ‘governance and civil society’.

1.13 Figure 4 provides a list of the largest current DFID S&J programmes by lifetime budget (although DRC and Ethiopia are currently suspended and Libya has been heavily curtailed). In recent years, there has been a trend towards more and higher value programmes, reflecting the increasing prominence of fragile and conflict-affected states in the UK bilateral aid programme.

1.14 Most of DFID’s S&J portfolio is focussed on criminal justice, with a concentration of funding on policing. There is also a substantial strand of programming on community justice, which can include supporting paralegal services, informal dispute resolution and ‘legal empowerment’ (empowering communities to use the justice system to claim their rights). There is less support for the formal justice sector and for civil justice. In recent times, DFID has begun to explore wider justice themes linked to the Prime Minister’s ‘golden thread’ agenda¹⁵ (see Figure 5 on page 10), including land tenure, economic law and commercial justice. Programming in these areas, however, remains small.

1.15 DFID also engages in international advocacy on S&J. In recent years, it has lobbied in international forums, such as the United Nations, to secure the inclusion of S&J goals in the post-2015 international development agenda.

Other UK S&J assistance

1.16 According to figures provided by DFID, the Conflict Pool spent £42.4 million on S&J programming in 2013-14 (excluding the Security, Justice and Defence Programme in Libya, which also appears in DFID’s figures). We note, however, that this sum uses a broader definition of S&J and includes expenditure that is not ODA-eligible, such as training for military forces.

1.17 Most Conflict Pool projects are much smaller than DFID’s programmes; in 2013-14, there were 116 projects, with an average budget of £365,000, as compared to an average DFID S&J programme of £20 million. Most Conflict Pool projects are implemented directly by the UK Government, NGOs or individual consultants, with only a small number tendered out for commercial delivery.

1.18 In 2015, the Conflict Pool will be replaced by the new CSSF, with an increased budget. The CSSF will fund activities in support of National Security Council country strategies, with the priorities decided by inter-departmental regional and country programme boards. This is part of a UK Government initiative to improve the co-ordination of UK engagement in strategically important countries. It is an important part of the strategic backdrop to UK S&J assistance.

1.19 We reviewed the Conflict Pool in 2012 and identified a range of issues regarding programming practices and results management.¹⁶ We have not repeated that assessment in this review.

1.20 The FCO also has a range of strategic programmes that fund S&J assistance. Its Counter-Terrorism Fund has an annual budget of £15 million, with a £7 million ODA target. The UK Government has a policy against co-operating on counter-terrorism with other countries where it is likely to lead to torture or abuse of suspects.¹⁷ S&J assistance under this fund goes to a number of countries to help them to develop the capacity to investigate, detain and try counter-terrorism suspects in accordance with human rights principles.

1.21 The FCO also has a Human Rights and Democracy Programme, of £6 million annually, which provides small grants to promote international human rights standards, including advocacy against torture and the death penalty. In recent years, part of this fund has been used to support the Preventing Sexual Violence in Conflict Initiative. In April 2013, under the UK Presidency, the G8 group of nations issued a Declaration that no post-conflict amnesty may be granted to people who have ordered or carried out rape.¹⁸ The Declaration has been endorsed by 155 countries. It was followed by a Global Summit in London in June 2014, hosted by the Foreign Secretary and UN Special Envoy Angelina Jolie.¹⁹

1.22 Other UK departments and agencies are active in international S&J assistance on a small scale, as implementers of programmes funded by others. The Ministry of Justice and the Home Office have a joint project team that implements European Union (EU) ‘twinning projects’ on justice and home affairs in EU accession countries. The Crown Prosecution Service (CPS) has around 20 officers abroad, helping to strengthen the capacity of criminal justice systems in areas such as asset recovery and fighting organised crime. It helped to deliver a DFID programme in Sierra Leone on criminal justice. The National Crime Agency (NCA) has a substantial overseas network engaged in both operational matters and in capacity building of partners. Both the CPS and the NCA have played roles in the implementation of Conflict Pool and DFID programmes.

1.23 The Stabilisation Unit is also an important part of the UK institutional architecture for S&J assistance. It is a tri-departmental unit, established by DFID, the FCO and the MOD in 2004, to boost the UK Government’s capacity to respond to instability overseas. It supports the rapid deployment of UK expertise in support of stabilisation. It has a Security and Justice Group, including a core team of experts and a roster of Senior S&J Advisers, which provides technical support to the UK Government on the design and delivery of S&J interventions. The Group supports DFID with the scoping, design and monitoring of its S&J programmes in a number of countries, as well as developing lesson-learning documents, such as its 2014 Policing the Context paper.²⁰

2 Findings: Objectives

2.2 This section reviews the objectives of UK S&J assistance, the quality of programme design and the level of coherence across UK Government agencies. We begin by looking at the policy goals underlying the S&J portfolio and whether there are strategies in place for delivering them in different country contexts. We then look at recurring patterns in the design of S&J programmes.

UK S&J assistance supports multiple, overlapping policy interests

2.3 The UK Government has a wide variety of policy goals that bear upon S&J. For DFID, the primary goal is to address poverty by reducing the vulnerability of the poor to insecurity and injustice. Its 1997 White Paper was an early statement of this goal, noting that ‘poor people, particularly women, are the most vulnerable to all forms of violence and abuse… because in very many cases systems of justice and government services do not fully extend to them’.²¹ A number of DFID documents²² cite the World Bank’s influential ‘Voices of the Poor’ study from 1999,²³ which found that insecurity and injustice are major concerns for poor people, of equal importance to hunger, unemployment and the lack of safe drinking water.

2.4 DFID also cites S&J assistance as a means of strengthening governance in developing countries, by promoting the accountability of government officials to the poor. According to a May 2008 paper, an effective justice system ‘is a guarantor of the rule of law, an essential element of democratic politics.’²⁴ This was, in turn, incorporated into DFID’s approach to state-building in post-conflict countries. Security, law and justice were defined as ‘core state functions’ which are essential for all states if they are to ‘govern their territories and operate at the most basic level’.²⁵

2.5 S&J assistance also supports conflict reduction. The UK has been a strong champion of including security within the international development agenda, based in large part on its experiences of intervening in the Sierra Leone conflict from 1996 onwards. DFID has been very influential in establishing security sector reform as a discrete area of development assistance, covering the restoration of civilian law enforcement, democratic control over the armed forces and the disarming and demobilising of former combatants.²⁶ In 1999, DFID prepared a policy statement on Poverty and the Security Sector, drawing the link between insecurity and poverty.²⁷

2.6 More recently, the Prime Minister’s ‘golden thread’ agenda makes reference to the rule of law as underpinning peace, open societies and economies, which are necessary in order to address the underlying causes of poverty²⁸ (see Figure 5 on page 10).

2.7 The 2011 Building Stability Overseas Strategy,²⁹ issued jointly by the FCO, the MOD and DFID, made the case that S&J provision is an important part of promoting stability overseas. There are also more specific UK interests at stake. The Prevent Strategy notes the importance of effective criminal justice systems in countering radicalisation and preventing terrorism.³⁰ The Serious and Organised Crime Strategy, likewise, notes the importance of building capacity in criminal justice in developing countries, in order to ensure effective collaboration in the fight against global crime.³¹

2.8 There are, therefore, multiple, overlapping UK policy agendas in the S&J arena, including poverty reduction, good governance, state-building, conflict reduction, the golden thread and UK national interests. With so many policy objectives at play, it can be difficult at times to identify which objective UK S&J assistance is pursuing at any given point.

The DFID S&J portfolio shows signs of strategic drift

2.9 With the exception of the agenda to tackle violence against women and girls, discussed below in paragraph 2.27, there has been little development of S&J policy or strategy in recent years. The clearest statement of DFID’s current approach is found in its July 2009 White Paper, where it committed to treating S&J as ‘a basic service’, on a par with health or education.³⁴ This helped to unify the various S&J activities under a common idea and to mainstream them within an aid programme that was, at that point, oriented towards service delivery for the Millennium Development Goals.

2.10 As a strategy, however, it also has its limitations. The emphasis on service delivery steers DFID towards a widespread assumption that the solution to insecurity and injustice, as experienced by the poor, is strengthening a specific set of S&J institutions and services, such as policing, courts and local tribunals. It works against approaching S&J as a set of social issues – for example, land tenure, labour rights or urban insecurity – requiring broad, multi-pronged interventions to address. Some of the experts whom we consulted stressed that poor people’s experiences of S&J have more to do with surrounding socio-economic conditions and cultural norms, than with the quality or accessibility of S&J services. As one NGO expert put it to us, ‘You don’t necessarily get security by doing security’. We saw good evidence of this broader, multidisciplinary approach in DFID’s programming on violence against women and girls but less in other areas of the S&J portfolio.

2.11 Most of DFID’s S&J portfolio focuses on the strengthening of S&J institutions as its starting point, rather than the need to address specific problems of insecurity or injustice. In our view, the portfolio would be strengthened by more attention to problem solving. A problem-solving approach entails multiple reinforcing interventions to tackle specific S&J challenges in particular locations or for particular groups of beneficiaries. It entails finding localised solutions and developing partnerships among different authorities and community groups. We see some signs of a move towards this in recent programme designs, such as the (now suspended) Ethiopia programme.³⁵ This has not, however, been clearly articulated as a strategy.

2.12 Beyond the idea of treating S&J as a service, DFID has no explicit strategy for the portfolio and little in the way of updated policies or approaches. We encountered little consensus, within DFID or among practitioners, as to which services to prioritise or what it takes to improve them.

2.13 DFID informs us that it has chosen not to develop an overarching strategy for its S&J assistance, preferring to allow country offices to identify their own solutions to local challenges and opportunities, in order to respond to context. In the absence of clear guidance, however, we found that DFID S&J programmes tend to be fairly similar in composition, suggesting a lack of adaptation to context.

S&J programmes are based on a menu of conventional activities

2.14 DFID S&J programmes appear to be constructed from a menu of conventional components, generally without a strong theory of change. There is limited variation in the mix of activities across countries, despite very different contexts.

2.15 Figure 6 shows the recurrent elements across the ten programmes in our sample. Some of this repetition comes from successful activities that have been replicated across countries. For example, the use of paralegals to tackle excessive pre-trial detention, which has worked well in Malawi, was taken to both Sierra Leone and Bangladesh – including using a Malawian NGO to deliver the training.

2.16 Other activities, however, recur across the portfolio without due consideration of their track record or their suitability to different country contexts. For example, DFID’s approach to community policing focusses more on supporting police with community outreach activities than on changing operational tactics (e.g. through crime prevention activities or increased patrolling in high-risk areas). The approach looks similar across country contexts, despite limited evidence of success (see paragraphs 4.11-4.17 on pages 26-28). Our interviews also suggested that there is little confidence among UK S&J experts that the demonstration of new policing approaches within model police stations is an effective means of promoting reform. Yet model police stations continue to be designed into new programmes, including the most recent design for Libya. DFID invests in internal affairs and professional standards units for police across many of its programmes, without much evidence that this contributes to improved police behaviour. Justice sector planning and coordination initiatives are another recurring activity, despite little history of success.

2.17 There are also gaps and omissions in the list of conventional activities that are difficult to explain. For example, despite DFID’s strong focus on criminal justice, with programmes in nine out of the ten countries we reviewed supporting criminal investigation by police, there is relatively little engagement across the portfolio with prosecutors, the judiciary, court administrations or the legal profession. Increasing police capacity to investigate crime without improving prosecutions and court processes is unlikely to improve the performance of the criminal justice system. While there may be good reasons for the omission, they are not explicit in programme design documents. Similarly, the balance is tipped strongly towards justice in rural areas, even though rapid urbanisation in many countries is generating pressing new S&J challenges. Certain justice issues, such as security of housing tenure, play only a minor part in the portfolio. Key stakeholders confirmed to us that these patterns of programming are a result of established preferences or ‘comfort zones’ among DFID and Stabilisation Unit staff and their regular consultant advisers.

2.18 Few of the programmes that we reviewed had explicit theories of change. Our own analysis of S&J programmes suggests a number of common assumptions underlying their design:

• building the capacity of central S&J institutions leads to improvements in S&J services and increased public trust and state legitimacy;
• training of police officers leads to improved police attitudes and behaviour;
• successful innovations introduced at pilot sites will result in national authorities replicating them at the national level;
• services to female victims of crime reduce the incidence of violence against women;
• improved community-police relations improve police responsiveness and reduce crime;
• better local dispute resolution processes reduce conflict within communities; and
• community awareness-raising and rights education encourage more women to access justice, leading to reduced violence against women.

2.19 We find it difficult to reconcile these assumptions with the pattern of results reported across the portfolio. In the community justice area, there is emerging evidence to support DFID’s theories of change, although there is still much to be learned about how to deliver and scale-up results in different contexts. Some of the work with central S&J institutions, however, seems to rest on assumptions that are implausible or contradicted by the evidence. For example, while victim support services for women can clearly help women in need, there is no evidence that they reduce the overall incidence of violence against women³⁶ (see the Malawi case in Figure 7 on page 14). Empirical studies from the Organisation for Economic Co-operation and Development (OECD) countries show – and some of DFID’s central documents confirm³⁷ – that entrenched police culture is not easily changed through training, particularly when externally initiated.³⁸ If police are trained in new approaches and then returned to their former working conditions, colleagues and superiors, they quickly revert to old patterns of behaviour. There is also little evidence across the portfolio that behavioural changes introduced in model police stations are replicated elsewhere, due to political, financial and organisational constraints.

2.20 We are concerned that the assumptions behind conventional programming choices are being left untested and often unstated. A well-managed portfolio should, in our view, include an active process of testing the evidence behind different theories of change, both from DFID’s own experience and from wider empirical evidence. The resulting conclusions should be translated into strategic guidance for country programmes, helping to drive continuous improvement.

Contextual analysis and use of evidence are often poor

2.21 The older design documents that we reviewed cited no empirical evidence to back their chosen interventions. Since the introduction of DFID business cases, this has somewhat improved. There is still a tendency, however, to use evidence selectively – and sometimes incorrectly – to justify programming choices (see Figure 7 on page 14). There are no DFID standards as to what constitutes sufficient evidence and the quality assurance of business cases does not appear to address the evidentiary basis of programming choices. Some of the DFID advisers that we spoke to confirmed that business cases are approached more as internal marketing documents than as opportunities to ensure robust design. There is also a widespread perception that DFID lacks technical depth in this area.

2.22 The quality of contextual analysis underlying S&J programmes is variable. In our view, tailoring programmes to the context requires an understanding of three different domains and their interaction:

• the national political, economic and social environment;
• the structure, history and interests of the S&J institutions; and
• the S&J needs of the intended beneficiaries and how they are currently served.

2.23 Business cases usually include analysis of one or more of these domains, sometimes to a good standard. For example, the Nepal business case contains a strong analysis of S&J institutions and of the drivers of insecurity for women and girls, while the Ethiopia programme has produced quality research on the S&J needs of intended beneficiaries. We also saw a range of analytical work commissioned following design. It is rare, however, for business cases to include an analysis of all three domains or for the analysis to be kept up to date through the life of the programme.

2.24 Contextual analysis should be followed by a process of adapting interventions to the particular challenges and opportunities identified in each country context. This calls for a good understanding not just of what has worked in other contexts but also why it has worked, allowing for an informed judgement about its transferability. It is this analytical foundation for programme designs that we find to be weak, leading to some poor programming choices (see Figure 7 on page 14).

DFID advisers feel under pressure to over-promise on results

2.25 We are concerned at a tendency to over-promise on results. Several DFID advisers and consultants told us that they felt that it was necessary to overstate the results that could be achieved in each programme cycle, in order to secure approval. According to one DFID adviser at country level, ‘we have to put in ambitious targets to look like we get good value for money… We are expected to put in linear, year-on-year milestones and targets that cannot be met.’

2.26 This is sometimes related to UK Government pressure to spend more in strategically important contexts. We were told that the design team in Libya, for example, was repeatedly asked to increase its level of ambition and expenditure, to match the UK Government’s commitment to supporting the country’s transition. Through successive iterations, the planned programme was scaled up to £62.5 million, making it the UK’s largest ever S&J programme. Its comprehensive, top-down, capacity-building approach, however, had little prospect of success in such a difficult environment. In the end, the programme was significantly scaled back and realigned in the face of deteriorating security conditions. In other reviews, we have found a similar tendency towards over-commitment in fragile contexts, based on overoptimistic assumptions of the time required to scale-up interventions.⁴⁵

2.27 In our view, the lessons from the case study countries clearly point to the conclusion that the most convincing designs are relatively modest in their objectives and focus on finding solutions to specific problems, rather than achieving across-the-board improvements in S&J institutions. The 15-year history of S&J assistance in Malawi is instructive. Over four generations of programming, the support has moved from capacity-building for specific institutions through an overambitious attempt to reform the criminal justice system as a whole, to arrive at a more focussed programme supporting the delivery of specific services for vulnerable people. The programme progressively moved away from major systemic reforms, in favour of niche interventions more tightly linked to specific problems, such as electoral violence and violence against women and children. This lesson on focus and selectivity, however, is yet to be articulated by DFID as a strategy for S&J programming or shared across the portfolio.

DFID has responded well to UK policy commitments on tackling violence against women and girls

2.28 The International Development Secretary, Rt. Hon. Justine Greening MP, has committed DFID publicly to tackling violence against women and girls around the world.⁴⁶ It is one of the pillars of DFID’s New Strategic Vision for Women and Girls, which contains a commitment to helping 10 million women across 15 countries to access justice through the courts, police and legal assistance.⁴⁷

2.29 We find that DFID has made good progress on developing strategies to support this commitment. Led by the Violence Against Women and Girls Team, which at the time of our review was based in CHASE, DFID has worked to map its existing programmes,⁴⁸ assemble evidence on what works and develop guidance notes that identify programming options and possible entry points, including in particular sectors, with an overarching theory of change.⁴⁹

2.30 One of the differences between this body of work and other DFID S&J guidance material is that it starts with a clearly defined problem – violence against women – and works back to possible solutions, giving it a more practical orientation. It also gives due attention to social norms and attitudes as barriers to change and a potential entry point for programming. In our view, DFID’s broader S&J portfolio would be strengthened by more detailed analysis of specific justice issues, such as land and housing tenure.

2.31 Across the portfolio, we saw a marked difference in the weight given in S&J programmes to the needs of women and girls since the Secretary of State’s commitments in this area. The programmes we reviewed in Bangladesh and Malawi had a strong focus on women and girls, including:

• providing medical, legal and social services for victims of violence;
• promoting women’s participation in and access to local justice mechanisms;
• recruiting female police;
• educating service providers on gender issues;
• raising women’s awareness of their legal rights; and
• working with women’s organisations to empower women.

2.32 There are important linkages between women’s vulnerability to violence and their broader legal status, which DFID is beginning to explore. During our field work, many of the women we met in poor communities described physical abuse, low status and discriminatory economic or social norms as closely connected problems. We saw evidence that approaching the problem of violence against women via economic empowerment, particularly around land rights, can be a productive strategy in some environments (see Figure 8 on page 31). In Bangladesh, DFID has begun to address other pressing justice issues for women, such as the labour rights of textile workers. It is also starting to explore the S&J challenges facing women in urban slums. We saw little attention, however, given to promoting attitude-change among men and boys, which our beneficiary consultations raised as a critical issue. There is also scope for the programmes to distinguish between different types of perpetrator (intimate partners, acquaintances and strangers) and target their interventions accordingly.

2.33 The DFID programmes we examined seem to have made relatively little effort to consult with women and girls or to include them in programme governance or monitoring arrangements. While DFID is certainly making women and girls a major focus of its programming, its approach would be stronger if it were more consultative and participatory. There is scope for DFID to be more creative in involving women and women’s organisations in the design, governance and monitoring of its S&J programmes.

The UK is yet to achieve a joined-up approach to shared international S&J challenges

2.34 Many of the stakeholders we interviewed pointed out the growing importance of international co-operation in the S&J arena on global threats such as terrorism, extremism, drug trafficking, money laundering and cyber-crime. These are ‘global public goods’, in that their impact is global and joint action across national boundaries is needed to address them. Effective international co-operation, in turn, often depends on building specific operational capacity in S&J institutions in other countries.

2.35 Global public goods in S&J are increasingly important to the UK Government. The National Security Council has signalled its desire for a more coherent UK approach in these areas. From 2015, CSSF funding will be accessible to a wider range of UK agencies to pursue these issues internationally.

2.36 DFID is obviously reluctant to allow other policy agendas to intrude on its S&J programming. We see this reluctance as legitimate; we would not like to see the poverty focus of DFID’s S&J portfolio become blurred with UK domestic interests.

2.37 Support for global public goods is, however, also a legitimate use of ODA. The NCA and the CPS make a persuasive case that problems such as organised crime and money laundering can be drivers of fragility and a brake on national development. There is likely to be an increased need in the future for focussed efforts to build capacity in developing countries to tackle specific global S&J threats.

2.38 In recent years, co-operation between DFID and other UK Government agencies in this arena has been growing. For example, DFID funds the Metropolitan Police to investigate money laundering linked to corruption in developing countries. There is also a range of co-operation at the country level. We note, however, that communication between DFID and UK S&J agencies is often poor. Other departments report that they find it difficult to engage with DFID without being treated as supplicants for funds. On the other hand, according to DFID, the interest of other departments in these issues is sporadic and influenced by the availability of budgets. The International Development Committee recently heard evidence that interdepartmental coherence is strong on some S&J issues, including on international campaigns against female genital mutilation, early child marriage and violence against women in conflict but weaker on drugs and arms trafficking, tax and anti-corruption. Overall, it concluded that the UK Government’s performance on policy coherence is ‘patchy’ and needs to improve.⁵⁰

2.39 One of the rationales for creating the CSSF is to promote greater UK Government coherence in this area. Given the current quality of communication, we believe that effective CSSF programming will need to be anchored in a wider, cross-HMG strategy for using S&J assistance to tackle global threats.

3 Findings: Delivery

Delivery

Assessment: Amber-Red

3.1 This section examines common delivery challenges associated with UK S&J assistance. It looks at DFID’s supervision of implementing partners and at how the corporate results agenda and the procurement process influence programme delivery. Finally, it assesses how well different UK Government agencies work together in the delivery of S&J assistance. We note that many of the delivery challenges explored here are not specific to the S&J portfolio. We are concerned, however, that this portfolio seems to be particularly affected by weaknesses in DFID systems for procurement and delivery oversight.

DFID generally makes sound choices of delivery partner

3.2 DFID delivers its S&J assistance through implementing partners, including United Nations (UN) agencies, NGOs, private contractors and, to a lesser extent, partner governments. All of the available delivery channels offer advantages and limitations, which need to be assessed in each instance. Generally, we found a good balance of delivery channels across the portfolio, reflecting a realistic appraisal of the delivery capacity available in each country.

3.3 Working through the UN, with its global mandate, can help to secure access in politically sensitive areas. In Bangladesh, the United Nations Development Programme (UNDP) enjoys a closer relationship with national S&J institutions than the bilateral donors. On the other hand, the UN’s need to protect its access can make it reluctant to challenge national authorities on areas like corruption and accountability. In Bangladesh, we were concerned that UNDP had become too close to its counterparts, particularly the police, to challenge them effectively. UN agencies sometimes have goals that are at variance with DFID’s and are more resistant to direction. In Malawi, the United Nations Children’s Fund (UNICEF) is funded by DFID to support services to women and children victims of crime. Given its mandate, UNICEF pays much more attention to children than to women, to the extent that one Malawi police counterpart reported feeling ‘locked out’ of DFID funding for women’s services. DFID, however, appears less able to take corrective action for a multilateral implementer.

3.4 Private contractors are generally preferred for large-scale programming. As discussed in paragraphs 3.22 to 3.28, large international companies are better equipped to manage DFID’s increasingly demanding contractual processes, particularly output-based contracting. They also have the resources to operate in insecure environments. As a result, the S&J portfolio is increasingly reliant on a small pool of large contractors.

3.5 In practice, as we found more generally in our review of DFID’s use of contractors,⁵¹ their performance varies widely. At their best, they can bring experience from around the world to bear on complex delivery challenges and harness the best of international and national expertise. The best implementation generally involves quality partnerships with local organisations that bring local knowledge and legitimacy. Few large companies, however, have standing capacity in S&J. They tend to recruit for each contract, resulting in inconsistent performance. Furthermore, we were informed that DFID terms of reference often set out precise requirements for members of delivery teams. According to implementers, these tend to favour technical expertise over management experience or knowledge of the country context. As a result, programmes often experience high turnover of personnel in the first year, as the contractors and DFID negotiate to get a suitable team in place.

3.6 We observed that national NGOs tend to have a clear advantage in the area of community justice. National NGOs in both Bangladesh and Malawi were able to offer thorough local knowledge, good working relationships with local stakeholders and an ability to mobilise communities without distorting incentives through payments and material benefits. In Bangladesh, the NGOs active in S&J were well networked with each other and willing to collaborate on delivery and joint advocacy. These advantages do not seem to be given sufficient weight in DFID’s procurement processes (see paragraph 3.27 on pages 21-22).

3.7 We were particularly impressed by an NGO grant-making fund run by the Manusher Jonno Foundation (MJF) in Bangladesh (£49.5 million over 5 years), which supports projects promoting human rights and good governance. It funds a number of partners active on justice issues, including violence against women, land rights and labour rights. MJF was established in 2002 as part of a DFID-funded CARE project, before becoming an independent organisation. It focusses on smaller and medium-sized NGOs able to operate in areas not covered by Bangladesh’s large NGOs.

3.8 We found MJF’s grant-making procedures to be very strong, comparing favourably with the contractor-managed programmes we examined in our 2013 review of DFID’s empowerment and accountability programming in Africa.⁵² The partner offers strong proposal assessment and due diligence, tailored capacity-building and robust monitoring arrangements. As a national NGO, it is cost-effective and well positioned to use its network to advance advocacy and policy dialogue on issues arising from its portfolio.

Supervision of programmes is inconsistent

3.9 Whatever the delivery channel, the level and quality of supervision by DFID are major factors in programme performance. Many stakeholders commented that DFID’s supervision of and level of engagement with its implementing partners is inconsistent. Given its importance and complexity, we are concerned that the S&J portfolio has not received the management focus that it needs. The problem may also reflect an overall lack of technical knowledge of the area.

3.10 While DFID governance advisers are required to have a broad knowledge of the S&J field, they offer different levels of practical experience. The level of oversight of programmes varies widely across different advisers and, therefore, often changes dramatically with staff turnover. We heard concerns that some DFID advisers were inclined to micromanage programmes while others were largely disengaged. Beyond mandatory contract-management processes, there appears to be no standard DFID approach to overseeing programmes, resulting in supervision that is often inadequate. The introduction of Senior Responsible Owners (SROs) under DFID’s new Smart Rules may help to address this – although there may still be a lack of continuity in SROs.⁵³

3.11 In Sudan, for example, there were five heads of office and three governance advisers during the design and implementation of a four-year programme. According to a number of stakeholders, this contributed to the poor performance of the programme and a breakdown in relations between DFID and the contractor. In DRC, Nigeria and Sierra Leone, implementers also reported disruptive changes in direction or approach following turnover in DFID staff.

3.12 We also found inconsistent engagement in building and maintaining relationships with counterparts. It is widely acknowledged that the political nature of S&J assistance calls for close attention to relationship-building and political risk management. Yet the nature of the relationships between DFID, implementers and counterparts often remains unclear. This carries the risk that relationships between the contractor and the counterparts break down or that the implementer becomes too close to the counterparts in an attempt to secure their support. In Bangladesh, we were concerned that a UNDP-run police programme, embedded in police headquarters in Dhaka, seemed to treat the Bangladesh National Police as its intended beneficiary, rather than the public. It also appeared that DFID was slow to react to clear signals that the political environment had become less conducive to reform.

3.13 Overall, we found that implementing partners were rarely challenged on their performance. Weak results data and overoptimistic reporting are routinely accepted. We are concerned that DFID staff are not encouraged to be proactive in identifying and resolving delivery challenges. According to one former DFID staff member, ‘nobody applauds you for noticing that things are going wrong’.

3.14 A consequence of weak supervision is that programmes are being run as a collection of discrete projects, rather than as an integrated whole. In both Bangladesh and Malawi, we found that different components and their implementers had limited interaction with each other and were not managing their activities as parts of a larger whole. Leveraging influence across multiple interventions is important for achieving results. We also saw few examples of interaction between S&J programmes and DFID’s programmes on public financial management reform. In Sierra Leone, DFID is working with S&J agencies to increase the quality of their budget submissions. In DRC, the programme made some progress on attracting funding for the police from regional and commune authorities, which was positive. On the whole, however, DFID does not give much attention to the budgetary side of S&J institutions, including the difficult problem of ‘right-sizing’ them according to the resources available in national budgets.

3.15 We found a mixed record on coordination between centrally managed programmes and country-level programming. For example, a project with Harvard University on S&J indicators⁵⁴ was initially not joined up with country programming in Sierra Leone or Nigeria, although this seems to have improved over time. The Security and Justice Innovation Fund is trialling innovative activities in a number of countries but is programmed without reference to the learning needs of country programmes. A new DFID protocol on co-ordination with centrally managed programmes may help to address this.⁵⁵

Implementing partners are under pressure to deliver ‘reach’ over depth

3.16 DFID set itself the corporate target of providing 10 million women and girls with improved access to security and justice services by 2015. Most of the stakeholders that we spoke to agreed that, despite measurement problems (see Figure 9 on page 32), this target had been useful in signalling the UK Government’s commitment to addressing the S&J needs of women and girls.

3.17 Overall, however, we noted a preference for programmes that attempted to cover a wide geographical area and reach a larger population with standardised and relatively superficial interventions, rather than trying to achieve greater depth in a more focussed geographical area. While this is not necessarily a product of the departmental target, implementing partners reported feeling under pressure to meet over-ambitious spending and output targets, which can compromise the delivery of long-term and sustainable results.

3.18 In Malawi, a component on primary justice was providing training to village tribunal members on human rights awareness, legal issues and record keeping. We found this to be a project with considerable potential. Its scale and pace of delivery, however, meant that only a once-off, single day of training, or less, was offered to each tribunal to tackle matters of considerable complexity. Our consultations showed that there was considerable unmet demand for more in-depth training and that recall of topics covered in the training was limited. We were also surprised to find that the content of the programme was always the same, despite working across varied cultural environments (including in matrilineal and patrilineal areas).

3.19 In Bangladesh, a five-year Community Legal Service programme started two years behind schedule, as a result of delays in procurement. Both the contractor and its NGO delivery partners expressed their concern at the pace with which they then had to scale-up their activities. This risked the sustainability of results from what looked to be a potentially strong programme.

3.20 While there was consistent feedback from implementing partners on this point, DFID told us that it was not intentionally encouraging reach over depth. This suggests a problem of miscommunication. It may also be caused by DFID’s reliance on ‘reach’ indicators to assess value for money. Reach indicators measure the numbers of people who have gained access to a particular service, usually because they live within a defined distance of the service delivery point. This can create unhelpful incentives for the implementing partners to spread services thinly across a wide geographical area, running the risk of generating superficially impressive beneficiary numbers at the expense of more meaningful impact.

3.21 Where DFID’s objectives are to achieve social change, such as empowering women or reducing gender-based violence, a greater intensity of effort may be required to generate meaningful results. These are deep-seated and complex problems that require sustained engagement at multiple levels. In such cases, aiming for wide geographic coverage may not represent the best value for money over the longer term.

DFID’s procurement system is undermining effective delivery

3.22 We encountered widespread concern among DFID staff and contractors that the current procurement system frequently undermines effective delivery. Both UK Government staff and contractors noted a number of recurring problems with procurement.

3.23 First, for reasons explored above, the initial programme designs are often overambitious in scope and timeframe, causing the procurement to proceed on the basis of unrealistic terms of reference. While those bidding for the contracts are permitted to challenge the design, in practice their incentive is to promise to deliver all the results and more, at a discounted price, in order to secure the contract.

3.24 The procurement process is often subject to delays of 12 to 24 months. This carries the risk that programme designs become dated, country (and DFID) ownership declines and continuity with any predecessor programme is lost. During inception, contractors are often required to rebuild consensus on the need for the programme. In Sierra Leone, for example, procurement delays and poor DFID management of the transition from one phase of programming to the next meant that the contractor faced open resistance from counterparts on its arrival. A great deal of time and effort was required to rebuild relationships.

3.25 Procurement panels are not able to take account of evidence of past contractor performance when making their assessments. Technical assessments are, therefore, made purely on the basis of written bids. We encountered considerable scepticism among stakeholders as to whether this allowed for proper assessment of technical competence. Recently, DFID has begun to use output-based contracting in the S&J area. This involves payment against the delivery of specified outputs, rather than according to the level of inputs. We recognise the usefulness of output-based contracting in driving value for money, in the right context. We are concerned, however, that DFID is yet to develop criteria for identifying the programmes that are suitable. S&J programmes, which need to work flexibly in the face of a complex political environment, are not obvious candidates.

3.26 In output-based contracting, the main activities and outputs are agreed during contractual negotiations. This undermines the value of an inception phase and may result in unhelpful rigidity in programme delivery. In a policing project in Malawi, for example, neither DFID nor the contractor could say for certain what the procedure would be for changing the outputs in the project logframe.

3.27 Finally, we are concerned that the tendency towards higher-value programmes with more complex procurement process advantages a limited number of large international firms, at the expense of small firms, NGOs and local partners. DFID procedures increasingly require a knowledge of sophisticated value-for-money metrics and contracting modalities. Bidding for an output-based contract to deliver S&J assistance in a volatile context requires the skills and financial capacity to estimate and absorb complex commercial risks. These skills do not necessarily match up with those required to deliver effective S&J assistance. The procurement process does not give enough emphasis to local networks and country knowledge, which we have seen are key factors for effective delivery.

3.28 These problems are not unique to S&J assistance. The S&J portfolio, does, however, seem to be particularly vulnerable to disruption from procurement-related issues.

The UK Government has recognised the importance of human rights risks but needs clearer principles on risk management

3.29 S&J assistance in many developing countries involves engaging with institutions with a poor human rights record. Indeed, reducing abuse of human rights is sometimes the rationale for UK support. Inevitably, this raises the possibility that the assistance may do harm or bring the UK Government into disrepute, calling for careful risk management.

3.30 The UK Government has developed a tool – the Overseas Security and Justice Assistance (OSJA) tool – to assist with assessing human rights and reputational risks.⁵⁶ It is mandatory for all UK agencies contemplating any form of support to S&J institutions abroad. It involves an assessment of the overall human rights situation in the country concerned; whether UK support in any way increases the risk of a human rights violation; and whether mitigating actions are available (for example, seeking assurances, training on human rights or additional monitoring and reporting). The guidance is procedural, rather than substantive, in that it sets out a decision-making process, rather than the principles to be followed. Serious risks that are open to mitigation must be approved by a head of department or overseas mission. If no mitigation is available, Ministers must be consulted. There is, however, no quality control of the underlying risk assessment.

3.31 We welcome the development of this tool, which clearly signals the UK Government’s commitment to taking human rights risk seriously. We note, however, that DFID still appears uncertain as to what types and level of risk are justifiable in which circumstances, resulting in inconsistent decision-making.

3.32 In Sudan, for example, there was a substantial problem with police brutality against street children in Khartoum. The DFID programme reportedly achieved some progress in training the police in new behaviours and strategies for engaging with street children. Nonetheless, DFID lacked confidence in the implementer’s ability to manage the human rights risks and requested that additional accountability and safeguarding mechanisms be introduced in this area. The programme ended up being terminated ahead of schedule, following violent suppression of protests in Khartoum and other cities in September 2013.

3.33 In November 2014, DFID suspended its Security Sector Accountability and Police Reform Project in DRC, following a finding by the UN that units of the Congolese National Police – although not those supported by DFID – had engaged in human rights violations, including extra-judicial killings.⁵⁷ DFID received some criticism in the media for waiting for almost a year after the incidents to respond.⁵⁸ According to DFID, it decided, in consultation with international partners, to delay a decision on suspension until the UN report was released, in order to give the DRC Government an opportunity to undertake a credible response. When no appropriate response was forthcoming, this was considered to be in violation of DFID’s Memorandum of Understanding with the DRC Government and the programme was suspended.

3.34 Both Sudan and DRC were examples of DFID trying explicitly through its programming to alleviate a known human rights problem. In such instances, DFID could be more forthright in defending its decision to engage with agencies with poor human rights records. It is more problematic when DFID helps to build capacity that might be misused, without a strong focus on safeguards and accountability. In Bangladesh, we were surprised to find that a UNDP-run DFID programme was helping to develop the intelligence functions of the Bangladesh National Police, including providing software and training to the Criminal Investigation Division on how to track mobile phones, analyse call data⁵⁹ and monitor social media. There are obviously legitimate uses for intelligence capacity in fighting crime. Indeed, through this assistance, the police claimed to have disrupted criminal gangs engaged in human trafficking and prostitution. Our concern was that this assistance might also be misused. We were informed both by DFID and UNDP that politicisation of the Bangladeshi police had increased in recent times. We saw evidence that the prison population spiked during periods of opposition activism. In a deteriorating political context, the intelligence capacity built by UK assistance could be used to monitor and suppress political opposition groups.

3.35 DFID Bangladesh completed an OSJA risk assessment and opted to proceed. It informed us that it does not work with the parts of the police that are allegedly used for political purposes. The communications tracking software was provided to carefully chosen units and subject to security protocols that limit access. We were concerned, however, that both trained personnel and software could be redeployed for other purposes. Since we raised our concern, the project has stopped all support for criminal intelligence units in the Bangladeshi police.

3.36 Clearly, support for S&J institutions can raise some genuine ethical dilemmas and the risk of doing harm is real. While the OSJA tool is welcome, this is an area that calls not just for procedural guidance but also for clearer principles on when and how to engage with S&J institutions with poor human rights records. DFID should be willing to work with high-risk partners, provided it achieves an appropriate balance between the likelihood of success and the risks of doing harm and gives due attention to developing accountability mechanisms. An explicit set of principles would enable DFID to be more robust in defence of its support where the benefits outweigh the risks.

Cross-departmental delivery mechanisms are underdeveloped

3.37 Other UK departments and agencies are eager to play a larger role in the delivery of S&J assistance. They offer a number of potential advantages over contractors. Current UK Government staff bring up-to-date knowledge of contemporary issues and approaches. They are able to form good working relationships with government counterparts in developing countries. Judges, government lawyers, police and corrections officers all seem to share a preference for receiving technical advice from peers over contractors. In sensitive political contexts, UK Government representatives may also have better political access.

3.38 In practice, however, both DFID and the Stabilisation Unit have experienced significant challenges with using staff from other departments. They are not necessarily cheaper, especially in insecure environments where the Government’s ‘platform costs’ (package of required support) may be higher than a contractor’s. Other departments often struggle to provide continuity in personnel over a sustained engagement. They also lack knowledge of developing country contexts and approaches to programming.

3.39 In short, other government departments are not geared up to deliver sustained, complex development programmes. They could, however, provide useful strategic inputs into larger programmes run by DFID or the FCO.

3.40 The first phase of the Libya programme represented an interesting experiment in cross-UK Government delivery. While the main programme was being designed, the Stabilisation Unit was engaged in an operational role, co-ordinating the deployment of a cross-departmental team of advisers to support Libyan S&J institutions. With no formal programme design but a high-level of supervision from the UK Government agencies present in-country, this delivery arrangement proved adept at responding to the very fluid Libyan environment. It also carried the advantage that the advisers could speak with the authority of the UK Government. Ultimately, however, once the engagement was scaled up, a decision was made to procure a private contractor and the cross-government delivery arrangement was brought to an end.

3.41 Though the Stabilisation Unit’s experience in Libya was not without challenges, it raises the interesting possibility that other, more flexible direct delivery arrangements might be more appropriate in fluid stabilisation contexts. There is potential for mixed delivery arrangements that use contractors for large-scale activities while drawing on strategic advisory inputs from across the UK Government. With its emphasis on joint delivery in support of shared country plans, the CSSF may provide an opportunity to develop such joint delivery platforms, under the oversight of the National Security Council.

4 Findings: Impact

Impact

Assessment: Amber-Red

4.1 This section looks at the achievement of impact for intended beneficiaries across the S&J portfolio. Using our two country case studies and reported results from across the portfolio, we look at patterns of achievement in the areas of policing, formal justice systems and community-level justice. We note that S&J reform is a long-term process and that the results of current programmes may take time to emerge. We place greater weight, therefore, on countries where there have been several generations of S&J programming, such as Bangladesh, Malawi, Nigeria and Sierra Leone. We note that many of the programmes we reviewed are achieving pockets of success at particular institutional sites or geographical locations. Our scoring, however, is based on whether programmes in their entirety have shown – or are likely to show – significant impact on security and justice outcomes for the intended beneficiaries.

Policing programmes are not achieving changes to service delivery, crime rates or community security

Capacity-building and training

4.2 The results data from across the ten programmes reveal a pattern of limited achievement from top-down capacity-building and institutional reform with the police, including:

• an improved legal framework for policing in DRC, although implementation of the new legislation has been limited;
• increased budgetary allocations for police in DRC at the provincial level and nationally in Sierra Leone;
• improved communications and transport in DRC;
• improved infrastructure, including the construction of 15 model police stations in Bangladesh;
• the recruitment of women police in Bangladesh, to reach 5% of the total, with a women’s police union to promote better working conditions. According to data provided by DFID, 7 out of 39 model police stations have women acting as Sub-Inspectors and 10 as Assistant Sub-Inspectors; and
• some limited accountability measures, including more oversight of police by provincial parliamentarians in DRC and complaints mechanisms and Citizen-Police Charters posted in public places in Sudan – although there is only limited evidence of any resulting changes in police behaviour or reductions in malfeasance.

4.3 The policing programmes have delivered training at a substantial scale in a range of subjects, including leadership, forensics, investigations, community policing and public order management. This includes building or rehabilitating training facilities, developing curricula, training trainers and using UK police training institutions to deliver specialist courses.

4.4 None of the programmes we reviewed has produced evidence that training – whether alone or in combination with other support – has resulted in overall improvements in police performance. For example, the conventional rationale for forensic training is that, if police are able to base criminal prosecutions on physical evidence, they are less reliant on confessions and, therefore, less likely to abuse suspects. We have not, however, found any empirical support for this assumption. In Bangladesh, police trained by the DFID police project collected fingerprints from 40,000 convicted prisoners and entered 24,000 of them into a fingerprint database. The police acknowledged, however, that their capacity to preserve a crime scene or to maintain a secure chain of custody over evidence from its collection to presentation in court was very limited. Furthermore, with a law of evidence dating from the 1850s, fingerprint evidence is not yet admissible in court. DFID makes the point that the introduction of modern policing practices has to start somewhere and then proceed piece by piece. We have not, however, come across any examples where enough of the pieces are in place to lead to changes in the functioning of the criminal justice system. Nor is there any evidence available of a reduction in the incidence of abuse of suspects.

4.5 We saw in Malawi, Sierra Leone and Nigeria that sustained DFID support has helped to put in place a small cadre of senior officers with an understanding of the principles and practices of modern policing. Although impossible to measure directly, it is likely that the existence of this group creates the potential for gradual change over the long term. It also has relationship benefits for the UK.

Model police stations

4.6 One of the strategies repeatedly used to promote change has been the development of ‘model police stations’. This usually involves building or renovating a station based on modern design principles, with new standard operating procedures and training for staff, with a strong focus on service orientation. The assumption is that, once the value of these innovations is demonstrated, the police senior command will be persuaded to implement them across the country.

4.7 This approach has been tried at various times in Bangladesh, Malawi, Nigeria, DRC and Sri Lanka. It has clearly proved challenging to deliver successful innovations even in the model police stations themselves. There is little evidence that reforms introduced in model police stations have been picked up and implemented more broadly, due to a combination of budget, organisational and political constraints. In Nigeria, for example, our review of anti-corruption programming found that, while police skills and practices had improved in model police stations, the cost of the model police station initiative is too high to have any prospect of being replicated nationally.⁶⁰ In DRC, an external review found that community policing pilots depended upon improved police salary and conditions, improved infrastructure, equipment and transport and a major investment in training, which the national government is unlikely to be willing or able to deliver from its own resources.⁶¹

4.8 In Bangladesh, the policing project built 15 model police stations, complete with a customer service area, barracks and toilets for women, separate holding cells for men and women, secure facilities for firearms, private interview rooms and accessibility ramps. The design principles behind the model police stations have been picked up by the Ministry of Home Affairs and are being used in the construction of 101 new police stations, at a cost of US$80 million from the national budget.

4.9 While this is an achievement, the existing data suggests that improving police facilities has not led to improvements in either police behaviour or service delivery. The most recent survey commissioned by UNDP reveals a consistent picture of increased crime and victimisation since 2010, combined with declining police performance, including:

• the proportion of citizens who have been victims of crime has doubled, to 48%;
• the proportion of victims who had reported the crime to police fell from 21.5% to 18%;
• the proportion of people interacting with police who paid a bribe rose from 7.2% to 23%; and
• confidence in police fell by three percentage points, although trust increased by four percentage points.⁶²

4.10 While the deterioration is highly likely to be due to factors beyond the control of the programme, it is telling that public perceptions of police performance are, by most measures, worse in model police stations than the national average – a result that calls for further investigation. The lack of progress was confirmed in our beneficiary consultations, with poor people, women and vulnerable groups still clearly reluctant to approach the police unaccompanied. One NGO representative described the model police stations as ‘old wine in new bottles’.

Community policing pilots

4.11 Given the difficulties of achieving systemic change in policing, DFID has focussed much of its efforts on promoting community policing. Although a contested term in the literature, ‘community policing’ usually refers both to an overarching policing philosophy and a specific operational tactic, based on community-police partnerships. In Bangladesh, DRC, Malawi, Nigeria and Sierra Leone, community-policing pilots have sought to open communication channels between police and the communities that they serve and, therefore, to improve community perceptions of the police.

4.12 In Bangladesh, the project has provided funding to Community Policing Forums in 70 Unions, including a monthly ‘open day’ at police stations. Our own observations suggested that relationships with communities were not greatly improved by these meetings. We saw no evidence of resulting changes in police practice. The project logframe measures improvements in police performance through increases in the proportion of victims of crime who report the crime to the police and the proportion of the public who feel safer due to Community Policing Forums. The latest survey shows that, while the proportion of the public who have been victims of crime has doubled since 2011, the proportion of those reporting to the police has declined by 16%. Only 11% of the survey respondents were aware of the Community Policing Forums. While most of these were positive about the Forums, the numbers are too small to indicate overall improvements in police performance.

4.13 In Malawi, community-policing structures were established across the country, with substantial DFID support, including some joint police-community patrolling. An external review found that there had been some resulting change in attitudes among the Malawi police but that changes in services had proved unsustainable following the end of external support. Coverage of community policing collapsed from 80-90% of the country in 2007 to just 20% in 2012.⁶³ According to the Malawi police, the structures can be revived when there is a pressing need. In Sierra Leone, Local Police Partnership Boards seem to have fared better in terms of sustainability but there are no reliable data on changes to the coverage or quality of policing services.

4.14 In DRC, a £60 million programme concentrated a significant share of its resources in promoting community policing (called ‘police de proximité’) in three target areas. Among other things, the programme constructed 23 major police stations and delivered cars, motorcycles, radios and mobile phones. A 2013 annual review found that there had been some positive examples of police responding to requests by the community, such as setting up a new police presence in an insecure quarter, shifting their patrol pattern in order to respond to high rates of rape in one location and working with local authorities to tidy up run-down areas. The review found, however, that this was happening ‘on an ad hoc basis, rather than yet being a more systematic change in police operations’.⁶⁴ The same annual review also found that, while 70% of police trained in community policing were initially positive about the approach, after one year this figure had fallen to 30%.⁶⁵ This raises a concern that any improvements in community-police relations may be transitory, even in the pilot sites.

4.15 The beneficiary consultations that we conducted in these pilot sites during our review of DFID’s scale-up in fragile states showed that the programme had increased the visibility of police in local communities.⁶⁶ This increased presence of police, however, had not improved security. Continued misconduct by police, including corruption and criminal activities (which beneficiaries linked to inadequate police pay), compromised the quality of service delivery. Importantly, women and girls identified the lack of electricity (a dark environment) and clean water (the need to walk long distances to wells), in addition to unequal access to education and other forms of gender inequality, as the primary causes of their vulnerability to sexual and other violence, rather than the lack of policing.

4.16 We are concerned that, in many instances, DFID is pursuing a narrow approach to community policing. Much of its community-policing work focusses directly on improving police-community relations – for example, by training specific community policing officers (Malawi) or holding police-community forums (Bangladesh). Improving community-police relations may be an important goal in its own right, particularly in post-conflict settings where public mistrust of the police is high. We are not aware of any evidence, however, showing that outreach activities can deliver sustained increases in trust and confidence in the police and thereby in the legitimacy of the state, without much broader changes in police behaviour. The literature does suggest an empirical link between how fairly police treat different groups in society – sometimes called ‘procedural justice’ – and the legitimacy of the police but this calls for deeper changes across the police.⁶⁷ Nor is there any evidence that this approach to community policing is likely to deliver improved police performance, unless accompanied by changes in police tactics.

4.17 DFID’s approach compares poorly to the more multi-dimensional approach taken by USAID in its programming in Central America, which involves working with a wider range of partners, including schools, churches and community groups.⁶⁸ Changes in police tactics (such as problem-solving or ‘hot spot’ policing, targeting specific crimes in particular locations⁶⁹) are combined with other interventions, such as youth diversion programmes or improved street lighting, to create a more multi-dimensional approach to community security. While this model would have to be adapted to the contexts in which DFID works, the comparison suggests that the weakness of DFID’s community security approach may be a one-dimensional focus on direct support for police.

Victim support services

4.18 The main results area for women and girls in the policing field has been victim support services. In Bangladesh, Malawi, Nepal, Nigeria and Sierra Leone, DFID has helped to develop dedicated police units that provide women and children victims of crime with temporary refuge, medical and legal assistance and referral to other services.

4.19 In Bangladesh, a UNDP-implemented DFID project has helped to establish eight Victim Support Units (VSUs) in new or refurbished premises, staffed by specially trained female officers. The clients may approach the VSU directly or be referred from a police station. The VSUs are functioning but with a low level of demand. Their major caseload is missing children, rather than violence against women. In Chittagong, the VSU we visited had received only 27 cases in the past six months, of which 14 were missing children. The Dhaka VSU had 638 cases in 2013, of which 309 were women victims who approached the unit directly. In a city of seven million, an average caseload of just six women per week suggests a lack of proper publicity and referral mechanisms. The latest survey shows that citizen awareness of the VSUs fell from 28% in 2011 to just 11% in 2014. It may also indicate that housing the VSUs with the police acts as a deterrent to women victims. The usual model in Western countries is to integrate police into public hospitals and medical clinics, to make them more accessible.

4.20 In Malawi, VSUs developed over several generations of programming were offering a range of services, including family, marriage and survivor counselling, mediation and some limited economic and practical support. In the absence of other social services at that level, they were filling an important gap, including helping destitute women abandoned by their husbands. The number of cases of sexual and gender-based violence brought to them seemed low, in the light of its prevalence. Some stakeholders suggested to us that the relatively low demand was linked to a decline in the surrounding community-policing structures, which had acted as a referral network.

4.21 DFID is also funding UNICEF to develop four One-Stop Centres based in public hospitals in Malawi. These offer medical, psycho-social, social welfare and legal services, although at present it is proving difficult to persuade other service providers, particularly social welfare, to participate. While the most developed of these Centres was functioning well, they are currently receiving an average of just 1.5 cases per day, most of whom are children. At the time of our visit, there had been no outreach campaigns or effort to develop referral mechanisms for the One-Stop Centres.

4.22 These victim support services offer an important service to survivors of violence. The relatively low level of demand in both countries, however, is striking. We were surprised that DFID was not doing more to understand the reasons for the low uptake of the services. For the time being, therefore, we conclude that these interventions offer relatively poor value for money but this could be improved with more attention to issues of location, staffing, publicity and referral.

Preventing harm in fragile contexts

4.23 We have not looked in detail in the course of this review at major post-conflict interventions. We are aware, however, that UK support for the security sector in fragile states can be critical to achieving stability. The UK’s large and highly visible intervention in Sierra Leone (where a former British police officer was appointed Inspector-General of Police) helped to stabilise and rebuild the security sector in that country. That experience, however, may be of limited relevance to the major conflict situations of today, such as Libya and Syria.

4.24 In South Sudan, according to informed observers, UK policing assistance, particularly its efforts to build a cohesive leadership team, helped prevent the police force from fragmenting along ethnic lines during the latest round of the conflict, unlike the military and other public institutions. We were not able to verify this.

4.25 In Malawi, DFID supported training on public-order management in advance of the 2014 elections. In July 2011, the Malawi Police Service had responded with excessive force during public demonstrations, leading to the loss of 20 lives. Faced with a close-fought election, DFID supported training of the police in planning for major events and in negotiation skills. Preventative outcomes are difficult to verify but it is likely that the training helped avoid major violence in what DFID correctly predicted would be a tense election.

4.26 These preventative outcomes can be very important in fragile contexts. With DFID committed to spending a high proportion of its bilateral aid in fragile states,⁷⁰ a relationship with the security sector which enables DFID to detect and respond to potential flashpoints or sources of violence, as it did in Malawi, would seem to be good risk management for the aid programme as a whole. Arguably, the major benefit of the UK’s capacity-building support for the security sector is to create the relationships that enable it to undertake preventative interventions when the occasion requires.

DFID assistance is working around, rather than fixing, dysfunctional criminal justice systems

4.27 As the UK S&J portfolio gives relatively little emphasis on reforming judicial systems, we would not expect to see major results. We note, however, that DFID programmes often support strategic planning, inter-agency co-ordination and legislative development (see Figure 6 on page 11). This is clearly a very difficult area in which to make progress. Across the countries that we have examined, there is little interest from politicians or governments in promoting an independent judiciary and apparently little interest within justice sector agencies in improving their own efficiency.

4.28 In the absence of systemic change, DFID programmes support NGO interventions to speed up the delivery of justice and to reduce the risk of human rights violations – a useful but limited result.

4.29 In Malawi, DFID pioneered the use of paralegal services (non-lawyers with basic training in legal procedure) to reduce excessive pre-trial detention and alleviate prison overcrowding. The paralegals distribute information, advise detainees on how to apply for bail and offer practical support, such as tracking down witnesses and sureties. In Malawi and many other developing countries, individuals often spend longer on remand than the maximum sentence allowed for the offence for which they were charged. Giving them access to bail is of great benefit to the individuals concerned and their families, while also helping to reduce overcrowding in prisons. (In Malawi, there are 14,500 inmates in a prison system built for 5,000. The resulting overcrowding is so severe that the prisoners do not have the space to sleep lying down. It leads to serious health and psychological problems for inmates.) The Malawi model has proved to be a cost-effective way of addressing prison overcrowding and has been exported to Bangladesh and Sierra Leone.

4.30 These paralegal services are, however, mainly addressing the symptoms, rather than the causes, of systemic problems. In both Bangladesh and Malawi, the programmes have established inter-agency committees that bring together the main criminal justice agencies to address blockages in the system. The paralegals use these committees to good effect to advance individual cases – for example, by persuading the agencies to share information, co-ordinate activities and resolve practical problems, such as securing transport for court officials and witnesses. We were not, however, able to identify examples of these piecemeal solutions leading to changes in the justice system itself. In Bangladesh, the only example offered to us was that investigating police officers had started to write their names and mobile phone numbers on case files, to make it easier to trace them when the case came to trial. Our visits, however, showed that this does not always happen in practice. As we note below in paragraph 4.40 on page 32, this raises difficult questions as to whether paralegal support will need to be provided indefinitely.

Some promising early results on community-level justice, particularly for women and girls

4.31 The most promising results across the S&J portfolio are, without question, at the level of local community justice. This is usually an area with fewer political interests at play, where DFID has more space to work. It is also an area where local NGOs have a comparative advantage, offering an effective delivery channel. In both Bangladesh and Malawi, we observed that DFID programmes were helping to improve dispute-resolution mechanisms, drawing on local and customary forms. This was accompanied by awareness raising and legal empowerment among local communities, with a good focus on women and girls (see Figure 8 on page 31).

4.32 In Malawi, the programme was strengthening traditional courts at the village level. It encouraged chiefs to hear cases in a panel with other village elders, including women. This had reportedly reduced the payment of bribes and encouraged more women to bring their complaints to the tribunal. The programme introduced casebooks for village courts to record their decisions. We were informed that this increased formality in the proceedings had helped to reduce arbitrary decision-making. Similarly, in Bangladesh, NGO grantees were working to revitalise and improve the capacity of local arbitration systems, coupled with awareness-raising for women and girls, particularly on marriage and family law. Similar activities in Sierra Leone are also showing promising results, particularly in respect of the registration of customary marriages.

4.33 Women have been important beneficiaries of this assistance. In Malawi, the local justice work is beginning to show signs of reducing ‘land grabbing’ – that is, the practice of dispossessing women of their property after divorce or widowhood. According to data generated by the programme, across more than 2,000 tribunals trained, 85% of women involved in land claims are winning their cases against attempts to dispossess them, against a baseline of 60%. Our consultations with women claimants confirmed that the situation had improved substantially as regards property claims, although the experiences of women seeking redress from violence was mixed. In Bangladesh, the programme was providing women with a means of redress against abuse, in particular dowry-related violence. In Sierra Leone, according to surveys conducted by the programme, DFID’s work with community mediators and paralegals has helped to increase women’s awareness of S&J services available within their communities. It is not yet known to what extent this is translating into greater uptake of the services.

4.34 These are cautious results in the face of deep-seated problems. In many instances, mediating cases of physical or economic abuse of women merely serves to restore calm, leaving the woman vulnerable to future violence and abuse. At their best, however, these programmes appear to be capable of enlisting local leaders in a process of reflecting on and adapting local cultural norms and practices, so as to be less harmful to women.

4.35 The nature of the results does raise questions, however, as to whether DFID should continue to spread its resources widely – so as to reach as many women as possible – or concentrate resources into more complex and sustained interventions. The results are highly context-specific and need to be delivered by sensitive, locally legitimate actors able to work with local leaders over a longer period. In the Malawi case, the length and intensity of engagement seemed low. While we believe that there is potential for DFID to programme interventions such as these at scale, we believe that more thought needs to be given to how to scale them without losing their transformative potential.

DFID’s reporting on aggregate results does not measure meaningful change for women and girls

4.36 According to its 2014 Annual Report,⁷¹ DFID has already achieved its 2015 global target of delivering access to improved S&J services to 10 million women and girls. As we noted above, we are satisfied that this target has helped to increase the focus on women and girls in DFID’s S&J programming.

4.37 The numbers reported, however, are not a very accurate reflection of actual impact, due to methodological problems with the indicator used. DFID seeks to estimate the numbers of women and girls who are potential users of improved S&J services, based on a series of questionable assumptions (see Figure 9 on page 32).

4.38 We acknowledge the importance of reaching as many beneficiaries as possible and of telling a clear story about results to the UK public. We are concerned, however, that the drive for impressive-sounding numbers may be contributing to the tendency we observed to focus on breadth over depth in programming. We encourage DFID to focus its reporting on the achievement of meaningful change for women and girls. DFID needs to understand and address the barriers to uptake of services by women, before it can meaningfully scale up.

The S&J portfolio faces significant barriers to sustainability

4.39 Sustainability is a profound challenge across the S&J portfolio. It is clear that, in very poor countries like Malawi and Sierra Leone, there is a high level of substitution for national expenditure. In Malawi, aid dependence is so acute that it risks undermining any results achieved with formal institutions. For example, previous S&J programmes had provided resources to the police and judiciary to clear a major backlog in homicide cases. Since then, the Government of Malawi has failed to provide enough resources to manage the regular caseload, with the result that a backlog soon began to build up again. Officials within government confirmed to us that this is, in large measure, a result of the expectations created by donor assistance. DFID attempted to address the question of financial sustainability through conditions on its general budget support programme (now discontinued) but without success.

4.40 According to the Malawi Police Service, a large majority of the aid-financed reforms undertaken over the past 15 years, including community policing, human resource management processes and fingerprint and criminal record databases, either collapsed or were substantially curtailed as soon as external funding was withdrawn.⁷⁴ Though the lack of sustainability is well known, DFID has no explicit strategy for addressing it. We saw no examples where proposals for new S&J services were shaped by analysis of the government’s willingness and capacity to pay for them. In its paralegal programming, DFID does not always distinguish between capacity development and the direct funding of service delivery. While paralegal services are a pragmatic and cost-effective solution to a pressing problem with pre-trial detention, DFID has not begun to discuss whether it should fund them indefinitely.

4.41 Across the portfolio, there is an apparent reluctance on DFID’s part to address explicitly the question of what levels and types of S&J service provision are achievable and sustainable in different political, geographical and socio-economic environments. This makes it difficult for DFID programmes to work towards S&J services that might be sustainable over the longer term.

4.42 It is also clear that, in low-income contexts like Malawi, DFID needs to minimise the use of external subsidies to motivate the participants in its programmes. Instead, it should seek to build its interventions around their existing interests and motivations.

4.43 The sustainability challenge is not just a financial one. It is also a question of whether there is a credible vision for reform of S&J institutions within the country in question, which DFID can support. We are concerned at a tendency for DFID to misjudge or make unrealistic assumptions about the location and depth of support for reform. Major institutional change is unlikely to be viable without support, both from senior political leaders and from within the country’s security sector, where the real locus of power may be difficult to ascertain. In the interim, it may be legitimate for DFID to invest smaller amounts in activities designed to explore, test or build potential political commitment to reform. Without that commitment, however, institutional capacity building is likely to produce only pockets of success, rather than transformational change. In DRC, for example, DFID’s own evaluations make it clear that there is limited commitment to police development from the country’s senior leaders.⁷⁵ As a result, there is little prospect that the programme’s achievements in its three pilot sites will be replicated at the national level. Too often, DFID seems to move straight into supporting ambitious reform programmes, despite an apparent lack of interest either from the political leadership or the S&J institutions themselves. In such cases, a more community-focussed approach, working with a wider range of partners, is more likely to produce meaningful results for the public.

5 Findings: Learning

5.1 In this section, we look at how well DFID performs at knowledge management, monitoring and evaluation and learning in the S&J field, both at the portfolio level and in individual programmes.

DFID has a range of central initiatives to promote innovation and learning

5.2 DFID has a good range of initiatives to support innovation and learning run by the S&J Team in CHASE, including:

• the Conflict, Crime and Violence Results Initiative (CCVRI, 2011-2014) was a consortium of six NGOs specialising in S&J who provided technical support to DFID on the challenges of measuring results, including collecting data and choosing indicators. They produced a range of guidance products and offered a helpdesk service to country offices needing specific advice. This service was used on 78 occasions, often to support the preparation of business cases;
• there is a second helpdesk facility on Violence Against Women and Girls;
• DFID contributes funding to a multi-donor International Security Sector Advisory Team in Geneva;
• in partnership with the Open Society Foundation, DFID supports the Global Legal Empowerment Initiative, which promotes evidence gathering and innovation in legal empowerment, through a £2 million grant to Namati, a global network organisation founded to promote grassroots legal advocacy;
• DFID provided £1.6 million in funding to a project by the Harvard Kennedy School to promote a country-led approach to measuring progress on S&J;
• the Security and Justice Innovation Fund is a £4.4 million challenge fund which receives applications for funding for innovative S&J initiatives. At the time of our review, it had two projects underway and another four in the pipeline;
• DFID has a strategic partnership with the Open Society Foundation to share knowledge and experience on S&J, as well as engage in joint advocacy; and
• CHASE manages Programme Partnership Arrangements⁷⁶ with five NGOs active on S&J. These provide core funding to the NGOs, to help to fund innovation and capacity development.

5.3 The S&J Team also manages a Community of Practice linking up DFID advisers interested in S&J. There is an email network used to disseminate information and guidance, which also allows for practical queries among country offices. There is an internal web page that gives access to resources and guidance material. The S&J Team provides advisory support to country teams when requested, including advising on business cases and participating in annual reviews. DFID advisers also allocate 10% of their time to supporting other country offices. What is missing from this community of practice, however, is a mechanism for linking up DFID advisers managing S&J programmes for the first time with more experienced advisers, as a source of advice and mentorship.

5.4 Many of the elements of an active knowledge-management strategy are, therefore, in place. In practice, however, the S&J Team has been consistently understaffed and overstretched. Much of its time is devoted to DFID’s central policy functions and representing DFID in international processes. Amongst other things, it has been playing a leading role in DFID’s campaign to secure the inclusion of S&J issues in the post-2015 international development agenda. In light of these commitments, it has struggled with the processes involved in managing central programmes and initiatives. The Security and Justice Innovation Fund, for example, has been delayed because of the difficulties of managing the due diligence reviews and other contractual processes from within the policy team.

5.5 As a result, there is scope to manage this portfolio of activities more proactively, with a view to maximising learning across the portfolio. Given that S&J is an area where the evidence base is known to be weak (see Figure 1 on page 2), we would have expected to see a process of identifying common knowledge gaps and untested assumptions across the country programmes and using central research and innovation funds to fill them. The S&J Team, however, has only enough capacity to respond to specific requests for assistance from country offices. As a result, it cannot steer learning across country programmes.

5.6 Some of the NGO partners that DFID funds through Programme Partnership Agreements (PPAs) are doing interesting and challenging work in the S&J area. Their interaction with CHASE, however, is primarily at the contract management level. While there are PPA learning groups, through which DFID and NGOs share experience,⁷⁷ research and innovation across PPA partners and DFID country programmes are not being linked up, so as to encourage collaboration and joint learning. One of the partners described this to us as ‘a massive missed opportunity’. We note, however, that the PPA holders were in agreement that the PPA had helped to promote learning on organisational issues, particularly results management.

5.7 In 2014, CHASE commissioned an ‘evaluability study’ to determine whether the S&J portfolio was in a shape that would allow for a macro-evaluation (that is, an evaluation of existing evaluations) of the portfolio as a whole. The study found that the available evaluation work was not robust enough to warrant a macro-evaluation. Instead, it recommended that CHASE invest in building an evidence base on what works, while helping country programmes to articulate their theories of change. The S&J Team is now considering how best to respond.

Programmes are not designed around active learning

5.8 We found that S&J programmes are not designed around an active learning process. In principle, DFID programmes should distinguish between interventions that are based on sound evidence and those that are experimental in nature.⁷⁸ While the former can be implemented immediately at scale, the latter should be tested through exploratory programming, supported by suitable monitoring and evaluation systems for capturing the results.

5.9 We found that S&J programmes are not distinguishing between established and experimental activities. There is very little use of piloting as a learning exercise within the portfolio. Sometimes activities that are implemented on a limited scale are described as pilots but are not part of a structured process of testing options and capturing lessons, for replication on a wider scale.

5.10 The community justice work, in particular, would lend itself to a more structured approach to learning. Programmes could try out a different mix of activities in different geographical areas in order to measure how the outcomes vary. A more deliberate approach to identifying what works would help DFID to identify the best package of interventions for delivery at scale.

5.11 We would also have expected to see DFID investing more effort in learning about how to deliver community S&J initiatives in urban environments. There was evidence in both Malawi and Bangladesh that interventions that had proved promising in rural environments, in relatively stable and homogenous communities, might be difficult to replicate in more fluid and diverse urban settings. Given the pace of urbanisation in many DFID priority countries, this is likely to be a key programming challenge for the future. We are informed that a forthcoming evaluation of the Malawi Justice for Vulnerable Groups programme will consider this issue.

5.12 We were concerned to find a number of cases where evidence suggesting that particular activities were not working was not being actively investigated. Examples include survey data in Bangladesh suggesting that model police stations were performing worse in important areas than the national average (see paragraphs 4.9-4.10 on page 26) or evidence on limited uptake of services at One-Stop Centres in Bangladesh and Malawi. We have noted our concern in other reviews about DFID’s failures to pick up and act on evidence that suggests that programmes are not working.⁷⁹

5.13 Similarly, in the primary justice programme in Malawi, our consultations suggested that the programme was causing some complex changes, such as a trend towards greater formalisation of village tribunals (see paragraph 4.32 on page 30), with unknown implications for the communities concerned. DFID was not encouraging its partners to investigate these trends to identify whether they were positive or harmful and whether they called for any adaptation in programming.

5.14 We also note that sharing of learning within DFID’s country portfolios seems to be lacking. DFID runs its multi-component programmes as separate activities, rather than as parts of a larger whole. Implementing partners seemed to have little knowledge of each other’s activities and little opportunity to share knowledge. We also noted that there seems to be little exchange of learning between country programmes and central initiatives with activities in-country or with related initiatives such as anti-corruption programming.

5.15 We did, however, see some useful examples of learning within and across programmes which could be built on and replicated. For example, the paralegal work in Malawi has steadily improved over time. It has developed an effective training programme that has been replicated in other countries. There has also been good learning within the Nigeria programme around training programmes, legislative advocacy and fostering NGO champions.

5.16 Overall, we see some evidence of a shift in the DFID portfolio from large, over-ambitious institutional development programmes towards more targeted interventions with better prospects of success. This seems to have come about, however, more through the progressive abandonment of unsuccessful activities, than through conscious learning and adaptation. Consequently, this shift in strategy has not been articulated by DFID and is not fully shared across country offices. The recent move towards larger S&J programmes in priority countries may also be creating pressures to default back to large-scale institutional development.

Monitoring and evaluation practices are often inadequate

5.17 We found that monitoring and evaluation practices at programme level were often technically inadequate. Despite the availability of technical support from the Conflict, Crime and Violence Results Initiative, we found numerous instances of inappropriate indicators or unreliable data-collection methods. For example, in Sierra Leone, an initiative to strengthen local courts was measured through an indicator on the number of local court cases overturned on review. The annual review found that the indicator was misleading and irrelevant, in that it might capture a stronger review process rather than improvement in the quality of local court judgments.⁸⁰ In Bangladesh, the paralegal programme was measuring the numbers of prisoners on remand but not their average of time on remand or the numbers held on remand beyond the legal time limits.

5.18 S&J programmes rely extensively on surveys to demonstrate impact. Most programmes we reviewed had carried out some kind of survey. We found many of the surveys to be poorly designed and difficult to relate to the programmes. Surveys were also generally poorly co-ordinated between DFID programmes and with other donors, making comparison between them unreliable. In countries where DFID makes a substantial commitment to S&J assistance, we would have expected to see DFID working with the partner government (if possible) and other donors to put in place a cycle of surveys capable of serving various needs, including both national policy-making and the monitoring of external assistance.

5.19 Data from surveys are often used selectively or misinterpreted, to create misleading impressions of the results. For example, a UNDP-commissioned survey in Bangladesh found that the proportion of people who reported paying bribes when interacting with the police increased from 7% to 23%. This was described as a 16% increase in bribe-paying, when in fact it represents a tripling.⁸¹ While this might be simple wording error, we found other instances where implementing partners had clearly downplayed negative findings or selected one or two positive-sounding results from a complex survey, creating a misleading impression of overall progress. It appears that implementing partners are not being challenged on their use of survey data. Information from surveys in this area rarely speaks for itself; rather, it flags up issues that require further investigation. DFID should be encouraging its staff and partners to interrogate the data and investigate further where necessary.

5.20 We found that annual reviews are of varied quality. Some of those we examined were serious efforts to probe the direction of a programme and examine its assumptions. Usually, these involved external reviewers. Others were surprisingly superficial, content to repeat the output-level results data offered by the implementing partner without questioning whether the programme is on track to achieve its outcomes and impacts. Our review on How DFID Learns noted the lack of any quality assurance process over annual reviews.⁸² We were also disturbed to hear that some DFID staff feel that they are not rewarded for pointing out when programmes are going wrong.

5.21 Programmes suffer from a lack of continuity in DFID advisory staff. Many of the long-running programmes have a succession of DFID advisers, resulting in repeated loss of institutional knowledge and, sometimes, ownership of the programme. There is also substantial loss of knowledge wherever there is a change of implementer from one phase of a programme to the next. Implementers usually have neither the incentive nor the opportunity to pass on existing knowledge. This is a process that is missing from DFID’s contract management.

5.22 There has been very little use of independent evaluation of S&J programmes in recent years. According to DFID, there are five external evaluations planned between 2014 and 2017. Some of the more recent programmes, including Ethiopia (now suspended) and Nepal, have built independent evaluation into the design of the programme, which is a welcome development. We note that, in some cases, the evaluation function is let as a separate contract to programme implementation. While this might strengthen the accountability function, it might come at the cost of learning if the evaluators and the implementers are too much at arm’s length. We would encourage DFID to explore the relative merits of integrated and external evaluation.

5.23 Overall, we found the approach to monitoring and evaluation across the portfolio to be focussed more on demonstrating results than on learning from experience in order to adjust and refine programming. We saw too many examples where poor quality monitoring data or reporting from implementers had been passed unchallenged by DFID. This may be the result of a lack of technical knowledge on the part of DFID staff or a lack of confidence to challenge implementers.

6 Conclusions and Recommendations

Conclusions

DFID S&J programming

6.1 The UK aid programme is increasingly focussed on countries affected by conflict and fragility. The Government’s commitment to spending 30% of UK aid in fragile states means that a high proportion of the bilateral aid programme will go to places affected by conflict and instability.

6.2 In view of this, we believe that S&J assistance is a necessary and important part of the UK aid portfolio and that its significance will continue to grow in the future. The UK cannot credibly work in situations of conflict and fragility without also working on conflict and fragility. S&J programming is a necessary part of that. Furthermore, security and justice are important development goals in their own right, although they are generally seen by the poor in much broader terms than the quality of security and justice services. In light of its importance to the wider portfolio, we believe that S&J assistance needs significantly more attention from DFID than it has received to date.

6.3 We have found that the S&J portfolio is showing promising results on addressing some of the S&J needs of women and girls, particularly through community-level initiatives. While addressing the physical and economic abuse of women and girls is a challenging and long-term agenda, we fully support DFID’s conviction that it should be integral to the UK development agenda. We believe that there are good foundations here on which DFID can continue to build.

6.4 Our review, however, has found that many of the standard activities in DFID’s S&J programmes are not making enough of a difference to the lives of the poor. Some of their objectives appear unrealistic, given the instruments available. We believe that there is a need for critical reflection, both on the overall goals of the portfolio and on what objectives are realistic in which country contexts.

6.5 We find that the portfolio is at its least convincing when it attempts to deliver generic capacity improvements to central S&J institutions, including the police and the judiciary, in the hope that this will translate into practical benefits for the poor. The institutions involved are large and cumbersome, with strong vested interests working against change. Political support for transforming their culture and ways of operating often proves short-lived or illusory. The complex nature of justice systems means that a lack of progress at a single institutional site can nullify progress at others. The gulf between formal S&J institutions and the poor, in terms of geography, cost, cultural norms and power relations, is very difficult to bridge.

6.6 As a result, in all but the most permissive environments, the case for investing in system-wide reform and capacity development is weak. The causal chain from institutional development to improved services and better S&J outcomes for the poor is too long and uncertain to be a credible foundation for programming. Furthermore, the level of insecurity and injustice faced by the poor may have little to do with the quality of policing or formal justice institutions. In that sense, treating S&J as a service sector, like health or education, may be unhelpful.

6.7 The portfolio is much more convincing when it focusses on addressing specific S&J challenges in particular locations or for particular groups of intended beneficiaries. A problem-solving approach leads naturally to a broader range of partnerships and entry points. Institutional development may be a result of problem-solving but is not its primary focus.

6.8 Some of DFID’s S&J assistance already fits this problem-solving model. Its work with paralegals, for example, is a cost-effective way of tackling some of the worst consequences of dysfunctional criminal justice systems, although it raises sustainability challenges. DFID has also helped to promote some useful niche services, such as support for women and children victims of crime.

6.9 Most of its policing assistance, however, is less effective. DFID’s work on community security, in particular, needs to be broader than the current, rather limited focus on community-police relations. We find that this is delivering few practical benefits to communities. To make a real difference to community security, DFID needs to build partnerships between communities and a wider range of institutions, including the police, in the search for practical solutions to local security challenges.

6.10 DFID has a range of promising initiatives in community justice and legal empowerment for the poor upon which it can build. They offer a potentially important means of empowering vulnerable groups and tackling cultural norms and practices that are harmful to women and girls. We are encouraged by the evidence that empowering women to assert their property and other economic rights and providing them with a forum in which to bring their complaints can make them less vulnerable to abuse.

6.11 The problem of abuse of women and girls is deeply entrenched and the results that we have observed were encouraging without yet being transformative. In many cases, mediating cases of abuse of women achieves only a short-term resolution, leaving them vulnerable to more abuse in the future. We also have some concerns about DFID’s approach to scaling up these kinds of intervention. While we believe that they can be delivered effectively at scale, care needs to be taken to ensure that the model retains enough depth and intensity to deliver sustainable results. Nonetheless, DFID’s community justice work is beginning to deliver meaningful results to beneficiaries. Had we been scoring this category of support on its own, it would have received ‘green-amber’ for objectives and impact.

6.12 One of the weaknesses of DFID’s portfolio is its relative lack of engagement with pressing S&J themes in each country context, such as labour rights, land tenure or insecurity in slum settings. Where these are addressed, it is through challenge funds to NGOs, rather than a central design feature of the programmes. This relative neglect is one of the consequences of a service-delivery approach to S&J. Addressing thematic S&J issues requires multi-layered interventions, working at the policy, legal and institutional levels, as well as directly with affected groups of citizens. It would also involve working directly with a wider range of community actors, such as faith groups, schools and youth associations. In Bangladesh, some of DFID’s NGO grantees are working in this way, on issues such as child labour and land rights for the poor. DFID Bangladesh is also developing a new programme on urban poverty that may work along these lines.

6.13 DFID has chosen not to have an overarching strategy for S&J assistance or much in the way of policies or approaches. We agree that interventions must be tailored for each country context, along all three dimensions (macro-political economy; S&J institutional configuration and community needs). The lack of a strategy, however, is resulting not in localised innovation but in programmes that appear to be assembled from a limited menu of conventional elements. As a result, they end up looking similar, even across very different country contexts.

6.14 We are concerned that DFID advisers feel a need to overstate the expected results from S&J programmes in order to secure approval for business cases. This carries a number of negative consequences for procurement and implementation.

6.15 DFID S&J programmes are inconsistent in their use of evidence on what works and in the coverage and quality of their contextual analysis. The quality assurance over business cases is not addressing this adequately. There is no guidance as to what constitutes sufficient evidence to ground programme choices. Programmes often fail to distinguish between tried and tested interventions and innovative programming, which needs careful monitoring and evaluation.

6.16 On the delivery side, high staff turnover, a lack of sector-specific knowledge and experience and an apparent lack of management attention are leading to weak supervision of implementing partners. In some cases, evidence suggesting that programmes are not achieving their intended impact or might be causing unintended negative impact is not being properly investigated and followed up. S&J is an area that needs close supervision, owing to the high levels of political risk and the importance of maintaining good relationships with stakeholders. In countries where there are several S&J projects, they tend to be managed as discrete entities, rather than as components of a joined-up portfolio.

6.17 We are concerned that setting targets in terms of the numbers of people reached with S&J services may be distorting programming choices. Implementers feel under pressure to spread their efforts thinly over a wide geographical area, with a risk of superficial results. We appreciate that DFID needs to demonstrate value for money by benefiting as many people as possible. Our observation, however, is that community justice interventions, in particular, need a certain level of intensity of engagement sustained over a sufficient period of time, in order to achieve lasting results. This calls for a more considered approach to scaling-up, based on packages of interventions that have been shown to deliver sustainable impact.

6.18 We encountered a clear consensus among DFID staff, consultants and contractors that the procurement system is not delivering good results. While the underlying causes are beyond the scope of this review, there are some pressing issues for DFID to address in the S&J portfolio, including delays in procurement and a tendency towards unrealistic designs. The process favours a small number of large firms at the expense of small and not-for-profit implementers, who may enjoy greater local knowledge and legitimacy. We also find that using output-based contracting in S&J programming risks undermining results by locking programmes into predetermined outputs and preventing them from taking a flexible and adaptive approach.

6.19 Finally, we conclude that DFID needs a more deliberate approach to learning across the portfolio. While there is a good range of central initiatives on knowledge management, they are not translating into a considered process of identifying and filling knowledge gaps in programming. Similarly, individual S&J programmes are not designed so as to test which programming options work best, in order to develop an evidence base and to identify options for scaling up.

Global threats and cross-government working

6.20 It seems likely that there will be increased demand in the future for the UK to use S&J assistance to counter global threats, such as terrorism, organised crime and money laundering. It is legitimate for the UK to use aid funds to address shared global threats, particularly those with the potential to destabilise developing countries. S&J assistance can help to build the national capacities required for effective international co-operation. This should be kept separate from DFID’s main S&J portfolio, however, to avoid compromising its pro-poor focus.

6.21 This is an area where UK specialist S&J agencies have relevant skills and experience to offer. The CSSF now provides an opportunity for these agencies to access funding for overseas assistance. Their specialist skills, however, will need to be combined with DFID’s knowledge of how to work effectively in developing countries. These agencies will also need to develop the ability to sustain engagements over an extended period.

6.22 At present, the UK Government lacks a clear strategy for using S&J assistance to counter global threats. Without such a strategy, there is a risk that the CSSF will encourage competition for resources, rather than effective joint working. There will need to be a concerted effort within the CSSF to build cross-department delivery platforms that draw on the comparative advantages of the different agencies. As the CSSF is still under development, we have not made a formal recommendation in this area but we propose to look again at the issue in the future.

Recommendations

6.23 There is a need for a significant rethink of DFID’s S&J portfolio. It should move away from investing broadly in the capacity of S&J institutions, towards addressing specific S&J challenges in particular contexts. The interventions need to be more focussed and realistic in scope. They need to utilise a wider range of entry points and partnerships, including but not limited to, S&J service providers. Where capacity-building is included, it should be tightly focussed on the achievement of specific outcomes for the intended beneficiaries. Problem-solving may also include helping to prevent S&J institutions themselves from being a source of conflict or instability, through interventions designed to reduce corruption or abusive behaviour. This may be part of the risk management for DFID country programmes in fragile contexts.

6.24 This focus on problem-solving would have important implications for UK policing assistance. Many elements of the current approach, including model police stations, community policing pilots and many of the standard training programmes, have a limited record of delivering real impact for the poor. DFID needs to rethink its approach to community security, working with a wider range of partners to develop tailored solutions to local problems.

6.25 We encourage DFID to continue to develop its approaches to addressing the S&J needs of women and girls, building on its successes to date. It should ensure that the provision of services to the survivors of violence is combined with other interventions, such as strengthening property and other economic rights, which can help to reduce women’s vulnerability over time. It should explore options for using S&J interventions to address harmful cultural practices, as it has begun to do in Malawi, including by working with men and boys.

6.26 Empirical evidence is essential to good S&J programming. DFID should do more to assemble evidence on the main causal linkages underlying its programme designs, to identify areas of certainty and knowledge gaps where more investment is needed on research and innovation.

6.27 It should set clearer guidelines and standards for the use of evidence and contextual analysis in programme design. It should specify the kinds of contextual analysis required to inform programming choices, including on macro-political conditions, S&J institutions and the S&J needs of the poor. Designers should be encouraged not just to identify empirical evidence on what works but also to consider whether the lessons are applicable, given the contextual analysis. It might be appropriate for DFID to invest research funds in exploring the transferability of empirical evidence across country contexts – including the applicability of Western criminology to developing countries.

6.28 DFID should ensure that programme designs are grounded in empirical evidence and sound contextual analysis. Furthermore, new programmes should clearly distinguish between well-established and innovative interventions. The latter should be designed around an active learning process, to build an evidence base. Programmes should be designed to capture lessons during implementation and adapt accordingly.

Annex

Figure A1: List of consultations and site visits

Abbreviations

CCVRI Conflict, Crime and Violence Results Initiative

CHASE DFID Conflict, Humanitarian and Security Department

CPS Crown Prosecution Service

CSSF Conflict, Stability and Security Fund

DFID Department for International Development

DRC Democratic Republic of the Congo

EU European Union

FCO Foreign and Commonwealth Office

HMG Her Majesty’s (UK) Government

MOD Ministry of Defence

NCA National Crime Authority

NGO Non-governmental organisation

ODA Official development assistance

OECD Organisation for Economic Co-operation and Development

OSJA Overseas Security and Justice Assistance

S&J Security and justice

VSU Victim Support Unit

UN United Nations

UNDP United Nations Development Programme

UNICEF United Nations Children’s Fund

Footnotes

1. Points extracted from a range of literature, including: N. Ball and L. van de Goor, The Challenges of Supporting Effective Security and Justice Development Programming, OECD Development Co-operation Working Papers, February 2013, http://www.oecd-ilibrary.org/docserver/download/5k49dffl6bmq.pdf?expires=1417212638&id=id&accname=guest&checksum=868C3C7AA4D58412A35E2AAAAAD54D9B; Policing the Context: Principles and Guidance to Inform International Policing Assistance, What Works Series, Stabilisation Unit, March 2014, http://sclr.stabilisationunit.gov.uk/su-publications/what-works-series/495-what-works-policing-the-context.html; S. Douglas, Gender Equality and Justice Programming: Equitable Access to Justice for Women, United Nations Development Programme, 2007, Primer in Gender and Democratic Governance No.2, http://www.coa.gov.ph/gad/downloads/undp/gendergovpr_justice.pdf; D. Desai, D. Isser and M. Woolcock, Rethinking Justice Reform in Fragile and Conflict-Affected States: The Capacity of Development Agencies and Lessons From Liberia and Afghanistan in H. Cissé, D.D. Bradlow, and B. Kingsbury (eds.), The World Bank Legal Review, World Bank, 2011, Volume 3, pages 241-261, https://openknowledge.worldbank.org/bitstream/handle/10986/2365/653710PUB0EPI2061563B09780821388631.pdf?sequence=1; and Challenges and Approaches: Strength of Evidence. Safety, Security and Justice: Topic Guide, GSDRC Applied Knowledge Services, undated, http://www.gsdrc.org/go/topic-guides/safety/-security-and-justice/challenges-and-approaches/strength-of-evidence.
2. DFID’s Results Framework: Managing and Reporting DFID Results, DFID, 2012, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/175715/DFID-external-results.pdf.
3. In Bangladesh, we met with beneficiary groups in and around Dhaka, Bogra, Chittagong and Jessore. In Malawi, we met with beneficiaries in and around Lilongwe, Mchinji, Kasungo, Blantyre, Rhumpi, Mthwalo and Mzuzu.
4. DFID’s Peace and Security Programme in Nepal, ICAI, February 2013, http://icai.independent.gov.uk/wp-content/uploads/2013/12/ICAI-report-DFIDs-Peace-and-Security-programme-in-Nepal.pdf.
5. DFID’s Support for Palestine Refugees through UNRWA, ICAI, September 2013, http://icai.independent.gov.uk/wp-content/uploads/2013/12/ICAI-UNRWA-report-FINAL-110913.pdf.
6. DFID’s Approach to Anti-Corruption and its Impact on the Poor, ICAI, October 2014, http://icai.independent.gov.uk/wp-content/uploads/2014/10/DFIDs-Approach-to-Anti-Corruption-and-its-Impact-on-the-Poor-FINAL.pdf.
7. Assessing the Impact of the Scale-Up of DFID’s Support to Fragile States, ICAI, February 2015, http://icai.independent.gov.uk/wp-content/uploads/2015/02/ICAI-Report-Assessing-the-Impact-of-the-Scale-up-of-DFID%E2%80%99s-Support-to-Fragile-States.pdf.
8. From Evidence to Policy: Addressing Gender-Based Violence against Women and Girls in Bangladesh, ICDDR,B and Population Council, January 2013, http://r4d.dfid.gov.uk/pdf/outputs/ORIE/VAWG_Bangladesh_Final_Report.pdf.
9. Justice Beyond Reach: A Short-Term Study on ‘Identification of High Risk Areas of Bangladesh’, Bangladesh Woman Lawyer’s Association, 2010, http://www.communitylegalservice.org/publication/study%20report@justice%20beyond%20reach%20%28final%29-%207.11.2010.pdf.
10. From Evidence to Policy: Addressing Gender-Based Violence against Women and Girls in Bangladesh, ICDDR,B and Population Council, January 2013, http://r4d.dfid.gov.uk/pdf/outputs/ORIE/VAWG_Bangladesh_Final_Report.pdf.
11. Presentation on DFID/UNICEF Justice for Vulnerable Groups Program (2012-2016), UNICEF, Lilongwe, Malawi, September 2014.
12. Justice for Vulnerable Groups in Malawi, Business Case and Intervention Summary, DFID, undated, page 3, http://iati.dfid.gov.uk/iati_documents/3755673.docx.
13. These figures do not capture S&J expenditure within programmes coded under other ‘input sectors’, such as ‘civilian peace-building, conflict prevention and resolution’.
14. See https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/268308/Table_17.csv/preview.
15. D. Cameron, Combating Poverty at Its Roots, The Wall Street Journal, 1 November 2012, http://online.wsj.com/news/articles/SB10001424052970204712904578090571423009066.
16. Evaluation of the Inter-Departmental Conflict Pool, ICAI, July 2012, http://icai.independent.gov.uk/wp-content/uploads/2012/07/Evaluation-of-the-Inter-Departmental-Conflict-Pool-ICAI-Report1.pdf.
17. Consolidated Guidance to Intelligence Officers and Service Personnel on the Detention and Interviewing of Detainees Overseas, and on the Passing and Receipt of Intelligence Relating to Detainees, HM Government, July 2010, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/62632/Consolidated_Guidance_November_2011.pdf.
18. See https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/274724/A_DECLARATION_OF_COMMITMENT_TO_END_SEXUAL_VIOLENCE_IN_CONFLICT.pdf.
19. See https://www.gov.uk/government/topical-events/sexual-violence-in-conflict.
20. Policing the Context: Principles and Guidance to Inform International Policing Assistance, What Works Series, Stabilisation Unit, March 2014, http://sclr.stabilisationunit.gov.uk/su-publications/what-works-series/495-what-works-policing-the-context.html.
21. Eliminating World Poverty: A Challenge for the 21st Century, DFID, November 1997, White Paper on International Development, page 11, http://www.bristol.ac.uk/poverty/downloads/keyofficialdocuments/Eliminating%20world%20poverty%20challenge.pdf.
22. For example, Justice and Poverty Reduction: Safety, Security and Access to Justice for All, DFID, 2000, page 3, http://www.gsdrc.org/docs/open/ssaj35.pdf.
23. Can Anyone Hear Us? Voices from 47 Countries, World Bank, 1999, Voices of the Poor, Volume I, http://siteresources.worldbank.org/INTPOVERTY/Resources/335642-1124115102975/1555199-1124115187705/vol1.pdf.
24. Justice and Accountability, DFID, May 2008, A DFID Practice Paper, page 1, http://webarchive.nationalarchives.gov.uk/+/http:/www.dfid.gov.uk/Documents/publications/briefing-justice-accountability.pdf.
25. Building Peaceful States and Societies, DFID, 2010, A DFID Practice Paper, page 7, http://www.gsdrc.org/docs/open/CON75.pdf.
26. DFID was a supporter to the first OECD-DAC manual on security sector reform. OECD DAC Handbook on Security Sector Reform: Supporting Security and Justice, OECD-DAC, 2007.
27. Poverty and the Security Sector, DFID, 1999, http://www.gsdrc.org/docs/open/cc111.pdf.
28. D. Cameron, Combating Poverty at its Roots, The Wall Street Journal, 1 November 2012, http://online.wsj.com/news/articles/SB10001424052970204712904578090571423009066.
29. Building Stability Overseas Strategy, DFID, FCO and MOD, 2011, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/32960/bsos-july-11.pdf.
30. Prevent Strategy, HM Government, June 2011, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/97976/prevent-strategy-review.pdf.
31. Serious and Organised Crime Strategy, HM Government, October 2013, page 41, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/248645/Serious_and_Organised_Crime_Strategy.pdf.
32. D. Cameron, Combating Poverty at its Roots, The Wall Street Journal, 1 November 2012, http://online.wsj.com/news/articles/SB10001424052970204712904578090571423009066.
33. See http://devtracker.dfid.gov.uk/projects/GB-1-203950/.
34. Eliminating World Poverty: Building our Common Future, DFID, July 2009, paragraph 4.24, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/229029/7656.pdf.
35. In the DRC programme, the community policing (police de proximité) pilots did attempt to address crime prevention and problem solving. The 2013 annual review, however, found that ‘they do not express the full implications of a preventive, problem-solving approach to policing and security, that is, proactively seeking to identify and reduce crime and insecurity by working with partners to identify manageable solutions’. Security Sector Accountability and Police Reform Programme (SSAPR) 2009-2014: 2013 Annual Review, UK Stabilisation Unit and International Security Sector Advisory Team, February 2014, page 39.
36. See, for example, N. Jubb et al., Regional Mapping Study of Women’s Police Stations in Latin America, Centro de Planificación y Estudios Sociales, Quito, 2008, http://www.wunrn.com/news/2008/12_08/12_22_08/122208_latin_files/Latin%20America-Regional%20Mapping%20Study%20of%20Women%27s%20Police%20Stations.pdf; and L. Heise, What Works to Prevent Partner Violence? An Evidence Overview, UN Women, September 2012, http://www.unwomen.org/~/media/headquarters/attachments/sections/csw/57/egm/egm-paper-lori-heisse%20pdf.pdf.
37. Policing the Context: Principles and Guidance to Inform International Policing Assistance, What Works Series, Stabilisation Unit, March 2014, page 40, http://sclr.stabilisationunit.gov.uk/su-publications/what-works-series/495-what-works-policing-the-context.html.
38. See, for example, R. Haarr, The Making of a Community Policing Officer: The Impact of Basic Training and Occupational Socialization on Police Recruits, December 2001, Police Quarterly, Volume 4, Number 4, pages 402-433, http://pqx.sagepub.com/content/4/4/402.short?rss=1&ssource=mfc.
39. Project Completion Review, Safety and Access to Justice Programme (SAJP).
40. Libya Security, Justice and Defence (SJD) Review: Phase I, Stabilisation Unit, July 2014, page 9.
41. Implementation Completion and Results Report on a Credit to the People’s Republic of Bangladesh for a Legal and Judicial Capacity Building Project, World Bank, May 2010; Dr. A. Nazrul et al., Evidence-Based Analysis of the Trial Courts of Bangladesh, prepared for UNDP and DFID, January 2011, page 37.
42. Assessing the Impact of the Scale-Up of DFID’s Support to Fragile States, ICAI, February 2015, http://icai.independent.gov.uk/wp-content/uploads/2015/02/ICAI-Report-Assessing-the-Impact-of-the-Scale-up-of-DFID%E2%80%99s-Support-to-Fragile-States.pdf.
43. Security Sector Accountability and Police Reform Programme (SSAPR) 2009-2014: 2013 Annual Review, UK Stabilisation Unit and International Security Sector Advisory Team, February 2014, page 11.
44. Justice for Vulnerable Groups in Malawi: Business Case and Intervention Summary, DFID, undated, http://iati.dfid.gov.uk/iati_documents/3755673.docx.
45. DFID’s Bilateral Support to Growth and Livelihoods in Afghanistan, ICAI, March 2014, http://icai.independent.gov.uk/wp-content/uploads/2014/03/ICAI-Report-DFID%E2%80%99s-Bilateral-Support-to-Growth-and-Livelihoods-in-Afghanistan.pdf; Assessing the Impact of the Scale-Up of DFID’s Support to Fragile States, ICAI, February 2015, http://icai.independent.gov.uk/wp-content/uploads/2015/02/ICAI-Report-Assessing-the-Impact-of-the-Scale-up-of-DFID%E2%80%99s-Support-to-Fragile-States.pdf.
46. J. Greening, Marking the International Day for the Elimination of Violence Against Women and Girls, speech at an event hosted by the Associate Parliamentary Group on Women, Peace and Security and the Gender and Development Network, 28 November 2012, https://www.gov.uk/government/speeches/justine-greening-eliminating-violence-against-women-and-girls.
47. A New Strategic Vision for Girls and Women: Stopping Poverty Before it Starts, DFID, 2011, page 3, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/67582/strategic-vision-girls-women.pdf.
48. Violence against Women and Girls: Map of DFID Programmes, Oxford Policy Management, October 2014, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/368932/Map-DFID-VAWG-prog-2014b.pdf.
49. A Theory of Change for Tackling Violence against Women and Girls, CHASE Guidance Note 1, June 2012, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/67336/how-to-note-vawg-1.pdf; A Practical Guide on Community Programming on Violence against Women and Girls, CHASE Guidance Note 2, May 2012, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/264401/VAWG-guidance-chase.pdf; and Addressing Violence against Women and Girls in Education Programming, Violence against Women and Girls Helpdesk, DFID, DFID Guidance Note – Part A, May 2014, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/318899/Education-guidance-note-partA.pdf.
50. The Future of UK Development Cooperation: Phase 2: Beyond Aid, International Development Committee, Tenth Report of Session 2014-15, January 2015, pages 16-23, http://www.publications.parliament.uk/pa/cm201415/cmselect/cmintdev/663/663.pdf.
51. DFID’s Use of Contractors to Deliver Aid Programmes, ICAI, May 2013, http://icai.independent.gov.uk/wp-content/uploads/2010/11/ICAI-REPORT-DFIDs-Use-of-Contractors-to-Deliver-Aid-Programmes.pdf.
52. DFID’s Empowerment and Accountability Programming in Ghana and Malawi, ICAI, October 2013, http://icai.independent.gov.uk/wp-content/uploads/2013/12/Empowerment-and-Accountability-081013-FINAL.pdf.
53. Rapid Review of DFID’s Smart Rules, ICAI, December 2014, http://icai.independent.gov.uk/wp-content/uploads/2014/12/ICAI-Smart-Rules-Report-FINAL.pdf.
54. Strengthening National Capacity in Development Countries to Design and Implement Sector Wide Indicators of Security and Justice, 2009-14, with a budget of £1.9 million, http://devtracker.dfid.gov.uk/projects/GB-1-200507/.
55. Centrally Managed Programmes: A Protocol, DFID, July 2014.
56. Overseas Security and Justice Assistance (OSJA): Human Rights Guidance, HM Government, February 2014 update, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/285812/OSJA_Guidance__2014_publication_.docx.
57. Report of the United Nations Joint Human Rights Office on Human Rights Violations Committed by Agents of the Congolese National Police During Operation Likofi in Kinshasa between 15 November 2013 and 15 February 2014, United Nations Organisation Stabilisation Mission in the Democratic Republic of Congo and United Nations Office of the High Commissioner for Human Rights, October 2014, http://www.ohchr.org/Documents/Countries/CD/LikofiReportOctober2014_en.pdf.
58. D. Blair, British aid funded African police force that “executed” children, The Telegraph, 27 November 2014, http://www.telegraph.co.uk/news/worldnews/africaandindianocean/democraticrepublicofcongo/11259137/British-aid-funded-African-police-force-that-executed-children.html.
59. This does not include the ability to listen to calls.
60. DFID’s Approach to Anti-Corruption and its Impact on the Poor, ICAI, October 2014, http://icai.independent.gov.uk/wp-content/uploads/2014/10/DFIDs-Approach-to-Anti-Corruption-and-its-Impact-on-the-Poor-FINAL.pdf.
61. Security Sector Accountability and Police Reform Programme (SSAPR) 2009-2014: 2013 Annual Review, UK Stabilisation Unit and International Security Sector Advisory Team, February 2014, page 8.
62. Public Opinion Survey on Personal Security and Police Performance 2014, SRG Bangladesh Ltd, commissioned by UNDP, September 2014.
63. P. Biesheuvel and A. McLean, Report of SJG Assessment Mission on DFID Malawi’s Support to Policing, Stabilisation Unit Security and Justice Group report for DFID Malawi, November 2012, pages 7-8.
64. Security Sector Accountability and Police Reform Programme (SSAPR) 2009-2014: 2013 Annual Review, UK Stabilisation Unit and International Security Sector Advisory Team, February 2014, page 39.
65. Security Sector Accountability and Police Reform Programme (SSAPR) 2009-2014: 2013 Annual Review, UK Stabilisation Unit and International Security Sector Advisory Team, February 2014, page 8.
66. Assessing the Impact of the Scale-Up of DFID’s Support to Fragile States, ICAI, February 2015, http://icai.independent.gov.uk/wp-content/uploads/2015/02/ICAI-Report-Assessing-the-Impact-of-the-Scale-up-of-DFID%E2%80%99s-Support-to-Fragile-States.pdf.
67. T.R. Tyler, Enhancing Police Legitimacy, The Annals of the American Academy of Political and Social Science, volume 593(1), 2004, pages 84-99; L.E. Wells, Type of contract and evaluations of police officers: The effects of procedural justice across three types of police-citizen contacts, Journal of Criminal Justice, volume 35(6), 2007, pages 612-621.
68. S. Berk-Seligson et al., Impact Evaluation of USAID’s Community-Based Crime and Violence Prevention Approach in Central America: Regional Report for El Salvador, Guatemala, Honduras and Panama, USAID, October 2014, http://www.usaid.gov/sites/default/files/documents/1862/USAID-LAPOP%20Crime%20Prevention%20Impact%20Evaluation%20-%20Regional%20Report%20-%20Final%20-%202014-10-29.pdf.
69. L. Sherman et al., Preventing Crime: What Works, What Doesn’t, What’s Promising, Report to the US Congress prepared for the National Institute of Justice, undated, www.ncjrs.gov/works/wholedoc.htm; C. Telep and D. Weisburd, What is Known about the Effectiveness of Police Practices? September 2011.
70. Assessing the Impact of the Scale-Up of DFID’s Support to Fragile States, ICAI, February 2015, http://icai.independent.gov.uk/wp-content/uploads/2015/02/ICAI-Report-Assessing-the-Impact-of-the-Scale-up-of-DFID%E2%80%99s-Support-to-Fragile-States.pdf.
71. Department for International Development Annual Reports and Accounts 2013-14, DFID, page 39, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/331591/annual-report-accounts-2013-14a.pdf.
72. Number of Women and Girls with Improved Access to Security and/or Improved Access to Justice Services through DFID Support, undated, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/361571/Access-security-justice22.pdf.
73. This can include the estimated percentage of annual survivors of violence against women and girls who have pursued justice through DFID-sponsored mechanisms, multiplied by the total number of women and girls living in the geographical areas benefiting from the programme.
74. Interviews with the Malawi Police Service.
75. Security Sector Accountability and Police Reform Programme (SSAPR) 2009-2014: 2013 Annual Review, UK Stabilisation Unit and International Security Sector Advisory Team, 2013, pages 5, 6, 8, 21-22, 28, 29, 32, 33 and 45.
76. We reviewed this funding instrument in 2013: DFID’s Support for Civil Society Organisations through Programme Partnership Arrangements, ICAI, May 2013, http://icai.independent.gov.uk/wp-content/uploads/2010/11/ICAI-REPORT-DFIDs-Support-for-CSOs-through-PPAs.pdf.
77. In an earlier report, we noted the value of these learning groups and encouraged DFID to develop them further: DFID’s Support for Civil Society Organisations through Programme Partnership Arrangements, ICAI, May 2013, paragraphs 2.86 to 2.90, http://icai.independent.gov.uk/wp-content/uploads/2010/11/ICAI-REPORT-DFIDs-Support-for-CSOs-through-PPAs.pdf.
78. The new DFID Smart Rules state: ‘Limited evidence or gaps in the evidence base are acknowledged at the design stage. Weak evidence does not necessarily mean that innovative programmes should not be carried out. However, a strong research and evidence plan (incorporating a combination of monitoring, evaluation and/or operational research) will be needed to help DFID learn and improve through the course of implementation.’ Smart Rules: Better Programme Delivery, DFID, 2014 edition, page 18, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/361330/Smart_Rules-oct14.pdf.
79. DFID’s Approach to Anti-Corruption and its Impact on the Poor, ICAI, October 2014, paragraph 2.35, http://icai.independent.gov.uk/wp-content/uploads/2014/10/DFIDs-Approach-to-Anti-Corruption-and-its-Impact-on-the-Poor-FINAL.pdf.
80. Access to Security and Justice Programme: Annual Review, DFID, March 2014, page 8.
81. Public Opinion Survey on Personal Security and Police Performance 2014 SRG Bangladesh Ltd., commissioned by UNDP, September 2014, page vii.
82. How DFID Learns, ICAI, April 2014, paragraphs 2.83-85, http://icai.independent.gov.uk/wp-content/uploads/2014/04/How-DFID-Learns-FINAL.pdf.