Independent Commission for Aid Impact (ICAI) Review of UK Development Assistance for Security and Justice

1. Introduction

1.1 The Independent Commission for Aid Impact (ICAI) is the independent body responsible for scrutinising UK aid. We focus on maximising the effectiveness of the UK aid budget for intended beneficiaries and on delivering value for money for UK taxpayers. We carry out independent reviews of aid programmes and of issues affecting the delivery of UK aid. We publish transparent, impartial and objective reports to provide evidence and clear recommendations to support UK Government decision-making and to strengthen the accountability of the aid programme. Our reports are written to be accessible to a general readership and we use a simple ‘traffic light’ system to report our judgement on each programme or topic we review.

1.2 We have decided to conduct a thematic review of UK development assistance for security and justice (S&J) and its contribution to UK international development objectives. Our focus will be primarily on DFID programmes but we will also examine a selection of S&J programmes by the inter-departmental Conflict Pool and other UK government departments as regards their contribution to international development objectives. We will review the portfolio from the perspective of its effectiveness in delivering improved justice and security for women and girls.

1.3 This Inception Report sets out the assessment questions, methodology and work plan for the assessment. It is, however, intended that the methodology and work plan be flexible enough to allow new questions and lines of inquiry to emerge over the course of the assessment.

2. Background

2.1 The background to this review is described in the Terms of Reference.¹

2.2 Since the Terms of Reference were completed, DFID has provided a written briefing on its S&J portfolio. According to DFID, it currently has 26 programmes in 14 fragile and conflict-affected countries, including one programme funded through the Conflict Pool. DFID states that it has provided 10.7 million women with improved access to S&J services, thereby meeting its commitments under the Bilateral Aid Review to reach 10 million women and girls. It estimates that its bilateral expenditure on S&J was £50 million in 2012-13, of which £16 million was legal and judicial reform and £34 million was security system management and reform. If civilian peacebuilding is included, the total increases to £88 million.²

2.3 DFID’s S&J portfolio is broad in scope, covering civil and criminal justice, public law (i.e. constitutions, administrative law and human rights standards), democratic control over the security sector and access to justice for communities. S&J is not a unified sector but a diverse collection of agencies, many of which enjoy autonomy from government. DFID notes that its approach is to provide support across national S&J systems, engaging with a range of ministries (interior, justice, defence), judiciaries, police services, prisons, anti-corruption agencies, local S&J providers, local government and civil society. It focusses on building effective, legitimate and accountable S&J institutions and on innovative approaches such as legal empowerment and community policing. It collaborates with other UK government departments and agencies through the Stabilisation Unit Security and Justice Group and the Justice Assistance Network.

2.4 DFID also has a range of centrally managed programmes that support research, evidence collection and the piloting of innovative approaches to S&J assistance. At the policy level, it has been a strong advocate for giving greater priority to S&J in the international development agenda, including by sponsoring the first OECD-DAC Handbook on Security System Reform in 2007,³ promoting S&J in the High-Level Panel Report on the Post-2015 international development framework and supporting international advocacy efforts by a range of NGO partners, including Open Society, Saferworld and The Asia Foundation.

3. Purpose of this review

3.1 To assess the relevance and effectiveness of UK S&J development assistance, with a particular focus on the needs of women and girls.

4. Relationship to other reviews

4.1 As described in the Terms of Reference⁴, this review will draw on the findings of prior and concurrent ICAI reviews that have examined S&J programmes, including in Democratic Republic of Congo, Nepal, Nigeria and the Palestinian Occupied Territories.

5. Methodology

Our approach

5.1 DFID’s approach to the provision of S&J assistance is based on the importance of S&J in building peaceful states and societies and on the importance of S&J services in overcoming conflict and fragility.⁵ These ideas, although influential internationally, are not based on a strong evidence base of past programming successes.

5.2 Our review will be a strategic assessment of the S&J portfolio as a whole. It will assess whether there is a coherent set of objectives and theories of change underlying DFID’s assistance in this area and whether the current portfolio is strategic in meeting the needs of its intended beneficiaries, particularly women and girls. We will assess the balance between tested and innovative approaches across the portfolio and the quality of DFID’s approach to knowledge generation and management. We will examine the delivery and impact of S&J programmes in two country case studies and make an assessment of DFID’s overall progress towards its results commitments.

5.3 The review will be primarily of DFID-funded programming but will also examine how S&J programmes by other government departments (including with funding from the Conflict Pool) contribute to the achievement of development objectives. Where other government departments also have other policy interests behind their international activities, such as counter-terrorism, we will not attempt to assess their overall effectiveness. We will focus on their contribution to development goals. We will, however, assess whether the different policy interests behind UK S&J assistance conflict with or detract from the development impact of the portfolio. We will review the division of roles and responsibilities, including between DFID and the Stabilisation Unit Security and Justice Group, as well as the quality of coordination between departments and agencies. We will consider the merits of using other government departments to deliver elements of DFID-funded programmes, as compared to other possible implementers. In these areas, we will draw on and contribute to findings from the ICAI reviews on the scaling up of assistance to fragile states and on anti-corruption.

5.4 We will examine the S&J portfolio from the perspective of its ability to deliver on DFID’s headline commitment of improving S&J services for women and girls. We will assess whether DFID reliably identifies the S&J needs of women and girls in particular country contexts and communities, whether its programming choices are relevant to their needs and preferences and whether it is able to overcome the challenges they face in accessing S&J services. While we will make an assessment of the overall impact of our sample programmes for all their intended beneficiaries, we will focus our consultations particularly on women and girls to determine whether the claimed results are making a meaningful difference to this group of intended beneficiaries. The focus on women and girls will provide a yardstick for assessing whether top-down capacity building support to central S&J institutions has translated into meaningful changes to services at the point of delivery and improved S&J outcomes.

5.5 To allow us to draw conclusions about the S&J portfolio as a whole, our methodology will have six components:

1. a literature review;
2. a strategic assessment of DFID’s overall approach to S&J assistance;
3. a desk review of a sample of DFID and Conflict Pool S&J programmes;
4. analysis of DFID’s approach to innovation and knowledge generation;
5. an assessment of the involvement of other government departments in S&J assistance; and
6. case studies of S&J programmes in two countries.

The methodology that follows is organised according to these five components.

5.6 The components of the review are designed to be mutually reinforcing, allowing us to draw robust conclusions about the portfolio as a whole. Our literature review will focus on identifying practical challenges in the delivery of improved S&J services to women and girls and the range of possible solutions to those challenges. Through the second and third components, we will develop a set of hypotheses as to how well DFID addresses these challenges, which will be tested through consultations with key stakeholder groups (principally NGO partners, experienced implementers and academics) and investigated in depth through case studies of programming in two countries, including through consultation with intended beneficiaries. Because the S&J area is relatively novel for development agencies and the evidence base is not well developed, we will pay particular attention to how well DFID manages innovation, research and knowledge management and whether, as a result, the portfolio is growing stronger over time.

5.7 Our methodology will be guided by an overall assessment framework, which is attached as Annex A1. It will be revised following the literature review and opening consultations with DFID. The components of the methodology are designed to collect the data required to answer the questions in the assessment framework.

i) Literature review

5.8 The literature review will focus on practical issues in the delivery of improved S&J for women and girls. It will cover three main areas:

1. identifying the S&J needs of women and girls. It will look at: the extent to which the S&J needs of women and girls in poor communities differ from those of men and boys; common forms of inequality or discrimination experienced by women and girls within S&J systems in developing countries; the practical barriers they face in accessing S&J services; the S&J dimensions tackling violence against women and girls; and barriers within legal and justice systems to the economic and social empowerment of women;
2. identifying entry points for S&J programmes. It will survey the literature to identify promising programming entry points (such as legal reform, judicial reform, police reform, criminal justice services for women, legal aid, legal empowerment, paralegals and local and non-state S&J providers). It will examine how suited they are to addressing the needs of women and girls; and
3. identifying common obstacles faced by S&J assistance programmes in delivering improved S&J outcomes for women and girls. It will look at common challenges to delivering improved S&J services, including those related to the operating environment (for example, political, legal and institutional constraints; cultural, geographical and financial barriers to access by the poor; corruption and vested interests; fragmented S&J systems; and legal pluralism) and to donor ways of working (for example, pressures to demonstrate measurable results, weaknesses in programme design and shortcomings in procurement processes). It will collect examples from the literature of possible ways of addressing these challenges.

5.9 The literature review has been designed so as to inform the development of the analytical frameworks we will use for the desk review and case study components.

ii) Strategic assessment of DFID’s overall approach to S&J assistance

5.10 We will assess DFID’s overall approach to S&J assistance, including its polices, strategies and guidance, its overall theory of change and the different approaches in use across its portfolio. This component of the review will involve:

• collection and reviews of S&J-related policies, strategies and guidance;
• interviews with DFID UK staff, including the S&J team and Violence Against Women team in the Conflict, Humanitarian and Security department (CHASE), the Governance, Open Societies and Anti-Corruption department and Africa and Asia regional cabinets;
• interviews with members of the Stabilisation Unit Security and Justice Group;
• interviews with NGOs holding DFID Programme Partnership Agreements and active in S&J assistance or violence against women and girls;
• interviews with UK-based academics engaged in policy-related research on S&J assistance;
• one or more focus groups with UK NGOs interested in S&J issues and violence against women;
• one or more focus groups with the main UK-based companies who implement DFID S&J programmes, together with independent consultants, supported by telephone interviews with firms and consultants not based in the UK;
• a portfolio mapping exercise, to identify the main approaches and entry points used for S&J programming across the portfolio; and
• a review of how DFID measures results across its S&J portfolio.

5.11 On the basis of this evidence, we will identify the key policy drivers behind the DFID portfolio and, to the extent possible, derive an overall theory of change linking DFID’s S&J assistance to particular development outcomes, such as creating more efficient markets, building effective states or empowering poor communities. As there are many possible rationales for providing S&J assistance, identifying the theory or theories in use within DFID will be an important starting point. We will test the plausibility of these theories against the available evidence, including the evidence that DFID has assembled to support its programming choices and the evidence from our literature review. This theory-based approach will enable us to formulate a set of hypotheses on strengths and weaknesses in DFID’s portfolio, to be tested through the desk review and case studies.

5.12 We will also examine the methods that DFID uses to define and measure aggregate results across the S&J portfolio, towards its Bilateral Aid Review target of improved services to 10 million women and girls. We will assess whether the reported results are a meaningful measure of the real impact of the programmes on women and girls and whether the setting of global targets distorts programming choices.

iii) Desk review of a sample of S&J programmes

5.13 We will conduct a desk review of a sample of current and recently completed DFID and Conflict Pool S&J programmes. The desk review will identify recurrent patterns in programmes, as designed and implemented. It will provide an evidence base for testing hypotheses on the strengths and weaknesses of DFID’s approach. It will collect evidence on results, as reported by the programmes themselves, DFID annual reviews and any external evaluations.

5.14 The primary challenge with conducting a desk review of S&J programmes is that the programmes often differ dramatically in practice from their original design. Data collected purely from design documents is, therefore, unreliable. We will, therefore, focus our desk review on telephone interviews with three main categories: the responsible DFID advisor; the programme implementer and (where possible to identify) the lead person on the programme design. These will be semi-structured interviews, with a set of common questions designed to capture comparable data across programmes, followed by a period of open discussion. The interviews will be complemented by the collection and review of design documents, background analytical work, annual reviews and any external evaluations.

5.15 For each programme, we will capture data using a common assessment framework. The assessment framework captures core information about the programme (budget, dates, implementing partners), the programme design and evidence base supporting it, the delivery options and the adequacy of monitoring and evaluation arrangements. We will collect any results data at output and outcome levels that the programme has produced but will not independently verify these figures. While the desk reviews will be shared across team members, a standard assessment framework will ensure a common approach and facilitate the aggregation of results.

5.16 We propose to select eight programmes for inclusion in the desk review. Each will be allocated three to four days of time for interviews and documentary analysis. The sample will be selected to provide a good cross-section of DFID and Conflict Pool S&J programming, so that the main types of programme and operating environment are covered. The selection criteria will include:

• scope (subject matter, size and complexity of programme);
• duration and continuity of support (maturity of programme, history of previous programming);
• geographical location (covering Africa, Asia and Europe);
• type of recipient country (post-conflict, fragile, developing);
• implementing partners (consulting companies, NGOs, multilateral agencies);
• funding source (DFID, Conflict Pool); and
• level of innovation (programmes that DFID identifies as innovative will be included).

5.17 When combined with the two country case studies, which will be examined in detail under the final component, our sample covers 10 of the 15 countries in which DFID currently has S&J programmes. As part of the desk review, we will also incorporate findings from past or concurrent ICAI reviews of S&J programmes, including in the Democratic Republic of Congo, Nepal, Nigeria and the Palestinian Occupied Territories.

iv) Analysis of DFID’s approach to innovation and knowledge generation

5.18 The policy and technical lead on S&J within DFID is held by CHASE. CHASE manages a number of central programmes that are designed either to improve the quality of DFID’s S&J portfolio or to promote DFID’s S&J objectives among international partners. They include programmes that support the piloting of innovative programming approaches in S&J and funding for a network of specialist organisations (the Conflict, Crime and Violence Results Initiative, £1.6 million, 2011-14) that develops guidance materials and provides help desk support for DFID country offices on results management.

5.19 We will conduct desk reviews of these programmes and assess how they contribute to improving the quality of the S&J portfolio. Where central programmes have activities in our case study countries, we will visit them. For programmes that pilot new approaches, we will assess the activities that are planned or under way for their novelty and their potential to strengthen DFID’s programming approaches. We will assess whether mechanisms are in place which make lessons learned from successful and unsuccessful innovations available to DFID country programmes and partners. We will assess the quality and level of utilisation of the guidance material and help desk services provided by the Conflict, Crime and Violence Results Initiative.

5.20 We will assess DFID’s efforts to assemble an evidence base on what works in S&J programming and to make this evidence available to its country teams in a useful form. We will consider whether DFID has accurately identified and communicated any weaknesses in the evidence base and whether it has appropriate strategies in place to build the evidence base over time. Through the desk review and case studies, we will assess whether the lessons learned from internal and external reviews have influenced the programmes in question and whether they have been collected centrally and disseminated in order to inform future programmes. We will assess whether DFID properly differentiates in its programme designs between interventions based on established evidence and those that are innovative in nature and whether it has monitoring and evaluation strategies in place that are appropriate to each. We will consider the design of DFID’s planned macro-evaluation of its S&J portfolio for its potential contribution to learning and knowledge management.

5.21 This review will not assess the results of DFID’s international advocacy on S&J.

v) Assessment of the involvement of other government departments in S&J

5.22 According to DFID, a number of other UK government departments play a role in the delivery of S&J assistance, either from their own budgets or as implementers of DFID-funded programmes. These include the Foreign and Commonwealth Office, the Ministry of Defence, the Ministry of Justice, the Crown Prosecution Service, the Home Office and the National Crime Agency. A substantial number of S&J programmes are also funded through the inter-departmental Conflict Pool. The UK Government has made efforts to improve coordination in this area, under the leadership of the National Security Council, including through the announcement of a new Conflict, Stability and Security Fund to replace the Conflict Pool.

5.23 This review will encompass the contribution that other government departments make to S&J programming, insofar as they fall within the definition of ODA. We acknowledge that other government departments have responsibilities to promote UK policy interests internationally and may be only partially engaged in promoting international development. Our review will focus solely on their contribution to international development goals. For that reason, we will not score individual activities undertaken by other departments. Rather, we will rate the overall contribution of their S&J assistance to the promotion of international development.

5.24 For each of the departments named above, we will map the extent and nature of their international S&J assistance. This will include a rapid assessment of their activities against our assessment framework, through desk reviews, as well as site visits to any activities in our case study countries. Particular attention will be given to whether the programmes have clear objectives and designs that are tailored to the country context. We will test the effectiveness and value for money of using other government departments to deliver S&J assistance, as compared to other possible delivery options.

5.25 Our review will encompass the quality of cross-government coordination in S&J assistance. We will consider the roles and responsibilities of the different actors, including CHASE and the Stabilisation Unit S&J network. This will include exploring the effectiveness of the Justice Assistance Network. We will assess whether DFID provides coordination or quality control over assistance programmes by other departments.

vi) Case studies

5.26 We will conduct detailed case studies of UK S&J assistance in two countries. The case studies will encompass all UK S&J assistance activities in those countries, including activities funded by DFID, the Conflict Pool or other government departments. The case studies will explore all of the questions in our assessment framework. The questions in our assessment framework under Objectives, Impact and Learning will be answered, to the extent possible, in respect of both active and completed programmes in that country. More detailed questions on Delivery will be answered in respect of the main current programmes.

5.27 Prior to the fieldwork, we will develop additional assessment tools to guide our research, such as interview guides for particular groups of informants. On the basis of other components of the methodology, we will formulate a set of hypotheses on the strengths and weaknesses of DFID programming for testing through the case studies.

5.28 The case studies will involve the following activities:

• a brief review of literature on the country in question, including political economy analysis and any analysis of its S&J challenges and institutions;
• a review of the DFID country strategy and poverty diagnostics;
• a review of programme documents, including programme memoranda or business cases, diagnostic and analytical work generated by the programme, monitoring and evaluation frameworks, baseline reports, annual reviews, activity and financial reports and any external evaluations;
• briefings from and interviews with DFID country office staff;
• interviews with implementing partners;
• interviews with counterpart institutions and other national government officials;
• interviews with national NGOs, think tanks, academics, journalists and other informed observers;
• site visits to areas where the programmes are active; and
• beneficiary consultations, with a particular focus on women and girls.

5.29 Our assessment of programme impact will start with the results reported by the programmes themselves, which will then be tested and validated through feedback from counterparts, independent observers and intended beneficiaries. We will pay particular attention to the relevance, significance and sustainability of the claimed results for women and girls in target communities, as well as to any unintended or unreported positive or negative impacts.

5.30 Our site visits and beneficiary consultations will focus both on the effectiveness of activities specifically directed at women and girls and on the impact of other activities for this particular group of intended beneficiaries. So far as the programmes in question work with central S&J institutions through reform or capacity building, we will examine both their direct outcomes for the target institution and whether they have resulted in improvements to the quality of service delivery for the intended beneficiaries. We will also assess whether other types of impact have been achieved, such as contributions to wider peacebuilding and state-building goals. This will be tested primarily through key stakeholder interviews.

5.31 We will collect feedback from women and girls (and to a lesser extent men and boys) through focus groups, individual interviews and discussions with representative organisations such as women’s groups. The research will be qualitative in nature. It will explore the S&J challenges faced by women and girls in different communities, to develop an evidence base against which to test the relevance of UK S&J programming. It will collect feedback on the experience of women and girls as to whether they have experienced improvements in access to or the quality of S&J services and whether this has addressed their needs and priorities. The qualitative research with intended beneficiaries may also generate insights into the development significance of S&J interventions – for example, where they have led to improvements in women’s access to property or their ability to pursue livelihood activities. If so, we will flag these as provisional findings requiring more detailed investigation.

5.32 The beneficiary consultations will take place in areas that have been targeted by the programmes, which will be identified prior to the country visits. The focus groups and interviews will be carried out by the review team itself, supported by interpreters. The team has extensive experience of conducting primary research on S&J issues in many developing countries. Furthermore, one of the team has done extensive field research on women’s access to justice in Sierra Leone. To support the team, we will engage national consultants in both case study countries with experience of primary research on S&J, who are able to advise on the cultural context.

5.33 The methods of selecting participants for focus groups and interviews will vary according to the context. In rural contexts, unannounced visits and focus groups organised spontaneously in consultation with local leaders are likely to offer the best method of collecting unmediated feedback from women and girls. In urban settings, we may identify representative organisations, such as women’s groups, to assist with organising focus groups. We may also target particular groups of beneficiaries, such as market traders. A detailed primary research plan will be prepared in advance of the field visit and adapted as necessary. So far as possible, the selection and conduct of the primary research will be done independently of DFID and its implementing partners, although we may accept some logistical support.

5.34 Research with minors and vulnerable individuals, such as victims of violence, entails strict ethical responsibilities. We will adhere strictly to the ICAI interviewee protection policy,⁶ together with generally accepted standards for interviewing survivors of traumatic events.⁷ This will include:

• ensuring that interviewees understand the nature and purpose of the review and give their informed consent to being interviewed;
• making it clear to interviewees that we cannot offer help in respect of individual cases;
• avoiding interviewing intended beneficiaries in circumstances that might expose them to personal insecurity or threat of harm;
• anonymising beneficiary feedback and keeping interviews strictly confidential; and
• obtaining the permission of responsible adults before interviewing minors and ensuring that adults from the community are present during any interviews with minors.

We will develop interview protocols prior to the field research and ensure that national consultants and interpreters are aware of them.

5.35 Some of the interviews will concern the experiences of women and girls in accessing S&J services after having been victims of crime. While we will not ask directly about their experiences of victimisation, it is possible that accounts of traumatic experiences may, nonetheless, emerge. We will be sensitive to the risk of distress or harm to the interviewees from recounting their experiences. We will discontinue any lines of questioning that appear to be causing distress and ensure that interviews conclude on positive or neutral topics. In village contexts, we will ask local women leaders to assist us by providing support to any interviewees that become distressed.

5.36 Our main criteria for selecting the country case studies are:

1. a sufficient level of UK S&J assistance over a sustained period, with a range of activities;
2. a range of country contexts, including a post-conflict state-building context and a more stable developing country context;
3. a mixture of rural and urban contexts for the delivery of assistance; and
4. the programmes in question have not previously been reviewed by ICAI.

5.37 On the basis of these criteria, we have provisionally identified Sierra Leone and Bangladesh as the case study countries. Sierra Leone has received a wide range of UK S&J assistance since the conclusion of the civil war (1991-2002) and represents the most mature example of post-conflict state-building S&J support in the portfolio. (South Sudan, as a more recent example, has been excluded as a possibility due to the current conflict). Bangladesh is a more stable developing country context with a much stronger economy but a range of S&J challenges, including those associated with rapid urbanisation.

5.38 Figure 1 describes the main current and recently closed DFID S&J programmes in those countries.

Figure 1: Recent S&J programmes in Bangladesh and Sierra Leone

5.39 In addition, DFID’s Global Legal Empowerment Initiative, a centrally managed programme that pilots grassroots legal empowerment initiatives, has activities in both Bangladesh and Sierra Leone. The Crown Prosecution Service has been involved as an implementer of DFID past programmes in Sierra Leone.

6. Roles and responsibilities

6.1 The Team Leader will be the primary point of contact with DFID. KPMG will provide oversight of this review under the overall leadership of the ICAI Project Director. Supplementary analysis and peer review will be provided by KPMG staff. The Lead Commissioner is Diana Good.

6.2 The team will comprise the following members:

Team Leader

With over 15 years of experience in development consulting, including policy advice, research and analysis, programme design and evaluation, he has worked for a variety of clients on diverse issues including governance and civil society programming and aid effectiveness. He is an authority in international law and human rights and has written widely on post-conflict reconstruction, state-building and the restitution of property. He will have overall management responsibility for all stages of the process including the country case studies and will ensure delivery of the outputs.

Principal Consultant

He is a New York-based independent consultant with a Ph.D. in political science and 20 years’ experience in security and justice programming. His past work experience includes positions with the United Nations (UN) Department for Peacekeeping Operations in New York, the UN Mission in Bosnia and Herzegovina and UNDP New York. He has provided design, analysis and evaluation support to over 40 security and justice assistance programmes in Africa, the Middle East, Europe, Latin America, Asia and the Pacific. He has conducted major thematic evaluations of security and justice assistance for a number of donors and international organisations, including the Australian Government and the European Union. He has an extensive list of publications on a range of S&J themes. He will lead on the portfolio review and take part in all the field work.

Principal Consultant

She is a Research Fellow with the Overseas Development Institute (ODI) in the Politics and Governance Programme, focussing on security and justice programming. She completed a Ph.D. on the challenges of engaging informal actors in policing and justice reform programmes in Sierra Leone. She has a long list of publications to her name, including on post-conflict peacebuilding and informal governance practice. She provides technical assistance and capacity building support to the g7+ (an alliance of fragile and conflict-affected states) and its Secretariat, based in East Timor. She has a strong research interest in violence against women and women’s access to justice. She is an experienced field researcher and will lead on the design and delivery of beneficiary consultations for the review, as well as participating in all aspects of the field research.

Senior Advisor

She is a Research Fellow in the ODI Politics and Governance Team. She has D.Phil in Politics and a distinguished publications record in accountability, rule of law, justice sector reform and legal empowerment, with a particular focus on Latin America. She has a distinguished academic career, having worked for the Universities of London and Salamanca and an extensive list of publications. She contributed to the OECD-DAC guidance on state-building in fragile states, led a report on children and women’s rights in Kenya and was lead researcher for an evaluation of UN Women and UNDP programming on women’s political participation in sub-Saharan Africa. She has a strong interest in evaluation methodology. She will act as adviser to the team on methodology and support the analytical process.

Researcher

She is an ODI researcher with fifteen years’ experience in academia and research institutions in the areas of gender equality, S&J reform, violence against women, peacebuilding and democratic reform. She has prior experience as an independent consultant for clients such as DFID, the OECD, UN Women and Cordaid. She headed the gender and peacebuilding team at International Alert from 2007 to 2009 and the gender and development section of the OECD Development Centre from 2010-11. She completed her Ph.D. at the London School of Economics in 2010 on the integration of gender into the UN’s peacebuilding work in Sierra Leone and has published a co-edited volume with Routledge on women, peace and security issue.

6.3 We have decided not to constitute an independent advisory panel for the review. We found it difficult to identify suitable senior advisers with practical knowledge of the field who were not also involved with the delivery of UK S&J assistance in some capacity. We will, nevertheless, consult broadly with experts and NGOs.

7. Management and reporting

We will produce a first draft report for review by the ICAI Secretariat and Commissioners by w/c 10 November 2014, with time for subsequent revision and review prior to completion and sign off in w/c 9 February 2015.

8. Expected outputs and time frame

8.1 The main deliverables will be:

9. Risks and mitigation

9.1 The following sets out the key risks and mitigating actions for this evaluation.

10. How this ICAI review will make a difference

10.1 S&J assistance is a relatively new part of the international development agenda, in which DFID has been a pioneer. There are multiple, conflicting theories as to how S&J contributes to international development and the evidence base underlying them is relatively weak. In recent years, there has been a crisis of confidence among many practitioners in the area as to whether traditional approaches to capacity building of S&J institutions are delivering real outcomes to the public. This has led to a proliferation of new approaches, such as legal empowerment.

10.2 Despite this uncertainty, S&J is growing in prominence in the international development agenda. The High-Level Panel of Eminent Persons on the Post-2015 Development Agenda has proposed the building of responsive and legitimate institutions that promote the rule of law and access to justice as one of five ‘transformative shifts’ for the new international development agenda that will follow the Millennium Development Goals.⁹ In 2012, the Prime Minister, the Rt. Hon. David Cameron MP, listed an independent judiciary, the rule of law, the rights of individuals and democratic control of the military as part of the ‘golden thread’ that links ‘successful countries and sustainable economies all over the world’.¹⁰ The International Development Secretary, the Rt. Hon. Justine Greening MP, has committed DFID to tackling violence against women and girls around the world.

10.3 S&J assistance is a core element of DFID’s approach to peacebuilding and state-building in fragile and conflict-affected states. DFID’s approach paper on state-building identifies S&J as one of the core services that any state must establish in order to be stable and legitimate.¹¹ As DFID has scaled up its spending in fragile and conflict-affected states, its S&J portfolio has increased. There are, however, some significant questions as to whether the objectives of its S&J assistance can realistically be achieved in difficult operating environments with local institutional capacity and, frequently, substantial political obstacles to progress on S&J.

10.4 DFID formulated its core strategies and policies between 2002 and 2009 and these have not been updated since. It is, therefore, possible that its approach is dated and lacks a plausible theory of change and clear evidence base.

10.5 This is, therefore, a timely moment for a strategic review of DFID’s S&J assistance. It will be a theory-based review, examining the underlying goals and theories of change of the portfolio and assessing whether these are supported by the composition of the programme, the evidence for international sources and the results of individual programmes. It will make a substantial contribution to DFID’s own learning in this area, as well as to external accountability.

10.6 We note that there was substantial interest in this topic expressed during our public consultations. This review will help to increase public understanding of the UK’s contribution in an area of considerable public interest.

Footnotes

1. http://icai.independent.gov.uk/wp-content/uploads/2014/04/ICAI-Review-of-UK-Development-Assistance-for-Security-and-Justice-Terms-of-Reference.pdf
2. ICAI’s thematic review of Security and Justice, DFID Security and Justice Team, 21 May 2014, paragraph 5.
3. OECD-DAC Handbook on Security System Reform: Supporting Security and Justice, OECD-DAC, 2007, http://www.oecd.org/development/incaf/38406485.pdf.
4. http://icai.independent.gov.uk/wp-content/uploads/2014/04/ICAI-Review-of-UK-Development-Assistance-for-Security-and-Justice-Terms-of-Reference.pdf
5. Building Peaceful States and Societies: A DFID Practice Paper, DFID, 2010, http://www.gsdrc.org/docs/open/CON75.pdf.
6. Interviewee protection policy, ICAI, http://icai.independent.gov.uk/wp-content/uploads/2011/11/Interviewee-protection-policy-2.pdf.
7. For example, WHO Ethical and safety recommendations for researching, documenting and monitoring sexual violence in emergencies, World Health Organisation, 2007, http://www.who.int/gender/documents/OMS_Ethics&Safety10Aug07.pdf.
8. Ebola virus disease, West Africa – update, World Health Organisation, 2014, http://www.who.int/csr/don/2014_06_04_ebola/en/.
9. A New Global Partnership: Eradicate Poverty and Transform Economies Through Sustainable Development, United Nations, 2013, http://www.post2015hlp.org/wp-content/uploads/2013/05/UN-Report.pdf.
10. Speech at Al Azhar University, David Cameron, 17 April 2012, https://www.gov.uk/government/speeches/transcript-prime-ministers-speech-at-al-azhar-university–10.
11. Building Peaceful States and Societies: A DFID Practice Paper, DFID, 2010, http://www.gsdrc.org/docs/open/CON75.pdf.