A Learning Review of DFID’s Efforts to Eliminate Violence Against Women and Girls

1. Purpose and rationale

This review assesses how well the Department for International Development (DFID) has responded to the UK Government’s commitment to tackle violence against women and girls (VAWG). It will review DFID’s efforts to build a portfolio of relevant and coherent programmes and assess how it is being positioned in order to achieve impact. It will also examine DFID’s efforts to build evidence on and assure value for money, particularly as the portfolio grows. DFID’s ambition is to promote long-lasting transformative change. The review will look closely at what DFID is learning about the scale, intensity and duration of interventions needed to bring this about. ICAI recognises the nascent and evolving nature of the VAWG portfolio. Review findings will be designed accordingly to inform DFID’s ongoing learning and evidence gathering process.

The review will build on ICAI’s review of UK Security and Justice Assistance¹. This assessed programming on access to justice and support services for victims of violence. The International Development Committee (IDC) published a report about DFID programming on VAWG in June 2013². Although the report concluded that DFID had a strong policy framework and some impressive programmes, it raised concerns that country programmes gave too little attention to changing social norms. This review will follow up on DFID’s response to the recommendations in the IDC report.

This review will not cover DFID’s work on VAWG in humanitarian emergencies, which is currently being reviewed by a separate House of Lords inquiry.³

2. Type of review

This is a learning review. ICAI learning reviews examine new or recent challenges for the UK aid programme, offering a snapshot of their effectiveness. They aim to inform their continuing development. Learning reviews focus on the generation and use of evidence by DFID and how well this translates into relevant and effective programming. They also provide scrutiny about how well programmes are performing and whether they are likely to deliver their intended impact.

In this review we aim to generate learning about how DFID is:

• Building an evidence base in a relatively new area and tackling evidence gaps.
• Creating a relevant portfolio of VAWG programmes designed to have significant impact.
• Adapting VAWG programmes to the national context.
• Identifying credible approaches to changing social norms.
• Taking successful pilots to scale.
• Integrating VAWG objectives across programmes in other areas, such as education, health and water and sanitation.
• Engaging with and influencing national stakeholders.
• Working internationally with others to influence global norms and processes.

As a learning review, we aim to identify how different approaches have emerged to a given aspect of VAWG in different contexts. We also examine how DFID has learnt from comparisons between them. The review is designed to capture the variation and diversity of approaches across the portfolio and compare and contrast across more and less successful programmes. We will review DFID’s pilot-to-programming pathways and the decision-making process by which DFID scales up successful pilots and closes unsuccessful ones.

We will explore DFID’s emerging approach of value for money and how it makes use of recent research. We will focus on choices about the scale and intensity of DFID’s programming and its approach to scaling up.

3. Scope

The review will cover DFID’s VAWG-focused programmes, those with a substantial VAWG component, and the development of the VAWG portfolio as a whole between 2010 and 2015. It will also assess DFID’s international advocacy in this area and its collaboration with partners in the UK and abroad.⁴ This will allow us to examine links between DFID’s international influencing work and VAWG programming.

The review will not directly examine programmes focused on security and justice as these have been covered by a previous ICAI review. Nor will it consider actions to address violence against women and girls in the UK more generally. It will however, consider DFID’s efforts to build domestic and international momentum behind their agendas concerning female genital mutilation (FGM), and child, early and forced marriage (CEFM) through the lens of the Girl Summit. The review will look at how DFID has promoted the inclusion of VAWG within the SDGs. It will also assess DFID’s collaboration with other HMG Departments in order to address VAWG at international levels. However it will not review initiatives driven by other departments, such as the FCO’s Preventing Sexual Violence in Conflict Initiative (PSVI), which is being covered by the House of Lords inquiry.

DFID’s VAWG work in humanitarian contexts and its programmes designed specifically to address VAWG in emergency situations will not be assessed as part of this review. This area is substantially different in aims, scope and type of programming and needs to be considered separately.

4. Background

Violence against women and girls is a global epidemic affecting an estimated one in three women worldwide.⁵ The types of violence suffered by women are diverse and VAWG should not be approached as a single phenomenon. Although many of the underlying drivers are shared, different forms of violence require different responses. There is an important lifecycle dimension to VAWG, stretching from female infanticide to abuse of widows. There are also specific issues experienced by disabled women and girls in relation to VAWG. DFID’s influential work on female genital mutilation/cutting (FGM/C), and child, early and forced marriage (CEFM) may lead to a focus on younger women and girls. However, it is also important to address sexual and domestic violence affecting older women.

The review recognises that VAWG is a complex area, involving significant sensitivities and nuances. We will seek to understand some of the assumptions underlying DFID’s Theory of Change, and how particular target groups and types of violence are defined and classified through the strategy and literature reviews.

Addressing violence against women and girls is recognised as a development goal in its own right, as well as an important precondition for achieving other development outcomes. It features as a sub-goal within the new Sustainable Development Goals (SDGs), adopted in September 2015.⁶

The UK Government has made major policy commitments to tackle VAWG internationally. DFID’s Secretary of State Justine Greening, has often repeated the Government’s commitment to addressing VAWG. Under her leadership, DFID has pursued a substantial advocacy programme to promote the inclusion of VAWG in the SDGs. It has also mobilised support for ending FGM/C and CEFM within a generation – most notably through the 2014 Girl Summit, co-hosted with UNICEF.

Following these policy commitments, DFID has significantly scaled up its VAWG programming, both in number and size, placing an increasing emphasis on prevention and long-term, multi-sectoral approaches. A 2014 mapping report⁷ showed that the number of programmes with a VAWG component since 2012 had risen from 63 to 109. The financial commitment for VAWG-focused programmes has also risen from £19.8 million (2012) to £131 million (2014), an increase of 563%. This represents only a portion of DFID’s total expenditure on VAWG.

VAWG is a relatively new area of programming for DFID and evidence to show what works is limited. DFID has made a substantial 5-year investment of £25 million in the global ‘What Works to Prevent Violence’ research and innovation programme in order to build the evidence base.

5. Theory of change

DFID has a theory of change for addressing VAWG,⁸ which was developed in collaboration with a range of stakeholders and published in June 2012 (see Annex 3). It includes a problem statement, barriers to progress, and desired outputs, outcomes and impacts. DFID’s goal is that “Women and girls are free from all forms of gender-based violence and from the threat of such violence.” It proposes four main types of intervention to address VAWG:

• Building political will and legal and institutional capacity: predicated on the principle that the state holds primary responsibility for action on VAWG. Interventions include strengthening government policy and legislation.
• Changing social norms, including behaviours and practices: challenging discriminatory gender norms and unequal power relations between women and men which lie at the root of violence.
• Supporting women and girls empowerment: promoting women’s political agency by supporting them to organise and mobilise against VAWG.
• Strengthening and expanding services including the availability and accessibility of health, education and social services for VAWG prevention and response.

The fourth area of service provision was partially covered under the 2015 ICAI Security and Justice review. This review will therefore focus more on the first three elements.⁹

The review will explore strengths and weaknesses in DFID’s overall theory of change, drawing on the literature review and interaction with stakeholders. We will also assess whether the theory accurately reflects the reality of DFID programming across the portfolio, and whether it is a helpful tool for programme development.

“Work with men and boys” is included as an element under interventions to change social norms. The international emphasis on work with men and boys has increased significantly over the past five years. This is in part a recognition that most political, religious and community leaders whose attitudes and actions need to change, are men.

The “empowerment of women and girls” pillar within the theory of change includes women’s rights, economic empowerment, education and political leadership. We will examine how this potential connection between VAWG and broader empowerment informs cross-programming working as part of our country case studies.

6. Existing evidence

What constitutes evidence in the area of violence against women and girls is still being debated. The DFID-funded programme “What works to prevent VAWG” has produced four evidence review papers¹⁰ which conclude that the gaps in the existing evidence of what works include:

• Tackling sexual violence;
• Men’s perpetration of violence, including pathways to abuse;
• Incidence and types of VAWG in fragile contexts;
• Effective protection, including at the community level;
• Risk factors, including by age group;
• The nature of the links between child abuse and subsequent VAWG.

Our literature review will draw on existing syntheses to produce a concise summary of evidence about what works in VAWG programming. It will assess the extent to which the literature supports the key assumptions underpinning DFID’s Theory of Change, and identify areas where evidence for violence prevention interventions is strong, where it is most contested and where it is weakest.

The 2013 IDC report also provides a range of useful data on DFID’s portfolio. By agreement with IDC, this review will follow-up on DFID’s response to the IDC’s recommendations.

There is a detailed portfolio mapping of DFID programmes up to July 2014¹¹ on which we can build during the study. A range of programme documents are publically available, including business cases and annual reviews. DFID informs us that approximately 40 of the programmes have ongoing or planned independent reviews or evaluations, but only a small number have been completed. These have been requested and will be analysed as part of the literature review. DFID has recently commissioned Itad to undertake a macro evaluation of its Strategic Vision for Girls and Women.¹² As far as possible we will draw upon this to inform our own review. We would expect to find studies and analytical work that have been used to inform the design of individual programmes. We will use this to test the strength of evidence behind DFID’s programming choices and the extent to which programming models have been adapted to fit the country contexts.

7. Review criteria and questions

The review is built around the OECD DAC Evaluation criteria of relevance and effectiveness. It will consider the following overall questions and sub-questions.

1. Relevance: to what extent is DFID’s VAWG portfolio relevant, coherent and plausible?

• How relevant is DFID’s VAWG programming to the needs and preferences of survivors and intended beneficiaries?
• How plausible are DFID’s theories of change for their respective objectives and contexts?
• To what extent is DFID’s programming designed at a scale and intensity likely to achieve sustainable impact and deliver value for money?

2. Effectiveness (use of evidence and learning): how effectively is DFID harnessing and applying learning in the development and scale up of VAWG interventions?

• How effectively do VAWG programmes make use of available empirical evidence and contextual analysis?
• How effectively is DFID identifying and addressing gaps in the evidence?
• How effective is DFID’s approach to piloting, replication and scale up?

3. Effectiveness (influence): how effectively has DFID influenced wider efforts to tackle VAWG at international and national levels?

• How effective has DFID been at securing and following up on international commitments on VAWG?
• How effectively has DFID coordinated with other UK government departments in tackling VAWG at an international level?
• How effectively is DFID linking up and aligning its VAWG programmes with its international influencing activities?

A detailed review framework is attached as Annex 1, cross-referencing the evaluation criteria and review questions to the different components of the review methodology.

8. ICAI themes and core issues

ICAI’s reviews fall within a framework consisting of four strategic themes¹³ that reflect UK aid’s challenges and priorities. This review relates primarily to ICAI’s ‘Transparency, Accountability, and Empowerment’ theme through its focus on women’s empowerment. The review also ties in to the ‘Leaving No One Behind’ theme through its focus on vulnerability, equity, youth and inclusion.

In terms of ICAI’s core issues¹⁴, the review addresses gender and equality, looking closely at a key aspect of DFID’s programming for women. It also looks at the quality of its processes for engaging with women and men in programme design and delivery. It links closely to coherence and partnerships, with components to address DFID’s collaboration with other UK Government departments, its international partnerships and its ability to wield influence on the international stage. There is a strong focus on research, evidence and learning, and exploring DFID’s processes for generating and using evidence in a new area of programming.

All ICAI reviews have an overarching interest in value for money. For the purposes of this learning review, we will examine how DFID is building its approach to value for money as its portfolio grows.

Evidence: in 2014 the What Works programme produced a summary of evidence based on the costs and cost-effectiveness of VAWG interventions, concluding that it was limited.¹⁵ Recognising this, we will consider how DFID is investing in data to inform value for money analysis, both ex ante and ex post. We will review the value for money methodology proposed under What Works and assess the extent to which it is used by DFID.

Application: we will assess the extent to which DFID draws upon this evidence of value for money in its decision-making on programme design and delivery channels. In particular, we look closely at the processes by which value for money is taken into account in the replication and scale up of programming.

Strategic approach: as an emerging portfolio which covers a wide range of activities, we are not seeking to assess the cost effectiveness of each programme. Instead we will make a judgement about DFID’s overall strategic approach to value for money in its VAWG portfolio. Where the evidence permits, we will assess the extent to which DFID draws upon its learning of value for money to inform investment decisions made at portfolio level, probing the impacts DFID expects to achieve compared with the level of resourcing.

9. Methodology

Fig 1. Summary of methodology components

As a thematic review of a substantial portfolio of programmes, the methodology has components at three levels:

1. A strategic review looking at DFID’s efforts to build an evidence base on what works, translate it into strategies and guidance and build a credible portfolio of programmes.
2. A desk review of a sample of programmes, to identify patterns and variations in DFID’s programming choices, to look at the generation and use of evidence and to assess the quality of programme designs.
3. Detailed case studies of programming in one or two countries, in order to explore how relevant they are to the national context, how well they interact with national stakeholders and beneficiary communities and how they link up with DFID programming in related areas, such health, education and livelihoods.

As a learning review, the methodology focuses on the early stages of the results chain, particularly the relevance and quality of programme designs. We will not look systematically at impact, although we will capture any evidence on emerging results from DFID’s own reporting. The methodology will consider the scale and intensity of DFID programming, including how it makes choices and justifies moving from piloting new initiatives to programming at scale. The review of DFID’s international influence is a discrete component, requiring a separate methodology.

The methodology will have five components.

i) Literature review

We will conduct a brief review of the literature on development programming on VAWG. The literature review will draw on existing syntheses, including DFID’s What Works initiative.¹⁶ It will provide a concise summary of key issues and conclusions emerging from both academic and grey literature, concluding where appropriate on the state of knowledge and the quality of evidence underlying the main conclusions (including the transferability of evidence from OECD countries to developing countries). The literature review will summarise available evidence from research on:

• The needs of women and girls, including survivors of violence;
• The assumptions and causal links in DFID’s Theory of Change;
• Common entry points for VAWG programming;
• Common areas of success and common obstacles in VAWG programming;
• The relevance of the international policy and legal environment to ending VAWG.

The literature review will be limited to research undertaken since 2005, relating to developing country contexts, and published in English. More details on areas to be covered under the literature review are given in Annex 4.

ii) Strategic review

We will conduct a strategic review of DFID’s spending decisions and programme approaches to VAWG. This will include:

• An updated mapping of the portfolio, to determine the patterns of expenditure and programming and how they have changed since 2010. This will enable us to assess (within certain limits¹⁷) the level of resources that have been mobilised to meet DFID’s VAWG commitment.
• A quality assessment of DFID’s VAWG strategies, guidance and overall theory of change. This will determine whether it is internally coherent and reflects the available evidence on what works and the opportunities and risks for VAWG programming identified in the literature review.
• An assessment of DFID’s approach to identifying and meeting gaps in the evidence base. This will include reviewing the processes and programmes involved in:
  1. Collecting and synthesising existing knowledge;
  2. DFID-funded research;
  3. DFID-funded innovation programmes;
  4. Knowledge management on VAWG across DFID country offices;
  5. Monitoring and evaluation;
  6. Capturing lessons from programmes and using them to inform future programming choices;
  7. Sharing knowledge and evidence with external partners.
• We will conduct a desk review of DFID’s ‘What Works to Prevent Violence Against Women and Girls: Research And Innovation Programme’, including looking at a sample of its innovation grants and impact evaluations.¹⁸ We will also examine the Global Girls Research Initiative, which contains activities on VAWG.
• A review and update of DFID’s responses to the recommendations from the 2013 IDC report. We will assess the adequacy of DFID’s management response. We will interview DFID about what has happened subsequent to each IDC recommendation, collecting documentary evidence and making judgments as to whether the underlying issues have been properly addressed.

The evidence for the strategic review will come from (a) a review of documents and data provided by DFID or obtained from DFID’s systems; and (b) key informant interviews, primarily in the UK but including telephone interviews with people in other countries as required. The key informant interviews will include:

• DFID staff at headquarters and country level;¹⁹
• UK development NGOs active in the area;²⁰
• Academic researchers and other independent experts.

The size and composition of this interview sample will be finalised after initial responses from DFID on their proposals for interviews, and drawing on outputs from the literature review on key informants and researchers.

iii) Review of DFID’s approach to international influence

We will assess DFID’s attempts to galvanise action on VAWG at the international level, and thereby influence its partner countries. DFID has not so far attempted to monitor or evaluate its international influence in this area. It recently commissioned a paper that mapped its influencing activities and proposed an overall theory of change and some results indicators.²¹

Without an existing record of influencing activities and results, it will be difficult to make a robust assessment of which international outcomes are attributable to DFID’s efforts.²² We can, however, identify the causal pathways by which DFID hopes to galvanise international action and examine how far it has progressed in some or all of these pathways. This will enable us to assess whether DFID has a strategic approach to influencing VAWG and the suitability of its own metrics for monitoring and assessing impact.

Drawing on the mapping exercise and draft theory of change, we will assess whether DFID has a coherent influencing strategy. This will include how clearly it has identified its objectives, the parties and behaviours it seeks to change and its influencing pathways. It will also include reviewing the quality of DFID’s collaboration with other UK Government (HMG) departments, in accordance with HMG commitments,²³ looking particularly at the areas of FGM/C and CEFM, where the UK international and domestic policy agendas intersect.

We will examine DFID’s effectiveness at achieving the following outputs/outcomes:

• Stronger strategic partnerships (focus: UN Women and/or UNICEF);
• International commitments by partner countries and multilateral organisations and follow-up actions (focus: Girl Summit, July 2014, and the follow-up National Action Plans and the Sustainable Development Goals);²⁴
• Financial commitments from other development partners (focus: financing commitments made at the Girl Summit);
• Increased voice and participation of women’s organisations in the global policy dialogue (focus: feedback from participants at the Girl Summit).

To answer these questions, we will rely on four types of evidence. First, we will trace the processes that led up to and followed the Girl Summit, and identify evidence of follow-up actions. Second, we will collect feedback from DFID’s peer organisations and other participants in the international processes (including HMG departments, development partners, representatives of partner countries, and international organisations) to include both the Girl Summit and SDGs. Third, we will include in our desk review sample a regional programme with explicit influencing goals: Toward Ending FGM/C in Africa and Beyond. We will also identify any VAWG links in the Global Programme to Accelerate Action to End Child Marriage. Both of these programmes have strong links to the Girl Summit. Fourth, in the case study countries, we will look for evidence from interviews and documentation of DFID’s international activities that have influence at the national level (such as National Action Plans from the Girl Summit).

iv) Programme desk reviews

We will carry out desk reviews of 23 DFID VAWG programmes, which represent one third of the programmes of interest to this review, including two global programmes, one regional programme and two Programme Partnership Arrangements (PPAs). The desk reviews will assess how well DFID’s guidance and theory of change have been translated into programme designs. They will examine whether business cases make proper use of the available evidence about the needs of those affected by VAWG and draw on adequate contextual analysis. We will assess their quality of engagement with survivors and other potential beneficiaries in programme design, implementation and monitoring. We will also assess how each programme deals with evidence gaps on beneficiary priorities and effective programme approaches in the country context. This will be achieved through research and piloting, and the strategy for moving from piloting to full-scale programming.

An analytical framework for the desk review will be developed. This will capture standardised information and allow comparable assessments to be made against common assessment criteria for each of the programmes.

The framework will capture the following data:

1. Types of VAWG intervention;
2. The results that the programmes seek to deliver;
3. Any programme-specific theory of change;
4. Delivery channels;
5. Monitoring and evaluation arrangements and expenditure;
6. Approach to maximising value for money;
7. Evidence of course corrections following Annual Reviews;
8. Any evidence on results and their sustainability;

The methods we will use while undertaking the desk reviews will include:

• Documentary analysis;
• Key informant interviews (using semi-structured interviews, including set questions where necessary in order to generate comparable data across the desk reviews);

The desk reviews will be based on documentation obtained from DFID²⁵ and telephone interviews with a small number of key stakeholders.²⁶ The findings of the desk reviews will then be analysed to map the different ways in which VAWG-related challenges have been addressed, and to identify recurrent patterns in programme design.

v) Case studies

The case studies provide a means of reviewing DFID’s approaches to ending VAWG across an entire country portfolio. We will test how DFID has drawn on its Theory of Change, research evidence and beneficiary feedback to design and scale up VAWG programmes. We will review the rationale and processes for decision making in programmes where it has been decided not to include a VAWG element. We will compare practice in each country with DFID’s own policy guidance. The case studies are not intended to generate new or representative data, rather to provide insights, illustrations and explanations that complement the broader analysis.

We will carry out detailed case studies of DFID VAWG programming in two countries, Ethiopia and India, involving visits by the review team to both countries. Case studies will gather evidence against a case study analytical framework, a modified version of the overall review framework, covering the relevance and effectiveness of VAWG programming.

The case study framework will include:

1. Types of intervention;
2. The objectives and types of impact that the programmes seek to deliver across the range of different types of VAWG;
3. Delivery channels;
4. The theory of change and how this is linked to the overall theory of change;
5. Monitoring and evaluation arrangements and their link to learning and knowledge management at the portfolio level;
6. Approach to maximising value for money;
7. Evidence of feedback and course corrections following Annual Reviews;
8. The relationships between VAWG interventions and other programming, such as in health, education or WASH;
9. Whether DFID has collected inputs and feedback from survivors of violence and other beneficiaries;
10. Whether programme designs are making progress on delivering the results expected at this stage of their implementation (activities, outputs and outcomes);
11. Whether any piloting components are well designed in terms of quality and intensity of monitoring and review mechanisms, and linked to processes for learning, replication and/or scale up;
12. Whether external counterparts (e.g. Government and key local stakeholders) view DFID’s approach as relevant and plausible;
13. Whether programmes link up with DFID’s international influencing activities.

The final case study approach, including the analytical framework, will be completed 3 weeks prior to the country visits taking place.

The methods used will include:

1. Documentary and data analysis, including gathering information from DFID’s country programme and project management documents, from counterparts (e.g. government and other local actors) and third parties (e.g. local academics).
2. Semi structured interviews with key stakeholders, including DFID, implementing partners, officials from counterpart institutions, other development partners and third parties such as civil society representatives. These will take place in DFID, counterpart HQs and to a limited extent at project implementation sites. We will interview implementing partners, local officials and community members, using semi-structured interviews. A local consultant will be engaged in each case study country to assist with providing access to local networks and knowledge.

We made a purposive choice of country case studies based primarily on the intensity and variety of VAWG programming. India and Ethiopia emerged as the top two candidates. India has some of the most substantial and mature of DFID’s VAWG programmes, including VAWG components with large multi-sectoral programmes in health, education and infrastructure²⁷. Ethiopia has substantial interventions which seek to address social norms around child marriage and FGM/C alongside a major programme promoting traditional justice solutions and community dialogues on VAWG. While this programme is under suspension, the design and early experience is of particular interest to the review. In addition, Ethiopia has a significant follow up national plan linked to the Girl Summit and held a national Girl Summit in June 2015.

Annex 1 sets out the review framework, showing how these methodological elements relate to the review questions.

10. Sampling strategy

There are three levels of sampling required for this review:

(i) A sample of programmes for desk review; (ii) a selection of DFID country programmes for case studies, and (iii) the choice of project sites to visit in each case study country. The sampling process for the first two levels is briefly described here, with further details in Annex 2. The third level will be decided prior to the country visits, when more detail has been obtained on the programmes being reviewed.

In choosing our sample, we used DFID’s 2014 mapping study, which identified 109 programmes with a VAWG element. This was brought up to date based on new information from DFID, yielding 115 programmes. From this, we eliminated programmes that were purely humanitarian in nature (which fall outside the scope of this review), where the VAWG element was too minor or limited to warrant individual review²⁸ or where it had already been visited by ICAI. This yielded a sampling frame of 68 programmes, including five run by DFID centrally, seven Programme Partnership Arrangements (PPAs),²⁹ eight regional programmes and 48 bilateral country programmes.

We have chosen to conduct desk reviews of 23 of these programmes, representing one-third of the total. We have included five central and regional programmes, as follows:

• Two global programmes focused on research and innovation;
• One regional programme linked to DFID’s campaigning around Girl Summit;
• Two PPAs with strong but contrasting VAWG approaches.

Of the 48 bilateral programmes, we used a form of intensity sampling³⁰ to identify those most likely to generate useful lessons for this learning review. We identified 27 ‘high interest’ programmes, representing those that involved sustained, intensive or innovative interventions in one or more of the three pillars of DFID’s theory of change that are of most interest to this review. From those 27 programmes, we chose one from each of the 11 countries in which they occur (where there was more than one, we made a random choice).

Finally, to balance any bias introduced by the purposive sample, we added a further seven programmes chosen at random from the 20 that were not identified as high-interest. The resulting sample of 23 (see Annex 2) represents a good spread of intervention types, countries and delivery channels, enabling us to assess the variety within DFID’s VAWG portfolio.

For the case studies, we have chosen to do two country visits, looking at the full range of VAWG programming in each country, including by central and regional programmes. In selecting countries, we focused on those that displayed the greatest variety and intensity of VAWG-related activities. We selected the following criterion:

• The countries should have at least one high-interest programme;

These 11 countries were then ranked through a scoring system, to generate a short-list of five countries (Ethiopia, India, Rwanda, South Africa and Zambia).

11. Limitations to the methodology

As a learning review, this methodology is only designed to probe the early stages of the results chain. VAWG remains a relatively new area of programming for DFID. Few programmes have reached maturity or completion, or generated impact data. The methodology will not attempt to generate new data on impact and sustainability, although it will capture any evidence about emerging results from DFID’s own reporting, and how well these link with on-going learning to deliver on impact and sustainability upstream.

While we will carry out a mapping of the main patterns in DFID’s expenditure and programming on VAWG, we will not repeat the detailed mapping work that has already been carried out on behalf of DFID. While we can track the main trends in expenditure on VAWG programming since 2011, we cannot accurately identify the total expenditure on VAWG, as we have no way of quantifying VAWG expenditure within wider programmes.

The methodology is based upon a purposive sample, though this can generate some risk of bias through researcher preferences. In this case, the bias may be towards the positive – that is, the sample favours more substantial VAWG interventions and is likely to represent DFID’s VAWG programming at its best. In a learning review, this is not a significant problem, as we are capturing learning across the portfolio, rather than assessing its average performance. We are mindful that substantial learning can come from programme failures, as well as successes. However, we have verified that the sample includes programmes rated ‘B’ or below in the latest DFID annual review. Nonetheless, to balance the risk of bias in the sample, we elected to include a random component, to provide greater breadth.

We will conduct desk reviews of approximately a third of DFID’s most substantial VAWG programmes. This should be sufficient to identify recurrent patterns and most of the variation within DFID’s portfolio. However, it is not suitable for making definitive judgements about the performance of the portfolio as a whole.

The two cases studies will provide an opportunity to explore certain programmes in detail, capturing lessons from those contexts. It will also enable us to test areas of enquiry formulated during the desk reviews. The findings from the case studies will only be illustrative of the programmes we include and cannot be directly generalised to the suite of programmes within the country or the VAWG portfolio as a whole.

12. Ethical considerations

ICAI reviews are undertaken with integrity and transparency. When commissioning and conducting reviews we are guided by ethical guidelines and code of conduct of professional bodies such as OECD DAC³¹. There are significant ethical and cultural issues concerning disclosure of violence and abuse during interviews with survivors and potential beneficiaries. We will ensure that cultural sensitivities will be respected throughout our review. We will engage an experienced national consultant from each of the two case study countries to advise us in this respect. We will undertake all interviews on the basis of informed consent and the results opinions and information will be anonymous, unless the explicit consent of the individual in question is obtained. We will focus our field level consultations on local civil society organisations and key informants who work directly with women and survivors.

The perspective of women and survivors will be collected indirectly, through key stakeholder interviews and by drawing on secondary sources (academic studies and reports by NGOs and other development partners). We expect that these will provide us with sufficient and appropriate evidence from the intended beneficiaries of the DFID programmes. They will also substantially inform our findings and enable their voice to come through the review.

13. Quality assurance and peer review

This review will be carried out under the guidance of ICAI Lead Commissioner, Tina Fahm. Francesca Del Mese is Peer Commissioner. The quality of the review will be assessed by the Review Oversight Unit of ICAI’s Secretariat, using OECD DAC evaluation standards.

This review will be externally peer reviewed at three points: an initial assessment of this paper (the comments from which have been incorporated in this final version); emerging findings; and draft report. The peer reviewer is Professor Liz Kelly from London Metropolitan University. Professor Kelly is both a VAWG sector/thematic expert and a highly experienced researcher. She is Professor of Sexualised Violence and Director of the Child and Woman Abuse Studies Unit. She has been active in the field of violence against women and children for almost 30 years. She is the author of Surviving Sexual Violence (Polity Press, 1988) and over 70 book chapters and journal articles.

14. Risk management

The main risks to the successful delivery of the review and how they will be mitigated and managed are summarised in the table below.

15. Timeline and deliverables

The timings of the main phase and deliverables is summarised as follows: