DFID’s Support for Civil Society Organisations through Programme Partnership Arrangements

Executive Summary

The Independent Commission for Aid Impact (ICAI) is the independent body responsible for scrutinising UK aid. We focus on maximising the effectiveness of the UK aid budget for intended beneficiaries and on delivering value for money for UK taxpayers. We carry out independent reviews of aid programmes and of issues affecting the delivery of UK aid. We publish transparent, impartial and objective reports to provide evidence and clear recommendations to support UK Government decision-making and to strengthen the accountability of the aid programme. Our reports are written to be accessible to a general readership and we use a simple ‘traffic light’ system to report our judgement on each programme or topic we review.

Green: The programme performs well overall against ICAI’s criteria for effectiveness and value for money. Some improvements are needed.

Green-Amber: The programme performs relatively well overall against ICAI’s criteria for effectiveness and value for money. Improvements should be made.

Amber-Red: The programme performs relatively poorly overall against ICAI’s criteria for effectiveness and value for money. Significant improvements should be made.

Red: The programme performs poorly overall against ICAI’s criteria for effectiveness and value for money. Immediate and major changes need to be made.

This report examines the Department for International Development’s (DFID’s) Programme Partnership Arrangements (PPAs) – one of the principal mechanisms through which it funds civil society organisations (CSOs).¹ In the current funding round (2011-14), DFID will provide a total of £120 million a year to 41 organisations, with grants ranging from £151,000 to £11 million. Through the PPAs, DFID supports CSOs that share its objectives and have strong delivery capacity. It provides CSOs with ‘unrestricted’ funding, giving them the flexibility to follow agreed strategic priorities. We assess the effectiveness and value for money of the PPA instrument, looking at partner selection, reporting and accountability. We comment in particular on six case study CSOs.

1 Introduction

1.1 This review considers the Department for International Development’s (DFID’s) £120 million annual funding of civil society organisations (CSOs²) through 41 Programme Partnership Arrangements (PPAs). It considers the performance of the grant-making mechanism as a whole and in respect of six case studies: Christian Aid; Action Aid; WWF; a consortium led by Restless Development; Conciliation Resources; and the Ethical Trading Initiative (ETI).³ This represents a spread of large, medium and small organisations with a variety of specialisations and focusses.

1.2 The present round of PPA funding began in 2011 and is intended to run until 2014. While it is too early to draw conclusions as to the impact of the current PPAs, there is evidence on which we can assess their prospects for success. PPAs are not a new mechanism. DFID has been funding through PPAs since 2000; four out of our six case study CSOs have previously held PPAs.⁴ While this evaluation is primarily concerned with the current round, it also draws on learning from PPAs since 2000.

1.3 In this review, we do not offer an overall judgement on the merits of PPAs as a type of assistance, as it is not our remit to make recommendations on policy. The purpose of this evaluation is to assess the delivery, effectiveness and impact of DFID’s PPAs, with a view to improving implementation of the current round and helping to shape any similar CSO funding instrument in the future. We concentrated on DFID’s decision-making processes, on the quality of its performance management and on the impact of PPA funding.

CSOs working with DFID

The role of CSOs in international development

1.4 Civil society is a vital actor in international development. An important share of global aid flows is spent through CSOs. CSOs play several key roles in the development process. In a 2006 report, the National Audit Office (NAO) concluded that CSOs strengthen the voice of the poor, promote awareness in the UK of development issues, advocate for change, hold governments to account, provide humanitarian assistance, deliver services and build capacity in developing countries.⁵

1.5 The UK has a particularly vibrant community of CSOs active in development.⁶ They relate to DFID in different ways, which sometimes overlap:

• Delivering programmes: DFID engages CSOs to deliver programmes across the world, usually through direct grants but also, on occasion, through a competitive tender;
• As partners: DFID and CSOs work as partners in pursuit of joint objectives, such as the global campaigns to eradicate landmines and polio.⁷ DFID supports CSOs (such as Conciliation Resources) that can work in areas that other agencies (for instance governments) can find hard to access. They collaborate on generating knowledge and developing policy. Restless Development and Christian Aid, for example, have both worked with DFID on the development of the successor to the Millennium Development Goals after 2015; and
• As challengers: CSOs continuously lobby DFID and the UK Government, as well as recipient governments, for changes to policy. Christian Aid, for example, is campaigning for more action on corporate tax avoidance.⁸ DFID reports that it received over 100,000 letters, emails and petition signatures from the public in 2012, many as a result of CSO campaigns.⁹

CSOs have multiple entry points to DFID

1.6 DFID’s Civil Society Department (CSD), predominantly based in East Kilbride within the Policy and Research Division, is responsible for DFID’s overall policy towards CSOs.¹⁰ CSD and the Conflict, Humanitarian and Security Department (CHASE), based in London, oversee most of the funding that DFID provides centrally to CSOs, including through PPAs.

1.7 These are not, however, the only points of contact between DFID and CSOs. One CSO might relate to many different DFID departments and staff, both centrally and in country offices. This is particularly the case for large international CSOs with multiple interests.

CSOs are an important funding channel for DFID

1.8 DFID spent at least £694 million through CSOs in 2011-12 (see Figure A1 in the annex). DFID staff informed us that the total is likely to be higher, since projects funded primarily through other channels (e.g. partner governments or multilateral organisations) may not report on the proportion of funds spent through CSOs.

1.9 There are two main channels within DFID for this spending:

• DFID country offices: of the £327 million spent through CSOs in 2011-12 by DFID’s country offices, £154 million went to Africa, £102 million to South Asia and the remaining £71 million to other countries (Pakistan, Afghanistan, Libya and Yemen); and
• DFID headquarters: each year, DFID central departments oversee £367 million spending through CSOs. DFID’s Policy and Research Directorate accounts for £265 million, of which the majority (£201 million) is managed by CSD, including £120 million in annual funding for PPAs.

1.10 PPAs are, therefore, DFID’s largest central funding channel for CSOs. Figure A2 in the annex lists all centrally administered funding mechanisms used by DFID.

Programme Partnership Arrangements

Flexible funding to CSOs

1.11 DFID funding to CSOs falls into two categories. Using terminology from UK charities law, these are:

• restricted funds, which can only be used for a named purpose;¹¹ and
• unrestricted funds (also called ‘core funds’), which can be used as flexible funding at the discretion of the CSO.¹²

1.12 It will be noted that the majority of the funding arrangements set out in Figure A2 in the annex are for restricted funding, whereas the PPAs (and the Strategic Grant Agreement for VSO) are for unrestricted funding.

1.13 Since the 1990s, DFID (and its predecessor, the Overseas Development Administration) has provided unrestricted funds to a small number of CSOs that were seen as strategic partners. Predictable and flexible finance was seen as a means of supporting their organisational development and allowing them to pursue their own development priorities. From 2000, these block grants developed into PPAs, with a typical length of 3 to 4 years. These are now in their fourth round of funding (2011-14).

1.14 In principle, the value of an unrestricted funding instrument like a PPA is that it strengthens CSOs’ capacity. Flexible funding enables CSOs to set their own priorities and to develop their own areas of comparative advantage. It fosters risk-taking and innovation, allowing them to pilot untested initiatives that donors might not wish to fund directly.¹³ It enables them to invest in their own organisational development, including their governance and management structures, their fundraising capacity and their systems for learning. According to CSOs, it is very difficult to cover any of these items through conventional, restricted funding. Although it may take time, these strategic changes are expected to lead to improved impacts for intended beneficiaries.

1.15 Unrestricted funding also, in principle, allows for a different kind of relationship between donor and recipient. It demonstrates a level of trust by DFID in the CSOs’ mission, integrity and capacity, allowing for a more genuine partnership. Potentially, it enables DFID to influence the overall strategy and programming choices of the recipient, thereby influencing a much larger pool of funding. It also gives DFID a basis on which to influence CSOs’ organisational development – for example, by sharpening their focus on achieving concrete results.

1.16 DFID does not generally deliver aid itself, using others, such as CSOs, to do so. Channelling funding through CSOs enables the reach of UK aid to be extended. In particular, CSOs will often have infrastructure and delivery capabilities in fragile and conflict states, which are now a priority in DFID’s strategic approach.

1.17 While DFID’s criteria have changed over the years, PPAs have generally supported organisations that share DFID’s priorities and values, have high standards of corporate governance and offer an extensive reach in poor countries (or within the UK, for building support for development). These have usually, but by no means exclusively, been large CSOs. Small organisations are now equally capable of being eligible for such funding, to their and DFID’s potential benefit.

The current round of PPAs (2011-14)

1.18 The current round of PPAs began as DFID was seeking to measure results better and improve accountability. DFID’s thinking on these issues was evolving as it put the current PPAs in place. At the same time, there was a general uncertainty about the future of PPAs (see paragraphs 2.7 to 2.9 on page 8). Consequently, the 2011-2014 PPAs were subject to a variety of different emphases than their predecessors. Evidence from DFID’s documentation also indicates that the corporate expectations from these PPAs were subject to critical shifts as they were being put in place (see paragraph 2.11 on page 8).

1.19 A notable break from the past was that the 2011-14 round was the first in which CSOs openly competed for funds. Any CSO could apply, including those based outside the UK (again a first for this round). For the first time, DFID introduced two new independent layers: a due diligence process and an evaluation manager to assess CSO performance. It was also the case that the level of funding could be adjusted in the third year, depending on the level of achievement.

1.20 Figure 1 on page 5 lists the CSOs that were awarded PPAs for 2011-14. Of the £120 million total, £20 million is reserved for Conflict, Humanitarian, Security & Justice (CHSJ) PPAs, administered by CHASE.¹⁴ The remainder is for general PPAs. Four CSOs receive both General and CHSJ PPAs.

Our approach

1.21 This evaluation looked in detail at the six PPA recipients shaded in Figure 1. These were chosen as broadly representative of the types and size of CSOs currently funded through PPAs.

1.22 DFID’s documentation shows some changes in emphasis in its objectives for PPAs over time. We, therefore, used the justification for funding made to the Secretary of State in July 2010, as the key statement of intent to Ministers, to frame our discussion of the PPAs’ achievements (see paragraph 2.11 on page 8).

1.23 We have drawn on a range of other relevant reviews, such as:

• a 2006 NAO report;¹⁶
• evaluations of previous rounds of PPAs (both individual grants and the mechanism as a whole);
• annual review reports from individual PPA recipients;
• Independent Performance Reviews (IPRs) of each of the PPAs undertaken in 2012; and
• a mid-term review prepared in 2012 by Coffey, DFID’s independent evaluation manager, assessing performance to October 2012.

1.24 Given the wealth of existing review material (the IPRs alone ran to over 5,300 pages), our evaluation methodology was designed to build on existing knowledge. It sought to avoid unnecessary additional burden on DFID or the CSOs in a process that has already been heavily scrutinised by DFID’s contractors over the course of 18 months.

1.25 For each of our six case study CSOs, we interviewed representatives and assessed their programme management capacity, focussing in particular on financial reporting.

1.26 We interviewed DFID staff, consultants who had undertaken IPRs, DFID’s independent evaluation manager, the NAO and a range of informed observers. We met with representatives of CSOs who benefit from PPAs but are not case-study organisations and also with others who failed to obtain PPAs in the 2011-14 round.

1.27 Investigative work for this review was undertaken by Agulhas Applied Knowledge, one of the organisations in ICAI’s contractor consortium, under direct contract to ICAI. ICAI’s lead contractor, KPMG, had provided due diligence assessments to DFID for these PPAs and was, therefore, excluded from any involvement in this report.

1.28 This study took place in parallel with ICAI’s review of DFID’s use of contractors to deliver programmes. That report covers, as one of its case studies, KPMG’s contract with DFID during 2010-13 to provide due diligence for CSOs. We used Concerto LLP, not our contractor consortium, to undertake that review.¹⁷ The two investigative teams shared information and co-ordinated findings.

2 Findings

Objectives

2.1 In this section, we examine the strategy behind DFID’s use of PPAs. We discuss DFID’s lack of clarity of purpose for this round of PPA funding. We also note how DFID did not explicitly match its strategic objectives to those of the grantees and how the process of CSO selection left room for improvement.

DFID’s design of the PPA instrument

DFID’s objectives for its work with CSOs

2.2 DFID has five overarching corporate objectives for its work with CSOs. These are that CSOs:

• ‘deliver goods and services effectively and efficiently;
• empower citizens in developing countries to do things for themselves;
• enable civil society to influence, advocate and hold to account national, regional and international institutions including improving aid effectiveness;
• build and maintain capacity and space for active civil society; and
• build support for development in the UK.’¹⁸

DFID sought to use PPAs as a complementary mechanism to other approaches

2.3 The PPAs were developed as one funding channel that DFID could use to support CSOs to achieve these objectives. PPAs were seen by DFID as having particular added value over other, more restricted funding mechanisms. They were complementary and distinctive from grants that were narrowly focussed on achieving specific deliverables.

2.4 The NAO acknowledged such potential advantages of PPAs in its 2006 report, finding that they ‘offer the prospect of a more strategic approach to long-term development challenges, better co-ordination and reduced transaction costs – where they can be based on well-specified and monitorable strategic objectives’.¹⁹ It noted that the benefits can include greater predictability, flexibility and reduced administration costs for both DFID and the recipient. A report by the Organisation for Economic Cooperation and Development (OECD) also commends the way that DFID’s PPAs allow recipients to ‘focus on strategic and substantive issues instead of constantly chasing funds’.²⁰ PPAs were thus inherently intended to be a mechanism that encouraged freedom and innovation.

The use of PPAs has merit but is unproven

2.5 There is, however, nothing automatic about the benefits of unrestricted funding. Whether benefits are achieved largely depends on the design of the PPAs and on the quality of the partnership they create. The 2006 NAO report also notes a number of potential disadvantages of unrestricted funding, in particular that DFID has less control over the use of the funds and finds it more difficult to assess their impact. Overall, the NAO concluded that the PPA mechanism is most likely to be ‘suitable for well established CSOs where DFID has assessed governance arrangements to be strong’.²¹

2.6 The NAO recommended that DFID review all its CSO funding mechanisms to identify ‘better evidence of the circumstances in which different approaches are likely to be best’.²² DFID undertook an internal portfolio review of its CSO projects in 2010 but was not able to find enough evidence to conclude on the comparative effectiveness of different civil society funding instruments. Nor was it able to conclude on the merits of civil society versus other funding channels. DFID therefore embarked on the current phase of PPAs believing, although without clear evidence, that the mechanism worked to improve aid effectiveness.²³

Altered corporate priorities affected the PPA’s strategic direction

2.7 Following the change of government in 2010, the then Secretary of State for International Development indicated that the 2011-14 round of PPA funding would be the last (a final decision on this remains pending). His preference was for restricted funding linked more closely to specific outputs and outcomes, emphasising the importance of holding CSOs accountable for their use of public funds.

2.8 This shift in policy direction brought some clear benefits to the design of the current PPAs. It sharpened the focus of both DFID and PPA recipients on the need for greater rigour in measuring results.

2.9 It also created an uncertain policy context for the current round of PPAs. This produced a lack of clarity about the use of unrestricted funding and the medium-term future of the approach. This, in turn, affected the coherence of the instrument.

DFID failed to design the PPAs around clear development objectives

2.10 We note that DFID does not appear to have used its corporate results framework (set out in Figure 2) to set priorities for this round of PPAs. This was published in December 2010, prior to the agreements being finalised with CSOs. DFID uses this framework to manage and report on its results. It was developed while DFID was putting in place the PPAs. We would have expected the two processes to be co-ordinated. DFID did not consistently set out what development outcomes or results it wanted the funding of CSOs through PPAs to support.

2.11 We have seen a range of documents setting out different objectives for PPAs at different times. The most authoritative of these was a submission to the Secretary of State in July 2010, on which the approval to implement this round of PPAs was given. We have, therefore, used this as the key statement of DFID’s intent against which to frame our comments on the PPAs’ impact. It stated that the 2011-14 round of PPA funding would focus on four key areas:

• Innovation: encouraging CSOs to test and scale up new ideas;
• Results: supporting CSOs with monitoring and evaluation systems and the ability to demonstrate results and value for money;
• Efficiency: supporting CSOs to lead reform, put into practice high standards of transparency and demonstrate efficiency; and
• Leverage: supporting partners to drive improvements and set high standards of excellence across the CSO sector.

2.12 We think that these objectives have merit. DFID is a funder, not a deliverer, of aid. These objectives are seeking to improve the quality of DFID’s aid delivery. The PPA mechanism enables DFID to align with CSOs as important deliverers of aid, supporting their innovation and capacity building. The objectives do not, however, set out what development outcomes DFID wanted PPAs to support. These were left for the CSOs themselves to define (see paragraph 2.26 on page 10).

2.13 Within DFID’s corporate results framework (set out in Figure 2 on page 8), the objectives set out above could be seen to fall in level 3 under the headings of ‘Portfolio Quality’ and ‘Monitoring and Evaluation’. The logic of DFID’s results framework is that improving the quality of DFID-funded aid will, in turn, contribute to improving DFID’s results and global development outcomes.

2.14 Our view is that DFID should have been more explicit about what it hoped to achieve with the PPA instrument. It should have identified which corporate results and outcomes it wanted the PPAs to support in order to improve the targeting of funds. Consequently, we think that DFID failed to ensure that the design maximised the potential benefits of this type of funding.

DFID did not set out clearly how PPAs enable CSOs to deliver more effectively

2.15 DFID did not set out a clear ‘theory of change’²⁵ or programme logic for how the PPAs would contribute to improving the quality of delivery and, in turn, to improved development results. While DFID did not formally require a theory of change until January 2011, we expect any expenditure of public money to be based on a logical and clear justification. In fact, the causal chain from improving how CSOs operate to development impact is long and complex, with multiple links that are difficult to observe and verify. This makes it all the more important that the assumptions are made clear, so that the instrument can be managed appropriately and its results tested. The absence of a theory of change or programme logic was, therefore, a significant omission.

2.16 After the PPA awards had been made, DFID sought to address this gap by hiring an independent evaluation manager, Coffey International Development, appointed in January 2011. Its role has included the drafting (in consultation with DFID and CSOs) of two high-level theories of change. These are framed as questions, concerning (a) why DFID should support civil society; and (b) how DFID should fund CSOs. Coffey is testing hypotheses that may enable it to answer these questions, using evidence both from the PPAs and from DFID’s Global Poverty Action Fund. Given the sequence of events set out above, this process could only provide a retrospective justification for DFID’s current work with CSOs.

2.17 The lack of clarity on how PPAs would work resulted in some unresolved tensions between objectives. Coffey’s Mid-Term Review of the PPAs notes, in particular, the challenge of DFID demanding concrete results while also seeking to drive innovation. Our view is that innovation necessarily involves risk and uncertainty and concrete results cannot be guaranteed in the short term.

Objective-setting and funding decisions for organisations

The competitive process for choosing partners left scope for improvement

2.18 In 2006, the NAO recommended that selection of grantees should include ‘greater use of competition by specifying the changes [DFID] wants to engender, and letting CSOs bid for associated support’.²⁶ DFID responded to this only in part. It introduced a competitive application process but without first identifying the specific goals it wanted to achieve or how much funding was available to each organisation. Instead, it was left to CSO applicants to identify the objectives in limbo.

2.19 DFID put in place a two-stage application process: following an initial concept note, short-listed applicants were invited to submit a detailed proposal. The issues they were asked to address in their applications are set out in Figure 3.

2.20 All applicants went through a single assessment process. While we agree that an element of competition was useful, it is surprising that a small organisation such as the Ethical Trading Initiative (income £1.6 million) should be assessed through the same process as an organisation the size of Christian Aid (income £95.5 million).

2.21 The assessment was based only on documents submitted. DFID’s assessors were told not to use their prior knowledge of the organisation to avoid privileging previous PPA holders. This appears to be a rather artificial mechanism for achieving a level playing field. We would have expected DFID to use all the information available to it about the applicants, including past PPA or project performance (whether funded by DFID, for instance via country offices where a reported £327 million was spent on CSOs in 2011-12, or by other donors), governance (including an assessment of leadership) and future potential.

2.22 DFID did not meet with representatives from applicant organisations at any time during the process. DFID informed us that this was due to resource constraints.

2.23 The time frames for the application process were short (see Figure 7 on page 14). According to several CSOs, this adversely affected the quality of their applications. Respondents from organisations with a long-standing relationship with DFID told us they found the application process easier. They felt that they were able to match the wording of their applications more closely to DFID’s expectations.

The selection process involved a mixture of objective and discretionary elements

2.24 DFID scored CSOs based on their applications. The resulting ranking was then adjusted in order to balance the selection across ‘niches’ or programming areas.²⁷ DFID also sought to ensure strategic fit (we could find no criteria for assessing that), geographical representation and inclusion. A shortlist (with indicative funding) was then put to the Secretary of State, leading (after some further modification) to the final selection.

2.25 The process of modifying the overall list to achieve ‘balance’ allowed an element of discretion. Some lower-ranked organisations were included in the final selection, while some former PPA recipients with higher scores were excluded. If DFID had used the lowest score of the CSOs it finally funded as the base level, an additional 13 CSOs would have received PPAs.

DFID retrospectively identified which development outcomes individual PPAs should support

2.26 In their applications for PPAs, CSOs were asked to set out their objectives in the form of a logical framework.²⁸ These were then finalised during 2011 and incorporated into the grant agreements. Only at this stage did DFID begin to match individual PPAs with its particular strategic priorities. DFID’s expected results from the PPA round were, therefore, built from the bottom up, through decisions on individual applications, rather than as part of an overall strategy (see Figure 4 for illustrative results for the case study CSOs).²⁹

2.27 Our view is that DFID should have set out which strategic priorities it wished the PPAs to contribute to first, then ensured that the individual CSOs’ objectives matched these. This would not have undermined the benefits of funding through an unrestricted mechanism; CSOs would have had full flexibility in defining how they supported such corporate objectives. This approach would, however, have ensured that PPA funding was more explicitly matched to what DFID prioritised. We note that DFID’s experts in the various programming areas were not generally involved in setting the objectives.

2.28 Many of the grantees went on to develop theories of change for their PPAs once they were already underway (for instance, in preparation for the 2012 Independent Performance Reviews). This involved revising their logical frameworks, in some cases several times. Through this process, individual CSOs have gradually achieved more clarity as to what they hope to achieve through their PPAs. The DFID evaluation manager has been tasked with aggregating these objectives into a set of theories of change for the instrument as a whole (see paragraph 2.16 on page 9).

2.29 The whole strategy-setting process has, therefore, been approached retrospectively to make up for the original lack of clarity. As the original goals of the PPA were not clearly identified, the instrument has not been designed in a strategic manner, to maximise the potential advantages of unrestricted funding of this type and to avoid the risks to effective delivery.

DFID’s funding decisions were not transparent, predictable or linked to results

2.30 The funding envelope for the 2011-14 PPAs was £120 million per year. DFID’s decision as to how to allocate these funds across the successful applicants was not based on clear and objective criteria (see Figure 5). The level of funding given to each CSO was not linked to the achievement of any defined set of results, nor was the process transparent to the participants. CSOs have told us that they have not been formally informed how their allocations were decided.

2.32 Figure 6 on page 13 shows the changes in annual funding for successful applicants that had also held PPAs under the previous round. Of these, 18 had their annual funding increased and 8 decreased, with an average increase of 67%. Eight CSOs had their allocation more than doubled. In some cases, funding was up to four times as large as CSOs had been expecting. In some cases it was less than the CSOs had planned for.

2.33 The lack of predictability on the scale of funding hampered CSOs’ ability to plan ahead. This was exacerbated by the grant process, in many cases, not corresponding with the CSOs’ own budget and planning cycles.

VSO was treated as a special case

2.34 DFID found additional unrestricted funding to allow it to accommodate Voluntary Services Overseas (VSO), an organisation that funds volunteer development work internationally. VSO had previously held a PPA with DFID worth £31.66 million in 2009-10, representing 59% of VSO’s total income that year. VSO applied for PPA funding under the 2011-14 round. There was a concern, however, that the ceiling of 40% of turnover represented a substantial reduction from its previous funding levels. There was direct lobbying from VSO and interest from MPs³³ and VSO was removed from the PPA mechanism on 3 December 2010 (11 days before DFID’s deadline to notify its PPA applicants) and awarded a Strategic Grant Arrangement (SGA). This operates exactly as a PPA, allows for unrestricted funding and Coffey International includes VSO in its reporting of the operation of the PPA mechanism. It is structured to reduce further VSO’s dependence on DFID. The approval of VSO’s funding (£78 million) had the effect of increasing DFID’s overall unrestricted funding for CSOs by 22% for the period 2011-14. A smaller, restricted SGA of £543,520 was also awarded to the network organisation Bond, which was unsuccessful in its application for a PPA in the current round.³⁴

Delivery

2.35 As funder, DFID is not directly involved in the delivery of activities under the PPAs. In this section we look at the delivery of the funding mechanism itself, including the partnerships that were established with grantees.

Implementation

2.36 Figure 7 on page 14 sets out the timeframe over which the PPAs have been delivered to date. We note that CSOs were not informed clearly at the start of the process of all the key steps or timings, which in some cases affected their planning (particularly financial).

2.37 Our view is that this timing was too compressed at the initial stages. For instance, if compared with the rules for procurement that DFID usually abides by, the first stage of an open procurement should last at least 36 days (where there has been prior notice of the intended procurement).³⁵ In this case, CSOs had 24 days (in August when many staff were away) to respond to DFID’s request for concept notes. As has been noted in the Objectives section, our view is that the sequencing of some of this process was also out of step, particularly the setting of individual PPA targets prior to the development of a theory of change.

Due diligence process contributed to organisational development

2.38 DFID informed applicants of its decision and how much they would receive on 14 December 2010. It then began a process of preparing grant agreements. Most organisations received their first funding in Summer 2011 (e.g. Christian Aid’s initial tranche arrived in August).

2.39 Finalisation of the grant documentation required an agreement on objectives, an environmental assessment and a ‘due diligence’ review of each applicant’s organisational capacity. Organisations had not been previously advised that these would form part of the approval process. The latter was DFID’s main tool for the risk management of each grant. DFID does not maintain a separate risk register for the PPA mechanism.

2.40 The due diligence process was completed by KPMG during February and March 2011.³⁶ Overall, the process was the same for all CSOs, irrespective of their size or prior relationship with DFID. KPMG tailored its approach for each CSO, including to the nature of the work and size of grant being considered. The results were not initially routinely shared with CSOs to support their own learning but this was subsequently corrected.

2.41 All but one of our case study CSOs found the due diligence assessment process itself helpful in identifying areas for improvement. These included updating risk registers, strengthening procurement policies and defining key performance indicators. One CSO reported to us that the process added no value and was ‘a missed opportunity’. We are also aware of non case study CSOs that found the process too burdensome.

Reporting and accountability

2.42 Funding under PPAs is provided on a quarterly basis. CSOs informed us that the first payment was considerably delayed, due to additional information requests from DFID. Subsequent payments have been made according to schedule.

2.43 CSOs provide annual activity and financial reports to DFID. They report on the activities set out in their performance framework (logframes) and on the finances of the organisation as a whole. CSOs are also required to submit audited Statutory Accounts to DFID.³⁷

2.44 Visibility of costs has improved during this round of PPAs and may improve further when the International Aid Transparency Initiative (IATI) is implemented. For our six case study organisations, support costs for the PPAs were between 3% and 14% of total expenditure, in line with the overall support costs reported in the case study CSOs’ Statutory Accounts (see Figure A3 in the annex for how they allocated their funding).

2.45 Two of the three larger organisations (WWF and Christian Aid) decided to treat their PPAs as restricted funds.³⁸ Although this was not a DFID requirement, they did this partly in response to a perceived demand from DFID for measurable results. By allocating the PPA funds to specific activities, it was easier to link them to a particular set of development results. This suggests that the accountability mechanisms around the PPAs may be working at cross purposes to the strategic objectives of the instrument.

Relationship to other funding sources

2.46 Among our six case study organisations, the flexible funding offered by a PPA is most significant for the three smaller CSOs. It represents a higher proportion of their turnover, ranging from 14% to 25%, compared to 5% to 8% for the larger organisations. The larger CSOs in our case study sample also have a greater diversity of funding sources, giving them inherently greater flexibility. As the VSO example illustrates (see paragraph 2.34 on page 12), however, this is not always the case.

2.47 It is notable that Christian Aid receives considerably more income from DFID through other mechanisms (£10.6 million) than from its PPA (£7.3 million). In contrast, some CSOs were excluded from receiving other funding from DFID, such as from the Civil Society Challenge Fund, once they were awarded a PPA.

Management of the partnership

2.48 PPAs are managed by the Civil Society Department (CSD), predominantly based in East Kilbride. All of the grant recipients we interviewed noted that CSD was perceived as peripheral to DFID’s main departments. Other stakeholders (including DFID staff) agreed with this assessment but noted that, recently, CSD had been achieving a higher profile. Given the role of civil society in UK aid, we think that managing the relationships with CSOs should be clearly seen as part of DFID’s core business.

2.49 CSD staff ensure performance monitoring takes place and authorise the transfer of funds to CSOs. Six programme managers in CSD (two of whom are part time) share the responsibility for managing PPAs, together with other central grant instruments. In addition, eight programme staff and twelve advisers in CHASE support PPAs as a small component of their roles. A knowledge and learning adviser facilitates learning partnerships among PPA holders and other CSOs. A results and evaluation adviser was appointed in March 2012.

DFID has prioritised accountability over partnership

2.50 CSD has not led the engagement with CSOs on the substantive contribution that they make to DFID’s development goals. DFID’s technical experts in the relevant areas were not involved in setting PPA objectives and, to date, have played little role in managing the PPA partnership. This has been a key shortcoming.

2.51 It is notable that CHASE chose to adjust the management of the Conflict, Humanitarian, Security & Justice PPAs. In August 2012, CHASE assigned both a programme manager and a specialist advisor to engage with each CSO. This improved the quality of collaboration between CHASE and its PPA holders, thereby increasing the value that DFID derives from these relationships.

2.52 In principle, PPA recipients were selected in part because of their ability to work collaboratively with DFID. It does not appear, however, that PPAs have contributed to the quality of this collaboration. Many of the CSO respondents informed us that they could contribute more to DFID’s work. One chief executive from a case study organisation stated that ‘the policy people no longer recognise the value of a relationship with us’.

2.53 All of this suggests that the nature of the partnership that DFID hoped to achieve with PPA holders was not clearly identified. Instead, it was left to develop by chance. This has been a key weakness of the delivery of this round of PPAs. DFID could have tasked its advisory staff to pursue a more substantive relationship, as CHASE has now done.

Impact

2.54 As the current round of PPAs only began in mid-2011, it is premature to conclude on their impact for intended beneficiaries. In this section, we consider the impact of the funding on the organisational effectiveness of the CSOs. We also assess some of the outputs achieved to date.

Impact on organisational effectiveness

2.55 As set out in paragraph 2.11 on page 8, DFID’s rationale for providing this round of PPAs when seeking Ministerial approval included improving four dimensions of CSOs’ organisational effectiveness: innovation; leverage; results; and efficiency. We look at the organisational outcomes of the PPAs in each of these areas.³⁹ It is notable that these elements were not included in DFID’s logframe monitoring of PPA recipients.

Innovation

2.56 While DFID does not require CSOs to report meaningfully on innovation in their annual reviews, this was assessed as part of the Independent Performance Review (IPR) process.⁴⁰ Those reviews and our own observations both suggest that the PPAs are encouraging innovation, to different degrees across the case study organisations. For example, Christian Aid has used £400,000 of PPA funds to establish a Learning, Development and Innovation Fund focussed on its PPA themes. It is also scaling up its use of Participatory Vulnerability Capacity Assessments to make its learning process more systematic. Conciliation Resources is using PPA funding to expand its engagement with the private sector. ActionAid and WWF are both using PPA funds to strengthen learning across their programmes.

2.57 We agree with the Mid-Term Review’s observation that there is an unresolved tension between DFID’s demand for concrete results and its desire for innovation. Innovation is often risky and vulnerable to failure and its impact can take time to become evident. This is another case where poorly designed accountability can work against achieving strategic impact.

Leverage

2.58 Reporting indicates that PPA funding has enabled 80% of holders to leverage additional sources of income. They have done this by hiring new staff members, including as fundraisers, or using PPA money as seed funding. Figure 8 on page 17 sets out examples of where case study CSOs report that PPA funding has assisted them in leveraging additional resources.

2.59 One of our case study CSOs obtained £1.25 million from another donor, who chose to fund the CSO in part because it held a DFID PPA. In this way, the PPAs have an accreditation effect, signalling to other donors that the recipients are well-governed and effective organisations. ActionAid reports it has used PPA funding to strengthen its secretariat, which has in turn enabled affiliates in emerging markets to raise an additional £2.2 million. Similarly, the Ethical Trading Initiative and Conciliation Resources informed us that PPA support had helped them to exceed their fundraising targets.

2.60 Signals from DFID that this would be the final round of PPAs have also encouraged the grantees to pursue alternative funding options. Restless Development, like many other recipients, has used the PPA to diversify its income base, putting in place a new revenue model. It has developed a five-year business plan that includes a targeted funding limit from any one source of 30% of turnover, prioritises in-country funding and targets the private sector. Other CSOs have initiated similar fundraising strategies.

Managing for results

2.61 Another priority for DFID was that the PPAs strengthen the CSOs’ attention to results. This has been a key focus of this round, with impacts beyond the funded CSOs (notably as a result of the Learning Partnership – see paragraphs 2.86 to 2.90 on page 21).

2.62 All of our case study organisations have increased their focus on results. Conciliation Resources has used PPA funds to appoint a Director of Planning and Organisational Performance. It is also putting in place new corporate performance management systems, including a corporate performance dashboard and balanced scorecard reporting. Christian Aid has used PPA funds to pilot new agreements with its partners, which are now being rolled out across the globe. WWF is also using PPA funds to improve reporting systems and tools and to promote beneficiary participation in the monitoring and evaluation of its work.

Efficiency

2.63 The recipients are improving their approaches to value for money. Christian Aid, ActionAid, Restless Development and WWF are all using PPA funding to test new value for money assessment tools. Bond, the network of UK NGOs working in international development, is helping to co-ordinate this work and share the lessons more widely. The New Economics Foundation is providing an advisory role.

2.64 PPA funding was also used by the smaller CSOs to support improvements in financial reporting systems. This included rolling out new financial procedures and reporting templates to overseas branches. The result was more rigorous comparison across branches, leading to better cost control and efficiency improvements.

Other impacts

2.65 PPA holders report that the process has, in turn, led them to renegotiate their relationships with CSO partners in developing countries. These secondary partnerships have been strengthened to include greater transparency and accountability, in large part to meet the PPA requirements.

2.66 The funding is also helping to improve accountability to funders, the public and intended beneficiaries. DFID has stated that, by April 2013, its grantees must comply with International Aid Transparency Initiative standards for disclosure of information on their funding and activities.⁴² One of the organisations we examined is using PPA funding to help embed the Humanitarian Accountability Partnership standard across its work.⁴³

Delivery of intended outputs to date

2.67 While still early days, CSOs report tangible outputs from the PPAs to DFID. These are the outputs set out in the logical frameworks that the CSOs report against annually to DFID. We are not, however, convinced that these intended outputs fully represent all that the DFID funding is in fact achieving (for instance using PPA funding to leverage new funds that are then used for other activities). We are also not convinced that DFID is measuring activities at the appropriate level for what is intended to be a strategic instrument.

2.68 Some examples of achievements from the six case study CSOs are provided in Figure 9 (see Figure A5 in the annex for outputs from the remaining case study organisations). We report these results without having undertaken our own verification. DFID’s own process of monitoring has been extensive (see paragraph 2.73 on page 19).⁴⁴ This has included verification by the evaluation manager of CSOs’ early results as set out in the IPRs.

2.69 These activities suggest that a range of impacts is likely to be delivered to the intended beneficiaries, across the diverse areas on which PPA holders focus. All of our case study organisations, except ActionAid, appear to be substantially on track to deliver their intended outputs.⁴⁵ Figure 9 indicates a pattern of overachievement against Year 1 targets. If this were to be sustained in the remaining two years, it would indicate that the targets being set are not sufficiently stretching.

2.70 In all cases, the PPAs are driving improvements in the ability of CSOs to demonstrate results. In addition, CSOs are seeking to improve the quality of attribution of achievements to PPA funding. It is likely that these changes will lead to improved results for intended beneficiaries. PPAs also have an influence beyond the recipient organisations to their local partners. There is evidence of useful organisational innovations being passed on to the recipients’ partners in developing countries.

2.71 Our view, however, is that seeking to identify and demonstrate results early in the life of what is intended to be a strategic instrument might be counter-productive. CSOs may be encouraged to fund activities that can readily provide reportable results, rather than activities that seek to result in meaningful longer-term change.

Learning

2.72 In this section, we assess how DFID is managing the monitoring and evaluation of its PPAs. We also consider how PPAs are contributing to learning, both by the CSOs and by DFID.

Monitoring and evaluation

Monitoring is not fully appropriate to PPAs’ strategic purpose

2.73 These PPAs have been subject to considerable scrutiny. Within the first year, CSOs underwent due diligence assessments and had to report on the PPAs’ environmental impact. DFID has also put in place three levels of monitoring for PPAs:

• Annual Self-Assessment Reviews: CSOs use a format provided by DFID to report progress against logical framework targets and on (selected) improvements to organisational effectiveness. DFID provides comments in response;
• Independent Performance Reviews (IPRs): Experts hired by CSOs review the progress of each organisation, including a verification of the annual review results. Their findings provide information that assists the evaluation manager to assess the PPAs as a whole. So far, each PPA has been through one IPR, in mid-2012, with another planned for 2014; and
• Mid-Term and End-of-Project Reviews: These are synthesis reports undertaken by the evaluation manager, drawing on information from annual reviews and IPRs. These reviews provide the evidence for evaluating the instrument as a whole and DFID’s overall engagement with CSOs.

2.74 DFID has not clearly differentiated its monitoring of PPAs from project-based funding. A key justification for PPA funding is that the mechanism is strategic and significantly different from project-based funding and, therefore, the emphasis of monitoring should be appropriate to that. Within 18 months of the PPAs beginning, however, the level of scrutiny has been considerable.⁴⁷

2.75 Monitoring and evaluation should effectively encourage mutual learning. While a number of individuals in CSD are engaged with the CSOs, the outsourcing of much of its monitoring and evaluation of PPAs appears to distance DFID from PPAs.

2.76 In 2006, the NAO said that ‘DFID must develop new tools to measure the effectiveness of its funding of CSOs’. It particularly noted the requirement for this to be put in place because ‘donors are shifting their emphasis towards building capacity and promoting accountability in developing countries’.⁴⁷

2.77 The principal tool used for monitoring performance is the logical framework. This tool is best suited for monitoring projects with clearly defined outputs and relatively predictable results. It is less suitable for more open-ended processes such as policy advocacy. CSO respondents (whether they were case study organisations or not) consistently reported that the logical frameworks do not sufficiently enable them to describe the impacts they wished to achieve. This particularly referred to their longer-term advocacy objectives, echoing the findings of the 2006 NAO report.⁴⁸

2.78 Monitoring based on logical frameworks can, therefore, provide only a partial picture of the impact of PPA recipients. DFID has, as a result, asked CSOs to report on such additional factors in their annual reviews. We note also that monitoring by beneficiaries is not prioritised, which is a key deficit.

2.79 DFID’s corporate objectives for CSOs stress their ability to ‘influence, advocate and hold to account national, regional and international institutions’. This is an important aspect of the role played by civil society in the development process. It includes activities such as helping to mobilise poor people to hold their governments to account. Activities of this kind are difficult to incorporate within logical frameworks. The changes they bring about are difficult to measure or verify. We note that DFID has been encouraging its partners to develop innovative methods for capturing this type of impact.

2.80 CSOs were not asked to specify their own organisational development as an output of the PPAs. They are not included in PPA logical frameworks. Given the purpose of the instrument, this seems a significant omission. DFID did, however, ask PPA holders to report in their annual reviews on how the PPAs are helping them to strengthen their performance in areas such as value for money, relevance, lessons learned and transparency.

2.81 The Mid-Term Review has sought to assess some outcomes of the PPAs at too early a stage. DFID tasked the evaluation manager with undertaking assessments of CSO performance, to inform DFID’s performance-based funding decisions for the third year of the PPAs. This assessment was based on evidence from the IPRs, gathered after only one year of PPA funding. Some organisations had yet to implement the changes enabled by their PPA. Given the very different starting points of the organisations, undertaking assessments so early was not a reasonable assessment method.⁴⁹

The monitoring burden has been disproportionate

2.82 DFID’s need for accountability has resulted in monitoring and evaluation processes that have proved to be unwieldy and burdensome. PPAs were originally intended to reduce transaction costs for both DFID and CSOs, thereby improving the effectiveness of delivery. One CSO noted in its annual review, however, that ‘evaluation and reporting requirements continue to place significant burdens on… programmes and partners’. Another CSO calculated that engaging with DFID had taken more than 18 weeks of staff time in one year. The Evaluation Manual was particularly criticised by PPA holders and IPR evaluators. A typical observation was that it was ‘over-long, over-engineered, poorly thought through and internally inconsistent – an extraordinary behemoth of a document’.

DFID’s outsourced evaluation function does not provide best value

2.83 DFID chose to outsource the role of managing the evaluation process to an external contractor, in part as a result of internal resource constraints. While we commend DFID for prioritising the evaluation function, this arrangement has not fully served the needs of either DFID or CSOs. The terms of reference stressed the need for the contractor to be independent (as befits an evaluator) while tasking it with supporting the grantees to put in place monitoring and evaluation. These tasks have proved incompatible.

2.84 In practice, Coffey, the contractor, has prioritised maintaining independence over building the relationships that would be required for effective support and guidance. For instance, Coffey set out a format and issued written guidance for IPRs but did not respond to (written or verbal) requests for clarification on these from the expert reviewers hired by CSOs. This then potentially weakened the performance of these reviews.

2.85 Our view is that DFID’s terms of reference for the role were over-ambitious, complex and somewhat contradictory. Any contractor would have struggled to deliver all the expected tasks effectively.

Learning by CSOs and DFID

The Learning Partnership has been very productive

2.86 A highly successful Learning Partnership has been established to support this round of PPAs. It is notable that, while learning was a component of the PPA approach, the creation of the Learning Partnership did not form part of the original design for these PPAs. It emerged out of previous experience with Latin American CSOs and has substantially been driven by a key member of the CSD team, the Knowledge and Learning adviser. Figure 10 describes how it is organised.

2.87 In the 18 months that it has been operating, the Learning Partnership has already led to some changes in CSO practice. One of the most advanced Learning Groups has been on Measuring Results in Empowerment and Accountability. It organised an event in 2012 chaired by DFID’s Director-General for Policy and Global Issues to identify shared concerns and priorities. It then formed seven thematic work streams and commissioned a review from the Institute of Development Studies on ways of evaluating empowerment.

2.88 The Learning Partnership is encouraging innovation among PPA holders and the wider CSO community. It is notable that DFID’s emphasis on measuring results has stimulated innovative thinking by PPA recipients, especially in areas where measuring results and impact is difficult (such as improving organisational effectiveness). It has also helped to change practice. For instance, both CAFOD and Oxfam report that they are now more open and rigorous when reporting on and learning from unsuccessful activities.

2.89 We assess the Learning Partnership to be an excellent example of DFID and PPA holders engaging at a more strategic level. By bringing PPA grantees and other CSOs into thematic groups, learning is shared among PPA recipients. An evidence base is being developed on successful approaches in areas identified as learning priorities.

2.90 There is, however, considerable potential for this process to enrich DFID and CSO learning further. By involving a broader spectrum of DFID staff, particularly technical experts and country-based personnel, learning would be more clearly embedded across DFID (as well as among CSOs).

DFID is not learning systematically from PPA holders

2.91 We are concerned that DFID is not using all available opportunities to learn from the CSOs it funds. They have much to contribute on a wide range of issues, such as delivering in difficult environments and to hard-to-reach groups, building CSO capacity in developing countries, working with the private sector, multi-stakeholder collaborations, developing delivery standards and leveraging funding and influence. The Learning Partnership demonstrates the creativity and reach of the CSOs and their potential when they work collaboratively.

2.92 By choosing to relate to PPA holders primarily as service deliverers who are accountable to it, DFID is missing out on further opportunities to learn. We would expect a clearer engagement with CSOs as partners (especially given the name of this instrument is Partnership Arrangement). We also think this would increase aspects of the value for money that DFID obtains from the PPAs, by directly building its knowledge and skills.

2.93 In practice, the operation of the contracting out of the evaluation function for these PPAs (both overall and also for the IPRs) has outsourced key aspects of the learning process (for both DFID and CSOs). We are concerned that, since DFID technical staff were not systematically involved in the evaluation of CSOs, opportunities for corporate learning are being missed. It is notable that such technical experts were also not systematically involved in the selection of the CSOs.

3 Conclusions and Recommendations

Conclusions

3.1 We recognise that a vibrant civil society sector is an essential part of the UK aid landscape. A strategic investment in this sector to strengthen CSO capacity and effectiveness through flexible funding is, in principle, an appropriate use of DFID funding.

3.2 To maximise its effect, DFID should have been more explicit about what it hoped to achieve with the PPA instrument and more strategic with its selection criteria. First, it should have identified which corporate priorities it wanted the PPAs to support. It should then have designed a competitive grant-making process designed to maximise that contribution, with fair and transparent competition.

3.3 As the current round of PPA funding only began in 2011, it is too early to draw conclusions on its impact. The evidence from our case studies and other reports suggests, however, that most PPA holders are delivering effectively on their expected outputs to date.

3.4 We found that PPAs have been driving important changes in the way that CSOs operate. In particular, they are improving the quality of performance management and accountability for results. We think it is likely that these changes will lead to improved results for intended beneficiaries, not just from PPA funding but across their full range of activities.

3.5 The complexity of DFID’s processes for reporting results and ensuring accountability has affected the ability of recipients to manage PPA funds flexibly. While the focus on demonstrating impact is important, it has been implemented in such a way that some of the grantees have chosen to use and manage the PPA funds more like conventional restricted funding. This diminishes the additional value of the instrument. Finding an appropriate balance between flexibility and accountability would have required greater clarity by DFID about the overall presumed causal chain (or theory of change) linking PPA funding and improved development impacts. The focus at this stage should be on identifying the planned and actual trajectory of impact rather than encouraging the publication of very short-term outputs.

3.6 The PPAs are influencing changes in CSO systems beyond the recipient organisations. There is evidence of useful organisational innovations being passed on to the recipients’ partners in developing countries. The Learning Partnership has been particularly effective at developing and sharing knowledge and approaches across the CSO community. It has proved to be an important driver of innovation.

3.7 We are concerned that the PPAs have not, in practice, operated as partnerships. The CSOs were selected on the basis that they share objectives with DFID. They have knowledge, influence and expertise that could contribute to DFID’s work. DFID failed, however, to define what it hoped to gain from these partnerships and, as a result, has gained less than it might have done.

3.8 The drive for increased accountability, amongst other factors, has worked to the detriment of collaboration. In some cases, DFID has limited its engagement with PPA recipients to a purely administrative level, with the frequency of contacts between the CSOs and DFID’s advisory staff diminishing. DFID has emphasised independence to the cost of real engagement and partnership.

3.9 To strengthen collaboration, DFID should have separated the administrative function of overseeing PPAs from its engagement with the partners at a substantive level. This would have improved the quality of monitoring, learning, knowledge exchange and joint working.

3.10 Overall, we conclude that PPAs have substantial potential as a funding mechanism. The evidence suggests that PPAs can enhance CSOs’ capacity to deliver development results. They help to expand the overall reach of UK aid, including into areas where DFID cannot easily deliver through its own direct financing.

3.11 To achieve this potential, the flexibility of unrestricted funding needs to be combined with accountability mechanisms that support the strategic goals of the instrument. The shift towards accountability for results in this round of funding has been an important driver of innovation for the recipients. What was missing, however, was a clear vision for how PPAs contribute to greater aid effectiveness, both as a funding channel and by building genuine collaboration between DFID and its major CSO partners.

3.12 The following recommendations are, therefore, intended to encourage DFID to focus the PPAs on their strategic purpose and to improve the nature of the partnership, whilst enabling accountability to be maintained at the appropriate level for a strategic instrument.

Recommendations

Recommendation 1: If DFID decides to continue with PPAs, or a similar grant-making instrument, it should use the intervening period to develop a more strategic, transparent and fair process for selecting CSOs and allocating funding. DFID should consider, both for this round and for any future rounds, extending the PPAs to more than three years to allow the strategic and innovative aspects of this unrestricted funding to develop.

3.13 Staff throughout DFID (in the UK and elsewhere) and in recipient CSOs should have a common and consistent understanding of the PPAs’ purpose and operation. This should include DFID:

• clearly setting out which strategic priorities it wishes CSOs to support;
• informing applicants in advance how they will be assessed, giving details on the expected organisational capacities as well as clarity on DFID’s priority areas;
• phasing the selection by first identifying the strategic fit of partners with DFID’s priorities, then inviting applicants to compete under particular categories;
• giving applicants sufficient time and guidance for the application process, to allow for fair competition;
• implementing more rounded processes for assessing each applicant’s track record, trajectory and potential contribution;
• providing clear guidance, from the outset, on how performance will be assessed;
• linking funding allocations to objectives more clearly; and
• increasing the predictability of funding, including by harmonising grants with each partner’s budgeting and planning cycle.

3.14 Once DFID is clear on the level of flexibility it wishes to allow the recipients of PPAs to have, it should consider extending the length of the funding cycle, if appropriate, from three to four or five years for the current round of PPAs. This will enable civil society organisations to work over a time frame more conducive to getting results which are truly transformative for intended beneficiaries, encourage innovation and enable leverage.

Recommendation 2: DFID should assign a technical counterpart to each PPA to ensure that both it and CSOs obtain full value from the partnership.

3.15 DFID should put in place new guidance on the management of PPAs to ensure that both parties receive the full potential value from the partnership. It should assign substantive experts from its own staff as technical counterparts for each grantee (as is done for the PPAs administered by CHASE). These counterparts should be given responsibility for dialogue, collaboration and mutual learning, including on policy issues and broader influencing strategies. For these staff, building quality engagement with PPA holders should be part of their job descriptions and performance objectives. They should also be encouraged to work and learn together in order to build DFID capacity and understanding of the role of civil society and the value it can add to DFID. DFID country offices and core policy teams should engage more closely with the PPA-funded CSOs.

Recommendation 3: DFID should re-design the monitoring and evaluation system for PPAs so that it is less cumbersome and better suited to the long-term strategic nature of this funding.

3.16 DFID should ensure that monitoring is proportional to the size and capacity of partners and captures the strategic and transformative impacts of the PPAs. This entails a revision of the evaluation strategy and changes to the way in which the evaluation manager operates. It also requires monitoring tools that can measure the effectiveness of this type of funding (as the NAO recommended in 2006⁴⁷). We do not think that the current logical framework effectively supports monitoring of the strategic benefits of PPAs. Other approaches could also be investigated, including increased beneficiary monitoring of CSOs’ activities and delivery. DFID should ensure that insights from the Learning Groups on evaluation of hard-to-measure areas, including the impact of strategic funding, feed into this process. Insights from DFID country offices and technical experts should be included.

Recommendation 4: DFID should strengthen the role of the Learning Groups, in order to ensure that lessons learned are shared more widely within DFID and with civil society partners.

3.17 DFID should increase its engagement with the Learning Groups, to track what outcomes they generate. It should develop new mechanisms for sharing learning with other stakeholders, including CSOs in developing countries. The technical counterparts for each PPA that we advise in Recommendation 2 should ensure that lessons arising from Learning Groups are shared systematically with DFID staff outside CSD.

Footnotes

1. DFID uses the term civil society organisation (CSO) to include non-governmental organisations (NGOs), which is often the more commonly used term.
2. DFID uses the term civil society organisation (CSO) to include non-governmental organisations (NGOs), which is often the more commonly used term.
3. See Figure A3 in the Annex for details of the income of these organisations.
4. Action Aid, Christian Aid, the Ethical Trading Initiative and WWF.
5. Department for International Development: Working with Non-Governmental and other Civil Society Organisations to Promote Development, The Comptroller and Auditor General, July 2006, http://www.nao.org.uk/wp-content/uploads/2006/07/05061311.pdf. For a wider discussion on the role of CSOs and associated challenges, see Banks, Nicola with David Hulme, The role of NGOs and civil society in development and poverty reduction, The Brooks World Poverty Institute, The University of Manchester, June 2012, http://www.bwpi.manchester.ac.uk/resources/Working-Papers/bwpi-wp-17112.pdf.
6. This round of PPAs was the first that was open to CSOs worldwide. Previously, only UK-based organisations could apply. In practice, applicants and successful grantees were predominantly from the UK.
7. Fifteen years after the landmine ban the number of new casualties halves, Reliefweb, December 2012, http://reliefweb.int/report/world/fifteen-years-after-landmine-ban-number-new-casualties-halves.
8. See examples of Christian Aid’s press releases on tax avoidance, such as http://www.christianaid.org.uk/pressoffice/pressreleases/december-2012/government-must-follow-aid-pledge-with-action-on-tax-and-environment.aspx, http://www.christianaid.org.uk/pressoffice/pressreleases/december-2012/uk-government-must-do-more-to-tackle-financial-secrecy.aspx and http://www.christianaid.org.uk/pressoffice/pressreleases/february-2013/help-combat-tax-avoidance-across-the-world-christian-aid-urges-uk-government.aspx.
9. On a DFID webpage dated 6 December 2012, DFID describes its response to 46 different civil society campaigns, of which 25 were led by PPA recipients: http://webarchive.nationalarchives.gov.uk/20130128103201/http://www.dfid.gov.uk/get-involved/your-campaigns/.
10. DFID informed us that 18 staff, including the head and deputy head of CSD, are based in East Kilbride. Four staff are permanently based in London.
11. CSOs have usually been funded by DFID through an ‘accountable grant’, i.e. a grant for a particular project or output for which the recipient must account to DFID.
12. The accounting requirements of the Charities Act 1993 and subsequent guidance require UK registered charities to identify funds they receive according to the terms under which they were given. See http://www.fit4funding.org.uk/support-pages/when-you-receive-funding/restricted-funding/ for a simple explanation from The Charities Information Bureau.
13. Better Aid: Civil Society and Aid Effectiveness – Findings Recommendations and Good Practice, OECD, 2009, http://books.google.co.uk/books?id=7VCMSWD1xzkC&lpg=PP1&dq=editions%3Avci1fYqf9_AC&pg=PP1#v=onepage&q&f=false.
14. These cover the themes of humanitarian; resilience; conflict and fragile states; and security and justice.
15. Funding figures shown are the annual average over the three years for each organisation. Actual amounts may vary once DFID makes performance-based adjustments. As a result, the total slightly exceeds DFID’s total plan of £120 million. Our case study organisations are shaded grey.
16. Department for International Development: Working with Non-Governmental and other Civil Society Organisations to promote development, The Comptroller and Auditor General, July 2006, http://www.nao.org.uk/wp-content/uploads/2006/07/05061311.pdf.
17. DFID’s use of contractors to deliver aid programmes, ICAI, May 2013, http://icai.independent.gov.uk/publications/icai-reports/.
18. Operational Plan 2011-2015 Civil Society Department, DFID, May 2012, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/67420/csd-2011.pdf.
19. Department for International Development: Working with Non-Governmental and other Civil Society Organisations to promote development, The Comptroller and Auditor General, July 2006, http://www.nao.org.uk/wp-content/uploads/2006/07/05061311.pdf.
20. Better Aid: Civil Society and Aid Effectiveness – Findings Recommendations and Good Practice, OECD, 2009, http://books.google.co.uk/books?id=7VCMSWD1xzkC&lpg=PP1&dq=editions%3Avci1fYqf9_AC&pg=PP1#v=onepage&q&f=false.
21. Department for International Development: Working with Non-Governmental and other Civil Society Organisations to promote development, The Comptroller and Auditor General, July 2006, http://www.nao.org.uk/wp-content/uploads/2006/07/05061311.pdf.
22. Department for International Development: Working with Non-Governmental and other Civil Society Organisations to promote development, Comptroller and Auditor General, July 2006, http://www.nao.org.uk/wp-content/uploads/2006/07/05061311.pdf.
23. We note that the PPA Learning Group has prepared a paper that uses the current round of PPAs to provide a case for the provision of unrestricted funding.
24. DFID’s Results Framework: Managing and reporting DFID results, DFID, 2010, http://www.dfid.gov.uk/Documents/publications1/DFID-external-results.pdf.
25. A ‘theory of change’ is a logical model setting out how a planned intervention will deliver its intended impact. Evaluations usually test whether this logic has proved to be sound.
26. The NAO recommended ‘making greater use of competition by specifying the changes [DFID] wants to engender, and letting CSOs bid for associated support. Where full competition is not possible, DFID should carry out appraisals of project costs aimed at securing economy.’ Department for International Development: Working with Non-Governmental and other Civil Society Organisations to promote development, The Comptroller and Auditor General, July 2006, page 7, http://www.nao.org.uk/wp-content/uploads/2006/07/05061311.pdf.
27. DFID used the term ‘niche’ to refer to elements such as agriculture, appropriate technology, children, disability, the elderly, faith groups, gender, HIV/AIDS, malaria, nutrition, reproductive health, trade, transparency, volunteering, water and sanitation, youth, co-ordination and cross-cutting areas. This list was determined after the applications had been received. Some of these correspond to the DFID bilateral and multilateral result areas.
28. Logical frameworks are DFID’s standard performance management tool. See How to Note – Guidance on using the revised logical framework, DFID, January 2011, http://www.dfid.gov.uk/Documents/publications1/how-to-guid-rev-log-fmwk.pdf.
29. Operational Plan 2011-2015 Civil Society Department, DFID, May 2012, page 4, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/67420/csd-2011.pdf.
30. The Hyogo framework seeks to reduce the impact of disasters by improving preparation and co-ordination: http://www.unisdr.org/we/coordinate/hfa.
31. Reducing Emissions from Deforestation and Forest Degradation (REDD) is a UN-led process to create a financial value for the carbon stored in forests. It offers incentives for developing countries to reduce emissions from forested lands and invest in low-carbon paths to sustainable development. ‘REDD+’ goes beyond deforestation and forest degradation and includes the role of conservation, sustainable management of forests and enhancement of forest carbon stocks. See http://www.un-redd.org.
32. The model was borrowed from that used in the Multilateral Aid Review (MAR), although with variations. The MAR was used to make changes to existing allocations, whereas this process was used to make complete allocations.
33. Hansard, Column 301, Q2 [29132] et seq., 8 December 2010, http://www.publications.parliament.uk/pa/cm201011/cmhansrd/cm101208/debtext/101208-0001.htm. It should be noted that DFID made the decision to remove VSO on 3 December.
34. Bond is a membership body for UK NGOs working in development: http://www.bond.org.uk.
35. DFID generally complies with the Official Journal of the European Union (OJEU) procurement guidelines, which sets out key timings for notification of procurement. The UK Government is currently negotiating with the EU for minimum timescales to be reduced to 35 days for open procurement.
36. For more details of our findings on KPMG’s contract with DFID to perform these due diligence assessments, see: DFID’s Use of Contractors to Deliver Programmes, ICAI, May 2013, http://icai.independent.gov.uk/publications/icai-reports/.
37. PPA Programme Managers are responsible for checking the Statutory Accounts using a standard DFID checklist. The Programme Manager, supported by the DFID Accountant responsible for CSD, is responsible for following up areas of concern with the recipient organisation. DFID has not ceased funding any PPA organisation on the basis of its Statutory Accounts.
38. Organisations can choose to treat their funds as restricted if this is implied by conditions of the funder.
39. The Mid-Term Review captures different dimensions: relevance; efficiency; effectiveness; sustainability and results.
40. DFID sees innovation as part of the ‘assumptions and risks’ section of the annual reviews. Given that innovation was one of the four key objectives used to justify this funding (see paragraph 2.11 on page 8), we would have expected this to have a higher and more prominent priority.
41. This included for Madagascar vanilla, Rajasthan sandstone and the garment industry in Tamil Nadu.
42. The International Aid Transparency Initiative standard for publishing information on aid flows and activities seeks to enable comparisons across donors: http://iatistandard.org.
43. The standard aims to make humanitarian interventions accountable to beneficiaries: http://www.hapinternational.org/about.aspx.
44. Intended results, targets and achievements for each organisation have been extracted from their logframes and latest Annual Reports (2011-12). These results have been verified through the 2012 Independent Performance Reviews (IPRs). We have not independently verified them.
45. The Independent Progress Review of ActionAid’s PPA concluded, however, that ‘results are likely to be achieved within the timeframe of the PPA’.
46. These are the intended results for the entire period of funding. For each year of a PPA, targets are set which seek to achieve these results. Only the first year’s results have been reported.
47. The IPRs alone have generated more than 5,300 pages of reporting on the first year of operation of the PPAs. With the Annual Reviews and Mid-Term Review, the total climbs to well over 6,000 pages.
48. Department for International Development: Working with Non-Governmental and other Civil Society Organisations to promote development, Comptroller and Auditor General, July 2006, page 5 (paragraphs 14-16), http://www.nao.org.uk/wp-content/uploads/2006/07/05061311.pdf
49. Department for International Development: Working with Non-Governmental and other Civil Society Organisations to Promote Development, The Comptroller and Auditor General, July 2006, page 5 (paragraph 16), http://www.nao.org.uk/wp-content/uploads/2006/07/05061311.pdf.
50. The Mid-Term Review set out the performance of all PPA recipients comparatively. This was unhelpful.

Abbreviations