DFID’s Use of Contractors to Deliver Programmes

1. Introduction

1.1 The Independent Commission for Aid Impact (ICAI) is the independent body responsible for scrutinising UK aid. We focus on maximising the effectiveness of the UK aid budget for intended beneficiaries and on delivering value for money for UK taxpayers. We carry out independent reviews of aid programmes and of issues affecting the delivery of UK aid. We publish transparent, impartial and objective reports to provide evidence and clear recommendations to support UK Government decision-making and to strengthen the accountability of the aid programme. Our reports are written to be accessible to a general readership and we use a simple ‘traffic light’ system to report our judgement on each programme or topic we review.

1.2 We wish to assess whether the Department for International Development’s (DFID’s) use of contractors to deliver programmes is providing impact and value for money. These Terms of Reference outline the purpose and nature of the evaluation and identify the main themes that it will investigate. A detailed methodology for the evaluation will be developed during an inception phase.

2. Background

2.1 DFID uses various terms to describe its expenditure on suppliers of different kinds of services, including:

• technical co-operation;
• technical assistance (a subset of technical co-operation); and
• consultancy.

Technical co-operation – essentially sourcing advice and skills – is the largest category, accounting for annual expenditure of approximately £468 million. Technical assistance is categorised as part of technical co-operation but is focussed on the direct provision of support to recipient governments. DFID defines ‘consultancy’ using a narrow definition of support carried out using its administration budget and accounting for less than £5 million of annual expenditure. All technical co-operation and technical assistance programmes are paid for through DFID’s programme budget.

2.2 The type of expenditure we wish to examine in this report – DFID’s use of contractors to deliver programmes – does not match any one of these categories exactly, although it is part of the technical co-operation portfolio. DFID does not capture data in a way which matches our chosen scope. In putting together these Terms of Reference, therefore, we have decided to use the technical co-operation category as a guide to the scale of expenditure. We have drawn this information from some National Audit Office (NAO) 2011 analysis of DFID’s technical co-operation expenditure as part of an IDC briefing.¹ As we undertake our own report, we will build a separate data set to cover the expenditure in which we are interested.

2.3 Technical co-operation is ‘the provision of advice and/or skills, in the form of specialist personnel, training and scholarship, grants for research and associated costs’.² DFID spent £468 million on technical co-operation from its bilateral aid budget in 2010-11.³ In recent evidence to the International Development Committee (IDC),⁴ DFID explained that it is essentially a commissioning organisation (unlike some other development organisations, such as the German Technical Co-operation Foundation, which employs 15,000 staff). While it has been employing more in-house experts in some areas, including public financial management, it is not able to employ cadres of staff to cover all areas of its work. Using technical co-operation also allows DFID to respond to evolving requests and needs over time.

2.4 Technical co-operation projects can vary from large projects delivered over several years to shorter-term engagement of individuals or small teams. A supplier contract can be with one lead organisation, a consortium of organisations or a self-employed individual. DFID procures services from suppliers to provide specialist skills in various areas, including to:

• ‘undertake activities to enhance the knowledge, intellectual skills, technical expertise or the productive capability of people in recipient countries;
• help deliver direct impacts on poor peoples’ lives through assisting the provision of services in developing countries;
• provide assistance in the form of training for persons from recipient countries, including through scholarship schemes;
• contribute to the design of some of the Department’s projects and programmes; and
• manage large strategic programmes on the Department’s behalf, for example the Global Poverty Action Fund worth £120 million’.¹

2.5 Excluding technical co-operation covered by memoranda of understanding (generally delivered for DFID by multilateral organisations), DFID spending in this area rose by 15% to £413 million in 2009-10, then remained static at £412 million in 2010-11 (around 10% of bilateral spending). This comprised £379 million of payments on contracts let through DFID’s central procurement team and a further £33 million on smaller-value contracts (under £100,000) which are let locally by country offices.

2.6 60% of the £412 million of spending in 2010-11 was spent in three sectors: government and state-building (30%), health (19%) and education (11%). Other areas include social services and infrastructure, research and humanitarian assistance. This spending is delivered in different ways by DFID:

• Country programme-specific: this accounts for 80% of the total. Five countries – Nigeria, Afghanistan, India, Nepal and Bangladesh – account for 57% of this spending, with Nigeria alone accounting for 28%. In Nigeria, 61% of DFID’s bilateral spending is technical co-operation, down from 82% in 2009-10. Figure 1 gives more detail on this expenditure. There are three other countries where technical co-operation makes up more than a quarter of the programme in 2010-11 – Afghanistan (31%), Sierra Leone (29%) and Nepal (27%);
• Regional spending: this makes up 5% of the total; and
• UK-based business unit spending: making up the remaining 15% of spending, this covers a range of activities, including those which benefit overseas programmes (such as the British Overseas Territories) and humanitarian assistance programmes.

2.7 The decision to use contractors of any kind is examined during DFID’s business case process. The main options available for sourcing technical co-operation services include:

• Competitive tendering processes: all tenders above the Official Journal of the European Union (OJEU) threshold (currently £113,057), must comply with the EU public procurement regulations. The awarding of these contracts is managed by DFID’s Procurement Group, working together with the relevant part of DFID. DFID awarded 117 of these contracts in 2010-11, with a total value of around £230 million. Below the OJEU threshold, the relevant part of DFID can run its own competitive process under its delegated authority;
• Framework Agreements: here, DFID can choose from pre-approved suppliers through the Government Procurement Service or from its own Framework Agreements, designed to support particular pillars of work; and
• Non-competitive tendering processes: these can be carried out for low-value contracts, typically below £25,000.

2.8 In 2010-11, DFID provided payments above £100,000 to 182 providers, although 63% of spending went to 20 of these, up eight percentage points on 2008-09.⁵ The top providers are set out in Figure 2.

2.9 In 2006, DFID produced guidance on How to Provide Technical Cooperation Personnel.¹¹ Donors collectively have agreed to certain principles in the provision of technical co-operation:

• a Paris Declaration (2005) commitment to ‘align their analytic and financial support with partners’ capacity development objectives and strategies, make effective use of existing capacities and harmonise support for capacity development accordingly’; and
• an Accra Agenda for Action (2008) commitment that ‘donors’ support for capacity development will be demand-driven and designed to support country ownership. To this end, developing countries and donors will i) jointly select and manage technical co-operation, and ii) promote the provision of technical co-operation by local and regional resources, including through South-South co-operation’.

3. Purpose of this evaluation

3.1 To assess to what extent DFID is achieving impact, value for money and sustainability in its use of contractors to deliver programmes.

4. Relationships to other evaluations/studies

Parliamentary committees

4.1 In December 2010, the Public Accounts Committee published a report entitled Central Government’s use of consultants and interims. Although we will not be including this type of expenditure in our work, the report has some findings which may also apply more generally to DFID’s use of contractors. The report concluded that:

‘Despite spending over £1 billion a year on consultants and interim staff, central government departments are largely in the dark about whether this represents value for money. There are of course legitimate reasons for a department to buy in specialist skills where they are in short supply internally. But departments have become too reliant on buying in core skills rather than developing them in their own staff.

‘What is unacceptable is the poor understanding of whether the extent of a department’s use of consultants is justified by the nature of its business.

‘It is a mark of departments’ poor understanding of spending on consultancy that some have reacted to cost pressures by cutting that spending in an uninformed way. This runs the risk that short-term savings could lead to increased costs and poor value for money for the taxpayer in the long term.’¹²

4.2 Regarding DFID in particular, IDC made the following recommendation for ICAI in February 2011:

‘The use of external suppliers to provide technical assistance fills an important skills gap. However, we are concerned that the use of such suppliers may affect the ability of developing countries to build up expertise. In addition, DFID needs to ensure that it is selecting the suppliers which are providing the best value for money and to examine whether, as a major purchaser of their services, DFID could do more to drive down fee rates. We will ask the new ICAI to examine whether external suppliers are providing value for money.’¹³

4.3 DFID’s management response to IDC’s report stated that:

‘DFID is continuing to strengthen its approach to procurement and the management of commercial aspects of project design in order to improve value for money from spending with suppliers on technical assistance. We encourage suppliers to use developing country sources on a sub-contract basis and frequently include specific commitments to this in technical assistance contracts. We are working to strengthen levels of commercial capability across the organisation.’¹⁴

4.4 In March 2012, IDC investigated DFID’s use of technical co-operation.¹⁵ DFID ‘informed us that it used a competitive system to drive down costs, but in some cases, the skills were in short supply and providers could charge a premium’. IDC made two recommendations on strengthening parliamentary capacity in recipient countries and another on technical co-operation in general:

‘The Department needs to improve its assessment of which types of projects and services it should use consultants for; in particular, DFID should assess more carefully the use of consultants to manage the Department’s own service delivery programmes. We also recommend that DFID identify expenditure on technical co-operation by purpose.’

ICAI’s consultation

4.5 In our consultation last year, when we asked the public which areas and countries they would like us to scrutinise, there were a number of comments requesting us to look into DFID’s use of consultants. Some of the areas highlighted were:

• the cost of consultants and the amount of profit they take in designing programmes;
• the salaries DFID pays for long-term consultants in developing country projects, including any associated distortion effects on national markets and economies; and
• the use of consultants within large programmes, including how risk issues impact on the cost of the contractor hired.

Evaluations

4.6 Various DFID evaluations draw conclusions on its technical co-operation, for example:

• in 2006, An Evaluation of DFID-Funded Technical Co-operation for Economic Management in Sub-Saharan Africa¹⁶ concluded that DFID support ‘has contributed in many cases towards the development of improved systems and procedures’ but ‘the realisation of this potential depends on a strengthening of government commitment to the use of these systems and procedures, as well as to measures to address constraints on the capacity of the organisations supported’. Therefore, ‘it is only in a minority of the cases reviewed that a capacity development impact can be identified’;
• DFID’s 2008 evaluation of its Sierra Leone country programme found that ‘DFID technical assistance is generally regarded as being of high quality, but over-use of gap-filling PIU [Project Independent Units] structures has drawn competent middle level staff out of government and limited the extent of increased capacity within GOSL [the Government of Sierra Leone]’;¹⁷ and
• DFID’s 2009 evaluation of its Afghanistan country programme found ‘the quality of technical assistance (TA) has been high, but there are drawbacks in terms of scope and sustainable results’. It also stated: ‘Technical Assistance (TA) does not automatically equate to capacity development, even if training is included. Without an explicit analysis of the incentives and disincentives for reform provided by TA, DFID has not been able to weigh up the value-added of TA compared to other aid instruments. Looking at the broader canvas of massive TA inputs by all donors – $1.6 billion since 2002 – the impact has been questioned.’¹⁸

4.7 In 2010, the Organisation for Economic Co-operation and Development Development Assistance Committee (DAC) published a peer review of the UK’s development assistance.¹⁹ It concluded that DFID is ‘relatively strategic’ in its approach to capacity development and manages technical co-operation flexibly in accordance with key donor principles. It also found that ‘DFID does however face challenges at the individual programme or project level, where the design of interventions is not systematically grounded in a robust approach to capacity assessment’. It recommended that DFID should ‘improve internal communication and guidance on capacity development; strengthen capacity assessments in the development of projects and programmes; and implement its commitment to support the development of capacities of non-state actors’.

5. Analytical Approach

5.1 This review will focus on DFID’s use of contractors through bilateral, regional and central programming. It will seek to identify whether DFID is achieving impact and value for money in its use of these contractors, using the key questions set out below. It will not cover expenditure on research or technical co-operation delivered through multilateral organisations.

5.2 We will use a range of case studies to assess how DFID determines the need to use contractors, selects them, negotiates terms and oversees their work. We will determine what added value contractors bring that DFID cannot itself provide and whether and, if so, how this contributes to wider development outcomes and to sustainability through the build-up of local expertise.

6. Indicative Questions

6.1 This review will use as its basis the standard ICAI guiding criteria and evaluation framework, which are focussed on four areas: objectives, delivery, impact and learning. The questions outlined below comprise those questions in our standard evaluation framework which are of particular interest in this review, as well as other pertinent questions we want to investigate. The full list of questions that we will consider in this review will be finalised during the inception phase.

6.2 Objectives

6.2.1 Does the use of contractors have clear, relevant and realistic objectives that focus on the desired impact? Is there clarity as to the specific objectives of the work, within the overall objectives?

6.2.2 Is there a clear and convincing plan, with evidence and assumptions, to show how the contractors’ involvement will work?

6.2.3 Does DFID use cost-benefit analysis to inform its plans to use contractors and, if so, what evidence does that provide?

6.3 Delivery

6.3.1 How do DFID’s procurement procedures ensure that the best contractors are being selected? How does this work in practice?

6.3.2 How explicitly does the use of contractors fit into the planned and actual delivery chains? What criteria does DFID apply to make these choices?

6.3.3 How does DFID ensure that it is getting value for money from contractors? How does DFID ensure an appropriate balance between price and quality?

6.3.4 Are contractors providing additional value that could not otherwise be achieved?

6.3.5 Are contractors delivering against their agreed objectives?

6.3.6 Are risks to the achievement of the objectives identified and managed effectively?

6.3.7 Does DFID have the expertise to manage contractors effectively and, if so, how well does it use this expertise?

6.4 Impact

6.4.1 How does DFID ensure that its contractors are delivering impact?

6.4.2 Is the use of contractors having a positive impact on development outcomes more broadly in the recipient country or region?

6.4.3 Is there evidence of long-term, sustainable impact?

6.4.4 Is there evidence of contractors helping to build capacity in the recipient country, thereby contributing to future aid independence?

6.5 Learning

6.5.1 Does DFID have appropriate procedures in place for monitoring the use of contractors?

6.5.2 How does evaluation of performance take place and how are the findings fed back into subsequent procurement processes?

6.5.3 Are lessons about using contractors effectively being learned within programmes and shared across DFID?

7. Outline Methodology

7.1 Given that we are investigating DFID’s use of contractors, a conflict of interest would arise if a consultancy firm that has provided services to DFID carried out the review. Of the organisations in our consortium, KPMG, Agulhas Applied Knowledge and the Swedish Institute for Public Administration (SIPU International) have all recently provided services to DFID; and the Center for Effective Global Action (CEGA), given its specialisms of evaluation techniques and economic analyses, does not carry out this type of work.

7.2 To manage this conflict of interest, therefore, we are looking to commission this work from another party, which has not been in receipt of UK Official Development Assistance (ODA) for at least the past five years.

7.3 Our review will have a number of elements, based around several case studies where DFID has used technical co-operation. Rather than reviewing all instances of technical co-operation at a high level, we will adopt a sampling approach so as to focus on a range of different:

• country and regional spending, including Nigeria, given the extent of DFID’s use of contractors there;
• contractors;
• sectors of spending; and
• sizes of contracts.

7.4 In some cases, we may decide to examine contractors involved in programmes that we have already examined, making use of our earlier findings.

7.5 We will carry out an overall literature review and interviews with DFID staff, experts and stakeholders in the UK. We may supplement this with a survey of DFID staff and other stakeholders about their experience of being involved with DFID-procured contractors. In addition, each case study will involve:

a) a review of evidence from DFID’s files and information systems, including financial information;

b) interviews with DFID staff and the service provider; and

c) depending on the programme, interviews with UK-based and international experts and the intended recipients of the contractor support.

7.6 One of the case studies will involve a visit to the country where the contracted work is being carried out.

8. Timing and Deliverables

8.1 The review will be overseen by Commissioners and implemented by the successful bidder to our open competition. The review will involve initial meetings with ICAI Commissioners and the Secretariat to agree the detailed scope. Once the fieldwork has been completed, the review will involve:

• meeting with Commissioners to present the initial findings of the review;
• producing an accessible draft report of the findings which is no more than 20 pages in length (with additional annexes if required);
• redrafting the report in response to Commissioner and Secretariat comments;
• making final changes to the report in response to the DFID fact-checking process; and
• providing advice after publication for an International Development Committee (IDC) hearing.

8.2 The review will take place during the third and fourth quarters of 2012, with a final report available by the first quarter of 2013.

9. Contact details

9.1 The named ICAI contact for this procurement exercise is:

Footnotes

1. Briefing to support the International Development Committee’s inquiry into DfID’s Annual Report and Accounts 2010-11 and Business Plan 2011-15, National Audit Office, October 2011, www.nao.org.uk//idoc.ashx?docId=6c168fd3-927b-4ec3-8999-bcac78fb5b04&version=-1.
2. DFID Annual Report and Accounts 2010-11 Volume 1, DFID, July 2011, www.dfid.gov.uk/Documents/publications1/departmental-report/2011/Annual-report-2011-vol1.pdf.
3. Statistics on International Development 2006/07 – 2010/11, DFID, October 2011, www.dfid.gov.uk/documents/publications1/sid2011/SID-2011.pdf.
4. Department for International Development Annual Report and Resource Accounts 2010-11 and Business Plan 2011-15, International Development Committee, March 2012, http://www.publications.parliament.uk/pa/cm201012/cmselect/cmintdev/1569/156908.htm.
5. Briefing to support the International Development Committee’s inquiry into DfID’s Annual Report and Accounts 2010-11 and Business Plan 2011-15, National Audit Office, October 2011, www.nao.org.uk//idoc.ashx?docId=6c168fd3-927b-4ec3-8999-bcac78fb5b04&version=-1.
6. http://www.adamsmithinternational.com/
7. www.crownagents.com
8. www.grminternational.com
9. www.britishcouncil.org
10. www.camb-ed.com
11. How To Note: How to provide technical cooperation personnel, DFID, June 2006, http://webarchive.nationalarchives.gov.uk/+/http://www.dfid.gov.uk/mdg/aid-effectiveness/tc-how-to.pdf.
12. Central Government’s use of consultants and interims, Public Accounts Committee, December 2010, www.publications.parliament.uk/pa/cm201011/cmselect/cmpubacc/610/610.pdf.
13. Department for International Development Annual Report & Resource Accounts 2009-10, International Development Committee, February 2011, http://www.publications.parliament.uk/pa/cm201011/cmselect/cmintdev/605/60510.htm.
14. Department for International Development Annual Report & Resource Accounts 2009–10: Government Response to the Committee’s Third Report of Session 2010–11, International Development Committee, May 2011, www.publications.parliament.uk/pa/cm201012/cmselect/cmintdev/1043/1043.pdf.
15. Department for International Development Annual Report and Resource Accounts 2010-11 and Business Plan 2011-15, International Development Committee, March 2012, http://www.publications.parliament.uk/pa/cm201012/cmselect/cmintdev/1569/156908.htm.
16. An Evaluation of DFID-Funded Technical Co-operation for Economic Management in Sub-Saharan Africa, DFID, June 2006, www.dfid.gov.uk/aboutdfid/performance/files/ev667.pdf.
17. Evaluation of DFID Country Programmes: Sierra Leone, DFID, September 2008, www.dfid.gov.uk/aboutdfid/performance/files/ev690.pdf.
18. Country Programme Evaluation Afghanistan, DFID, May 2009, www.dfid.gov.uk/Documents/publications1/evaluation/afghan_eval.pdf.
19. The United Kingdom: Development Assistance Committee (DAC) Peer Review, OECD DAC, 2010, www.oecd.org/dataoecd/49/20/45519815.pdf.