Achieving value for money through procurement

1. Purpose, scope and rationale

The UK Aid Strategy states: “We will ensure that every penny of money delivers value for taxpayers.”¹ DFID spent £1.4 billion through suppliers in 2016-17, or 14% of its total budget.² Over the same period, DFID awarded nearly 700 contracts to 170 unique suppliers.³ Procurement is therefore an important aspect of achieving value for money. In recent years, it has emerged as a subject of particular concern to both parliament and the public. The former International Development Secretary, Priti Patel, recently announced the results of a major review of DFID’s procurement practices, including a new code of conduct for suppliers.⁴

This review is the second of two undertaken by the Independent Commission for Aid Impact (ICAI) of DFID’s approach to the procurement of goods, works and services for the bilateral aid programme. The first review concerned DFID’s approach to its supplier market and was published in November 2017. This second review assesses whether DFID has maximised value for money from suppliers through its tendering and contract management practices.⁵ This review is intended to complement another ICAI review on DFID’s approach to value for money in programme and portfolio management, to be published in 2018.

This will be a performance review.⁶ It will provide parliament and the public with an assessment of whether DFID makes appropriate use of competitive procurement and whether its tendering and contract management practices secure quality programme delivery at competitive prices. It will assess whether DFID has adequate controls in place against uncompetitive practices and unethical behaviour.⁷

The scope of this review will be procurement within DFID’s bilateral aid programme over the period 2012-13 to 2016-17. It will include procurement of any goods, works or services in relation to the delivery of aid programmes, both above and below the European Union (EU) threshold (see Box 1).⁸ It will cover the full range of procurement and contract management practices, from defining supply need, sourcing strategies and delivery options throughout the contract award phase, to oversight and monitoring of suppliers and contract compliance. It will explore how well DFID captures and applies lessons on procurement. It will also consider DFID’s choices regarding whether or not to source through competitive procurement. It will not cover funding given to multilateral organisations, accountable grants, payments to partner governments or the procurement of goods and services for DFID’s administrative use.

Our overarching goal for the review is to establish whether DFID uses ethical and competitive procurement to ensure effective programme delivery at the best price.

2. Background

The amount of DFID’s total expenditure through contracts has increased in recent years, both in absolute terms and relative to DFID’s overall spend. In 2012-13, contracts represented 9% (£0.7 billion) of DFID’s expenditure. By 2016-17, this had increased to 14% (£1.4 billion).

This increase has coincided with heightened media interest in DFID’s use of suppliers, including alleged unethical practices by one of DFID’s top suppliers that have been investigated by the International Development Committee.⁹

DFID works within a legal and policy framework common to UK government departments (see Box 1). Procurement of contracts above the EU threshold (over which specific rules apply) is managed by DFID’s Procurement and Commercial Department, which also provides commercial advice and support to programme teams during business case development and the subsequent management of contracts. Day-to-day contract management activity is the responsibility of programme teams in country offices and central spending departments, supported by finance managers and commercial delivery managers. The Procurement and Commercial Department has articulated a ‘vision’ which focuses on three main themes: supporting DFID programme delivery through a first class commercial and procurement service within DFID; maximising and shaping markets; and strengthening its commercial influence and impact on the wider sector.¹⁰

3. Review questions

The review will be built around the evaluation criteria of relevance and effectiveness. It will address the questions and sub-questions in Table 1.

Table 1: Our review questions

4. Methodology

Our methodology builds and expands on the work undertaken by ICAI in the first procurement review. It will consist of four mutually reinforcing components designed to generate a holistic picture of DFID’s procurement practice:

• Literature review: an analysis of UK government rules, and other commonly used guidance and best practices from across government and the international development sphere.
• Strategic review: an assessment of DFID’s procurement policies, strategies, systems and processes across the whole procurement cycle, benchmarking these against the requirements and best practices identified in the literature review. This will include analysis of data from DFID’s management information systems to explore issues such as timeliness and performance against key performance indicators.
• Desk reviews of contracts: reviews of a purposive sample of DFID contracts, which feature different contract types and market conditions, to identify strengths and weaknesses in DFID’s procurement practice.
• Country case studies: visits to two DFID country offices to review procurement practices at the country level. Here we will also follow a subset of our sample contracts from tender through to contract delivery to assess how well any performance issues are identified and dealt with.

Component 1 – Literature review: A literature review carried out for the first procurement review analysed relevant UK government rules and guidance on procurement, and surveyed the field of supplier market shaping. Building on the material collected, we will analyse the literature for best practices in the aspects of tendering and contract management. Our methodology will include a review of available literature, including academic and other research reports, to identify current trends and good practices in procurement, tendering and contract management. This will include guidelines from the UK and Scottish governments, the Chartered Institute of Procurement and Supply, the Organisation for Economic Cooperation and Development and other sources. We will draw on insights into best practice from the private sector and from other international donors. We will also analyse previous ICAI, International Development Committee, National Audit Office and DFID reports for findings related to procurement, tendering and contract management. This will support our analysis of whether DFID is complying with UK government requirements and guidance, and whether it is integrating current best practice into its approach to tendering and contract management.

We will also explore the literature on payment by results in international development, to assess the evidence on whether, and in what conditions, it creates positive incentives for suppliers to maximise value for money. This will enable us to assess whether DFID’s use of payment by results contracting reflects the evidence on what works.

Component 2 – Strategic review: This component will explore DFID’s evolving approach to procurement by reviewing its key policies, strategies and guidance, and consulting with key DFID stakeholders at headquarters level as well as private sector stakeholders. The review will assess DFID’s tendering and contract management processes and procedures to consider their relevance and effectiveness, benchmarking them as appropriate against the principles of good practice identified in the literature review. The review will undertake a light-touch assessment of the following areas:

• The appropriateness of DFID’s procurement policies and strategies, and how well they support the objectives and priorities of the aid programme.
• The division of procurement and procurement-related functions and responsibilities, the number and qualifications of staff involved, and the results of DFID’s efforts in recent years to build up its commercial capability.
• Whether DFID’s systems and technology support integration across the organisation, reduce manual effort, improve the user experience, increase levels of compliance and create a ‘single source of truth’.
• The major changes in procurement policies and practices over the review period, and the extent to which DFID’s approach to procurement has been informed by previous reviews and reports, both internal and external.
• How well the commercial vision, strategy and practice meets DFID’s strategic objectives and have kept pace with the scaling up of the aid programme.
• How well DFID captures learning from its own activities and lessons from wider best practice, including other major international donors, the public and the private sector, and adapts its processes in response.
• How DFID identifies examples of poor value for money in procurement and takes corrective action.
• Early and predicted effects of the changes following DFID’s recent supplier review on its approach to procurement.

During our first procurement review, we obtained a selection of procurement data from DFID’s management information systems and carried out a range of financial and data analyses to identify patterns and trends in the supplier market, particularly around market diversity and concentration. For this review, we will build on this analysis with any additional data available over the lifetime of this review. We will also carry out further analysis, including examining the quality of DFID’s market segmentation work, its use of different tendering mechanisms, its tender cycle times and its performance metrics.

The strategic review will involve an examination of DFID documents, analysis of data from its management information systems, and interviews with key internal and external stakeholders. The stakeholder consultations will include:

• meetings with DFID personnel from the Procurement and Commercial Department, commercial delivery managers in spending departments, the Better Delivery Department, programme managers and other DFID management staff
• stakeholder workshops in the UK with the private sector and international non-governmental organisations (NGOs), both at the start and end of the evidence-gathering process
• individual interviews with a selection of existing and potential DFID suppliers, including at least three each of i) key suppliers ii) small and micro suppliers iii) ‘other’ suppliers and iv) unsuccessful bidders.

Consultations will be conducted in person (including workshops, roundtables or forum groups) or by telephone.

Component 3 – Desk reviews of contracts: We will conduct desk reviews of a core sample of 30-40 contracts (see below for the sampling strategy). The sample will cover the full range of contract types and market segments. Using a bespoke assessment tool, the desk reviews will follow the full procurement cycle, from needs identification and options appraisal through the tender process, letting of contracts, contract management and performance assessment. Each desk review will involve an examination of pre-tender, tender and contract documents, supplemented by interviews with programme teams, commercial delivery managers and, if appropriate, suppliers. We will identify any strengths and weaknesses at particular points in the procurement cycle and in relation to particular types of contracts.

The sampling process will involve checking a wider range of contracts in order to identify those most suitable for review. If it is not clear from our sample whether our findings are generalisable, we may carry out additional light-touch reviews of other contracts, to check for patterns and trends.

Component 4 – Country case studies: We will undertake visits to two countries, Nigeria and Tanzania (see section 5 for the rationale for choosing those two countries). The country case studies will include:

• a review of DFID procurement practices at country level, to extend the analysis conducted in the strategic review
• an assessment of the commercial and technical capacity and capability available in the two country offices, and staff understanding of good procurement practices
• a detailed review of the contracts in our desk review sample that relate to the two countries.

The detailed contract reviews undertaken in the case study countries will enable us to test the effectiveness and efficiency of contract delivery and performance at a greater level of depth than is possible from a desk review. Choosing specific contracts based on analysis of their risk levels, we will track contract supply chains from tender through first-tier supplier and sub-contractors to the point of delivery.

In addition to the contract documents collected through the desk reviews, the country case studies will involve in-depth interviews with DFID country office heads and staff examining their overall approach to procurement at programme and portfolio level. This will be supplemented by interviews in-country with DFID country office staff, implementing partners and other relevant stakeholders.

We will assess the quality of country commercial strategies, any associated guidance and tools, and how they are being implemented in practice. We will explore country office experience with the introduction of payment by results contracting and with other payment mechanisms, and assess the effectiveness of other procurement reforms.

5. Sampling approach

We have used a stratified sampling approach²⁰ to select the countries and contracts that we will look at in more detail. First, we identified seven contract categories to review (outlined below). From this, we identified the countries with the most contracts matching these criteria. Segmenting the sample in this way allows for greater representation of spend than choosing contracts at random.

Country selection

Based on DFID’s current contract data, we categorised all open and completed contracts between 2012-13 and 2016-17 according to the following criteria:²¹

1. High value: the top 30% of contracts by value
2. Medium value: the middle 40% of contracts by value
3. Kraljic²² strategic: procurements with a high risk of dependency on a small number of suppliers
4. Kraljic bottleneck: limited supply and risk of exposure to price increases or supply disruption
5. Key suppliers:²³ contracts awarded to key suppliers (accounting for approximately 40% of DFID’s contactor spend)
6. Mid-tier suppliers: suppliers accounting for the next 30% of DFID’s contractor spend
7. High and medium country spend: countries in the top 30% and middle 40% of expenditure through procurement

Using this analysis, we identified the top eight countries enabling the greatest coverage of the contracts in each category: Nigeria, Democratic Republic of Congo, Sierra Leone, Ethiopia, Pakistan, Tanzania, Nepal and Kenya. Our contract sample selection (see below) was based on these eight countries to enable us to build a picture of procurement practices in a range of countries and to identify potential trends, weaknesses or good practice.

In addition, we identified the two country offices that, combined, provided the best coverage of all seven criteria for our field visits:

Nigeria – a large programme in the West Africa region with a range of contracts, suppliers and contract types. Nigeria has faced issues in tackling corruption and increasing transparency.

Tanzania – a mid-sized programme in East Africa which has made significant progress in tackling poverty in recent years.

Together these countries accounted for 15% of DFID’s total expenditure through suppliers in 2015-16.

Contract selection

We selected our sample of aid delivery contracts from the eight countries (including the two case study countries) identified in the country selection process. This left us with 450 contracts from which we will select a stratified random sample of 30-40 contracts to include:

• approximately 60% from the two countries selected for case studies, so we can follow them in detail in our field visits, including contracts above and below the EU threshold to cover a range of contract sizes
• 40% spread across the remaining six countries to give us coverage across a range of country contexts, comprising only of contracts above the EU threshold, prioritising on larger contracts.

We will cross-check these to ensure we have a suitably wide range of contract types and payment mechanisms (e.g. payment by results).

In addition to the core sample of 30-40 contracts selected, we will add contracts for at least two centrally-managed programmes (to be identified following initial interviews with DFID on commercial challenges faced by central spending departments). We may also follow-up on additional contracts during the country visits if we become aware of any specific issues relevant to the review.

This sample size is approximately 5-6% of the total number of contracts awarded in the five years to 2016-17. We consider this sample size to be a reasonable balance between the cost and strength of evidence needed to give us useful insights across the range of procurement contexts and challenges that DFID faces.

6. Limitations to the methodology

DFID’s procurement and commercial operation is continuously developing and new policies, processes or procedures are likely to be launched during the review period, particularly following the conclusion of DFID’s supplier review. Our review will take account of these new developments as evidence of learning, and they will be a factor in our assessment of the relevance of DFID’s procurement strategy and approach. However, our assessment of effectiveness will involve looking at a sample of contracts over a period of five years and will therefore necessarily be backward looking. We will take account of whether issues emerging from past contracts have been correctly identified by DFID and addressed in recent reforms.

Our sample size and selection aims to give us insights into the effectiveness of procurement procedures across a range of DFID countries and contracts. It is not, however, fully substantive or statistically representative and therefore some sectors and contract types are likely to receive greater attention than others.

7. Risk management

8. Quality assurance

The review will be carried out under the guidance of ICAI lead commissioner Tina Fahm; the peer commissioner will be ICAI chief commissioner Dr Alison Evans; and the ICAI secretariat will provide support throughout the process. The review will be subject to quality assurance by the service provider consortium. Both the methodology and the final report will be peer reviewed by Annamaria La Chimia (associate professor, School of Law, University of Nottingham) and Chris Bedford (former director at the National Audit Office).

9. Timing and deliverables

The review will be completed within a year, starting from early September 2017.

Footnotes

1. UK aid: tackling global challenges in the national interest, HM Treasury and DFID, November 2015, p. 4, link. (hereafter: Aid Strategy).
2. Calculated from internal procurement data provided by DFID.
3. DFID’s approach to its supplier market, ICAI, 2017, not publicly available.
4. DFID Guidance: DFID’s Supplier Review, DFID and The Rt Hon Priti Patel MP, 3 October 2017, link.
5. Note that we will focus on aspects of contract management that relate to decisions made during the procurement process and compliance with contractual obligations and not wider aspects of programme management or effectiveness of delivery.
6. The different types of reviews are available on the ICAI website, link.
7. As described in DFID Supply Partner Code of Conduct, link.
8. EU Procurement Thresholds, Official Journal of the European Union, link.
9. Conduct of Adam Smith International, Seventh Special Report of Session 2016-17, International Development Committee, 12 February 2017, link.
10. Commercial Capability Blueprint, version 3.0, DFID, February 2017, not publicly available.
11. EU Procurement Thresholds, Official Journal of the European Union, link.
12. The Public Contracts Regulations 2015, Cabinet Office, February 2015, link.
13. Of which the most relevant is Directive 2014/24/EU on Public Procurement, February 2014, link.
14. EU procurement directives and the UK regulations, Cabinet Office and Crown Commercial Services, 1 March 2017 update, link.
15. Managing public money, HM Treasury, August 2015 update, link.
16. The Green Book: Appraisal and Evaluation in Central Government, HM Treasury, July 2011 edition, link.
17. Smart Rules: Better Programme Delivery, DFID, 1 October 2017 update, sections 2.4 and 4.3, link.
18. DFID’s Procurement and Commercial Vision, DFID, April 2016, link.
19. DFID Supply Partner Code of Conduct, DFID, October 2017, link.
20. A stratified sample is a sample drawn from a number of different segments of a population, rather than at random from the whole population. This allows for greater representation.
21. Note that these categories are not intended to be mutually exclusive as inclusion in more than one increases the likelihood of selection.
22. The Kraljic matrix is an influential tool in procurement that allows managers to segment contracts according to different criteria. For more information, see link.
23. DFID has established a Key Supplier Management Programme where selected major suppliers receive focused management by the Department across their portfolio of contracts. Suppliers on this programme are referred to by DFID and in this document as ‘key suppliers’.