Investigation into DFID’s Programme Controls and Assurance in Afghanistan
1. Introduction
1.1 The Independent Commission for Aid Impact (ICAI) is the independent body responsible for scrutinising UK aid. We focus on maximising the effectiveness of the UK aid budget for intended beneficiaries and on delivering value for money for UK taxpayers. We carry out independent reviews of aid programmes and of issues affecting the delivery of UK aid. We publish transparent, impartial and objective reports to provide evidence and clear recommendations to support UK Government decision-making and to strengthen the accountability of the aid programme. Our reports are written to be accessible to a general readership and we use a simple ‘traffic light’ system to report our judgement on each programme or topic we review.
1.2 We wish to assess the effectiveness of the steps taken by the Department for International Development (DFID) to help ensure that aid monies to Afghanistan reach the intended beneficiaries and are used for their intended purpose. This is an intentionally specific remit. We expect that Afghanistan will remain a priority for us in the future. The results from this investigation are likely to inform the scale and scope of future ICAI reviews on the Afghanistan programme.
1.3 These Terms of Reference outline the purpose and nature of the study and identify the main themes of the work. A detailed methodology for the study will be developed during an inception phase.
2. Background
2.1 UK efforts in Afghanistan are co-ordinated by the Foreign and Commonwealth Office (FCO), DFID and the Ministry of Defence (MOD). DFID figures from its 2009 Afghanistan programme evaluation show that in the last decade, the UK was the third largest development aid donor to the country, contributing about 10% of total Official Development Assistance (ODA) since 2003, just over £100 million per year since 2004-05. The majority of funding has been through central government, notably the Afghan Reconstruction Trust Fund (ARTF), which by early 2008 accounted for 47% of DFID’s Afghanistan budget.1 DFID plans that, between 2011-12 and 2014-15, total annual expenditure will be £178 million, to be spent in support of four strategic priorities: governance and security, education, wealth creation and humanitarian assistance.2
2.2 Afghanistan provides a challenging context for DFID’s work, specifically in managing risk and ensuring probity. DFID’s Operational Plan for Afghanistan 2011-15 states that ‘we have a robust system of checks and balances in place to ensure UK aid money helps only those it is intended to’.2 DFID’s country programme evaluation noted in 2009, however, that reliable data and evidence about the use of aid monies and the resulting outcomes in Afghanistan have often been scarce.1
2.3 Given the public interest in the UK’s engagement with Afghanistan – it was the top priority country of respondents to our public consultation – we wish to begin a process of investigating how UK aid is delivered in the country by looking at the assurance provided by the systems used to manage delivery.
3. Purpose of this study
3.1 To establish how effective DFID’s arrangements are for ensuring that aid monies are used for their intended purpose in Afghanistan.
4. Relationship to other evaluations/studies
4.1 DFID’s efforts in Afghanistan have been subject to considerable and regular scrutiny, not least from the UK Parliament through the International Development Committee.3
4.2 In 2008, the National Audit Office (NAO) published its report DFID Operating in Insecure Environments,4 which included Afghanistan. It came to some specific conclusions and made recommendations for DFID, including:
a) ‘There is limited research and experience on delivering effective aid in insecure environments, so the information on which DFID is able to base its decisions is weak.’
b) ‘DFID staff do not have enough practical guidance on working in insecure environments.’
c) ‘Successful projects have good design features that should be applied more consistently.’
d) ‘Monitoring in insecure environments is difficult, but DFID needs to identify and respond to problems as early as possible.’
e) ‘DFID needs better management information on its costs to inform its decisions and achieve value for money.’
4.3 In addition, NAO has recently investigated DFID’s financial management controls in general terms.5
4.4 DFID’s 2009 Afghanistan country programme evaluation1 highlighted the challenges of delivery and control which are notable given the context of the majority of DFID’s funds being put through the ARTF:
‘DFID’s practice of putting its aid funds through common systems adds to the usual problems of attribution in development aid. Evidence of the results of specific inputs in a multi-funded project is therefore often related more to aid effectiveness than to wider developmental impacts. Moreover, in all programmes, security constraints prevent staff monitoring either outputs or impact in any consistent manner.’
4.5 We wish to see how DFID is applying the lessons and recommendations of these reports. This review will also link to our November 2011 report on DFID’s anti-corruption strategy.6
5. Analytical Approach
5.1 This is an investigation which will focus on programme controls and systems. So whilst it will examine systems which will ultimately help develop efficiency, effectiveness and impact and test if these systems are fit for purpose, it will not in itself assess the efficiency, effectiveness and impact of the Afghanistan programme.
5.2 The study will take a systems-based approach and evaluate the effectiveness of the mechanisms put in place to manage and monitor the distribution and use of existing aid monies. It will also examine how considerations of assurance and control are built into DFID’s decision-making when planning aid support to Afghanistan and how the wider context in Afghanistan has been taken into account in using systems and controls.
5.3 The study will focus only on funds administered by DFID. Other UK Government funds such as those of the FCO, Conflict Pool and Strategic Programme Fund fall outside the scope of this first review.7 It will, however, include transfers made out of the bilateral programme through multilateral channels.
6. Indicative questions
6.1 The following set of questions is at the core of the review and they will be elaborated on in the inception report:
6.2 Planning
6.2.1 What are DFID’s key financial control systems and are they working in practice?
6.2.2 How have the financial controls been designed to work in Helmand? What evidence is there that these controls are working?
6.2.3 In establishing safeguards to mitigate risk, does DFID have evidence of what mechanisms are working best and why?
6.2.4 How have the needs of the intended beneficiaries of UK aid been taken into account in this process?
6.2.5 To what extent has DFID drawn on UK established practice for good governance, for example as set out in the Office of Government Commerce’s Gateway Process?
6.2.6 How closely is the work of the DFID’s Counter-Fraud Unit taken into account in establishing the programme assurance and controls?
6.2.7 To take account of the environment, how should DFID’s controls vary from, say, what might be expected in terms of corporate controls in a UK business?
6.2.8 What are the strategic imperatives that inform continuation of this programme?
6.2.9 How have the governance system and controls put in place by DFID been informed by the experience of other countries providing large amounts of aid to Afghanistan e.g. US, Japan, UAE?
6.2.10 How is the spending allocation affected by the fiduciary controls applicable to a particular means of aid delivery?
6.2.11 What evaluations and audits have been performed in the past six years and what have their key recommendations been? How effectively has DFID implemented recommendations on financial controls made in past evaluations?
Potential additional questions:
6.2.12 What is unique about this context that would substantively affect value for money in programme delivery?
6.2.13 What is the theory of change for the programmes?
6.2.14 How integrated are the programmes with wider activities, in particular military programmes and local government stability?
6.3 Monitoring
6.3.1 What substantive tests does DFID apply and who performs them? How robust are they? Is assurance being properly obtained?
6.3.2 How effectively has DFID’s Fiduciary Risk Assessment process been applied in Afghanistan?
6.3.3 Are robust and meaningful comparisons made between different projects a) in-country; and b) in other fragile states?
6.3.4 How are reports of leakage handled and investigated in the governance process?
6.3.5 How are the intelligence and information received about the misappropriation of aid monies used in considering how best to channel funds?
6.4 Evaluation
6.4.1 Does DFID have a clear picture of its delivery chain for each project and what checks and balances are in place throughout that chain?
6.4.2 How does DFID hold organisations in the delivery chain to account?
6.4.3 To what extent are the financial controls and accountabilities specified by DFID actually applied in country?
6.4.4 How effective has DFID been in putting controls into Afghan organisations such as local government?
6.4.5 How effectively has the anti-corruption policy been applied? This will include DFID’s understanding of the local incentive structure.
6.5 Overall assessment
6.5.1 Are these controls (however good) actually assisting with DFID’s delivery objectives?
6.5.2 What proportion of aid money is it appropriate to be at risk when balanced against the returns (positive outcomes) that can be achieved when funds are deployed as intended?
6.5.3 Will the programme controls in place now be as effective or will they need to be refreshed as planned military withdrawal goes ahead?
6.5.4 What is the overall effectiveness of programme assurance and controls and how might they be strengthened?
7. Outline Methodology
7.1 The main components of the approach will comprise:
- discussions with DFID staff, both in the UK and in Afghanistan;
- a review of the documented governance, programme assurance and control arrangements;
- a review of the application of the methods put in place to collect data and evidence about how aid monies are being put to use;
- a review of data and management information available to DFID about the performance of aid projects;
- a review of learning points from other major providers of aid to Afghanistan;
- discussions with stakeholders and organisations in the UK that are involved in the delivery chain of aid in Afghanistan;
- case studies of specific projects in each of the four strategic priority areas to illustrate the effectiveness of DFID controls; and
- consultations with NAO regarding the findings from their programme of work, for example on financial management.
8. Timing and deliverables
8.1 The review will be overseen by Commissioners and implemented by a small team from ICAI’s consortium. The review will take place during the final quarter of 2011, with a final report available in the first quarter of 2012.
Footnotes
1 Country Programme Evaluation Afghanistan, DFID, May 2009, EV696, www.dfid.gov.uk/Documents/publications1/evaluation/afghan_eval.pdf
2 DFID Afghanistan Operational Plan 2011-15, DFID, 2011, www.dfid.gov.uk/Documents/publications1/op/Afghanistan-2011.pdf.
3 Reconstructing Afghanistan, International Development Committee, February 2008, HC 95-I, www.publications.parliament.uk/pa/cm200708/cmselect/cmintdev/65/65i.pdf.
4 Department For International Development: Operating in Insecure Environments, National Audit Office, October 2008, www.nao.org.uk//idoc.ashx?docId=d79346cc-51fd-4d2e-8e74-2fe9c01c7783&version=-1.
5 Department for International Development: Financial Management Report, National Audit Office, April 2011, http://www.nao.org.uk//idoc.ashx?docId=8184a4cd-c1d9-4c5e-9e88-4d7e4cbe32e7&version=-1.
6 The Department for International Development’s Approach to Anti-Corruption, Independent Commission for Aid Impact, November 2011, http://icai.independent.gov.uk/wp-content/uploads/2010/11/DFIDs-Approach-to-Anti-Corruption.pdf.
7 This review is a first report; future work could potentially examine wider UK ODA including other UK Government funds, as well as the UK’s World Bank, EC and UN contributions to Afghanistan.