ICAI follow-up review of Year 5 reports

Executive summary

Each year, ICAI follows up on the response to the recommendations it made in previous reviews. The process is a key link in the accountability chain, providing Parliament and the public with an account of how well DFID and other government departments have responded to ICAI reviews.¹ It also provides feedback to the responsible staff, and serves as a useful learning exercise for ICAI in our quest to achieve greater uptake of our findings and recommendations.

This report summarises the results of our follow-up on five reviews:

• Assessing DFID’s results in water, sanitation and hygiene (May 2016)
• DFID’s efforts to eliminate violence against women and girls (May 2016)
• DFID’s approach to managing fiduciary risk in conflict-affected environments (August 2016)
• UK aid’s contribution to tackling tax avoidance and evasion (September 2016)
• How DFID works with multilateral agencies to achieve impact (June 2015)

The last of these was held over from the previous year, due to the delayed publication of DFID’s Multilateral Development Review. We also followed up on three outstanding issues identified in last year’s follow-up:

• Does DFID exercise adequate supervision of its development capital portfolio?
• Is DFID’s refreshed security and justice strategy appropriate?
• Does DFID’s new departmental results framework address ICAI’s past concerns?

In its response to our reviews, DFID indicates whether it accepts each recommendation in whole or in part. We explored whether its responses were reasonable and whether its management actions are making a difference.

DFID accepted the majority of our recommendations and has taken positive action in a range of areas. Our assessment of its response is summarised in Tables 1 and 2. Some of the highlights include:

• Concrete steps to improve the sustainability and monitoring of results of water, sanitation and hygiene programmes, including measures to ensure that vulnerable groups, such as the elderly or people with disabilities, share in the benefits.
• New guidance on scaling up programming on violence against women and girls, and a stronger approach to sharing learning across the department on what works.
• Some significant improvements in DFID’s approach to managing the risks of fraud and corruption in insecure environments when delivering remotely through partners.
• Measures to improve UK support to developing countries in fighting international tax avoidance and evasion.
• Some progress towards a more strategic approach to strengthening the multilateral aid system.

Table 1: Summary of DFID’s response to ICAI recommendations

Table 2: Outstanding issues from earlier reports

Conclusions

This is an encouraging set of responses from DFID in many areas. We have seen evidence of ICAI recommendations making a difference through a range of mechanisms, which are summarised in Table 3.

Table 3: Types of influence ICAI had through its recommendations

We also noted some recurring themes among the recommendations that DFID did not respond well to. DFID did not act on recommendations that involved reassigning staff – either to support its multilateral partnerships or to strengthen project management on high-risk programmes. It also has made limited progress on our suggestion to improve its communications with the UK public around multilateral aid. We believe that in making its case to the UK public for both development and humanitarian aid, it is important that DFID explains the channels through which it is spent.

We have also found that DFID is not well placed to respond to recommendations that require changes in approach across a sectoral or thematic portfolio. DFID prefers not to issue detailed strategies to guide its programming. With programme design mainly decentralised to country offices, its central policy teams have few other levers at their disposal to shape country programming. This makes it difficult for DFID to follow through on cross-cutting policy commitments, such as its ‘leave no one behind’ pledge.

1 Introduction

1.1 Each year, ICAI follows up on the recommendations it made in reviews published in the previous year. This is an essential link in the chain of accountability. The aid-spending departments that are the subject of a recommendation are required to produce departmental responses to the ICAI review, indicating whether they accept, partially accept or reject each recommendation and setting out the management actions that they propose to take in response. Our follow-up work tests the adequacy of the departmental response and the progress on implementing agreed management actions. We also look more broadly at whether DFID has recognised the concerns underlying our recommendations and taken action to address them.

1.2 We summarise the results of our follow-up in this report, in order to provide Parliament and the public with an account of how well the responsible departments have responded to ICAI reviews. The follow- up process also provides feedback to the responsible staff, and a learning opportunity for us, as we strive to make our reviews as impactful as possible.

1.3 ICAI’s fifth year of operations (July 2015-June 2016) was the first year for the phase II appointed commissioners (commissioners have a four-year tenure). ICAI has followed up on the response to the recommendations it made in reviews published up until September 2016. This means that the follow-up analysis was conducted five to nine months after the reviews were published. In addition, we followed up on the June 2015 ICAI report on DFID’s work with its multilateral partners; this follow-up had been deferred from the previous year in view of the delayed publication of the Multilateral Development Review. We also followed up on three issues that had been identified as outstanding in the previous year’s follow-up (see Table 4).

Table 4: Reviews covered in the Year 5 follow-up

1.4 This report is structured as follows. Chapter 2 summarises our methodology. Chapter 3 offers a summary of the results of our follow-up work on each report and outstanding issue. The final chapter then draws out some conclusions and lessons for ICAI.

2 Methodology

2.1 Our methodology for following up on past reviews comprises (i) an assessment of the coherence and completeness of the departmental response, including probing the reasons behind the rejection of any recommendations; (ii) an assessment of progress on implementing the agreed management actions and other relevant actions, the quality of the work that has been undertaken and whether it is likely to be effective in addressing the issues identified in the ICAI report.

2.2 We ask DFID or other relevant government departments to prepare a short summary of progress on each recommendation. We then validate that account through interviews with the responsible staff and by examining relevant documentation. Where the management action involves collaboration with external partners or where third parties are in a position to take a view on its effectiveness, we interview external stakeholders, including other UK government departments, multilateral partners and implementers. We also assess whether the ICAI analysis has had an influence beyond the specific issues raised in the recommendations. This includes contacting DFID country offices, particularly those visited during the review, in order to identify changes in practice.

2.3 Each follow-up process concludes with a formal meeting between a commissioner and the senior civil service counterpart in the responsible department. The meeting is an important part of the review cycle, helping to close the accountability loop.

2.4 Each year, we identify a number of issues that require further follow-up work in the next annual cycle. These are issues where we judge the departmental response to have been inadequate, and which we assess to be of continuing strategic importance.

2.5 The follow-up process also enables ICAI to keep abreast of developments in areas of interest for our future review programme, and to identify learning points for our own work.

3 Findings

3.1 This section presents the results of our follow-up investigations of DFID’s response to our reports and of three outstanding issues identified in last year’s follow-up. We present the results of each follow-up assessment in turn, focusing on the most significant results from, or gaps in, the response.

Assessing DFID’s results in water, sanitation and hygiene

3.2 In May 2016 we published our first impact report, exploring DFID’s results in water, sanitation and hygiene (WASH) over the period 2011 to 2015. We awarded a green-amber score, reflecting the fact that DFID had exceeded its target of reaching 60 million people with WASH interventions, and the evidence that its WASH programmes were contributing to wider development results, especially improved health outcomes. However, we identified a number of areas for improvement, including around results measurement and ensuring sustainable results. We made four recommendations in the areas shown in Table 5.

Table 5: Area of recommendations and DFID’s response

3.3 DFID’s initial departmental response to the review was disappointing. The narrative format combined explanatory argument with ongoing and planned actions, making it difficult to identify the proposed management actions. At the request of the International Development Committee, DFID subsequently revised its response to include more specific commitments. All the recommendations were accepted in full or in part, and DFID is engaging actively on most of the concerns we raised. However, there were also a few points of disagreement.

Monitoring results

3.4 On results measurement, we were concerned that DFID’s method of counting the beneficiaries of its WASH investments was based on estimates, rather than surveys of actual use, leaving it unable to identify whether vulnerable groups (for example the elderly or people with disabilities) were being reached. We also noted that DFID was not monitoring wider development impacts, beyond WASH access, such as health results or school attendance. Our view was that collecting wider impact data, at least on a sample basis, during the life of a WASH project would enable DFID to adapt its interventions in order to maximise results.

3.5 DFID agreed with the need to improve results monitoring, but not with all the details in our recommendation. It has developed a new methodology for collecting WASH results, based on surveys of actual use, with greater disaggregation of data. This will support an increased focus on reaching women and people with disabilities, in the first instance, and potentially other marginalised groups. On wider development results, DFID agreed to push ahead with collecting more impact data and strengthening its evaluation work. However, it disagreed with the suggestion of routine monitoring of health and school attendance data, citing practical challenges and the difficulties of attributing changes to WASH programming. DFID justified this stance on the basis that the links between WASH access and wider development results were already sufficiently established in the international literature.

Ensuring sustainability

3.6 We found that DFID had fallen behind international best practice in the WASH sector by failing to monitor whether results were being sustained beyond the life of its programmes. We proposed paying greater attention to sustainability in programme design – including by extending monitoring beyond the standard three-to-five year programme cycle.

3.7 DFID has responded quite well to this recommendation. It has been active in promoting improved standards on sustainability with other bilateral donors and multilateral agencies, such as the World Bank and Unicef, and it is in discussion with the UK Water Network on drafting a joint position paper on sustainability. We were pleased to find that DFID had used our report to draw partners’ attention to the importance of embedding sustainability into programme delivery and monitoring arrangements. In its own programming DFID is now challenging its implementers to pay more attention to sustainability in programme design and to include sustainability indicators in their results frameworks.

3.8 DFID has not yet taken up our suggestion of extending monitoring beyond the programme cycle, citing practical constraints in the way it funds its programmes. However, it is planning to undertake a thorough review of whether the benefits of its WASH investments in recent years have been sustained. It is also planning to invest £1.5 million in evaluating future programmes.

Improving value for money

3.9 We found that DFID had begun collecting data on the value for money of its WASH programming, but had not yet come up with a workable method of comparing the return across its investments. While acknowledging the technical challenges involved, we recommended that DFID press ahead with developing an appropriate value for money framework. DFID has updated its value for money guidance, drawing on a range of research, and is developing a set of value for money indicators, covering not just the economy of inputs but the full range of results data. This is part of a wider DFID initiative to strengthen its use of value for money metrics, and one of the most significant responses to our WASH report.

3.10 DFID has been working with Unicef – one of the leading global implementers of WASH programming – to introduce 12 core value for money indicators. DFID is also encouraging other bilateral donors and foundations to adopt these and hopes that they will develop into a global standard.

3.11 There remain tensions to be managed between a more active stance on value for money and DFID’s commitment to ‘leaving no one behind’.² DFID recognises that provision of WASH services to marginalised groups involves higher unit costs, and is building equity into its value for money framework.

Improving learning

3.12 We found that DFID was not systematically capturing and disseminating lessons learned from its WASH programmes to inform future programme designs. We therefore recommended a more active approach to learning – including joint learning with related sectors, such as health and education, to improve understanding of how investments in these areas interact.

3.13 While DFID recognised the concern and accepted our recommendation, its response has been disappointing, with no major new learning initiatives. DFID is making useful investments in knowledge generation, but faces challenges in using that knowledge systematically to guide programming choices. DFID’s response on this issue reflects a wider challenge for DFID’s central policy teams in providing effective guidance and technical support for programmes designed in country offices.

Conclusion

3.14 This is a strong response by DFID in most respects. We are particularly encouraged that DFID is tackling the two challenges of sustainability and value for money more actively, including by influencing practices and standards across the sector. We also welcome the commitment to improving the quality of results data. In its single departmental plan, DFID has set itself the target of providing 60 million people with sustainable access to clean water and sanitation.³ If implemented effectively, the measures described here will help to ensure that this investment makes a lasting difference to the lives of beneficiaries.

DFID’s efforts to eliminate violence against women and girls

3.15 The UK government has made a strong commitment to leading efforts to tackle violence against women and girls (VAWG) around the world. As this is a relatively new area for the aid programme, we chose to make this the subject of our first learning review, published in May 2016. We assessed DFID’s efforts to build an evidence base on what works, and its progress in developing a relevant and credible programming response. We awarded DFID an overall green rating for its work in this area, in light of its active learning stance, a strong research portfolio and some good-quality programming, combined with a strong leadership role at the international level. However, we noted that the programmes remained small relative to the problem they sought to address, and that the key challenge for the future would be taking successful initiatives to scale. We made four recommendations, in the areas indicated in Table 6.

Table 6: Area of recommendations and DFID’s response

3.16 On the whole, DFID has responded reasonably well to our findings and recommendations, despite a disappointing initial departmental reaction. However, we are concerned that the scale and intensity of VAWG programming does not appear to have increased since the review was published.

Scaling up programming

3.17 The future challenge of scaling up VAWG programming was a central concern of our review. We found that DFID had developed some good-quality pilots and learning programmes, but had no clear plans for how to implement them at scale. We also found that quality tended to fall away when VAWG activities were incorporated into wider programmes. DFID has now prepared a draft theory of change for scale-up, which among other things considers how VAWG can be successfully addressed within large sectoral programmes.

3.18 While this is a useful initiative, we are concerned that a recent provisional mapping suggests that the number of DFID country programmes with VAWG elements has declined. This may be linked to DFID’s programming cycle; we were told that new programmes may be developed over the coming year. It is important that DFID maintains the intensity of its work in this area in order to build on the solid foundations it has established.

3.19 During our field work in Ethiopia, we visited the Girl Effect programme, which was using innovative branding techniques to encourage girls to challenge the acceptability of violence. We were disappointed to learn that DFID has withdrawn its support for this programme, which was subject to critical media reporting, for reasons other than evidence of unsatisfactory results. While it was too early to assess the results of that particular initiative, in our report we identified the use of marketing techniques to challenge social norms as an important area for further learning. It would have been preferable, in our view, to give this initiative more time to demonstrate its effectiveness.

3.20 DFID has identified working with multilateral partners as an important route for scaling up VAWG programming. It has encouraged the World Bank to develop a new gender strategy and is conducting a formal dialogue with UN Women to address organisational issues identified in the Multilateral Development Review.⁴ DFID has also worked with other donors to raise the profile of VAWG within OECD donor forums.

Improving learning

3.21 In our review, we noted the need to improve learning – not just among specialist VAWG practitioners, but across DFID, to support scaling up. In response, DFID has appointed a knowledge and learning adviser for the VAWG portfolio. So far, the adviser has reviewed the VAWG community of practice and help desk facility to identify how they can be improved, and is preparing a learning plan. The policy team conducted a VAWG session at the professional conference of DFID social development advisers in November 2016, and plans further sessions with the economists and education advisers. There has also been an increase in internal communication on VAWG, including dissemination of ‘how to’ notes. These are useful initiatives and we encourage DFID to maintain this momentum.

Disseminating results

3.22 In our review, we found that DFID had positioned itself well as a leading global investor in VAWG research. Its £25 million WhatWorks research and innovation programme was making a major contribution to knowledge in the field. However, we found that its work was not well known beyond DFID, and therefore recommended an increased focus on dissemination. WhatWorks has now reoriented itself towards outreach and uptake. It has produced a research uptake and stakeholder engagement strategy, which will be regularly updated, and included uptake measures in its results framework. We found that the programme’s dissemination approach, while thorough, was not particularly innovative. DFID and its implementers could do more to understand the evidentiary needs of its target audiences, including civil society, and make use of a wider variety of communication techniques.

Tracking expenditure and results

3.23 DFID has been less convincing in its response to our recommendation that it track VAWG expenditure and results more systematically through its management information system. DFID only partially accepted this recommendation, as it had decided not to introduce a VAWG target into its new single departmental plan. However, it is beginning to collect data on VAWG expenditure more systematically, and has put in place plans to update its mapping of VAWG programming on a six-monthly basis.

Conclusion

3.24 DFID informs us that ICAI’s award of a green rating has helped to raise the profile and status of the VAWG portfolio and given the central policy team a platform to raise VAWG issues across the department and with international partners. There has been a good response to the recommendations on learning and the dissemination of research. On the key issue of scaling up, while we welcome the efforts of the central policy team, we are concerned that we cannot yet see evidence of a greater intensity of effort across country programmes, which is essential to achieving transformative impact. DFID appears to face a significant challenge in maintaining its commitment to cross-cutting issues like VAWG across a decentralised programme. We have therefore decided to revisit this issue in next year’s follow-up work, looking in particular at what progress has been made in integrating VAWG initiatives into DFID’s wider sector programmes and into the Conflict, Stability and Security Fund portfolio.

DFID’s approach to managing fiduciary risk in conflict-affected environments

3.25 DFID has committed to spending half of its budget in fragile and conflict-affected states, where the risks involved in delivering aid, including risks of corruption and diversion of aid, are high. In August 2016, we published a performance review of how well DFID manages fiduciary risk in such environments. We awarded DFID an overall green-amber score, in recognition of its active approach to managing fiduciary risks, including fraud and corruption, and of a series of useful ongoing reforms on risk management. However, we also identified a number of areas where improvements were needed. We made four recommendations, in the areas indicated in Table 7.

Table 7: Area of recommendations and DFID’s response

3.26 DFID reacted well to the review, accepting the analysis and all the recommendations. So far it has taken positive action in response to the first two recommendations, as part of the ongoing process of strengthening its approach to risk management, but progress on the remaining two has been limited. Our report has also encouraged DFID’s Internal Audit Department to look in more detail at risk management practices in conflict-affected countries.

Fiduciary risks relate to the funds not being used for the intended purposes and/or not being properly accounted for.
Smart Rules: Better Programme Delivery, DFID, April 2017, p. 18

Risk appetite and risk-return balance

3.27 At the time of our review, DFID was in the process of introducing a new risk management framework and accompanying guidance. However, it did not yet have a consistent approach to defining its risk appetite, or tools for balancing risk and return across its country portfolios. This led to considerable diversity of approach across country offices, and differences in the interpretation of how to reconcile DFID’s high risk appetite and its zero tolerance approach to fraud and corruption.⁵ We therefore recommended that DFID accelerate the introduction of new policies and guidance in this area.

3.28 In December 2016, DFID approved a new risk appetite statement and risk management policy, with accompanying tools and training materials. The new policy breaks down risk appetite by type of risk, noting DFID’s willingness to work in high-risk environments but emphasising its low appetite for reputational and fiduciary risk. The country office staff we spoke to confirmed that the guidance and training had given them a better understanding of the department’s approach.

3.29 DFID has also piloted a new ‘risk and development return’ tool for portfolio management. Country business plans now include some analysis of how to balance risk and development return across their portfolios.

We will do all we reasonably can to ensure our funds are not used fraudulently…
We have a zero tolerance approach towards, and will work to prevent, fraud and corruption, reporting and acting upon all suspicions.

Risk appetite statement and risk management policy, DFID, December 2016 (unpublished)

Risk transfer

3.30 Our 2016 review found that DFID did not have a clear position on risk transfer down the delivery chain, where its primary implementers work through subcontractors. Staff were unsure of the extent to which risks were transferred to subcontracted parties and the consequences of transferring risk down the delivery chain. We recommended that DFID clarify its rules around risk transfer and the residual responsibilities of DFID managers.

3.31 DFID’s new risk management policy states that partners are responsible for the robust monitoring of their subcontractors, including taking a zero tolerance approach to fraud and corruption. Programme managers are now required to map the delivery chain for each bilateral programme and a new smart guide on how to do so was adopted on 1 April 2017 (later than planned). It includes a section on matters to consider when transferring risk management responsibilities, including the capacity of third parties to absorb risks and the potential costs. This new guidance is a welcome development that, once implemented, should improve DFID’s management of complex delivery chains, particularly in insecure environments where the risks tend to be greatest.

Monitoring of multilateral partners

3.32 Where DFID engages multilateral agencies as implementers of bilateral programmes, there are limits on its ability to ensure adequate fiduciary risk management. In our 2015 report, we found that oversight arrangements were negotiated at the local level, rather than based on an objective risk assessment. We therefore recommended that DFID explore ways of improving the transparency of multilateral partners and its ability to monitor their management of fiduciary risks.

3.33 DFID accepted this recommendation and its new smart guide requires delivery chain mapping for multilateral partners. However, the core issue of DFID’s entitlement to supervise multilateral delivery partners remains unresolved. Although there have been efforts to improve the accountability of multilaterals at the central level through the Multilateral Development Review process, DFID’s oversight of multilateral partners at country level remains fairly ad hoc. DFID acknowledged that this is an area that it is still working on.

Allocation of staff

3.34 The review found that DFID did not have a system for assigning experienced staff to manage its riskiest programmes. Furthermore, when staff were redeployed rapidly in response to humanitarian crises, programme management skills were not prioritised. We therefore recommended that DFID introduce a system for matching fiduciary risk management skills to its highest-risk programmes in conflict-affected countries.

3.35 While DFID accepted this recommendation, its departmental response was not specific as to what measures would be undertaken and progress since then has been limited. DFID has incorporated risk management into staff training and established a risk management community of practice. Staff report that this has helped to improve risk management capabilities. DFID has also established a crisis response network, which is a register of staff able to offer surge support in humanitarian emergencies. Members of the network identify their programme management experience, which in principle makes it possible to assign them accordingly. While these measures may prove helpful, they have not addressed the substance of the recommendation. This appears to reflect a wider uncertainty within DFID over whether to create a specialised programme management cadre or to mainstream programme management functions across the whole department.

Conclusion

3.36 This has been a mixed response from DFID. We have seen positive action on the first two recommendations, with new policies, guidance and staff training. These have been well received by staff, and are reportedly beginning to improve risk management practice in the field. On the rules governing risk transfer through complex delivery chains, oversight of multilateral partners and matching staff skills with programme risk levels, the response has been uneven. We recognise that it takes time to change business processes, and that some issues, particularly the oversight of multilateral partners at country level, require sensitive handling. We nevertheless expect to see DFID continuing to push forward in these areas. We therefore propose to revisit the question of delivery chain and multilateral partner oversight in next year’s follow-up.

UK aid’s contribution to tackling tax avoidance and evasion

3.37 One of the themes for ICAI’s work programme is ‘beyond aid’ which, among other things, explores the UK’s efforts to influence how the international system interacts with developing countries. In September 2016 we published a review of how DFID had influenced the international tax system as part of its efforts to help developing countries tackle international tax avoidance and evasion. The review also explored how DFID has worked with HM Revenue & Customs (HMRC) to strengthen revenue authorities in developing countries – given the commitment in the UK aid strategy to making more use of the expertise available across government in the aid programme.⁶ It was a learning review, in view of the fact that this was a relatively new area of engagement for both departments, where the evidence on what worked was limited. We awarded DFID an overall amber-red score for its work. We found that while DFID had enjoyed some early success in helping developing countries participate in international tax processes, its efforts were not sufficiently grounded in the needs and priorities of developing countries, or informed by research or learning from its country engagements. We made four recommendations, in the areas indicated in Table 8.

Table 8: Area of recommendations and DFID’s response

3.38 DFID has generally reacted very positively to the report: three of the recommendations were accepted and there has been significant progress in some areas. HMRC also informs us that it found the analysis useful.

Ending global poverty can only be achieved through ensuring the international system works for developing countries.

DFID submission to the International Development Committee⁷

Prioritising and sequencing international tax initiatives

3.39 The review found that DFID’s influencing work on international tax and its capacity building support on international tax initiatives were not based on sufficient analysis of developing country needs and interests. As a result, there was a risk of developing countries being encouraged to implement technically challenging initiatives without enough attention to the costs and benefits, and without sequencing them with the development of their national tax systems.

3.40 DFID acknowledged these concerns and has begun working with international partners on two initiatives that will help to address them. First, it is working with the OECD to develop a new cross- border tax diagnostic tool, which will help partner countries identify what they stand to gain from implementing particular tax initiatives, given their economic conditions, national tax systems and institutional capacity. Second, DFID is working with the International Monetary Fund to develop medium-term revenue strategies, which partner countries can use to sequence their international and domestic tax reforms. Both initiatives are still in the development phase and it is too soon to assess how well they will work in practice. In principle, however, they represent a strong response to the recommendation.

3.41 We also found that DFID’s international tax work was not properly informed by experience from its country-level tax programmes. There is now a community of practice linking DFID and HMRC tax and development practitioners in headquarters and partner countries. Among other learning functions, it serves as a forum for bringing the country office perspective into the new international tax arena.

Cross-government working

3.42 UK departments that are new to development cooperation face a steep learning curve on how to use their technical knowledge for effective capacity building. We found that DFID’s support for HMRC’s Capacity Building Unit was a good model for combining technical expertise from another department with DFID’s knowledge of good development practice. However, we recommended that DFID be more active in ensuring that other departments engaged in capacity building draw on its experience and in- country knowledge.

3.43 At the time of our review, DFID, HMRC and the Treasury were working on a new strategic framework for tax capacity building, which was launched at an international event in October 2016. The document offers a comprehensive guide to programming on tax and development, with useful advice on various issues raised in our report, such as the need for political economy analysis, sequencing of reforms, matching reform initiatives to existing capacity levels and working collaboratively with DFID country offices. We find the document to be a useful summation of good practice.

3.44 DFID continues to offer technical support to the HMRC Capacity Building Unit, particularly in the area of results measurement. It is also designing a new resource centre for tax and development, which will include a panel of experts that both DFID and HMRC practitioners can draw on to support their work.

3.45 We are also interested to note that DFID is developing a new ‘GREAT for Partnership’ programme, which will help to promote, broker and support partnerships between UK government agencies and their counterparts in developing countries. It will focus initially on three sectors: extractive industries, health and anti-corruption and financial accountability. There are plans eventually to scale up the programme further, so that it can act as a central hub for government partnerships in developing countries. This suggests that DFID is beginning to marshal the resources it will need to support new entrants in the capacity building arena.

A strategic approach to influencing

3.46 Our review found that DFID had been more opportunistic than strategic in its approach to international influencing on tax. We concluded that, if DFID considered ‘beyond aid’ initiatives to be an important part of its development agenda, it should approach the work more systematically, with clear strategies and objectives, adequate staffing and systems for monitoring and reporting on its results. While DFID accepted the recommendation, the management actions it proposed were noncommittal and its response since then has been disappointing. DFID informs us that it will continue to play a leading international role on tax and domestic resource mobilisation but does not see the need for a more formal planning to its influencing work.

Policy coherence for development

3.47 ‘Policy coherence for development’ is the principle that donor countries should strive for greater coherence between their development policy and other policy domains, such as trade, so as to create more opportunities for developing countries.⁸ The International Development Committee has encouraged DFID to do more in this area.⁹ We recommended that DFID take a more active stance on policy coherence in the tax arena, including by assessing the impact of UK tax policies and practices on developing countries and deciding whether to raise any points of tension with the Treasury.

3.48 The government decided to reject this recommendation. DFID informed us that, while there is good cross-government collaboration on tax and development, it does not have a role on domestic tax policy. In addition, it argued that ‘spillover analysis’ is a resource intensive exercise and that its efforts are better spent building capacity to tackle multinational tax at the country level. We are not entirely convinced by this argument. The UK government is not in a position to judge the impact of its tax policies on developing countries unless it does the analysis. We note that policy coherence for development is one of the commitments made under the Addis Tax Initiative, of which DFID was a sponsor.

All participants will ensure that relevant domestic tax policies reflect the joint objective of supporting improvements in domestic resource mobilisation in partner countries and applying principles of transparency, efficiency, effectiveness and fairness.

The Addis Tax Initiative – Declaration, Commitment 3, July 2015, link

Conclusion

3.49 This was a strong response from both DFID and HMRC in a number of respects. In particular, the steps towards better prioritisation and sequencing of international tax initiatives and strengthening inter- departmental cooperation on capacity building were well conceived and have the potential to improve practice. We were encouraged to hear from stakeholders that HMRC’s capacity building work is steadily improving. The experience shows that cross-government partnerships can add real value to the aid programme, but need time and effort to become effective.

3.50 It is not clear to us, however, why DFID is resistant to the idea of approaching international influencing in a more structured way. ‘Beyond aid’ initiatives can be as important a contribution to international development as aid programmes, and should be treated with the same seriousness. We also believe that, as the UK increases its international engagement in preparation for leaving the European Union, DFID may need to take a more active stance on policy coherence for development.

How DFID works with multilateral agencies

3.51 The multilateral system receives approximately 40% of UK aid in core contributions, and another 20% through contracted projects. ICAI’s review of how DFID works with its multilateral partners was published in June 2015, under the previous commissioners. It awarded DFID an overall green-amber score, recognising its influence within the multilateral system and its success in increasing its partners’ focus on results and value for money. ICAI made recommendations in seven areas, summarised in Table 9. In 2016, we decided to postpone the follow-up of this report by a year, as DFID’s Multilateral Development Review (MDR) had not yet been released. It was published in December 2016.¹⁰

Table 9: Area of recommendations and DFID’s response

3.52 DFID’s formal reaction to the review was disappointing. Even though ICAI had engaged in open and constructive discussion of the recommendations with DFID officials, the department decided to reject four of the seven recommendations. Since the publication of our report, DFID’s stance towards the multilateral system has changed in important ways – partly as a consequence of the decision to leave the European Union. We are pleased to note that many of DFID’s subsequent actions have been consistent with our analysis and recommendations.

Global and in-country strategies on working with multilaterals

3.53 The review concluded that DFID’s lack of a global strategy for the multilateral system was a significant weakness. It left staff uncertain as to how to engage with key multilateral partners, at both global and country levels. It also meant that DFID’s influencing focused on business processes within individual agencies (especially results management and value for money), rather than strategic issues. While DFID rejected the recommendation, the 2016 MDR contains important elements of this kind of strategy. DFID has now articulated reform objectives for the multilateral system as a whole and not just for individual agencies. The MDR commits DFID to promoting greater coherence of the multilateral system in specific sectors, such as health and humanitarian aid. New performance agreements with multilateral partners include mechanisms for judging how effectively they work with others. DFID has also introduced a key relationship management approach to its partnerships. These measures create a useful platform for a more strategic engagement in the future.

3.54 The review was also concerned about the lack of explicit objectives in DFID’s country strategies for engaging with multilaterals, and about a consequent disconnect between its relationships with key multilateral agencies at the global and country levels. ICAI suggested that working with multilaterals be integrated more fully into DFID’s diagnostic work and country plans, and that DFID engage with its multilateral partners on development strategy as well as on business processes. While this recommendation was also rejected, there have nonetheless been some significant and positive developments. The MDR process included a greater focus on ensuring coherence between bilateral and multilateral aid, and a director-level coherence group has been created to take this forward. DFID now has two smart guides to help staff work effectively with multilaterals. Heads of country offices are now more involved in multilateral reform. DFID is also beginning to coordinate its multilateral engagements at a regional level – for example, multilateral departments are working with country offices to address common performance concerns for multilateral development banks in Asia and for Unicef in Africa. However, DFID offices are still not required to articulate clear objectives for influencing multilateral partners at the country level. We believe more could be done to integrate the multilateral system into country planning.

Staffing levels and collaborating with others

3.55 While multilateral agencies spend around 60% of DFID’s budget, they receive a far smaller proportion of management and staff time. The ICAI review pointed out that DFID did not nominate senior staff to manage its most important multilateral relationships, and was not staffing its influencing work adequately or at the right levels of seniority. DFID rejected the finding that its multilateral partnerships were under-resourced. Its introduction of the key relationship management approach is a step in the right direction, but has not fully addressed ICAI’s core concern.

3.56 The review noted that DFID enjoys a strong leadership role among bilateral donors on multilateral reform, but is perceived as often preferring to work alone – even though experience demonstrates the value of joint working. ICAI recommended that DFID increase its collaboration with other bilateral donors. While DFID rejected the recommendation, it has improved its collaboration in a number of areas, such as performance-based funding of multilaterals. The secretary of state has convened a forum for political leaders from major donor countries to promote UN reform. As the MDR itself notes, the Brexit vote has spurred the UK government to be even more engaged internationally and to take on a stronger leadership role.¹¹

Transparency, accountability and integrated working

3.57 The report found that DFID had made significant progress in encouraging its multilateral partners to improve their focus on results and value for money, but less progress on promoting transparency and accountability. ICAI suggested that greater transparency would lessen the need for DFID to negotiate additional oversight arrangements. DFID informs us that multilateral transparency is also a ministerial priority, and that an internal task force has been established to address this issue. DFID made transparency a major focus of the MDR process. It is pushing for more publication of financial information, expenditure and results data – including by improving the quality and accessibility of reporting under the International Aid Transparency Initiative¹² – and for increased accountability to beneficiaries through more effective feedback mechanisms. These initiatives have the potential to drive significant improvements in the multilateral system, and we look forward to seeing how they develop.

3.58 ICAI also recommended that DFID do more to promote integrated working among multilateral agencies at the country level. Drawing on the experience of the ‘cluster system’ for humanitarian aid, ICAI suggested that coalitions of multilateral agencies might be formed around the Sustainable Development Goals, to better coordinate development assistance. This has also been a focus of the MDR process, but achieving meaningful change will require sustained attention. So far, progress on integration in development assistance lags behind coordination in the humanitarian sphere.

Communicating with taxpayers

3.59 The 2015 review found that DFID said very little in its public communications about UK aid spending through multilateral agencies and what this achieves. Beyond humanitarian response, public knowledge of the multilateral system appeared to be low. ICAI suggested that DFID communicate more effectively to taxpayers about the role, impact and importance of multilaterals. DFID partially accepted this recommendation. DFID informs us that, from 2017, it has adopted a much more proactive approach to communicating positive messages about the aid programme to the public, to counteract negative media coverage. DFID staff also noted the need for a clearer narrative on the value of UK multilateral aid and the UK’s role in multilateral reform, but that it is challenging to communicate this in an accessible way. So far, there has been limited progress on this beyond the publication of the MDR itself.

Conclusion

3.60 DFID was initially unwilling to accept most of the recommendations from this review. However, the context has changed significantly since the review was published and many of DFID’s subsequent actions have been consistent with the thrust of ICAI’s recommendations. DFID has put in place several elements of a strategy for engaging with the multilateral system. It is trying to make its relationships with key multilateral partners more strategic in nature, and to achieve greater coherence across its multilateral and bilateral portfolios. DFID has positioned itself well to push for progress on greater transparency. Overall, the new political context since the Brexit vote is encouraging DFID to become more outward-leaning in its orientation.

Outstanding issues from the 2016 follow-up

3.61 In last year’s follow-up work, we introduced a new feature. We identified three strategic issues where DFID’s progress on implementing our recommendations had been inadequate and flagged them for further follow-up this year. Those issues were:

• the governance arrangements for DFID’s development capital portfolio
• the refresh of DFID’s approach to security and justice
• the new departmental results framework.

In this section, we present the results of our additional follow-up work in these areas.

Governance of development capital

3.62 In recent years, DFID has increased the share of its budget spent as development capital: equity investments, loans and guarantees for businesses in developing countries. These financing instruments are still relatively new for DFID, requiring different skills and management arrangements. In our May 2015 report on Business in Development, we expressed concern about the adequacy of DFID’s management and oversight arrangements for its development capital portfolio. The bulk of the portfolio is managed by CDC, a specialist development finance institution, so we recommended that DFID bring additional private-sector expertise onto its investment committee to ensure effective oversight of CDC.

3.63 Since then, Parliament has increased CDC’s permitted capital from its former limit of £1.5 billion to £6 billion, with the option of extending this by a further £6 billion.¹³

3.64 We find that DFID has now improved the oversight of its development capital in a number of ways. It commissioned an independent review of CDC’s performance over the past five years, which found that CDC had met the targets in its investment policy but recommended improvements to its monitoring and reporting framework. The National Audit Office also conducted an audit, finding that governance arrangements were thorough but that DFID should be more explicit about its role in investment decisions.¹⁴ These two reports informed a new CDC strategy and a new investment policy for 2017 to 2021 (not yet published). A new memorandum of understanding has been agreed between DFID and UK Government Investments, the government’s centre of expertise on corporate governance and finance, which will support DFID in its oversight of CDC. In response to ICAI’s recommendation, DFID also appointed two new non-executive directors to its investment committee to bring in additional risk management, investment and corporate finance experience. We find this to be a suitable response to the concerns raised in the review.

A new security and justice strategy

3.65 In March 2015, ICAI released a critical report on DFID’s approach to security and justice programming. It found that institution building in the police and judiciary rarely led to improvements in security and justice services or outcomes for the poor. ICAI recommended that DFID review its approach, moving away from capacity building of service providers towards a more problem-based approach aimed at resolving discrete security and justice challenges.

3.66 In response, DFID committed to producing an updated and consolidated security and justice position paper. At the time of our follow-up in 2016, a draft had been produced, but the security and justice policy team had been disbanded and it was unclear who was taking the work forward. The division of roles between DFID and the Conflict, Stability and Security Fund (CSSF) was still emerging. We therefore decided to follow this up again in 2017.

3.67 DFID decided to delay publication of the paper until it had completed its new building stability framework, which governs its work on conflict and fragility as a whole. Following approval of the new framework in December 2016, the security and justice position paper was revised and at the time of writing is awaiting approval. The new draft is a brief document of just six pages, in keeping with DFID’s current preference for short strategies. It acknowledges the ICAI criticism that past support was too ambitious, poorly evidenced and not properly attuned to the political context. As ICAI recommended, it calls for a more focused and realistic approach, based on tackling specific security and justice challenges. It outlines some useful principles, such as being politically informed and using adaptive management techniques. However, the content is not very substantial.

3.68 We are informed that most new security and justice programming is now funded from the CSSF, with technical support provided by the Stabilisation Unit. It is therefore unclear who stands behind the new guidance and to whom it is addressed. We were, however, pleased to hear of an interesting new initiative on organised crime, which follows the problem-solving approach recommended by ICAI (see Box 3).

DFID’s results framework

3.69 ICAI’s June 2015 report on DFID’s approach to delivering impact summarised concerns from several ICAI reviews about the limitations of DFID’s departmental results framework 2011-2015 – particularly its over-reliance on ‘reach indicators’,¹⁸ which specify the number of beneficiaries to be provided with different types of assistance. While acknowledging its role in communicating the scale and ambition of the UK aid programme, ICAI was concerned that the results framework gave a misleading or incomplete picture of results. The use of reach indicators also risked distorting the incentives of staff and partners to focus on maximising numbers directly reached, without adequately considering quality of interventions and impact. ICAI therefore recommended that DFID develop a new results framework that would capture a wider range of results.

3.70 While DFID accepted the analysis, it took a different view on the appropriate solution. It has opted to simplify results reporting at the departmental level, in order to reduce the burden on country offices. At the same time, it is investing in building up a better picture of results from the bottom up, through its Aid Management Platform.¹⁹

3.71 The new single departmental plan, adopted in February 2016, updated in September 2016 and currently undergoing further revision, now takes the place of the results framework.²⁰ It contains 25 commitments with 14 indicators, but only five ‘reach’ targets, compared to 21 in the previous departmental results framework. There have been improvements in the quality of the indicators that address some of ICAI’s past concerns. For example, the WASH indicator now requires qualitative information on the sustainability of results before they can be counted, while nutrition interventions are classed as high, medium or low intensity, giving a more accurate total.²¹ There is a target on the number of countries who will be supported to improve the transparency of their public finances.²² There are additional indicators covering the average quality of DFID interventions in general (measured by performance against output targets) and specifically the impact of development capital interventions.

3.72 Five of the 14 indicators focus on spending. While these provide a useful picture of strategic emphasis (for example share of total spending in fragile and conflict-affected states; spend on climate change), they are not consistently complemented by indicators of quality or the impact of that spending. We also remain concerned that equity is not explicitly addressed in the single departmental plan.

3.73 DFID is also developing a results module for its Aid Management Platform that will capture results from individual programmes. During 2017, the system will begin storing output and outcome data from programme logframes, in principle giving DFID the capacity to aggregate results data at various levels (for example by country, sector or delivery partner). However, aggregating results data across programmes will only work if DFID is able to introduce greater standardisation in its indicators. Work is underway in some sectors and thematic areas to develop menus of standard indicators, but remains at an early stage. At this point, DFID does not yet know the extent to which it will be able to make use of aggregated data from the system.

3.74 Overall, we are concerned that DFID’s system for reporting on aggregate results (ie above programme level) remains unclear. DFID is still working on developing a method of reporting on its contribution to the Sustainable Development Goals – that is, the 17 goals agreed in 2015 for ending poverty, protecting the planet and ensuring prosperity for all.²³ Other donors are reportedly also struggling with this. While country offices are required to report on their contributions to single department plan indicators, there is no other standard reporting of whether they have achieved the results expected from their devolved budgets. With the number of aggregate results indicators in the single departmental plan significantly reduced, this leaves open the question of how DFID will convey the magnitude of its results to Parliament or the public. While DFID does communicate its results in various other ways – such as by describing particular types of impact or through human interest stories – this is a different issue from the communication of aggregate results.

3.75 We see these as potential gaps in DFID’s business processes. We will therefore come back to this in next year’s follow-up to see how the position has evolved.

4 Lessons learned

4.1 Overall, we find that DFID’s response to the reports we have followed up on has been positive in most areas. A substantial majority of our recommendations were accepted, in whole or in part. While it is too early to assess their effectiveness, many of DFID’s management actions are well considered and have the potential to make a real difference. However, there have also been areas where DFID did not agree with our analysis and, in some cases, is unable to move forward. In this concluding section, we look at some of the factors that contributed to stronger or weaker responses.

4.2 In the last two years, we have introduced a number of changes to the way we formulate recommendations. We have simplified our recommendations, limiting them to a single issue each, in order to make them clearer and easier to follow, and have made them less prescriptive, recognising that the responsible departments are generally better placed to identify the appropriate solutions. Our recommendations are now accompanied by problem statements that make it clear which issues from our analysis each recommendation is intended to address.

4.3 We believe that these innovations have been helpful and we found a higher level of understanding of our recommendations this year. However, we continue to receive feedback from DFID staff that they would like more interaction with ICAI around the formulation of recommendations. While we are not willing to negotiate our recommendations, we accept that more communication with DFID towards the end of the review cycle would be helpful. We have tested an approach to this on a recent review.

4.4 For our first two reviews, on WASH and VAWG, we were disappointed at DFID’s departmental response. At DFID’s request, the format was changed from a grid of management actions with a clear implementation timetable to a more discursive response. We found that the proposed management actions were too vague, while much of the discussion consisted of justifications of past actions or descriptions of ongoing work. At the urging of the International Development Committee,²⁴ DFID revised its responses to both reviews, and the responses to subsequent reviews have been of better quality.

4.5 It is very important for accountability purposes that DFID is clear about which aspects of the recommendations it accepts, and commits itself to specific actions (even if the details are not yet decided) that can subsequently be verified. We note, however, that the content of DFID’s departmental response is not necessarily a predictor of its subsequent actions. We found instances of positive actions on recommendations even after they had been rejected.

4.6 Among the reports we have followed up on, VAWG was the only topic to receive an overall green rating on our four-point scale. We did not award any overall red ratings. We were interested in DFID’s feedback that the positive report and learning frontiers had helped to give the VAWG agenda more profile and momentum across the department. It has reportedly attracted more staff to engage, and encouraged greater learning and innovation, which is an encouraging response.

4.7 Looking across the reviews, we can see ICAI recommendations making a difference through a number of mechanisms:

• improved programming guidance
• more disaggregation of results data, to support the ‘leave no one behind’ commitment
• greater attention to value for money, including data collection
• a greater focus on the sustainability of results
• new research and analysis
• stronger learning mechanisms
• improved staff training
• better monitoring processes and higher quality results data
• changes to core business processes
• new initiatives to influence multilaterals
• contributions to international dialogue and standard setting
• improved collaboration between UK government departments engaged in delivering aid.

4.8 We were pleased to come across some instances of DFID using ICAI reports in its dialogue with other donors and delivery partners. In this way, ICAI has the potential to influence the development of international norms and standards for aid delivery in particular sectors or areas.

4.9 Among the areas where DFID did not respond as well to ICAI recommendations, we have identified a number of patterns.

4.10 We found that DFID has been slow to act on recommendations that involved staff allocation – for example, to support its multilateral partnerships or strengthen programme management for high- risk programmes. There is continuing uncertainty as to whether DFID should develop a specialised programme management cadre, or instead seek to mainstream programme management skills across the department.

4.11 Perhaps most importantly, we found that DFID is not well positioned to implement recommendations that call for a change in approach across a sectoral or thematic portfolio. DFID currently prefers not to issue detailed strategies to guide its programming. As a result, in a decentralised department, central policy teams have few levers at their disposal to influence programming choices made at country level. This is a structural problem that makes it difficult for DFID to follow through on cross-cutting commitments, such as mainstreaming VAWG across sector programmes or its ‘leave no one behind’ pledge. We will explore this issue further in forthcoming reviews.

4.12 We have identified three outstanding issues of strategic importance which merit further follow-up in 2018:

• On VAWG: progress on mainstreaming VAWG initiatives into DFID sectoral programmes and the Conflict, Stability and Security Fund portfolio.
• On fiduciary risk management: progress on strengthening oversight of implementers in complex delivery chains, and in improving transparency and oversight arrangements for multilateral agencies delivering bilateral programmes.
• On results management: progress on aggregating and reporting on development results above the programme level.

Footnotes

1. While ICAI’s mandate extends to aid spent by any government department, the recommendations covered in this report were all addressed to DFID.
2. Leaving no one behind: Our promise, DFID, updated 10 January 2017, link.
3. Single departmental plan: 2015 to 2020, DFID, updated 1 September 2016, link.
4. Gender Equality, Poverty Reduction and Inclusive Growth: Gender Strategy 2016-2023, World Bank Group, 2016, link.
5. Smart Rules: Better Programme Delivery, DFID, April 2016, p. 31, link.
6. UK aid: tackling global challenges in the national interest, HM Treasury and DFID, November 2015, p. 10, link.
7. Quoted in The Future of UK Development Co-operation: Phase 2: Beyond Aid, International Development Committee, Tenth Report of Session 2014-15, January 2015, p. 6, link.
8. Busan Partnership for Effective Development Cooperation, December 2011, p. 2, link.
9. The Future of UK Development Co-operation: Phase 2: Beyond Aid, International Development Committee, Tenth Report of Session 2014-15, January 2015, link.
10. Raising the standard: the Multilateral Development Review 2016, DFID, December 2016, link.
11. Raising the standard: the Multilateral Development Review 2016, DFID, December 2016, p. 5, link.
12. The International Aid Transparency Initiative (IATI) is a voluntary initiative among donors, partner countries and civil society organisations to increase the transparency of aid flows by publishing expenditure data under the open access IATI Standard, link. The non-governmental organisation Publish What You Fund produces an annual Aid Transparency Index, scoring development agencies on their transparency. The results can be found here: link.
13. The Commonwealth Development Corporation Act 2017, February 2017, link.
14. Department for International Development: investing through CDC, National Audit Office, November 2016, p. 2, link.
15. Following a change of strategy in 2012, investments in China are being phased out. The South African portfolio includes work with South African businesses active across the Southern Africa region.
16. Growing in ambition and impact: CDC Group plc Annual Review 2015, CDC, 2015, p. 14, link.
17. Department for International Development: investing through CDC, National Audit Office, November 2016, link.
18. DFID’s Results Framework: Managing and reporting DFID results, DFID, 2011, link.
19. The Aid Management Platform is a tool that enables DFID staff to view data on expenditure from DFID’s management information systems in formats that facilitate management decision making at various levels.
20. Single departmental plan: 2015 to 2020, DFID, updated September 2016, link.
21. Methodology notes for the single department plan indicators are available online, link.
22. See footnote 21.
23. Sustainable Development Goals, UN website, accessed June 2017, link.
24. Oral evidence: ICAI’s Review on water, sanitation and hygiene, HC 102, International Development Committee, June 2016, link. Oral evidence: ICAI’s Review on violence against women and girls, HC 101, International Development Committee, July 2016, link.