ICAI follow-up of: Assessing DFID’s results in water, sanitation and hygiene

Executive summary

Each year, ICAI follows up on the response to the recommendations it made in reviews published the previous year. The process is a key link in the accountability chain, providing Parliament and the public with an account of how well DFID and other government departments have responded to ICAI reviews. It also provides feedback to the responsible staff, and serves as a useful learning exercise for ICAI in our quest to achieve greater uptake of our findings and recommendations.

This document is a summary which focused only on the results of our follow up of Assessing DFID’s results in water, sanitation and hygiene. The full Follow Up report of all our year five reviews, including overall conclusions from the process and details of our methodology, can be found on our website.

Findings

Assessing DFID’s results in water, sanitation and hygiene

In May 2016 we published our first impact report, exploring DFID’s results in water, sanitation and hygiene (WASH) over the period 2011 to 2015. We awarded a green-amber score, reflecting the fact that DFID had exceeded its target of reaching 60 million people with WASH interventions, and the evidence that its WASH programmes were contributing to wider development results, especially improved health outcomes. However, we identified a number of areas for improvement, including around results measurement and ensuring sustainable results. We made four recommendations in the areas shown in Table 1.

Table 1: Area of recommendations and DFID’s response

DFID’s initial departmental response to the review was disappointing. The narrative format combined explanatory argument with ongoing and planned actions, making it difficult to identify the proposed management actions. At the request of the International Development Committee, DFID subsequently revised its response to include more specific commitments. All the recommendations were accepted in full or in part, and DFID is engaging actively on most of the concerns we raised. However, there were also a few points of disagreement.

Monitoring results

On results measurement, we were concerned that DFID’s method of counting the beneficiaries of its WASH investments was based on estimates, rather than surveys of actual use, leaving it unable to identify whether vulnerable groups (for example the elderly or people with disabilities) were being reached. We also noted that DFID was not monitoring wider development impacts, beyond WASH access, such as health results or school attendance. Our view was that collecting wider impact data, at least on a sample basis, during the life of a WASH project would enable DFID to adapt its interventions in order to maximise results.
DFID agreed with the need to improve results monitoring, but not with all the details in our recommendation. It has developed a new methodology for collecting WASH results, based on surveys of actual use, with greater disaggregation of data. This will support an increased focus on reaching women and people with disabilities, in the first instance, and potentially other marginalised groups. On wider development results, DFID agreed to push ahead with collecting more impact data and strengthening its evaluation work. However, it disagreed with the suggestion of routine monitoring of health and school attendance data, citing practical challenges and the difficulties of attributing changes to WASH programming. DFID justified this stance on the basis that the links between WASH access and wider development results were already sufficiently established in the international literature.

Ensuring sustainability

We found that DFID had fallen behind international best practice in the WASH sector by failing to monitor whether results were being sustained beyond the life of its programmes. We proposed paying greater attention to sustainability in programme design – including by extending monitoring beyond the standard three-to-five year programme cycle.

DFID has responded quite well to this recommendation. It has been active in promoting improved standards on sustainability with other bilateral donors and multilateral agencies, such as the World Bank and Unicef, and it is in discussion with the UK Water Network on drafting a joint position paper on sustainability. We were pleased to find that DFID had used our report to draw partners’ attention to the importance of embedding sustainability into programme delivery and monitoring arrangements. In its own programming DFID is now challenging its implementers to pay more attention to sustainability in programme design and to include sustainability indicators in their results frameworks.

DFID has not yet taken up our suggestion of extending monitoring beyond the programme cycle, citing practical constraints in the way it funds its programmes. However, it is planning to undertake a thorough review of whether the benefits of its WASH investments in recent years have been sustained. It is also planning to invest £1.5 million in evaluating future programmes.

Improving value for money

We found that DFID had begun collecting data on the value for money of its WASH programming, but had not yet come up with a workable method of comparing the return across its investments. While acknowledging the technical challenges involved, we recommended that DFID press ahead with developing an appropriate value for money framework. DFID has updated its value for money guidance, drawing on a range of research, and is developing a set of value for money indicators, covering not just the economy of inputs but the full range of results data. This is part of a wider DFID initiative to strengthen its use of value for money metrics, and one of the most significant responses to our WASH report.

DFID has been working with Unicef – one of the leading global implementers of WASH programming – to introduce 12 core value for money indicators. DFID is also encouraging other bilateral donors and foundations to adopt these and hopes that they will develop into a global standard.

There remain tensions to be managed between a more active stance on value for money and DFID’s commitment to ‘leaving no one behind’.¹ DFID recognises that provision of WASH services to marginalised groups involves higher unit costs, and is building equity into its value for money framework.

Improving learning

We found that DFID was not systematically capturing and disseminating lessons learned from its WASH programmes to inform future programme designs. We therefore recommended a more active approach to learning – including joint learning with related sectors, such as health and education, to improve understanding of how investments in these areas interact.

While DFID recognised the concern and accepted our recommendation, its response has been disappointing, with no major new learning initiatives. DFID is making useful investments in knowledge generation, but faces challenges in using that knowledge systematically to guide programming choices. DFID’s response on this issue reflects a wider challenge for DFID’s central policy teams in providing effective guidance and technical support for programmes designed in country offices.

Conclusion

This is a strong response by DFID in most respects. We are particularly encouraged that DFID is tackling the two challenges of sustainability and value for money more actively, including by influencing practices and standards across the sector. We also welcome the commitment to improving the quality of results data. In its single departmental plan, DFID has set itself the target of providing 60 million people with sustainable access to clean water and sanitation.² If implemented effectively, the measures described here will help to ensure that this investment makes a lasting difference to the lives of beneficiaries.

Footnotes

¹ Leaving no one behind: Our promise, DFID, updated 10 January 2017, link.

² Single departmental plan: 2015 to 2020, DFID, updated 1 September 2016, link.