Chief Commissioner Speech at Crowe Clark Whitehill INGO conference 28 November 2013

28 Nov 2013

Focussing on the Beneficiary – ICAI’s Approach to Reviewing UK Government Aid Spend

Good morning and thank you for inviting me here today. I have the privilege of being the Chief Commissioner of the Independent Commission for Aid Impact. The last time that I was speaking at this conference, in early 2012, ICAI had just been established and we had delivered our first four reports.

Now, in 2013, we are about to deliver our thirtieth report and have undertaken studies on Official Development Assistance expenditure right across the UK Government portfolio. We have been to countries in Africa and Asia and beyond, examined programmes across many different sectors and delivered by multilaterals, contractors and, of course, NGOs. Indeed some of your organisations have enjoyed being part of the ICAI experience!

Our approach to reviews is set out in our first report, ICAI’s Approach to Effectiveness and Value for Money, where we give our view that effectiveness and value for money are inextricably linked. Fundamentally, how can a programme be value for money if it is not effective; and if there is poor value for money, is the programme being as effective as it could be?

We recognised that aid organisations, including DFID, take a range of different approaches to defining and ensuring effectiveness and value for money.

Our approach has been to assess the results that have been achieved on the ground. In our view:

■          effectiveness involves achieving a sustained and positive impact for intended beneficiaries; and

■          value for money is the best use of resources to deliver the desired impact.

In forming our ratings for programmes we consider key stages in the aid project lifecycle:

• Objectives – Does the programme have realistic and appropriate objectives and a clear plan as to how and why the planned intervention will have the intended impact?
• Delivery – Does the programme have robust delivery arrangements which meet the desired objectives and demonstrate good governance and management throughout the delivery chain?
• Impact – Is the programme having a transformational, positive and lasting impact on the lives of the intended beneficiaries and is it transparent and accountable?
• Learning – Does the programme incorporate learning to improve future aid delivery?

We stand by this approach. We also have refined our methodology and will soon publish an updated assessment framework, which builds upon our experience.

Mark Lowcock, Permanent Secretary of DFID, recognised in front of the International Development Committee that ICAI has “identified the important set of issues around beneficiary monitoring and feedback, and using the views of people whose lives DFID is trying to improve to drive the nature of programme development and prioritisation, and the way programmes work in practice”.

Over the past year we have seen some very good examples of where the beneficiary has been at the heart of the design and implementation of programmes and it is here that we have seen the most sustainable interventions. I would like to share some of our findings, on some of these projects, with you.

Our review of DFID’s Water, Sanitation and Hygiene Programming in Sudan demonstrated where beneficiary focus worked effectively in one programme and contrasted strongly with others where that focus was missing.

The Integrated Relief and Recovery programme is implemented by Tearfund, working with the Government of Sudan and local community groups funded through a NGO accountable grant. It is an emergency and early recovery programme, which focuses on improving health and well-being through interventions to improve water supply. Its objectives are to reduce malnutrition and mortality among children under five through various measures, including access to safe water.

From the beginning of the project in 2007, Tearfund consulted with the beneficiaries leading to an allocation of 40% of resources to WASH as the best means of achieving its goals. The programme focussed on newer Internally Displaced Person settlements and rural communities which were underserved with clean water sources.

We also found that Tearfund were responsive to the changing strategic context of the Darfur conflict. Their projects minimised dependence on aid and invested in building sustainable capacity in its target communities.

They built exit strategies into their engagement with communities, limiting financial support for the operation and maintenance of new water facilities to six months. Tearfund provides support in phases; enters into handover arrangements with communities and water authorities and terminates emergency support once basic needs are met.

It also has a high level of beneficiary engagement through ‘village development committees’ and ‘water-user groups’, together with the training of maintenance staff. Its sanitation activities focussed on boosting community demand for sanitation. This beneficiary engagement appears to be producing more sustainable results.

Additionally, the project focusses on environmental sustainability. Through its integrated approach, it has built hygienic slaughter yards, introduced waste carts and disposal pits, constructed more than a kilometre of drainage systems and conducted more than 240 neighbourhood clean-up campaigns.

Because of its greater focus on beneficiary engagement and sustainability, we found the project to be an appropriate set of interventions for a protracted conflict.

We did not just however examine that programme. We looked at other, less successful, programmes, such as the UNOPS managed Darfur Urban Water Supply (DUWS) project. This project was designed to address the impact of mass urbanisation on urban water supply in a drought-prone country context. The project constructed mechanised boreholes to increase the overall water supply to four urban centres.

Implementation of the DUWS project was held up, mainly by delays in obtaining permits from the Government of Sudan for import of materials and construction; this is evidence of insufficient investment by UNOPS in building relations with the national authorities. Additionally, there was an assumption that the Government of Sudan would invest in extending the piped network to more of the urban population. This investment was not made.

As most urban households are not connected to the piped network, we observed in El Fasher that the main beneficiaries of any increased water supply were private companies, government institutions and international organisations.

Had the project been set a more beneficiary-focussed objective, such as lowering the price of water for poor households, it would have been obliged to engage directly with the operation of the urban water market. Without such engagement, we found that the project had a sound aim but an inadequate design for delivering it.

The stark contrast shown in the fortunes of these two projects is in part due to their differing beneficiary focus. Tearfund designed and implemented a project that understood the needs of the beneficiary in a changing context from end-to-end. The UNOPS project, while having noble intentions, was dogged by lack of understanding of the context, particularly for the intended beneficiary; lack of buy in from the Sudan Government; and poor design. This and other projects that we looked at led to an assessment overall of Amber-Red but had it been a report purely on Tearfund’s project it would have been Green.

Our report on Empowerment and Accountability in Ghana and Malawi examined a direct grant of £2.5 million given to a group of Malawian CSOs to run Kalondolondo, a project promoting citizen monitoring of public services through the use of community scorecards.

The scorecards were used to collect feedback on the delivery of development projects and public services, such as building schools, HIV-AIDS services and the use of local development funds. The process generated evidence on local problems and issues, which are then raised with the responsible authorities through community meetings.

Additionally Kalondolondo also shared its findings with national authorities, often inviting local community members to testify to their accuracy. This influencing strategy is effective, mainly for resolving problems in the delivery of existing government programmes, in cases where national authorities share an interest in improving services.

A good example is where Kalondolondo surveyed the Farm Input Subsidy Programme, through which the poorest rural households across Malawi receive coupons for the purchase of subsidised seed and fertiliser. According to the communities, the programme had major corruption problems, with subsidised inputs often acquired fraudulently by merchants, causing the intended beneficiaries to miss out. Kalondolondo produced evidence that a poorly organised distribution system was exacerbating the problem. Following its representations to national authorities (although not solely attributable to Kalondolondo), the distribution system was modified.

Now, all beneficiaries from a particular group of villages are scheduled to purchase their inputs on a particular day, in alphabetical order, with the community itself policing the process. Because community members are well known to each other, interlopers are easily detected and excluded. The communities we visited reported that this had resulted in major improvements to the programme.

The best example that we have seen of beneficiary involvement is the Western Orissa Livelihoods Project (WORLP) in India, which sought to reduce poverty by improving communities’ water resources, agriculture and incomes. It built infrastructure such as embankments, water storage ponds and irrigation channels. It also provided loans and grants to the poor for community-based businesses. The project took place in four of the poorest of Odisha’s 30 districts, between 2001 and 2011.

The project was designed in close consultation with communities. Communities held meetings to decide whether to participate in the project. Organisations of beneficiaries implemented activities and set local priorities, particularly for funding. Beneficiary groups were also engaged in monitoring project progress.

But encouraging beneficiaries to participate and building community institutions requires time. The project was designed with a ten-year timespan. Such long-term planning is currently unusual for DFID but we think that this timespan was appropriate for the project and it may be for other, similar projects that plan to overcome poverty by involving participants deeply in delivery.

Delivery was built around beneficiary involvement, focussing on those who were socially and economically marginalised. Each community had a detailed ‘micro-plan’ to prioritise project interventions. Such plans were negotiated and were the product of considerable debate within communities. This process itself built the capacity of communities to take control of their own livelihoods.

The beneficiaries were organised into groups, through the project, so that they could establish common interests. Such groups included:

■          Watershed Associations and Watershed Development Committees: these planned and implemented schemes. Members were mainly small farmers;

■          User Groups: these were responsible for managing and maintaining the physical infrastructure created in the project. Members were mainly families who owned land;

■          Self-Help Groups: these ran savings and credit schemes and developed micro-enterprises. Members were mainly women and the poorest households; and

■          Common Interest Groups: these developed community resources to improve incomes. Members were mainly men and women from the poorest households.

The project’s anti-corruption approach emphasised community-level accountability and was facilitated by being fully transparent to beneficiaries in villages. It was reinforced by the detailed record-keeping, control mechanisms and comprehensive audits. The NGO and district teams were particularly responsible for ensuring accountability systems were in place.  We found evidence of two cases of corruption. Both involved the community committee secretaries forging authorising signatures to draw money out of project accounts. The communities had noticed the discrepancy in funds and involved the police. All funds were recovered and, in both cases, the secretaries were prosecuted and imprisoned.

We think that that project’s transparency of funding to beneficiaries for these activities was exemplary. Direct funding of communities with transparency helped to reduce fraud and made it possible for communities to tackle cases of fraud when they took place.

In assessing the overall project we saw much that we would consider to begood practice and saw evidence that the project has been sustained once the project was completed. The involvement of beneficiaries from objective setting through to delivery and learning contributed to our overall rating of Green, the first one that we awarded and one of only two so far.

Our other Green rated report, on Health Programming in Burma, also considered the needs of the intended beneficiary.

Following Cyclone Nargis in May 2008, DFID Burma led the way in conducting a needs assessment in the Irrawaddy Delta, building on good relationships with the local communities, including local government officials.

This resulted in the creation of the Delta Maternal Health programme, which aimed to improve maternal and child health in the townships most affected by Cyclone Nargis. It also aimed to increase access to quality maternal and child health services, working through township health plans. These township plans sought to address the needs of intended beneficiaries at the local level, with flexibility to be updated to reflect changing priorities.

In the Delta Maternal Health programme, DFID has used its knowledge of what intended beneficiaries need, from the work it did with them following Cyclone Nargis, on how best to deliver health services. It has laid the foundation for stronger state–citizen relationships and capacity-building in the future. By developing relationships at the township level, DFID Burma aims to integrate services with those provided by the regional and national authorities.

So, as you can see from these examples, effective beneficiary engagement and involvement improves programmes which, thereby, are more likely to succeed.

There are several other recurrent factors that we identify in our reports that, where handled well, can contribute to successful interventions.

For example, getting the Theory of Change right is critical to the ensuing success of each programme. It needs to take account of many variables, including the country context, drivers and available levers.

We saw contrasting approaches to education theories of change in DFID’s Nigeria and Pakistan programmes. In Pakistan, DFID’s approach comprised financial aid, extensive political dialogue, technical support, funding for research and innovation and a set of interventions designed to generate social pressure for change.

In practical terms, the programme involves a series of interventions collectively designed to improve learning outcomes. These include a drive on teacher attendance, the introduction of lesson plans, teacher performance incentive schemes, upgrading of school facilities and work with the examination board to raise the standards of assessment.

In contrast, we found that DFID’s approach in Nigeria targeted the improvement of the education system as a whole, without prioritising either the ultimate objective of pupil learning or support for weaker schools.

Applying learning can also make the difference in successful interventions. We noted in our Horn of Africa report, strong evidence of learning from previous interventions within each of the affected countries, as well as from other humanitarian interventions. We have seen, however, evidence of lessons and experience between country offices and central teams not being shared. This is a critical issue and we are currently undertaking a report to examine and address this challenge.

An additional factor in successful interventions is staff. DFID needs personnel that have the right capabilities to: commission and manage other delivery partners; build networks; and, critically, manage programmes well.  Staff churn is also a challenge. This was proved by the Health team in Burma, which built tremendous knowledge and networks in-country which have enabled them to deliver beyond expectations.

Several members of the team, after many years in the country now are moving on; but the success, in some ways, is down to that experience and local knowledge gained.

I am now going to take a couple of minutes to talk about one of the reports where we focussed less on the beneficiary but I know will be of interest to many of you, this is our report on the Support for Civil Society Organisations through Programme Partnership Agreements.

Our findings showed that PPAs are helping CSOs to drive innovation in their organisations and that the quality of performance management and accountability for results is improving. We believe that it is likely that these changes will lead to improved results for intended beneficiaries.

But we did identify issues that those of you that hold a PPA will recognise:

• The disproportionate burden of monitoring (particularly for smaller CSOs);
• Outside of DFID’s CHASE department, where technical counterparts were appointed, there is a lack of collaboration and contact points;
• The initial uncertainty surrounding PPAs in 2010 and the lack of strategic direction, once the decision was taken to hold a funding round.

Our recommendations sought to address these issues and DFID agreed with us except on the point about technical counterparts, where DFID believed that the current mechanisms were sufficient. We would, of course, welcome your views on that!

One particular improvement in this PPA round that we observed was the Learning Partnership, which has led to greater innovation especially in the area of measuring results and impact.

For example, both CAFOD and Oxfam report that they are learning from unsuccessful activities. This shared learning and consequent evidence base has potential to lead to greater impact for intended beneficiaries.

Finally, please indulge me while I reflect on some of the impacts that ICAI has had since its creation in 2011 as a result of our year one reports and recommendations. For example:

• in Zimbabwe, DFID has negotiated significant reductions in administrative percentages paid to distribution organisations, which, in turn, has meant UK aid has reached an additional 170,000 people;
• our report on education in East Africa has been used in discussion with the Government of Tanzania and among top DFID officials to help to drive a positive change of approach towards learning outcomes; and
• Girl Hub officials have introduced a new and improved approach to girl safety, including, for the first time, a child protection policy.

These are improvements to the lives of many, many thousands of intended beneficiaries and a record of which we are proud. As we follow up on our reports in the future, I believe that this will continue and will justify our belief that that our approach to focus on the beneficiary is right and will show the tax payer that the UK Aid budget is being scrutinised effectively.

Chairman, ladies and gentlemen – thank you very much for your attention.