ICAI’s approach to effectiveness and value for money

Our approach

There is much discussion in the aid sector about what VFM (value for money) means and how that relates to the concept of effectiveness in the context of international development. The Commissioners’ view is that, far from being separate and distinct concepts, VFM and effectiveness are closely interrelated.

The purpose of this study was to develop, in collaboration with the Commissioners, an approach to assessing and forming a judgement on the VFM and effectiveness of UK official development assistance (ODA). The study drew on current theory and practice but also considered emerging thinking and the future challenges facing the aid programme. Most importantly, through the use of illustrative case studies, it both shows how VFM and effectiveness have typically been assessed to date and how ICAI will make judgements in the future.

Review questions

Key questions that the study sought to address include:

The Department for International Development’s (DFID) and other donors’ current approaches 

• How are DFID and other donors approaching the measurement of VFM and effectiveness in development assistance?
• What questions is DFID asking about VFM and effectiveness when planning the aid programme?
• How is DFID now evaluating programmes which were set up under a different set of criteria and how is DFID seeking to change programmes as a result?
• How does DFID understand and define costs in relation to aid programmes?

Defining and measuring VFM and effectiveness

• To what extent has a consensus emerged on how VFM and effectiveness should be defined and assessed in an aid context?
• What performance measures should be used to define economy, efficiency, effectiveness and equity?
• What progress has been made in developing standardised output measures for different types of development result? Which areas of the aid programme lend themselves to quantitative measurement of results?
• How are the beneficiaries of aid consulted and involved in the delivery of VFM and effectiveness (e.g. through the design and ownership of aid and through feedback)?
• How do the concepts of VFM and effectiveness need to adapt to help evaluate the changing strategy and focus of UK aid?

Assessing VFM and effectiveness

• Should an assessment of VFM and effectiveness vary according to whether it is bilateral aid, multilateral aid or budget support aid and if so, how?
• What aspects need particular emphasis when assessing VFM and effectiveness in an aid context (e.g. sustainability)?
• How do the governance arrangements at all levels in aid programmes affect their VFM and effectiveness?
• What techniques have emerged for analysing VFM and effectiveness at different levels, e.g. project, bilateral portfolio, multilateral portfolio or corporate?
• Over what period of time is it sensible to form judgements about VFM and effectiveness – how long should we wait after aid has been delivered?
• What methods should be applied to assess VFM and effectiveness? Methods currently in use include randomised controlled trials and community impact assessments.
• What data should (ideally) be available from recipient communities in order to inform donors’ judgements about VFM and effectiveness?

Meeting ICAI’s requirement

• Overall, what approach best meets the remit of ICAI?
• What kinds of VFM reviews and evaluations might ICAI usefully conduct to complement and/or challenge existing reforms?

The approach developed supports our role in scrutinising the VFM and effectiveness of UK aid and provides guidance on how individual assessments can be undertaken.