ICAI review of transparency in UK aid

Abbreviations

AAA                             Accra Agenda for Action

AIMS                           Aid information management system

AMP                            Aid Management Platform

ATI                               Aid Transparency Index

BPEDC                         Busan Partnership for Effective Development Cooperation

CSO                             Civil society organisation

DFID                            UK’s Department for International Development

FCDO                           Foreign, Commonwealth and Development Office

FCO                             Foreign and Commonwealth Office

IATI                              International Aid Transparency Initiative

ICT                               Information and communication technology

NGO                            Non-governmental organisation

PrOF                            Programme Operating Framework

1. Introduction

This literature review accompanies and contributed to the general evidence gathering for the ICAI rapid review of Transparency in UK aid.[1] This rapid review assesses the compliance of the Foreign, Commonwealth and Development Office (FCDO) and its predecessor departments – the Department for International Development (DFID) and the Foreign and Commonwealth Office (FCO) – with aid transparency standards and explores the extent to which the departments’ efforts to promote transparency have led to greater accountability and effectiveness.

It is broadly recognised that aid transparency – information on what aid is being spent where and by whom, and its results[2] – is valuable in its own right, given the inherent importance of openness to those who are affected by and contribute to aid (Carothers and Brechenmacher, 2014; McGee, 2010; Glennie et al., 2021). There is also emerging literature on the role of aid transparency in promoting better development outcomes, through improving the accountability and effectiveness of aid. This literature review explores the theoretical and empirical literature linking aid transparency to accountability and effectiveness outcomes, to provide the background for the rapid review’s response to the following sub-question (under the ‘Effectiveness’ element): 2c) To what degree have transparency efforts helped to improve the accountability and, subsequently, the effectiveness of UK aid?

There is extensive literature exploring the concept of accountability in the public realm and how it links to transparency. A common view is that accountability requires agents, such as public officials, to be answerable to principals, such as taxpayers, for their performance and behaviour, and that principals also need the power to sanction or change behaviour (referred to as ‘enforceability’ in the literature) from agents (Gaventa and McGee, 2010; Ocampo and Arteaga, 2014). Based on this, aid transparency provides the foundation for accountability by allowing principals for aid (such as those affected by and contributing to aid) to understand the performance and behaviour of agents for aid (that is, donor officials and delivery partners) (McGee, 2013). This literature review refers to this framing of the concepts of and links between aid transparency and accountability.

This literature review has consulted 79 published sources, covering English-language academic literature, international policy documents, UK government corporate documents and some analysis by policy experts and non-governmental organisations. These sources mainly focus explicitly on aid transparency, but also draw on some relevant literature on the links between transparency, accountability and effective resource use in relation to development. This review does not draw on literature focused on the use of aid to support accountability and effective resource use (such as through building civil society capacity) in partner countries, as this is outside its scope.

The methodology for locating literature on aid transparency involved using Google Scholar (using search terms relevant to the topic)[3] to identify all relevant literature. The main limitation of this review is that it only consults English-language sources.

In exploring the aid transparency literature, we begin by examining the policy literature to identify the key channels through which aid transparency is theorised to link to accountability and effectiveness (Section 2), before more fully exploring the theoretical and empirical literature in relation to each of these channels (Section 3) and ending with a synthesis of the analysis and concluding remarks (Section 4).

[1] Transparency in UK aid: rapid review, Independent Commission for Aid Impact, October 2022, link.
[2] This definition is based on the definition used by the aid transparency campaign Publish What You Fund – see Why transparency matters, Publish What You Fund, 2022.
[3] These included ‘aid’, ‘development cooperation’ and ‘aid effectiveness’, AND ‘transparency’, ‘reporting’ and ‘accountability’.

2. The policy origins of the aid transparency agenda and its link to accountability and effectiveness

This section presents an overview of global and UK policy literature on aid transparency and attempts to identify how the importance of aid transparency was conceptualised, and the channels through which this policy agenda was intended to promote accountability for and the effectiveness of aid.

Aid transparency was first identified as an explicit priority for the international aid community in the agreement emerging from the Second High Level Forum on Effectiveness (held in Paris in 2005) – the Paris Declaration on Aid Effectiveness. As part of the commitments on ‘mutual accountability’ between aid donors and recipients, the Paris Declaration stated that “Donors commit to: Provide timely, transparent and comprehensive information on aid flows so as to enable partner authorities to present comprehensive budget reports to their legislatures and citizens” (OECD, 2005:8). The framework of indicators developed to monitor implementation of the Paris Declaration included a focus on tracking the proportion of aid to the ‘government sector’ captured in government budgets but did not track reporting on aid flows beyond the government sector by donors (OECD, 2005: 9).

The agreement emerging from the Third High Level Forum on Aid Effectiveness (held in Accra, Ghana in 2008) – the Accra Agenda for Action (AAA) – placed a stronger emphasis on the importance of aid transparency and noted that “greater transparency and accountability for the use of development resources – domestic as well as external – are powerful drivers of progress” (OECD, 2008:5). In addressing transparency, the AAA committed donors to “publicly disclose regular, detailed and timely information on volume, allocation and, when available, results of development expenditure” in order to support “more accurate budget, accounting and audit by developing countries” (OECD, 2008:6). However, no specific indicators were introduced to monitor the implementation of these added commitments by donors.

The approach to aid transparency promoted by the Paris Declaration and the AAA puts partner country governments at the centre of the aid relationship, and aims to use public institutions, processes and oversight (especially around domestic resource management) to promote accountability to the legislature and citizens.

The agreement emerging from the global summit on aid held in Busan, South Korea in 2011 – the Busan Partnership for Effective Development Cooperation (BPEDC) – represented a further evolution in the approach to aid transparency. The BPEDC identified “transparency and accountability to each other” as one of four core principles that were critical to delivering results, and emphasised that this should be focused on “the intended beneficiaries of our co-operation”, as well as “citizens, organisations, constituents and shareholders” (OECD, 2011:3). The transparency section of the BPEDC also stated that emphasis would be placed on efforts to “strengthen the capacities of all relevant stakeholders to make better use of this information in decision-making and to promote accountability” (OECD, 2011:6). This perspective on aid transparency identifies the importance of making aid information available to all relevant stakeholders, so that they can play a role in promoting accountability for and the effectiveness of aid.

The BPEDC also committed the international aid community to pursue efforts to “[implement] a common, open standard for electronic publication of timely, comprehensive and forward-looking information on resources provided through development co-operation”. In addition, there was an emphasis on ensuring that this standard met “the information needs of developing countries and non-state actors, consistent with national requirements” (OECD, 2011:6-7). This commitment was monitored through an indicator which tracked the emphasis of donor reporting on aid flows in terms of its timeliness, level of detail and forward-looking focus (OECD-UNDP, 2014:133). This commitment generated momentum around launching and expanding support for the International Aid Transparency Initiative (IATI), which developed a common standard for publication of aid data that donors began to apply in 2011.

IATI, which has since expanded to include over 1,000 publishers of aid information, has pursued a pluralist approach to aid transparency which emphasises that “[each] partner country has its own specific information needs, with various stakeholders requiring aid information for different purposes”. A briefing published by IATI, which introduces the work of the initiative, identifies the types of information, needs and uses that each distinct group of stakeholders has (see Box 1).

Box 1 – Aid stakeholder groups and their information needs

  • Finance and budget ministries need data that can be easily integrated into their planning and budgeting systems, to improve planning and accounting processes. Their priority is timely, accurate and comprehensive data. They also need forward-looking information to ensure effective budget projections.
  • Line ministries require similar information, but with more specific focus on aid that is directed to their particular sectors, for example health or education. Information on expected and actual results is critical, and detailed geographic information on aid projects is required to enable better coordination and planning of services.
  • Parliamentarians play a vital role in holding governments to account for the resources and services provided to their constituents. For accountability mechanisms to work effectively, parliamentarians need access to detailed, timely, comprehensive and consistent data from both national governments and donors.
  • Civil society organisations use aid information to support their advocacy and hold service providers to account. They have a key role to play in tracking aid expenditure through the system; to do this they need information on geographic locations of projects and details of implementing organisations.
  • Donor agencies and international non-governmental organisations operating in each country need better information about aid allocation to assist with better coordination.
  • Implementing agencies and organisations often struggle to plan and coordinate their efforts effectively with other stakeholders. Timely information regarding geographic location, sector and project descriptions can provide implementers with the data that they require to plan and coordinate well, avoiding duplication of effort.

Source: IATI (n.d.)

The illustration (in Box 1) of the ecosystem for aid transparency presented by global aid policy and IATI seems to identify pathways through which the sharing of information about aid should promote accountability and effectiveness. With regards to accountability, the vision is that aid transparency will allow those actors with a role in overseeing resource use in partner countries – namely legislatures and civil society organisations – to use this information to ensure that accountability mechanisms function and that they can hold relevant actors accountable for their use of aid (alongside domestic resources). The hope is that closer scrutiny of the use of aid will lead to more effective use of development finance.

It is also envisaged that aid transparency can promote greater awareness of activities undertaken across the aid system, so that different actors can work together more effectively. This involves ensuring that government ministries (especially finance ministries) can take into account an understanding of how donors are planning to use their aid, so that they can plan to use their own resources in a way that complements donors’ efforts. It also involves allowing donors and implementing agencies to understand what their counterparts are doing, so that they can find ways to coordinate and complement each other’s work.

Many of these themes about how aid transparency promotes accountability and effectiveness also appear in UK government policy documents since 2010, with some additional nuances.

In 2010, the UK’s Department for International Development (DFID)[4] adopted an Aid Transparency Guarantee, with a central commitment to “publish detailed information about all new DFID projects and programmes on our website, in a common standard with other donors”, as well as a commitment to require DFID partners to apply transparency standards. In introducing this guarantee, DFID made clear that the main targets of transparency were both “UK taxpayers and citizens in poor countries”, so that they “can more easily hold DFID and recipient governments to account for using aid money wisely” (DFID, 2010).

In 2012, DFID introduced a further set of aid transparency commitments, requiring organisations receiving and managing DFID funds “to release open data on how this money is spent in a common, standard, reusable format” and to “require this of sub-contractors and sub-agencies – right through the aid chain” (DFID, 2012a). In a speech introducing this requirement, the then secretary of state, Justine Greening, made it clear that its aim was to support transparency to “UK taxpayers”, but also to stakeholders in partner countries, as “[transparency] means that recipient governments can plan and manage the resources coming into their country. It empowers citizens and parliamentarians in these countries to hold governments and donors to account” (DFID, 2012b).

This emphasis on the importance of publishing aid information for both UK taxpayers and partner country stakeholders was also reflected in DFID’s management systems. Its Smart rules, which were introduced in 2014 and which guided DFID’s programmes until the department was merged with the Foreign and Commonwealth Office (FCO) to form the Foreign, Commonwealth and Development Office (FCDO) in September 2020, stated that transparency was a core principle and that “British taxpayers and beneficiaries have a right to know what we’re doing, why we’re doing it, how we’re doing it, how much it will cost and what it will achieve” (DFID, 2020:9).

In April 2021, FCDO published its Programme Operating Framework (PrOF), which aims to guide programme delivery in the new department. In setting out the case for aid transparency, the FCDO PrOF states that transparency helps the UK government to be more accountable, efficient and effective through the channels listed in Box 2 below.

Box 2: FCDO PrOF – Case for aid transparency and channels through which it promotes accountable, efficient and effective aid

  • Supporting evidence-based decision-making by feeding into FCDO management information
  • Improving engagement with programme constituents (beneficiaries), enabling empowerment of choice and control in programmes
  • Providing better oversight and coordination of spend
  • Reducing duplication by sharing information with others.
  • Delivers comprehensive, relevant and accessible aid information to the public domain via devtracker and gov.uk.
  • It enables sharing of information with countries where ODA spend supports better outcomes
  • It helps us track funds to downstream partners and helps address corruption

Source: FCDO (2021)

The UK policy documents make it clear that transparency commitments are important for citizens in donor countries, to enable accountability to taxpayers, as well as for governments and citizens in recipient countries.

Overall, global and UK policy documents identify four main sets of actors who play a role in the relationship between aid transparency, accountability and effectiveness, including:

  • Partner country governments – who can use aid information to inform their own policies and engagement with donors, and also to engage directly with citizens and their representatives.
  • Partner country citizens and their representatives – who can use aid information to engage with aid delivery partners, their governments and donors.
  • Donor country citizens and their representatives – who can use aid information to engage with their governments and donor agencies.
  • Donor and delivery agencies – who can use aid information to inform their policies, and whose behaviour is shaped by aid transparency efforts.

The subsequent review of academic and empirical literature on the links between aid transparency, accountability and effectiveness is organised around these four sets of actors. The next section explores both the theory and the empirical evidence as to how each group of actors makes use of aid information to pursue accountability and promote effectiveness.

[4] In September 2020 this department was merged with the Foreign and Commonwealth Office to form the Foreign, Commonwealth and Development Office.

3. The academic literature linking aid transparency to accountability and effectiveness

3.1     Partner country governments

Theory

The theoretical literature suggests that because aid spent in partner countries is significant to a range of government policy- and decision-making processes, improved reporting on aid flows to governments can help to make these processes more effective and accountable, thereby supporting improvements in overall resource use.

Where aid flows into a country are substantial, these can have consequences for macroeconomic variables such as the exchange rate, debt levels and the money supply (Isard et al., 2006). As a result, accurate and timely information on aid flows is vital to efforts by recipient governments to manage these macroeconomic challenges and design policies to address them effectively (Moon and Williamson, 2010).

Where donors are providing financial aid directly to governments, it is especially important to ensure that timely information on aid flows is provided to finance ministries, to feed into budget processes (McGee, 2013). Even where donors are working outside of government systems or through other partners, sharing information about the amount and purpose of aid flows can help governments decide where best to focus their resources and improve their overall allocation (Moon and Williamson, 2010). The evaluation of the Paris Declaration noted that in a number of the country studies governments highlighted concerns about aid not reported to them for inclusion in their budgets, and that this challenge needed further attention (Wood et al., 2011). One area in particular where government knowledge of donor spending plans is important is in relation to capital projects, as there are likely to be recurrent budget implications for sustaining these projects that governments need to plan and budget resources for (CABRI, 2008a).

Aid transparency can also support the effective functioning of governments by enabling reporting on aid flows and utilisation to parliaments, which helps promote democratic accountability for aid (IATI, n.d.). Best practice on ensuring that aid information is reported to parliaments involves ensuring that the donors report this information in a suitable format and at the right time to be included in the revenue and appropriations approved by parliament (CABRI, 2008a). In most countries, aid loans have to be approved by parliaments, and therefore the key gap has been in ensuring that grants are effectively reported to parliaments (CABRI, 2008a).

Although aid dependency has generally been falling in recent years (ActionAid, 2015), aid is still an important resource for low-income countries (ERD, 2015). Some authors suggest that donor efforts to better share aid information for use by governments will be most impactful in these countries (Moon and Williamson, 2010).

Practice and constraints

There are indications that aid transparency initiatives, such as the International Aid Transparency Initiative (IATI), are helping partner country governments access improved information on donor activities and carry out basic aid management tasks more effectively. Recent examples include the following:

  • Chad – In 2017, Chad’s Ministry of Finance established the Observatoire Tchadien des Finances Publiques to assist efforts to improve transparency in relation to public finances. This body has used IATI data to produce a report on international finance. The first of these reports was published in October 2020 and was presented to the national assembly as an appendix to the Finance Bill for the 2021 financial year (IATI, 2021).
  • Lesotho – Since 2017-18, the Ministry of Development Planning has used IATI data to produce an annual mapping of donor resources, published as the Lesotho development cooperation report. IATI has provided the ministry with a wider range of data, beyond just looking at aid to the public sector. This has helped to inform policy dialogue with donors and better inform discussions about coordination (IATI, 2021).
  • Madagascar – Initially, IATI data was used to check information being reported through the government’s aid management platform (AMP). Work  to formally link the AMP to IATI began in 2015, so that IATI data could be used to update the AMP (IATI, 2021). With the support of IATI data, Madagascar’s government has been able to get a fuller picture of aid flows into the country. This has even helped it to identify and begin formal collaboration with development partners operating in Madagascar that it had not previously known about (IATI, 2022).
  • Nigeria – The Federal Ministry of Finance, Budget and National Planning used IATI data to produce its Development cooperation report 2016-20 and to inform the development of its medium-term expenditure framework (IATI, 2020).

The formal evaluation of the implementation of the Paris Declaration (as well as the Accra Agenda for Action), undertaken in 2011, identified some signs of progress on donor reporting to partner country governments and links to parliamentary accountability. Most of the country evaluations that fed into the overall evaluation explored progress on donor transparency, and in half of these cases transparency was shown to have increased through establishing national databases on aid flows. The evaluation also found that in these same cases where donor reporting had improved, “[accountability] to (and thus through) parliaments is found to have been enhanced… although the degrees of improvement vary widely” (Wood et al., 2011:37).

No other substantive studies were seen to explore whether and how the reporting of aid information to partner country governments has helped to improve accountability and effectiveness. Overall, however, the literature does suggest that only limited progress has been made to date in using this channel to strengthen the accountability and effectiveness of aid.

Despite repeated commitments made by donors to report their aid to partner countries for inclusion in their budget processes, there has to date been only limited action to address this challenge, and performance may have worsened since the evaluation of the Paris Declaration. Surveys monitoring the delivery of global aid reform commitments have shown that the proportion of aid to the government sector reported in partner country budgets initially improved during 2011-14, remained largely stable during 2014-16, but then suffered a significant decline during 2016-18, with both development partners and partner governments reportedly contributing to this outcome (OECD-UNDP, 2019). As a result, by 2018, less than two-thirds of aid for the government sector was reported in partner country budgets (OECD-UNDP, 2019). It is also the case that these statistics do not take into account how effectively information on aid managed outside of government and by other partners is made available to feed into budget discussions (CABRI, 2008b).

Research on the use of aid information management systems (AIMS) – information and communication technology (ICT) applications which have been used to support information-sharing on aid at the country level – across developing countries has also identified similar reporting problems. Park (2017) explored the historical experience of 75 different AIMS that have been in operation across 70 countries since they began to emerge in the mid-1990s, and found that only 16% of these were “relatively active and being used”. The rest had either not been updated by donors within the last six months or had been shut down.

A range of factors have been found to obstruct efforts to link aid reporting to partner country budget processes. An analysis of challenges in delivering on international commitments to report ‘aid on budget’ found that the “ability or willingness of donors to provide meaningful and complete information on time to inform the budget preparation process varies across countries”. It also noted that even when this information is provided to partner countries, it may not end up in their budgets, partly because of concerns about how aligned this aid is to government plans and whether it will be delivered on time, but also because of institutional challenges and weak information management systems in partner countries (OECD, 2012:49).

Research exploring country-level experiences with incorporating information on aid flows into budgets has found that incentives do not always work in favour of including aid data in budgets. Studies in Ethiopia, Mozambique and Uganda identified that some government agencies were reluctant to report the aid targeted at their activities because of concerns that they would receive reduced levels of resources from government budgets (CABRI, 2008a:30-32). It was also noted in the Uganda case study that donors face disincentives to report project aid because these resources were taken into account in agreeing budget ceilings and therefore reporting this planned spending could lead to lower domestic spending in relevant sectors (Williamson, 2008).

Analysis also suggests that where a formal AIMS is being used, a higher share of development cooperation is recorded in the national budget (OECD-UNDP, 2019). However, research into the use of AIMS across 70 countries found that these had all too often failed because “[in general], the problem-solving capacity of ICTs as a tool were over-estimated, while the political and institutional arrangements required for implementing and maintaining the systems were underestimated” (Park, 2017:2587).

The literature also finds that even when aid is recorded in partner country budgets, the impact in terms of promoting accountability for and the effectiveness of this aid is shaped by weaknesses in the accountability of budget and other relevant government processes. One study in Kenya concluded that “[lack of] parliamentary scrutiny of proposals regarding the use of external funding has less to do with transparency on the proposed use than with weak and even inappropriate engagement of parliament in the budget process” (Fölscher, 2008a:16). Another study in Ethiopia stated that, with regard to parliamentary oversight, “in practice little time is given to this activity” (Bladon, 2008:17). Research also identified capacity constraints faced by parliamentarians in Ethiopia (Bladon, 2008), Kenya (Fölscher, 2008a), Mali (Betley, 2008) and Tanzania (Fölscher, 2008b), limiting the role they could play in oversight of aid spending. These dynamics therefore work to undermine the ability of citizens to pursue answerability and enforcement in relation to their government’s use of aid.

Insights from research on budget and accountability processes in partner countries point to similar conclusions about weak budget processes causing obstacles to accountability for aid. Studies have found that in poorer countries with high levels of dependence on external resources there are major challenges in ensuring that actual spending matches approved budgets (de Renzio et al., 2019). The 2021 Open Budget Survey found major weaknesses in the accountability of budget systems in low-income settings, particularly in terms of public participation in budget decision-making, and issues related to the transparency and oversight of budgets (IBP, 2021:16). Similarly, an analysis of the effectiveness of transparency and accountability initiatives focusing on budget processes concluded that efforts to improve the transparency and accountability of budgets cannot alone address the most pressing development challenges, and that there need to be links to wider efforts to promote accountability and good governance (Carlitz, 2010).

3.2 Partner country citizens and other affected populations

Theory

Although the field of aid transparency is a relatively new area, initiatives for promoting transparency and accountability in relation to other development resources and sectors have a longer history, dating back to at least the 1990s. A wide range of these initiatives have been implemented in recent decades, with a strong emphasis on supporting citizens to play a role in improving service-delivery outcomes, reducing corruption and making budget processes more accountable (Gaventa and McGee, 2010). Aid transparency initiatives are seen by some as a subset of this wider movement on aid accountability, with a role in ensuring that aid empowers communities, promotes participatory development and ultimately supports better development outcomes (McGee 2013; Glennie at al., 2021).

There is extensive literature exploring the theoretical pathways through which citizens in partner countries can influence the use of domestic public resources (World Bank, 2004; Joshi, 2010; Hause et al., 2022). However, it is also clear from the literature that in the aid sector these same models of accountability do not apply. First, donor governments are primarily accountable to their own citizens and their representative institutions in spending aid, and formal mechanisms through which intended beneficiaries can secure effective accountability from donors do not exist (Gibson et al., 2002; Martens et al., 2002; McGee, 2013). Second, there is the widely acknowledged dynamic that the organisations implementing aid programmes for donors are all too often more focused on upwards accountability to these donors than on downwards accountability to the communities they are working in, and are therefore constrained in their responses to them (Devine, 2003; Elber and Arts, 2011; Morfit, 2011). These dynamics mean that the pathways through which aid transparency can hope to empower citizens (and other stakeholders) in partner countries are complex and uncertain (McGee, 2013). This analysis points towards a theoretical concern with the degree to which donors and their delivery partners are ‘answerable’ to partner country citizens, and whether there is any mechanism for citizens to ‘enforce’ behaviour on these actors.

The aid transparency literature claims that granting citizens access to data could make donors more accountable to citizens. McGee (2013) noted that citizens, who are the intended beneficiaries of aid, are best placed to provide feedback on whether aid programmes are working, and how they should evolve to secure greater impact. Kharas (2010) emphasised that among the benefits provided by aid transparency initiatives is the opportunity to empower citizens to oversee and comment on spending development projects that are ostensibly for their benefit. Glennie et al., (2021) noted that by engaging and empowering civil society actors, aid data transparency could improve outcomes for the most vulnerable. Gutman and Horton (2016) emphasised how aid information could support civil society to be involved in tracking the implementation of projects and playing a role in directing their impacts. Herring (2015) noted that transparency helps to maximise the chances of delivering aid in a more citizen-driven way by providing opportunities for voice and feedback.

In the literature on humanitarian aid there is a distinct emphasis on the role that aid transparency can play in promoting effective support for, and accountability to, refugees and other people who are often not citizens of the locations to which they are displaced, and therefore have limited formal rights. For these groups of stakeholders, aid transparency is vital to efforts to address the significant power imbalances they face in the aid system, and to support them to participate in decisions which affect their lives (Martin et al., 2021; IASC, n.d).

Practice and constraints

Overall, a small number of studies have explored the use of aid data by citizens and their representatives to promote accountability for aid spending, providing a limited evidence base for exploring the extent and outcomes of such activity.

The 2011 Paris Declaration evaluation only generated direct findings on consultation and participation by citizens and organised civil society in aid matters in one-third of the country evaluations and identified positive progress in a very small number of cases, with direct positive impacts identified in only one case (Wood et al., 2011).

Two case studies of partner country non-governmental organisations (NGOs) using IATI and other aid data for monitoring have been documented by IATI. The first involved the National Taxpayers Association Kenya, together with Integrity Action, piloting the use of information on UK aid to monitor its support for NGO projects, which reportedly had very positive results. The second case involved Aidspan, a Kenya-based NGO, attempting to use IATI data to act as an independent watchdog for the Global Fund to Fight AIDS, Tuberculosis and Malaria, although data quality issues limited the impact of this effort (both documented in Ntawiha and Zellman, 2017).

A number of sources also confirm that civil society groups are demanding better access to aid information. Consultations organised by IATI to inform the development of the initiative indicated that civil society organisations (CSOs) were eager to secure access to basic project information, information on funding types and processes, information on cooperating agencies, and monitoring and evaluation reports (IATI, 2009). A pilot study of USAID’s implementation of IATI identified interest from CSOs in Zambia in gaining access to better information on aid flows (USAID, 2015). Similarly, an IATI study of aid information needs in Malawi noted that the major NGO network working on budget accountability was eager to access better aid data to support its work to secure accountability for off-budget resource (IATI, 2019).

There is also extensive research suggesting that a wide range of factors continue to constrain the ability of citizens to play this role. One constraint is the limited engagement by those who publish aid data with citizens’ groups, to understand and respond to their aid data needs. The first formal evaluation of IATI concluded that there had been limited effort to engage the users of aid since the initiative was established in 2008. It also noted that, as a result, the data generated by the initiative were “found insufficient for civil society organisations to monitor the use of donor funds by government or to track the flow of funds from the initial donor to the final recipient”. This evaluation suggested that greater disaggregation to provide information on the location, population groups reached and modalities used, among other areas, was needed to be able to meet the needs of civil society users (Davies, 2015:6).

A recent evaluation of international aid transparency initiatives, which engaged a wide range of aid information producers and users, reached a similar conclusion. Glennie et al. (2021) concluded that donors “regard the ‘public back home’ as the primary group interested in receiving information on aid spending”, which has led to limited attention on the aid information needs of partner country citizens (Glennie et al., 2021:10). This study found that, as a result of this lack of attention, there has been limited awareness among civil society groups of aid information sources. There has also been limited effort to build their capacity to use aid information, and the format of aid information makes it challenging for these groups to use (Glennie et al., 2021).

Gutman and Horton (2015) noted that there had been limited transparency of information during the implementation of aid programmes, making it more challenging for affected communities to engage with donors to influence projects. Quak (2020) documented how, as of early 2020, only a few donor agencies had begun to actively engage with efforts to promote traceability along their aid delivery chains, and just three of over 1,000 organisations reporting to IATI were actually publishing full information on their receipts and disbursements of aid systematically, indicating limited willingness on the part of the aid sector to allow more detailed scrutiny of its activities. Glennie et al. (2021) highlighted that, to date, limited information on the results of aid activities had been published through aid transparency initiatives, making it harder to hold donors accountable for their effectiveness.

The literature links this to donors’ limited understanding of the contexts in which users are operating. McGee’s 2013 analysis of aid accountability initiatives concluded that these had been based on limited articulation of the links between inputs, outputs and planned impacts, and that there had only been limited conceptualisation about who aid users were, what their needs were and of the circumstances and everyday realities they faced (McGee, 2013:2). McGee concluded by noting that all too often there had been an expectation by donors that, once these transparency initiatives were in place, users would automatically appear and make use of information, and that this expectation was misplaced (McGee, 2013:21-22).

Similarly, Joshi (2010) noted that one of the challenges in evaluating the impact of transparency and accountability initiatives was that there was often limited understanding of the theories of change underpinning these initiatives, with assumptions not well articulated and a “lack of attention to histories and patterns of citizen-state relationships” (Joshi, 2010:22).

The literature exploring the role of partner country citizens in pursuing accountability for government spending illustrates that the outcomes of such initiatives are uncertain and dependent on the responsiveness of local authorities to citizen scrutiny and engagement. Björkman and Svensson (2007) found that supporting communities to utilise citizen report cards to scrutinise the performance of local health facilities in rural Uganda led to improvements in the quality and quantity of the health service provision in treatment communities. In contrast, Banerjee et al. (2008) carried out research on the effects of citizen-led efforts to monitor the presence of government nurses in public health facilities, which was initially extremely effective in incentivising the nurses to attend their posts but was quickly undermined by new employment flexibilities introduced by the local health administration. Similarly, Olken (2007) documented the results of an experiment to evaluate the impact of grassroots monitoring of 600 infrastructure projects in Indonesia. The experiment found that these efforts had only had a modest effect (compared to top-down bureaucratic monitoring), seemingly because of issues related to elite capture undermining scrutiny of local government. This evidence suggests that citizens gaining access to and utilising aid information to pursue accountability from aid implementing agencies face these same types of challenges in securing improved delivery and outcomes.

3.3 Donor citizens and their representatives

Theory

As already noted (in Sub-section 3.2), it is widely acknowledged in the literature that the primary accountability relationship for any public spending is between citizens and their representatives (Gibson et al., 2002; Martens et al., 2002; McGee, 2013). This conclusion highlights the important role that aid transparency can play in enabling donor country citizens to influence the use of aid (IATI, 2017), with Barder (2011) going so far as to say that because of the dynamics of accountability for aid, “citizens of donor countries are, in the end, the only people who can insist on aid working better”.

The theoretical literature notes that aid transparency may be vital to efforts to build public trust and support in donor countries on aid. Collins et al. (2009) emphasised that governments need to share information about how aid is spent in order to make the case to their citizens for sustaining aid, with Ghosh and Kharas (2011) noting that taxpayers need to know how their taxes are being used for this purpose.

Publish What You Fund (2017) noted that aid information is vital for donor country parliaments to be able to play their role as overseers of public funds. For Development Initiatives Poverty Research (2009), donor country NGOs need aid data to be able to monitor the action of donors to deliver on their commitments related to aid volumes and effectiveness, with Ntawiha and Zellman (2017) adding an emphasis on the role for academia and think tanks (in both donor and partner countries) to undertake analysis on aid.

However, it is also acknowledged that donor country stakeholders should not substitute for partner country stakeholders in pursuing accountability for aid (Mulley, 2010; Glennie et al., 2021), partly because these stakeholders do not receive the benefits of aid and therefore cannot adequately monitor its delivery (Christensen et al., 2010).

Practice and constraints

No empirical studies were identified which explored how effectively donor country citizens and their representatives have used access to aid information to promote accountability for and the effectiveness of aid, or whether aid transparency has promoted public support for aid (Mills, 2017).

There is, however, research exploring the degree to which donor country citizens and their representatives influence aid delivered by their national governments, with mixed findings. In their analysis of patterns of aid spending during economic downturns, Heinrich et al. (2016) concluded that these downturns lead to reduced public support for helping the poor abroad, which in turn influences government decisions to cut this support. Milner’s (2006) statistical exploration of the drivers of decisions by 20 Organisation for Economic Cooperation and Development governments to deliver aid through multilateral organisations found that, in cases where the public were more sceptical about the benefits of aid, governments were more likely to delegate aid delivery to multilateral organisations. In contrast, based on case studies of six donor governments, Greenhill and Rabinowitz (2016) concluded that parliamentary and public opinion does not appear to have a significant direct influence on decisions to deliver aid through multilateral organisations. Similarly, Day’s (2018) study of the influence of public opinion on significant changes to aid policy in Australia and the UK concluded that these policy changes were driven largely by top-down political processes rather than bottom-up citizen-led processes. These mixed findings suggest that the ability of citizens and their stakeholders to use aid information to influence major shifts in aid policy and delivery may be limited in practice.

Donor country NGOs have played an important part in initiating and promoting recent aid transparency initiatives. IATI was established by a group of bilateral, multilateral and philanthropic donor agencies (IATI, 2008), and has since played a central role in promoting improved aid transparency across the aid community (Davies, 2015). A number of complementary initiatives have also been established by donor country actors, including the Aid Transparency Index (ATI), which was developed by the NGO Publish What You Fund and has since been used to comparatively assess the performance of donors in meeting IATI standards (Honig and Weaver, 2019). Findings show that ATI has helped to drive up levels of aid transparency across the donor community, through inciting external political pressure and by translating broad political mandates and commitments regarding transparency into distinct operational policies that can be enacted through internal reforms (Honig and Weaver, 2019).

3.4     Donor agencies and delivery partners

Theory

The theoretical literature suggests that aid transparency has the potential to support improvements in the functioning of donor agencies through a number of channels. Mulley (2010) suggested that one of the problems caused by a lack of transparency and accountability in the aid sector is the inefficiency resulting from by donors not being able to coordinate their activities and work together more effectively. Christensen et al. (2010:1) argued that the consensus in the aid sector is that aid transparency “enables better coordination among donors” and “makes greater specialisation between donor agencies possible”. Linders (2013) noted that aid transparency initiatives can substitute for a lack of top-down coordination mechanisms and can allow donors to apply a bottom-up approach to identify how they can coordinate their aid more effectively.

Other sources also emphasise the potential role that aid transparency can play in supporting learning within and across donor agencies. Christensen et al. (2010) noted a common view in the donor community that aid transparency supports an accumulation of research, shared evaluations and best practices that can support learning. Barder (2009) emphasised how transparency, information flows and feedback can help to incentivise a greater emphasis on results and learning across the aid sector.

Finally, Ghosh and Kharas (2011) suggested that aid transparency can help donor agencies to make efficiency savings when managing access to aid information. This is due to the proposition that there are lower costs involved in undertaking systematic disclosure of information compared with responding to information on request.

Practice and constraints

The empirical research provides only limited insight into how aid information generated through aid transparency initiatives is being used internally by donor agencies to help improve the efficiency and effectiveness of their aid spending.

An illustration of how information generated by aid transparency efforts can support donor agencies to improve their own internal operations is provided by a case study of Oxfam Novib’s adoption of IATI standards for reporting across its partners. The study reported that IATI’s standards had led to “the improvement of internal information on resources and results to inform decisions” (IATI, 2017). The Netherlands Ministry of Foreign Affairs is also reportedly trying to use IATI data to better track delivery down its aid programme delivery chain, with the aim of improving understanding of how its aid is being spent and what results are being achieved (IATI, 2020). However, it is not clear in either of these cases whether the use of IATI data has had a substantive impact on the efficiency of the organisations’ operations and the effectiveness of their aid programmes.

Collin et al. (2009) estimated that by applying systematic approaches to publishing aid information, such as approaches promoted through the IATI system, donors could save approximately $7 million a year due to the reduced burden of information requests they would be responding to. However, no studies were identified which have tested these estimates of the savings from systematic publishing in practice.

No studies were identified which explored whether and how donors are using publicly available information (available through IATI and other mechanisms) on each other’s aid activities to make more effective decisions about how to collaborate and coordinate their assistance.

4. Synthesis and general reflection

This literature review has illustrated that there are multiple theoretical channels through which aid transparency could theoretically promote accountability and effectiveness, and these are distinct for each of the main groups of potential users of aid data. For partner country stakeholders, these channels are more complex and uncertain, given the lack of formal channels that these actors have to pursue genuine accountability from donors and delivery partners.

While the theoretical literature helps to identify some of the channels through which aid information can be used by different stakeholders to support their efforts to promote accountability and effectiveness, theories of change for fully elaborating these channels and the factors that determine the outcomes of these efforts are not fully developed. For example, there are good reasons to qualify the role that getting ‘aid on budget’ can play in promoting aid effectiveness, including the often limited quality and accountability of government systems in aid-dependent countries. However, such factors are not adequately explored in the theoretical literature.

Empirical research exploring the links between aid transparency, accountability and effectiveness is limited, although it has been central to the aid reform agenda for more than a decade. The available literature faces similar challenges to explorations of the benefits of transparency in other areas of public policy areas: it is challenging to identify the complex links between aid transparency, accountability and effectiveness through empirical work, especially when transparency is just one of a number of factors that facilitate the development of accountable and effective public institutions.

Although empirical literature on the impact of transparency on aid is still a relatively new area, relevant literature suggests that the potential impact is likely to have been constrained by a number of factors. First, there has been only limited progress to date across the aid sector in ensuring that comprehensive, timely and robust aid information is published, so that it can be used by aid stakeholders. Second, there has been limited effort to understand the distinct aid information needs of different groups of stakeholders so that these can be met, thereby effectively empowering these stakeholders to play their part in promoting aid effectiveness.

Finally, this literature review illustrates that the main stakeholder groups struggling to access and effectively use aid information are those based in developing countries, and significant gaps remain in efforts to ensure that aid information is made available to these stakeholders.

These conclusions do not mean that the literature suggests there is limited value in ongoing investments to promote aid transparency, for a number of reasons. First, this conclusion does not in any way challenge the value of aid transparency as an important principle in and of itself, as noted in the introduction. Second, there is some tentative empirical research which has found that aid transparency is associated with both improved outcomes of aid projects (Honig et al., 2022)[5]  and reduced levels of corruption in partner countries (Christensen, 2010). Finally, there is evidence that stakeholders across the aid system recognise that transparency provides the foundation for the accountability and effectiveness of aid (Wood et al., 2011:64).

[5] This study analyses a data set on the performance of approximately 20,000 aid projects financed by 12 donor agencies in 183 countries, and finds that the adoption of access to information policies by aid agencies is associated with better project outcomes when these policies include independent appeals processes for denied information requests

Bibliography

  1. (2015). Real aid: ending aid dependency. ActionAid, September 2011, link.
  2. Banerjee, A. V., Duflo, E. and Glennerster, R. (2008). Putting a band-aid on a corpse: incentives for nurses in the Indian public health care system. Journal of the European Economic Association, Volume 6, Issue 2-3, 1 May 2008, link.
  3. Barder, O. (2009). Beyond planning: markets and networks for better aid. Centre for Global Development, Working Paper 185. Washington, D.C.: CGD, link.
  4. Barder, O. (2011). 8 lessons on aid transparency. Owen Abroad blog, April 2011, link.
  5. Betley, M. (2008). Putting aid on budget. Mali case study. Final April 2008. Oxford: Mokoro Ltd, link.
  6. Björkman, M. and Svensson, J. (2007). Power to the people: evidence from a randomized field experiment of a community-based monitoring project in Uganda. Policy Research Working Paper, No. 4268. World Bank, Washington, D.C., link.
  7. Bladon, R. (2008). Putting aid on budget. Ethiopia case study – working paper. Final April 2008. Oxford: Mokoro Ltd, link.
  8. Collaborative Africa Budget Reform Initiative (CABRI). (2008a). Putting aid on budget – good practice note: using country budget systems. CABRI, June 2008, link.
  9. Collaborative Africa Budget Reform Initiative (CABRI). (2008b). Putting aid on budget: synthesis report. CABRI, April 2008, link.
  10. Carlitz, R. (2013). Improving transparency and accountability in the budget process: an assessment of recent initiatives. Development Policy Review, August 2013, link.
  11. Carothers, T. and Brechenmacher. S. (2014). Transparency, participation, and inclusion: a new development consensus? Carnegie Endowment for International Peace, October 2014, link.
  12. Carter, B. (2013). Transparency and accountability. GSDHC Applied Knowledge Services, January 2014, link.
  13. Christensen, Z., Nielsen, R., Nielsen, D. and Tierney, M. (2010). Transparency squared: the effects of donor transparency on recipient corruption levels. Paper prepared for application to participate in the 4th Annual Conference on the Political Economy of International Organizations for 2011, link.
  14. Collin, M., Zubairi, A., Nielson, D. and Barder, O. (2009). The costs and benefits of aid transparency. A draft analytical framework, link.
  15. Davies, I. (2015). External independent evaluation of the International Aid Transparency Initiative: synthesis report. October 2015, link.
  16. Day, B. (2018). Political attention and aid policy change: lessons for aid advocacy from the aid spending expansions in Australia and the United Kingdom. Development Policy Centre, Australian National University, Discussion Paper 71, September 2018, link.
  17. de Renzio, P., Lakin, J. and Cho, C. (2019). Budget credibility across countries: how deviations are affecting spending on social priorities. International Budget Partnership, September 2019, link.
  18. Development Initiatives Poverty Research (DIPR). (2009). International Aid Transparency Initiative (IATI): scoping paper for consultation. Development Initiatives Poverty Research, April 2009, link.
  19. Devine, J. (2003). The paradox of sustainability: reflections on NGOs in Bangladesh. Annals of the American Academy of Political and Social Science, 590:227-42, link.
  20. Department for International Development (DFID). (2010). Aid transparency guarantee. DFID, June 2010, link.
  21. Department for International Development (DFID). (2012a). Aid transparency challenge. DFID, December 2012, link.
  22. Department for International Development (DFID). (2012b). Launching Aid transparency challenge, Speech by the Secretary of State for International Development, Justine Greening, MP. December 2012, link.
  23. Department for International Development (DFID). (2020). Smart rules: better programme delivery. DFID, Version XIII: effective 1 April 2020 until 1 October 2020, link.
  24. Elbers, W., and Arts, B. (2011). Keeping body and soul together: southern NGOs’ strategic responses to donor constraints. International Review of Administrative Sciences 77(4): 713-732, link.
  25. European Report on Development (ERD). (2015). Combining finance and policies to implement a transformative post-2015 development agenda. European Report on Development, Overseas Development Institute (ODI), 2015, link.
  26. Eyben, R. (2008). Power, mutual accountability and responsibility in the practice of international aid: a relational approach. IDS Working Paper 305, Brighton: Institute of Development Studies, link.
  27. Foreign, Commonwealth and Development Office (FCDO). (2021). FCDO Programme operating framework. FCDO, April 2021, link.
  28. Fölscher, A. (2008a). Putting aid on budget. Kenya case study – working paper. Final April 2008. Oxford: Mokoro Ltd, link.
  29. Fölscher, A. (2008). Putting aid on budget. Tanzania case study – working paper. Final April 2008. Oxford: Mokoro Ltd, link.
  30. Gaventa, J. and McGee, R. (2010). The impact of transparency and accountability initiatives. Institute of Development Studies (IDS), University of Sussex, 2010, link.
  31. Ghosh, A. and Kharas, H. (2011). The money trail: ranking donor transparency in foreign aid. World Development 39(11): 1918-1929, link.
  32. Gibson, C. C., Andersson, K. Ostrom, E. and Shivakumar, S. (2002). The Samaritan’s dilemma: the political economy of development aid. Oxford: Oxford University Press.
  33. Glennie, J. et al. (2021). Evaluating aid transparency: full report. Oxford Policy Management and Hewlett Foundation, January 2021, link.
  34. Greenhill, R. and Rabinowitz, G. (2016). Why do donors delegate to multilateral organisations? A synthesis of six country case studies. ODI, November 2016, link.
  35. Gutman, J. and Horton, C. (2016). Accessibility and effectiveness of donor disclosure policies: when disclosure clouds transparency. Brookings Institution, May 2015, link.
  36. Hall-Matthews, D. and Irby, E. (2016) Transformative transparency? A literature review. Integrity Action, unpublished.
  37. Haus, M., Wehner, J. and de Renzio, P. (2022). (When) do open budgets transform lives? Progress and next steps in fiscal openness research. International Budget Partnership, May 2022, link.
  38. Heinrich, T., Kobayashi, Y. and Bryant, K. A. (2016). Public opinion and foreign aid cuts in economic crises. World Development, Volume 77, pp. 66-79, January 2016, link.
  39. Herring, S. (2015). The business proposition of open aid data: why every US agency should default to transparency. Blog post on Publish What You Fund website (30 June), link.
  40. Honig, D. and Weaver, C. (2019). A race to the top? The Aid Transparency Index and the social power of global performance indicators. International Organization, July 2019, link.
  41. Honig, D., Lall, R. and Parks, B. C. (2022). When does transparency improve institutional performance? Evidence from 20,000 projects in 183 countries. American Journal of Political Science. February 2022, link.
  42. Inter-Agency Standing Committee (IASC). (n.d.). Accountability to affected populations (AAP): a brief overview. Inter-Agency Standing Committee, link.
  43. International Aid Transparency Initiative (IATI). (n.d.). IATI – Improving countries’ access to and use of aid information, IATI, link.
  44. International Aid Transparency Initiative (IATI). (2008). International Aid Transparency Initiative: Accra statement. 4 September 2008, link.
  45. International Aid Transparency Initiative (IATI). (2017). Oxfam Novib case study. IATI, 2017, link.
  46. International Aid Transparency Initiative (IATI). (2019). IATI date use case study: Malawi. IATI, link.
  47. International Aid Transparency Initiative (IATI). (2020). 2020 annual report: IATI strategic plan in action. IATI, 2020, link.
  48. International Aid Transparency Initiative (IATI). (2021). Data use in partner countries. Presentation at the IATI Virtual Community Exchange, 12-13 Oct 2021, link.
  49. International Aid Transparency Initiative (IATI). (2022). Madagascar case study. IATI, link.
  50. International Budget Partnership (IBP). (2021). Open budget survey 2021. IBP, 2021, link.
  51. Isard, P. et al. (2006). The macroeconomic management of foreign aid: opportunities and pitfalls. Washington, D.C.: International Monetary Fund, 2006, link.
  52. Joshi, A. (2010). Do they work? Assessing the impact of transparency and accountability initiatives in service delivery. Institute of Development Studies (IDS), University of Sussex, October 2010, link.
  53. Kharas, H. (2010). Transparency: changing the accountability, engagement and effectiveness of aid. Brookings Institution, October 2010, link.
  54. Linders, D. (2013). Towards open development: leveraging open data to improve the planning and coordination of international aid. Government Information Quarterly, Volume 30, Issue 4, October 2013, link.
  55. Martens, B., Mummert, U., Murrell, P., and Seabright, P. (2002). The institutional economics of foreign aid. New York: Cambridge University Press.
  56. Martin, D., Singer, S. and Mathias, B. (2021). Humanitarian accountability in displacement contexts: five years on from the Grand Bargain. Refugee Survey Quarterly, Volume 40, Issue 4, December 2021, link.
  57. McGee, R. (2010). Review of impact and effectiveness of transparency and accountability initiatives – Annex 5: Aid transparency. Prepared for the Transparency and Accountability Initiative Workshop, October 14-15, 2010, link.
  58. McGee, R. (2013), Aid transparency and accountability: ‘Build it and they’ll come’?. Development Policy Review, 31, link.
  59. Mills, L. (2017). Transparency and performance. K4D, April 2017, link.
  60. Moon, S. and Williamson, T. (2010). Greater aid transparency: crucial for aid effectiveness. ODI, Project Briefing no. 35, January 2010, link.
  61. Morfit, S. N. (2011). “AIDS is money”: how donor preferences reconfigure local realities, World Development, 39(1): 64-76, link.
  62. Mulley, S. (2010). Donor aid: new frontiers in transparency and accountability. Transparency and Accountability Initiative, November 2010, link.
  63. Ntawiha, W. and Zellman, C. (2017). Reaching the potential of IATI data. Development Initiatives, March 2017, link.
  64. Ocampo, J. A. and Arteaga, N. G. (2015). Accountable and effective development cooperation in a post-2015 era. Background study 3, Accountability for Development Cooperation, UN DCF, March 2014, link.
  65. Organisation for Economic Cooperation and Development (OECD). (2005). Paris Declaration on Aid Effectiveness. OECD Publishing, Paris, link.
  66. Organisation for Economic Cooperation and Development (OECD). (2008). Accra Agenda for Action. OECD Publishing, Paris, link.
  67. Organisation for Economic Cooperation and Development (OECD). (2011a). Busan Partnership for Effective Development Co-operation. Fourth High Level Forum on Aid Effectiveness, Busan, Republic of Korea, 29 November-1 December 2011, OECD Publishing, Paris, link.
  68. Organisation for Economic Cooperation and Development (OECD). (2012). Aid effectiveness 2011: progress in implementing the Paris Declaration. Better Aid, OECD Publishing, Paris, link.
  69. Organisation for Economic Cooperation and Development – United Nations Development Partnership (OECD-UNDP). (2014). Making development co-operation more effective: 2019 progress report, link.
  70. Organisation for Economic Cooperation and Development – United Nations Development Partnership (OECD-UNDP). (2019). Making development co-operation more effective: 2019 progress report, link.
  71. Olkun, B. A. (2007). Monitoring corruption: evidence from a field experiment in Indonesia. Journal of Political Economy, 2007, Volume 115, no. 2, link.
  72. Park, K. R. (2017). An analysis of aid information management systems (AIMS) in developing countries: explaining the last two decades. Proceedings of the 50th Hawaii International Conference on System Sciences, 2017, link.
  73. Publish What You Fund (PWYF). (2017). Briefing paper 1: Why aid transparency matters, and the global movement for aid transparency. PWYF, 2017, link.
  74. Publish What You Fund (PWYF). (2022). Why transparency matters. PWYF, link.
  75. Quak, E. J. (2020). Donor agencies’ efforts for improved transparency of delivery chains for aid programmes. K4D, April 2020, link.
  76. S. Agency for International Development (USAID). (2015). Aid transparency country pilot assessment: final report. USAID, May 2015, link.
  77. Williamson, T. (2008). Putting aid on budget. Uganda case study. Final April 2008. Oxford: Mokoro Ltd, link.
  78. Wood, B., Betts, J., Etta, F., Gayfer, J., Kabell, D., Ngwira, N., Sagasti, F. and Samaranayake, M. (2011). The evaluation of the Paris Declaration, Final report. Copenhagen, May 2011, link.
  79. World Bank (2004). World development report 2004: making services work for poor people. World Bank Group, 2014, link.