FCO and British Council Aid Responses to the Arab Spring Report

Objectives

Assessment: Green-Amber

2.1 In this section of the report, we examine whether the FCO and the British Council have sound strategies for allocating ODA in the MENA region.

The Arab Partnership Strategy is well articulated and based on sound principles

2.2 The FCO’s strategy in response to the Arab Spring drew on earlier FCO-commissioned research begun in 2009 into the causes of discontent in the region. This led to the launch of a pilot Arab Human Development Initiative in October 2010 to deepen the research and develop a UK response. The strategic planning done under this initiative fed into the design of the Arab Partnership from 2011.

2.3 In launching the Arab Partnership in May 2011, the UK Government sought to accomplish a significant shift in UK foreign policy towards the region. The Foreign Affairs Committee noted that, over many years, the UK had been ‘perceived as having prioritised its own interests, particularly in stability, commerce and counter-terrorism, over the promotion of more representative governments’.⁹ In a speech to the Kuwaiti Parliament in February 2011, the Prime Minister stated that the UK response to the Arab Spring would be based on the values of human rights and democracy.¹⁰

2.4 The Arab Partnership Strategy is focussed on promoting the ‘building blocks of democracy’ – a goal that is applicable both to the post-revolutionary contexts and to the more gradual reformers. It recognises that the changes underway in Arab societies are long-term in nature and will require sustained international engagement. Given that UK financial assistance is too small to be transformative on its own, the Arab Partnership aims to catalyse local change processes and to leverage support from other international sources. It is ‘demand-driven’, recognising that the UK cannot and should not dictate the shape or pace of reform but should instead support change processes that are already underway.

2.5 The Arab Partnership Strategy has three complementary components:

• Policy: promoting democratic values through diplomacy;
• Partnerships: working with and through international partners and multilateral organisations; and
• Programme: referring to projects that support reform.

2.6 We find these basic operating principles of the Arab Partnership to be clear and well-conceived. The values-based agenda is an appropriate one, even if it meets with some scepticism in a region where many remain suspicious of UK and Western motives and conflicting objectives, such as Western arms sales in the region.¹¹ Focussing on the ‘building blocks of democracy’ has enabled the FCO to work with both post-revolutionary regimes and the gradual reformers, without appearing partisan.

2.7 Care has been taken to ensure complementarity among UK programmes. DFID’s support for economic reform in the region under APEF falls within the Arab Partnership Strategy and answers to a cross-Whitehall MENA Strategic Programme Board. The tri-departmental Conflict Pool’s operations in the region also fall within the same governance structure and the roles of the three instruments are clearly set out in the Arab Partnership Strategy.

2.8 Influencing other international actors is also a key element of the Strategy. The goal is to secure transformative international support for democratic transition through UK influence within the G8, the European Union and the International Financial Institutions.

The choice of themes is appropriate but there is scope for stronger activity design

2.9 There is no fixed allocation of funds across the seven countries where APPF focusses (Egypt, Libya, Tunisia, Jordan, Morocco, Algeria and Syria¹²). Each is given an indicative importance rating, based on need, potential UK added value and the delivery environment. Tunisia, for example, is assessed as a key country for demonstrating the potential for democratic reform in the region and is open to international support. It has the largest portfolio, with commitments of £2.7 million in 2012-13. Egypt is another country of considerable importance but a more difficult environment in which to operate, receiving APPF commitments in 2012-13 of £2 million. There is also a portfolio of multi-country projects with 2012-13 APPF commitments of £4 million. The average project receives £335,000 a year. Given the small scale of the expenditure, particularly in a country the size of Egypt, the goal is to identify investments with the potential for catalytic impact or to leverage other funding.

2.10 The APPF Programme Strategy sets out four main areas of support:

• Political participation (comprising 35% of total expenditure in 2011-12 and 2012-13): this includes support for elections and election monitoring, voter education, political party support, social mobilisation and support for the participation of women and youth in public life;
• Public voice and freedom of expression (27% in 2011-12 and 43% in 2012-13): this includes media reform and development, social media and civil society development;
• Good governance (26% in 2011-12 and 15% in 2012-13): this includes support for parliaments and anti-corruption reforms; and
• Promoting inclusive growth (12% in 2011-12 and 7% in 2012-13)¹³: this goal was included in the strategy for the first year but then passed to APEF.

2.11 These themes are broadly appropriate, given the overall goal of supporting democratic transition. They target key transition processes, including elections, the establishment of new parliaments and constitution-drafting processes. The emphasis on public voice makes good sense in societies where freedom of expression has dramatically increased, requiring a transformation in the media landscape.

2.12 Under the good governance theme, APPF has found few examples of anti-corruption reforms to support in most of the countries, although it is working with Transparency International at the regional level. The strategy, moreover, does not explain sufficiently how small grants to civil society organisations (CSOs) can support the major governance reforms required across the region.

2.13 For each country with APPF funding, the FCO produces a country strategy, identifying which areas within the three APPF themes it will prioritise. The strategies are strong on contextual analysis but weaker in setting out how the proposed activities will lead to the desired results. The FCO does not use explicit ‘theories of change’ to describe the intended causal links from its interventions through to its intended impacts.

2.14 We found the Tunisia portfolio to be well conceived, targeting key change processes and working with a good range of national partners. The Egyptian portfolio is yet to find a clear strategic direction. The Government of Egypt does not welcome external support for political transition and there are restrictions on foreign funding of civil society. With the Government of Egypt focussed on managing continued unrest, there are few government-sponsored reforms for APPF to support. As a result, APPF is looking for viable entry points, with mixed success to date. We accept that a level of experimentation and risk is a necessary part of supporting political transition in Egypt.

2.15 The FCO actively solicits proposals from potential implementing agencies, both through publicity campaigns and direct approaches. Outreach to potential local partners has increased over time, with more information provided in Arabic. The FCO also helps implementing partners to shape proposals to meet their strategic priorities. The APPF does not currently design activities directly and then procure an implementer, although this option is not ruled out by the strategy.

2.16 Given APPF’s objectives, a responsive grant-making process makes good sense. It orients APPF towards supporting democratic transition processes that are already underway, rather than attempting to shape reforms directly – although this principle has been somewhat compromised by over-reliance on UK-based implementing partners (see paragraphs 2.30 to 2.33).

2.17 It does mean, however, that the strategic nature of the portfolio is limited by the spread and quality of proposals that are received. In Egypt, in particular, there are various priorities identified in the country strategy, such as support for anti-corruption and women’s equality, for which few adequate proposals have been received.

2.18 The FCO does not provide substantive guidance to APPF staff on their programming choices. It does not appear to draw on experience from democratic transitions in other regions, such as Eastern Europe, other than through the personal knowledge of its staff. The teams have access to advisory support from DFID’s MENA Department, including on governance and results management. While this has been useful, access to support from more specialised advisers, in areas such as media or parliamentary reform, during the appraisal of proposals would help to ensure that the teams benefit from the latest international experience.

The British Council provides a valuable complement

2.19 The British Council responded to the Arab Spring with a £1 million programme of research and consultations, to determine what role it could play in the transitions. For example, it commissioned research on art and social change across the region.¹⁴ It identified a number of areas where it could contribute, including youth skills and employability, social mobilisation and leadership training. It then worked to customise its global programmes for the changed context, particularly in the Education and Society field.

2.20 The British Council allocates its funds across countries according to three criteria: the UK’s national interest, inherent need of the recipient country and the potential for building linkages with the UK (for example, trade). It also bid for APPF funds, receiving grants for 8 projects totalling £2 million in 2011-12 and 2012-13 (in some cases, shared with partners). Its projects are small in scale, averaging £165,000 a year.

2.21 In Egypt and Tunisia, its programming has a strong youth focus which is helping to strengthen vocational training systems by sharing UK experience. In Tunisia, it is contributing to an innovative, multi-donor programme to promote entrepreneurship. Through APPF funding, it is teaching young people debating skills and organising a network of debating clubs. The beneficiaries described this as highly relevant to the democratic transition context, where it helps young people to learn advocacy skills and to see both sides of an argument. Its Active Citizens programme targets community leaders, particularly young people and women, with leadership training, helping them to launch diverse local community initiatives.

2.22 We found that the British Council had a clear sense of its potential contribution to the region and a relevant and useful set of objectives for its assistance. Its programmes are mostly well designed, with strong local partnerships to extend their reach. We saw one example, however, of a British Council project that seemed to us to be over-ambitious to the extent that it sought to address complex institutional and policy challenges (see paragraph 2.53).

Delivery

Assessment: Green-Amber

2.23 In this section, we review the delivery of FCO and British Council ODA programmes. We assess their governance and management processes and their methods for delivering activities. We assess whether their financial management and ODA reporting are sound.

APPF represents an improvement in FCO’s ODA delivery capacity

Collaboration between the FCO and DFID is strong

2.24 Compared to our findings on the Conflict Pool, which we reviewed in 2012,¹⁵ the Arab Partnership represents a strong model of inter-departmental coordination. Within the overall strategy, there is a clear division of labour between the FCO and DFID, based on individual comparative advantage. We were impressed by the way in which DFID resources have been used to boost FCO delivery capacity. This has included DFID’s contribution of £20 million to APPF, staff secondments and transfers and the provision of advisory staff as needed.

2.25 These collaborative structures are the more impressive considering that they were created in a situation of considerable urgency. The two departments have shown a commendable willingness to innovate and to adjust their ways of working as necessary.

2.26 The Foreign Affairs Committee noted that a decline in FCO staff numbers in MENA over many years, including a loss in Arabic language skills, had reduced the FCO’s capacity for information collection and analysis.¹⁶ The development of the Arab Partnership has been part of a concerted effort to rebuild that capacity. Feedback from the two embassies we visited was that the Arab Partnership itself had helped to boost FCO delivery capacity, both through the recruitment of additional staff and by providing new local partnerships that serve as a platform for diplomatic activity.

APPF’s grant-making model is flexible and responsive but needs to do better at supporting local advocates for democracy

2.27 APPF’s grant-making model aims to be fast, flexible, responsive, innovative, demand-led and partnership-based. For the most part, it achieves these objectives. The FCO was among the first donors to support political transition and was able to support some critical processes (for example, election observation in Egypt and constitution-drafting in Tunisia) before others were in a position to do so. It has proved adept at trying out new activities and has been quick to move on from unsuccessful experiments, as we discuss further in the Learning section.

2.28 In each annual funding round, APPF calls for proposals from potential implementing partners and makes funding decisions against a set of criteria. These include relevance, value for money, evidence of local demand or need, implementer capacity, sound project design, inclusiveness of women and youth, sustainability and a ‘potential multiplier effect’. While these are broadly appropriate, we consider that APPF would benefit from having more explicit criteria for distinguishing genuinely strategic interventions from those that are merely worthwhile. On occasion, the scale of projects is not well matched to their objectives. For example, a project on voter education in rural Tunisia was on too small a scale to have strategic impact. For activities that need to be delivered at a larger scale, APPF should take a more deliberate approach to leveraging funding from other sources.

2.29 The annual funding cycle initially proved to be a constraint on effectiveness. This was particularly the case in the first year, as the launch of the APPF only weeks before the beginning of the financial year meant that early projects had less than 12 months to complete their activities, due to the time required for project design. The FCO responded by introducing multi-annual funding for its larger projects, which has significantly improved delivery.

2.30 Implementing partners told us that the FCO is a good partner to work with. It does not insist on visible UK branding where that would be unhelpful to the implementing partner. It takes a close interest in the design and delivery of projects, replies quickly to communications and is willing to adjust activities as necessary. Some of the more successful projects ended up changing their planned activities and outputs substantially during the life of the project. The most successful designs had several delivery options available to them, enabling them to adjust rapidly in response to obstacles, opportunities or lessons learned. For example, the Global Partners project on parliamentary reform in Egypt was able to adjust the focus of its support following dissolution of the Egyptian Parliament, without losing its overall direction.¹⁷ APPF implementing partners may be international non-governmental organisations (INGOs), local civil society organisations (CSOs), national government institutions, international organisations or UK Government agencies (see Figure 3 on page 11 for an analysis of funding commitments by delivery channel). APPF has made extensive use of major UK ‘brands’ such as the BBC, Chatham House, the British Council and the Westminster Foundation for Democracy. The Foreign Affairs Committee praised this as raising the profile of UK support and strengthening the UK’s ‘soft power’.¹⁸

2.31 On the other hand, UK-based partners without a presence in-country have sometimes struggled to establish quality partnerships and respond quickly enough in a volatile political environment. An internal review of APPF’s first year of operations by the Stabilisation Unit came to a similar conclusion.¹⁹ The FCO responded by encouraging implementing partners to establish a presence in the region.

2.32 We also have some concerns as to whether support delivered from the UK is sufficiently anchored in local demand. Some partners, such as the Westminster Foundation for Democracy, appeared to have offered standardised packages of assistance without adapting them to local need. While counterparts are unlikely to turn down offers of assistance from well-known UK organisations, it does not follow that they are committed to the aims of the project. For example, the idea of involving the BBC in major reforms to national broadcasting agencies in both Egypt and Tunisia proved over-ambitious. The BBC’s engagement was discontinued in Egypt and in Tunisia was recast as technical support for specific television programmes.²⁰

2.33 In practice, the majority of APPF grants have been awarded to INGOs (70% of total funding commitments, see Figure 3). This reflects the inexperience of civil society in the region and the shortage of credible proposals. To address this gap, APPF usually requires INGOs to pair with a local CSO in a joint project, with the INGO managing the funds. This has been a useful way of channelling support to young CSOs with limited capacity but with the potential for development. For example, in Tunisia, APPF supports I-Watch, a new student organisation dedicated to monitoring political party finance. As I-Watch has, for the time being, no capacity to manage funds, the grant is managed by an INGO, the International Foundation for Electoral Systems, which disburses funds for particular I-Watch activities and supports its organisational development.

2.34 APPF also has ‘Tactical Funds’ which can be used for small grants to CSOs. Country programmes have the option of bidding for up to £50,000 in devolved funds, which can be used for small grants (up to £20,000) for pilot activities. In Tunisia, the FCO has used Tactical Funds to support local CSOs with activities that allow them to demonstrate their capacity to manage APPF grants in the future.

2.35 While APPF has made some effort to support ‘new voices’, its grant-making model is not well suited to funding local partners. Only 13% of funding commitments have gone directly to national CSOs. In fact, its procedures make no distinction between grants to major INGOs, such as the National Democratic Institute and Saferworld; and new CSO ventures. This is an area where the FCO could learn from DFID’s civil society funding instruments, which are able to identify different methods of funding for organisations appropriate to their level of capacity and can also provide funds linked to support for capacity development.

The Arab Partnership offers good value for money2.36 As discussed above, the Arab Partnership is a high-risk, high-return venture. In aiming for catalytic investments in key transition processes, it has the potential for substantial impact from relatively small expenditure. It also operates in an experimental manner in a volatile and unpredictable environment. Its failure rate is, therefore, inevitably high. Provided it remains flexible and learns quickly from experience, this venture capital-style programming model offers a good value for money proposition. Given the relatively high management costs for small-scale expenditure, this type of programme has largely disappeared from DFID’s portfolio as the average size of its activities has increased. This makes it an area of comparative advantage for the FCO, provided it is used strategically.

2.37 On the other hand, we are concerned about the top-heavy nature of APPF governance structures. Individual grants are first assessed by Programme Boards in-country and then reviewed again by a London-based Programme Board made up of senior officials. The FCO told us that this high level of centralisation was necessary while the programme was in its infancy and management capacity was being built up at the country level. We also recognise that the sensitive nature of the programme calls for close supervision. Over the longer run, however, it would represent both a stronger governance model and better value for money for the London-based Programme Board to focus on setting strategy, developing guidance and continuing to strengthen the mechanism, rather than managing the funds directly.²¹

2.38 Value for money is one of the selection criteria for individual grant applications but there is no established assessment methodology. Partners are required to provide activity-based budgets, which are transparent and enable ready comparison of unit costs across similar projects. We saw evidence that FCO project managers were making such comparisons and working with implementing partners to minimise costs.

The British Council has a much more advanced financial management system than the FCO

The FCO lacks adequate financial management systems

2.39 FCO financial systems were not designed for managing programmes and are a constraint on effective delivery on the ground. The FCO’s financial management system, PRISM, is a receipting system mainly designed for procuring items for the FCO’s own use. It has been adapted to include some programme management and project accounting capacity, although these additional modules are not used consistently across different FCO programmes and country posts. While it has been adapted to include some programme management and project accounting capacity, it does not facilitate effective real-time tracking of expenditure across the APPF portfolio on the ground. It requires programme managers to enter receipts from projects individually, rather than allowing automatic payment against invoices from implementing partners. This is burdensome for both FCO staff and implementing partners.²² The FCO’s financial systems for programme and project management should be consistently applied across the FCO’s network and Strategic Programmes, so that there is local ownership over programme management.

2.40 One consequence is that the relationship between FCO staff and implementing partners tends to be dominated by the need to chase financial reports and receipts. While the FCO allows partners to adjust their activities in response to changing circumstances, this flexibility is not built into the activity-based project budgets. As a result, partners may be required to rework their budgets on several occasions.

2.41 At the outset, there were insufficient FCO staff in-country with project management experience and the ability to operate PRISM. Staff lacked appropriate guidance on operating the system and suitable training programmes. We heard from one local staff member that she had developed her own project management guidelines for her team.

2.42 APPF spending is often unpredictable (for example funding may be allocated to support an election that is postponed on several occasions), requiring reallocation of funds within the financial year to avoid under-utilising the budget. To improve its ability to forecast expenditure, the APPF recently introduced monthly financial reporting on all projects. While this may be appropriate for small CSOs with lower capacity, feedback from larger INGO partners is that it is unnecessarily burdensome.

2.43 The FCO has no established standards for assessing the financial management capacity of its implementing partners at the application stage and does not specify any minimum capacities to qualify for APPF funding (although partners are required to have a bank account in the name of the organisation). When assessing applications, it relies mainly on the applicants’ reputation and record of past delivery. This means that the FCO is unable to adjust its financial reporting requirements to the individual capacities and risk profiles of its partners.

2.44 While the FCO has overall guidelines on preventing fraud and corruption, knowledge of these guidelines on the part of APPF staff appears to be limited. There are no specific guidelines for the APPF and partners are not required to have their own corruption or fraud reporting policies in place. The exposure is kept lower by the practice of making monthly payments in arrears against the presentation of invoices. We were informed that there had been no examples of fraud or corruption found in either Tunisia or Egypt. The FCO needs to develop a more systematic approach to assessing and mitigating fiduciary risks for individual implementing partners.

The British Council has strong governance and financial management

2.45 The British Council has a strong organisational structure based around Strategic Business Units and Regional Divisions. This structure facilitates clear lines of accountability. Its financial management system is much more advanced than the FCO’s. It has customised a system widely used by business so that it can be used to track financial performance and aggregate outcomes across Strategic Business Units, regions, countries and programmes. It is also used to manage client relations.

2.46 The British Council implements its projects directly. It rarely provides financial support to partners. Corruption risk is, therefore, limited to fraud within the organisation. There were no reported cases in Egypt or Tunisia.

The British Council delivers training and networking very effectively

2.47 The British Council has a distinctive delivery model. Over the past decade, it has consolidated its portfolio of activities into 18 global programmes. These provide models of assistance that country offices can adapt to local conditions. This approach reduces the level of investment needed in activity design, while still permitting local innovation. New global programmes are developed from time to time to reflect learning across the organisation.

2.48 As an agency dedicated to cultural relations, the British Council often works by pairing UK institutions with counterparts in developing countries, particularly in the education field. For example, it partners schools in Egypt and Tunisia with UK schools through its ‘Connecting Classrooms’ project. It pairs national agencies responsible for vocational education and training with UK colleges. It focusses on areas where the UK is seen as offering a distinct contribution. For example, in Egypt it is helping government agencies to introduce apprenticeships and a grading system for technical qualifications based on UK systems.

2.49 The British Council has very strong local partnerships. Its network of teaching centres provides a good base for building ties with diverse national organisations. It is skilled at engaging local organisations and individuals in its activities on a voluntary basis, in order to extend its reach. It does this through the use of rewards and incentives. Partners are given access to training and qualifications and are able to participate in British Council-sponsored events and networks.

2.50 The British Council pursues its development objectives primarily through the training and empowering of individuals. It has well established methods for doing so and receives consistently positive feedback from participants. It makes extensive use of ‘cascade’ training. For example, it is helping the Tunisian Agency for Vocational Training²³ to improve the standard of English teaching in vocational training centres. After providing initial training to a group of 54 English teachers, it selected 18 for ‘master training’, who are now able to provide continuing training to their peers. Cascade training provides a multiplier effect, increasing the numbers of individuals benefiting from British Council support.

2.51 Through its Active Citizens programme, the British Council uses cascade leadership training in support of social mobilisation. It identifies community leaders (for example youth workers, religious leaders and civil society activists) and develops their skills in network building and project management. As part of their training, participants identify and implement ‘social action projects’ in their own communities. As this training method is cascaded out into the community, it results in a mushrooming of useful, small-scale initiatives, from youth clubs to environmental campaigns and local income-generating activities.

2.52 The British Council activities we examined represent good value for money. For example, in Egypt, the Active Citizens programme costs just £10 per participant. The return on that investment appears to be impressive. We note, however, that activities delivered through a cascading model may require additional monitoring to ensure that quality does not fall away.²⁴

2.53 In Egypt, the British Council was unable to deliver an APPF-funded project to develop a national strategy for English language teaching because of insurmountable objections. The scale of the challenge involved was underestimated by both the British Council and the FCO. APPF’s project completion report notes that the British Council had been poor at managing risks around the relationship. Our view is that the British Council’s delivery model is less likely to be effective for projects which involve complex reforms or organisational change processes.²⁵

Systems for reporting expenditure on ODA are sound

2.54 Like other donor countries, the UK reports to the Organisation for Economic Cooperation and Development (OECD) on its ODA expenditure. We reviewed the systems of both the FCO and the British Council for ODA reporting and conducted our own testing of their combined 2012 ODA return. We found that both organisations have comprehensive systems in place with several layers of quality assurance (see page 28 in the Annex for more detail).

Impact

Assessment: Green-Amber

2.55 In this section, we review the emerging impact of FCO and DFID programming in the MENA region.

Promising signs of emerging impact from APPF

2.56 There are a number of methodological challenges involved in assessing the impact of APPF. It is a new portfolio, supporting transition processes that will continue for many years and are likely to be marked by setbacks, breakthroughs and periods of stasis. Both Tunisia and Egypt have experienced significant political instability in recent months and the prospects for successful democratic transition remain highly uncertain. In this context, the link between APPF activities and the intended results is therefore neither immediate nor linear. Many of the activities, such as cross-party dialogue or the development of parliamentary procedures, may bear fruit only some years down the track, when the right political context emerges. Even with the benefit of hindsight, it is likely to be difficult to attribute results with confidence to the APPF.

2.57 Furthermore, neither the FCO nor its implementing partners are accustomed to rigorous impact measurement. APPF activities are not generally designed so as to facilitate results measurement and there are very few impact indicators.

2.58 For all these reasons, it is only possible at this stage to make some preliminary observations on impact, based on output data, project narratives and feedback from intended beneficiaries. Our scoring is based on what impacts can reasonably be expected at this early stage and whether the programme has successfully identified interventions with the potential for transformative impact.

2.59 Tunisia has been APPF’s most successful country programme. There were 21 projects in the first two years,²⁶ with a total of £3.8 million in funding from the FCO and another £4.4 million in co-financing from other donors.

2.60 APPF successfully supported a number of key stages in Tunisia’s political transition. It achieved its goal of catalytic impact through its support for cross-party dialogue on a new constitution (see Figure 4 on page 15). Although the constitution is not yet adopted, parliamentarians we spoke to confirmed that APPF support had helped to secure agreement on some key principles. Through a UNDP-managed joint donor fund, APPF helped to ensure that the first election was free and fair, although capacity development results were limited by the fact that it worked with a temporary electoral commission rather than a permanent body. Since the election, it has made a contribution to the establishment of parliamentary committees.

2.61 Under its ‘public voice’ theme, APPF helped to prepare legislation on press freedom. The national media regulatory authority²⁷ confirmed that the technical input had been valuable. It contributed to improved coverage of the first election by the national broadcaster and has helped to improve the quality of television programmes on public affairs. It has sponsored public debates on a range of topics – drawing large audiences – and has funded the establishment of two youth-run internet radio stations. In the good governance field, it helped to develop the legal framework for a new Anti-Corruption Commission, to boost the capacity of public auditors working inside government (we found the public auditors themselves to be highly motivated to play their role in the transition) and to establish the first mechanism for monitoring political party finance.

2.62 These results are mostly still provisional in nature. The political upheavals following the assassination of an opposition leader on 6 February 2013 illustrated the continuing risk of setbacks. We nonetheless assess the Tunisia programme as having good potential for achieving impact. The activities have also achieved their goal of giving the UK a visible role in supporting the transition, thereby increasing the FCO’s access and influence.

2.63 In Egypt, since 2010-11 APPF has had 16 projects²⁸ costing £3.3 million, with an additional £2.2 million in co-financing. Under the political participation theme, APPF successfully arranged international observer missions for two elections and the constitutional referendum, increasing their credibility and reducing the potential for conflict. Through its efforts, a political consensus has emerged for regular international monitoring of elections. APPF has helped to prepare the ground for new parliamentary procedures, which may be adopted once a new parliament is elected. It will shortly begin training 700 women to stand for election as local councillors.

2.64 On the public voice theme, APPF funded the establishment of an independent news website, Aswat Masriya,²⁹ which was widely recognised as a source of objective reporting during the first elections and continues to receive around 170,000 visitors each month. It is helping a group of students from various universities to establish a ‘webzine’ focussing on parliament.³⁰ It has funded a series of high-profile television debates on political issues with a combined audience of nearly ten million viewers. It has begun to develop a network of debating associations in schools and universities, holding five high-profile youth debates that attracted online audiences of over six million. It is training 500 ‘citizen journalists’ (such as bloggers) on fact-based reporting and is providing them with a quality-controlled web platform. Under the governance theme, it is introducing social accountability mechanisms for two governorates, which are regional authorities outside Cairo.

2.65 The Egyptian programme has had a higher failure rate of projects, with a number discontinued in the second year. We consider this to be consistent with the high-risk nature of the portfolio, rather than evidence of an overall lack of success. We nonetheless conclude that the prospects for sustainable impact in Egypt are yet to be demonstrated. The programme will need to continue to seek out new entry points and opportunities.

2.66 It is not possible at this point to draw any conclusions on the sustainability of these results. Sustainability is a difficult standard to apply to this type of support. In complex transitions, it is often necessary to focus on short-term priorities rather than long-term results. As one senior diplomat put it to us, today’s problems need to be solved in order that tomorrow’s problems can be tackled. If the UK is able to contribute to a continuing positive trajectory of political change, then that offers a form of sustainability. Events in both Tunisia and Egypt since our visits make it clear that this positive trajectory is far from guaranteed. In both countries, political transition remains highly vulnerable to setbacks. While it is possible that the UK investments in strengthening national advocates of democracy will offer long-term results despite the volatility, this is a judgment that will have to be made in the future.

A useful contribution on international partnerships

2.67 The goal of securing effective international support for the Arab Spring is a key aspect of the Arab Partnership. There is a widespread perception among all the stakeholders we consulted that the overall international response has been disappointing. This is due in large part to the initial, misleading G8 announcement at Deauville, which created unrealistic expectations.

2.68 Against this background, the Arab Partnership has, nonetheless, contributed to some important results. The UK played an important role in extending the mandate of the European Bank for Reconstruction and Development (EBRD) to the MENA region (from Eastern Europe and Central Asia). This made an additional £2.2 billion in finance available to the region, with the first investments commencing in September 2012.

2.69 The UK has also been instrumental in the establishment and design of a MENA Transition Fund, managed by the World Bank. The Fund provides grants in support of institutional development and policy implementation for economic transition. Through APEF, DFID has provided a contribution of £16 million towards a proposed fund of £160 million (£110 million in pledges have been received to date).³¹

2.70 In 2013, the UK holds the chair of the G8. In this capacity, it is pursuing three initiatives under the Deauville process: an investment conference; an initiative on women’s empowerment; and a programme of mentoring for small and medium-sized enterprises. These are modest proposals but are helping to refocus the Deauville process back to concrete and realistic goals.

2.71 The goal of influencing the European Union (EU) to provide more funding to the region backed by strong political dialogue and robust conditionality has proved challenging. In May 2011, the EU’s High Representative of the Union for Foreign Affairs and Security Policy, Baroness Ashton, announced a refreshed Neighbourhood Policy, based on ‘money, markets and mobility’ (financial support, trade agreements and an easing of visa restrictions). Progress in all three areas, however, has been slow and we were unable to see any impact from FCO efforts. Indeed, attempts by the EU to apply political conditions to its support to Egypt have drawn criticism from many observers.

2.72 We were pleased to see, however, that the Arab Partnership has helped the UK Government to take a more joined-up approach to exercising its influence across different multilateral channels, rather than the traditional division of labour between the FCO and DFID. There are now regular communications between FCO posts in the region and the UK representatives at the EU and World Bank, to co-ordinate approaches.

British Council projects deliver well on skills training but wider impact is hard to judge

2.73 The British Council pursues its cultural relations mandate predominantly by educating and empowering individuals. Results are reported in terms of numbers of people trained and their feedback on the utility of the training. Where the programmes have broader goals, such as improving community resilience or reducing conflict risk, the British Council lacks rigorous methods for measuring wider impact, beyond the results for individual beneficiaries. This makes it difficult to gauge the wider significance of its projects.

2.74 For example, the ‘theory of change’ behind the Active Citizens programme is that training of local leaders or ‘change agents’ within marginalised communities will encourage them to improve their engagement in cross-cultural networks and community projects. This in turn will lead to improved confidence and understanding among different social groups, more participatory democracy and ultimately greater community resilience to conflict.

2.75 To that end, across MENA the British Council has so far engaged 18 local partners and more than 100 community-based organisations. From these, it has trained 96 facilitators who will cascade training to an additional 30 facilitators each.³² As part of their training, these facilitators will undertake social action projects in their own communities. At the time of our visit, these activities had only recently begun. The participants we spoke to reported improved social mobilisation and project management skills and high levels of motivation to put them to use in their communities. The programme had already begun to generate greater citizen engagement in local community initiatives. For example, we met participants who had organised the planting of fruit trees in their village, organised livelihood opportunities for carpet makers and provided services to blind girls in Alexandria. The programme appears, therefore, to be successful at generating more active citizens, at least in the short term. There is no way of knowing, however, whether these activities, though worthwhile in their own right, collectively contribute to the higher goals of participatory democracy and more peaceful communities.

2.76 In the APPF-funded Young Arab Voices project, the British Council has trained more than 2,500 young people in debating skills and has begun to establish a network of debating clubs in universities, schools and CSOs to cascade the training. The beneficiaries we met were enthusiastic about the potential of this initiative to foster a more constructive culture of political dialogue. In the coming phase, the project will introduce debating topics directly related to democratic transition. So far, it has organised more than 1,000 public debates, many of which attracted a wider audience via broadcast media and the internet.

2.77 In some APPF-funded projects, a lack of performance indicators focussed on results makes it difficult to draw conclusions about impact. For example, British Council research has identified that, across the region, English language skills are a factor in youth employability. Its English for Employability programme seeks to promote youth employment by improving the quality of English teaching. In Tunisia, it trained 54 English teachers from vocational training centres and selected 19 to become master trainers. It also helped to improve curricula and teaching materials. We are concerned, however, that the project is not attempting to measure its impact on the employability of students within the vocational training system, leaving the core project assumption untested.

2.78 In another youth employability initiative, the British Council is helping to improve vocational training in Egypt and Tunisia. It is developing partnerships between Egyptian and UK institutions with a view to piloting the introduction of apprenticeships, introducing quality assurance and grading of skills and improving employer engagement with the vocational training system to improve its relevance. It is also seeking to address negative perceptions of skills training among young people. To that end, it has created a social network programme using new media that has attracted 16,000 Facebook ‘likes’ and 591 Twitter followers. While it is too early to assess the results of these initiatives, we are concerned that the project has too many scattered components to be delivered effectively and remains vulnerable to the shifting priorities of the Government of Egypt.

Learning

Assessment: Amber-Red

2.79 This section considers whether there are appropriate reporting and impact assessment arrangements in place in FCO and British Council programmes. It assesses FCO central knowledge sharing and the level of adaptation in FCO programmes in response to lesson learning.

APPF shows good capacity to learn but limited capacity to measure results

APPF is not set up for impact measurement

2.80 Measuring the results of APPF poses significant challenges. To achieve its objectives, it needs to be quick and flexible in its interventions and to be willing to invest in political processes, the outcomes of which may only become apparent over time. We would not like to see an overly rigid results measurement system that undermined the distinctive strengths of the instrument.

2.81 There is, nonetheless, considerable scope for improvement on current practices. While the larger APPF projects are required to have logical frameworks (that is, programme plans setting out activities and objectives in a specific format), there are very few quantitative impact indicators across the portfolio. Reporting is almost solely at output level or in narrative form. We saw no examples of independent validation of results (such as by forming a committee of media professionals to assess the results of media activities). There is insufficient investment in data collection (such as using surveys or opinion polls to track changes in public attitudes).

2.82 Each Project Completion Report usefully sets out lessons learned. There is no process, however, of collecting and analysing these lessons, sharing them across country programmes or using them to shape future project designs.

2.83 More fundamentally, the APPF approach to results measurement is limited to capturing the results of individual projects. While this is necessary, it is not sufficient for measuring progress towards the higher-level objectives set out in each country strategy. The monitoring system should track not just whether the individual projects are delivering as intended but whether APPF as a whole is succeeding in bringing about the changes required to achieve its goals.

FCO is not accustomed to sharing knowledge with partners

2.84 The FCO has a range of mechanisms for collecting and generating knowledge around APPF goals. Its MENA Research Group, consisting of seven analysts, acts as an in-house learning resource. It maintains links with UK academics who specialise in Arab politics and societies and organises periodic events (such as a recent roundtable on political Islam following the Arab Spring). It prepares summaries of articles from international think tanks. It publishes periodic issue papers, some of which are directly related to APPF programming (including an analysis of the new Egyptian constitution). We received some feedback to the effect that staffing constraints in the FCO had reduced its capacity to interact with the wider foreign policy research and academic community.

2.85 Learning is also supported by regular political reporting from the region. Staying up to date with the transition processes that APPF seeks to influence is the FCO’s core business. Country strategies are refreshed annually. Generally, we found the programming to be flexible and responsive in a dynamic environment.

2.86 It does not appear that the FCO draws on lessons from other political transitions around the world, particularly in technical areas like media reform. The FCO expects its implementing partners to possess this knowledge and in some cases they clearly do. It nonetheless needs the capacity to challenge its partners on their programming choices.

2.87 A notable omission from the current APPF approach is that implementing partners are not being encouraged to network, in order to share lessons or develop complementary approaches, even where they are working on common objectives. This means that opportunities may be missed to increase the leverage or influence of APPF projects. Partners told us that they would welcome this. By contrast, we saw a good example of the British Council’s partners working together in an effective network in the Active Citizens programme in Egypt.

There is good evidence of learning and adaptation at the organisational level

2.88 At the organisational level, we saw good evidence of flexibility and willingness to learn for such a young programme. The FCO has carried out a series of internal reviews of the Arab Partnership. APPF systems and processes have developed rapidly to meet delivery challenges, with senior management paying close attention to ensuring that lessons are translated into action.

2.89 Examples of adaptations in response to lessons learned include the introduction of country strategies, the introduction of the Tactical Fund for rapid response purposes (see paragraph 2.34 on page 10), the shift to multi-annual funding (paragraph 2.29 on page 9), increases in staff project management capacity in country, the shift from over-reliance on UK-based implementing partners (paragraphs 2.31 to 2.33 on page 10), more use of Arab language materials to reach out to local implementing partners and a shift in the focus of programming beyond the capitals. This is an impressive range of measures in a short time and suggests good capacity to learn.

The British Council is strong on beneficiary feedback but weaker on impact measurement

2.90 The British Council systematically collects feedback from beneficiaries of its programmes as its primary tool for measuring results. It administers questionnaires following each individual event, such as a training session. It also commissions a global Annual Impact Survey, conducted by Ipsos MORI. The survey covers 5,000 people who have been involved with British Council projects over the past three years, out of an estimated 6.5 million around the world (2012).³³ Through an online questionnaire, respondents are asked to rate the impact of the programme on their professional life and their organisation and about their awareness of – and links to – the UK. They can also opt in to a follow-up telephone interview, in which they provide further, qualitative information on the long-term impact of British Council activities.

2.91 In 2011-12, 55% of people stated that British Council programmes have had a ‘significant impact’ on their professional life (and 92% ‘some impact’). The British Council has an overall customer satisfaction target of 83%, which was met in 2011-12. The Education and Society business unit scores particularly highly at 80% on the net promoter score index (i.e. the percentage of customers willing to recommend a brand minus the percentage unwilling to do so).

2.92 We welcome this systematic process of collecting feedback from beneficiaries. We would like to see more of this practice in the UK aid programme.

2.93 We note, however, that the respondents are those who have received direct benefits from British Council programmes without paying for them. As such, there is a high likelihood of positive feedback. We are concerned that the survey is used more to prepare promotional material than for either accountability or learning.³⁴

2.94 As noted, the British Council does not put equivalent effort into measuring impacts beyond the immediate results for individuals. While individual programmes report on results at the outcome level, the practice of external evaluation is not well developed. While a focus on the individual is understandable for training programmes, it means that the ‘theories of change’ underlying British Council programmes are often left untested. While we appreciate that British Council programmes are often small and that monitoring needs to be proportionate, greater investment in results measurement would enable the British Council to be more accountable about its impact.