How DFID Learns

1. Introduction

1.1 The Independent Commission for Aid Impact (ICAI) is the independent body responsible for scrutinising UK aid. We focus on maximising the effectiveness of the UK aid budget for intended beneficiaries and on delivering value for money for UK taxpayers. We carry out independent reviews of aid programmes and of issues affecting the delivery of UK aid. We publish transparent, impartial and objective reports to provide evidence and clear recommendations to support UK Government decision-making and to strengthen the accountability of the aid programme. Our reports are written to be accessible to a general readership and we use a simple ‘traffic light’ system to report our judgement on each programme or topic we review.

1.2 Our reports on DFID’s programmes typically focus on four areas of assessment: objectives, delivery, impact and learning. By learning, we mean the extent to which DFID gains and uses the knowledge it garners in the course of its work (and the work of others in related areas) to influence its actions and strategies.

1.3 From the 27 ICAI studies published to date,¹ we have seen that DFID staff do not always fully use available knowledge when making decisions. We want to understand why this happens, as we have seen how learning can enhance the impact and effectiveness of aid. We have also seen how failure to use learning can inhibit the effectiveness and impact of UK aid. We noted in our 2011-12 Annual Report that, ‘with DFID’s technical expertise and standing, we would expect to see better sharing and lesson learning about what is both good and poor practice’.² The International Development Committee (IDC) is also interested in learning and has requested that we look into how DFID’s staff learn and how the organisation enables them to do so.

1.4 This inception report sets out the assessment questions, methodology and a work plan for the delivery of the review. It is, however, intended that the methodology and work plan are flexible enough to allow for new issues and questions that emerge over the course of the review.

2. Background

2.1 Please see Terms of Reference for background.³

3. Purpose of this review

3.1 To assess how effectively DFID and its staff learn in order to improve the value for money and impact of aid programmes, taking account of DFID’s increasing focus on fragile states.

Approach

3.2 Our review will examine what difference learning makes to DFID’s work in reducing poverty. The focus will be on DFID’s staff and their experience and practice. At the same time, we will look at the corporate enabling environment for individuals’ learning.

3.3 We will focus our analysis on four key activities that DFID undertakes. These are:

• making programme choices;
• creating theories of change;
• choosing delivery mechanisms; and
• adapting and improving implementation of its activities.

3.4 We will investigate how DFID is using learning to carry out these activities.

4. Relationship to other reviews

4.1 Considerable data exists relating to how DFID learns; this has been collected by ICAI, by DFID itself and by third parties. Our approach will be, in the first instance, to draw upon this material (primarily that created since 2010). We will augment this by collecting our own primary data only where there is insufficient evidence from the existing material to answer our questions or where we have concerns about the quality of that data.

ICAI’s own reports

4.2 We will draw upon the evidence of ICAI’s 27 studies. These have considered 120 DFID programmes across 38 countries/territories, including undertaking visits to 24 DFID country offices. Our approach will be first to distil learning-related lessons from all the reports. We will also draw upon the lessons learned from ICAI’s process of follow-up with DFID that takes place after our reports have been published. See Figure 1 for an overview of reports published to date.

4.3 Our initial analysis of the 27 reports indicates a consistent set of very broad findings that relate to learning. See Figure 2 for a brief summary.⁴

Case studies derived from ICAI’s reports

4.4 Building on ICAI’s analysis to date, we will specifically identify 12 areas of interest from our previous reports as case studies. These will be selected to provide insights into the activities set out in section 4.2 above. Our decision on these case studies is informed by the findings of each report. We will gather information on each case study in order to help answer our evaluation questions. See Figure 3 for a provisional list of these case studies.

DFID’s own survey data

4.5 In addition to our existing information, we will draw upon data from DFID’s internal staff surveys undertaken between 2010 and 2013. With varying coverage and depth, these provide considerable insight into how DFID learns. Figure 4 summarises the surveys of which we are currently aware.⁶

Studies of DFID by others

4.6 We will also draw evidence from others’ assessments of DFID’s learning. For instance, a study by the Overseas Development Institute (ODI) from 2010⁸ identified the following three domains for learning in DFID:

• For research and evaluation outputs: ODI noted that ‘the question of whether lessons are learned focusses on how influential that work is, whether findings and recommendations are taken up in policy and programming and acted upon’;
• For decision-making and action: ‘the question of lesson learning becomes a matter of looking at the extent to which evidence (and in particular, that emerging from DFID’s research and evaluation) feeds into and informs the process of policy making and programming’; and
• Concerning DFID as a learning organisation: ‘the question of lesson learning focusses on how knowledge within DFID is captured, shared and used, as and where it is needed’.

4.7 The ODI study suggests that DFID is more comfortable with using the findings of research and evaluation than it is with organisational learning (see bullet three of Figure 5). Similarly, it is much better at using research and evaluation findings during or as part of, a project cycle, than in more complex and emergent decision-making processes. ODI concluded that initiatives that promote a sense of ownership of research and evaluations and those that support the development and strengthening of interpersonal learning networks were effective. We are particularly interested in DFID’s corporate learning, noting that, in 1990, Senge identified that organisational learning is ‘only successful when it is based on an understanding of how the whole organisational system is connected, rather than a focus on individual parts’.⁹

Research external to DFID

4.8 In addition, we will draw upon reviews of other organisations. The topic of learning is one that many development agencies seek to address. For instance, a 2007 report for the Government of Sweden noted that ‘despite increasingly rigorous feedback systems, development agencies continue to be criticised for their inability to incorporate past experience. They are routinely accused of learning too little, too slowly – or learning the wrong things, from the wrong sources’.¹⁰

5. Methodology

Conceptual framework

5.1 There is considerable general literature on how organisations learn and manage their knowledge.¹¹ The literature indicates that learning requires innovation, creativity, openness and commitment. This needs to be balanced with some structure, stability and continuity as well as focussing on achieving the organisation’s aims.¹²

5.2 We want to understand how people in DFID gain knowledge and whether and how this influences their decision-making. To enable us to interpret the evidence gathered for each of the four activities set out in paragraph 3.3 above, we therefore wish to identify the different aspects of how learning takes place. This will allow the assessment framework questions to be answered (see 5.12 below). The following sets out the characteristics that we consider are required for an organisation such as DFID to learn effectively. It is based on lessons from the general literature (some of which is referenced below).

• Clarity of Purpose;
• Connectivity;
• Creation;
• Capture;
• Communication; and
• Challenge.

Clarity of purpose: This will identify the ‘for what’.

5.3 We will seek to identify, whether DFID is clear about what learning it needs to adapt and develop in order to achieve its objectives. We are particularly interested in whether there is such clarity at all levels of the organisation, including individual staff.¹³

Connectivity: This will identify the mechanism that allows knowledge to flow through DFID.

5.4 We are interested in how well DFID’s staff are linked together; within professional groups, vertically between policy, programme management and delivery, as well as horizontally between people doing similar tasks in different places, particularly country offices. We will capture how DFID’s staff connect to intended beneficiaries, delivery agents, peer organisations and other sources of knowledge outside DFID. Such connections are through both formal and informal mechanisms and networks. We believe that the ideal state is that individuals participate in open communications that allow the transfer of knowledge and experience. This relationship requires mutual support amongst the individual, the organisation and their supporting networks.’¹⁴

Creation: This will address where and how DFID staff acquire knowledge, where new ideas are created and from where they emerge.

5.5 A key issue here is identifying the relative impact of different sources that DFID uses to create knowledge, including research institutions, academics, opinion formers and communities of practice. We will include looking at whether there are opportunities for discovery through, for example, testing, trial and error and role-playing scenarios.¹⁵

5.6 When considering how DFID learns we wish to specifically make a distinction between ‘knowledge’ and ‘know-how’.¹⁶ In a paper presented to the World Bank in 1998, Steve Denning, then the World Bank’s Programme Director for Knowledge Management, also suggested the following: ‘Knowledge then is more than just information (‘Know What’). It is also about experience of what works (‘Know How’), the reasons for doing things (‘Know Why’) and who can help through contacts and networks (‘Know Who’).¹⁷

Capture: This will identify how DFID captures knowledge that can be used for learning and how lesson-learning happens before, during and after activities take place.

5.7 We wish in particular to identify how individuals and DFID as a whole learn from both success and failure and the impact this has on their decision-making processes and creative cycles. We will specifically look at DFID’s experience of learning from intended beneficiaries and its readiness to do so.¹⁸

Communication: This will consider whether and how the right information gets to where it is needed.

5.8 We want to know what the processes are that are actively used to communicate knowledge around the department, rather than just how people are linked together (see points on connectivity above). We are conscious that lessons from the literature show that people learn best from each other.¹⁹

Challenge: This will identify how DFID challenges staff to ensure that they apply learning to improve delivery and impact.

5.9 We are interested in DFID’s corporate culture of learning, the examples set by managers and the corporate expectations for learning. We are particularly interested in how individual staff are managed to ensure that their and DFID’s learning improves performance.

Sources of Evidence

5.10 Our approach uses mixed methods to gather evidence. Through the surveys, it will use quantitative tools to assess qualitative findings from respondents. The surveys will also ensure coverage of DFID staff. Semi-structured interviews of individuals and focus group discussions, as well as case studies, will be used to drill down into particular themes and issues. Where possible, we will use evidence from currently available data in order to reduce the burden on DFID and other respondents.

5.11 The different methods we will use to gather evidence are set out below.

1. Literature review:
   1. general literature on learning;
   2. private sector approaches;
   3. literature specific to development agencies;
   4. literature specific to DFID’s approach to learning; and
   5. comparison with approaches used within other organisations.
2. ICAI review findings:
   1. documentary analysis of all 27 reviews published by ICAI to date;
   2. analysis of learning from ICAI’s follow-up processes to date; and
   3. qualitative assessments from a focus group of ICAI study team leaders.
3. DFID’s documentation (leading to compilation of foundation evidence pack):
   1. desk-based Quest review of relevant DFID policy and guidance documents;
   2. desk-based Quest review of DFID decisions regarding learning; and
   3. desk-based review of DFID’s spending on learning.
4. External sources of learning used by DFID:
   1. desk-based review of sources, knowledge and information identified by staff as providing them with information used for decision-making (e.g. resources and research centres); and
   2. mapping of sources.
5. DFID’s own survey data:
   1. analysis of data from DFID internal surveys undertaken during 2011, 2012 and 2013 (see paragraph 4.5 above).
6. ICAI’s own survey of staff:
   1. we will only undertake our own survey if evidence from DFID’s own data is insufficient.
7. Semi-structured interviews with DFID staff:
   1. focus group discussions with groups of staff through the organisation (horizontally through geographical or thematic areas and vertically through programme and delivery areas);
   2. decision-makers responsible for corporate learning; and
   3. specific staff as required to investigate issues arising from the surveys.
8. Semi-structured interviews with external informants:
   1. external researchers; and
   2. other stakeholders (such as representatives from BOND).²⁰
9. Case studies (see paragraph 4.4 above) of examples of learning relating to specific decision points (i.e. making better programme choices, creating theories of change, choosing delivery mechanisms and adapting and improving during implementation of activities):
   1. individual case studies will primarily be derived from ICAI reports to date;
   2. evidence will be gathered mainly from documentary evidence and in-depth telephone interviews via video conference with decision-makers and stakeholders, as well as with ICAI team members; and
   3. specific case studies on India and Nepal, to discuss learning issues including following up on relevant findings from our previous reports. This will involve focus groups and interviews with DFID staff, following up on previous ICAI findings and talking to third parties in each country. We have decided that it will be possible for us to do this remotely, as we will be able to conduct focus groups and interviews using video conferencing. If, however, these case studies do not produce the necessary material we may need to revisit the decision not to visit these countries; we do not consider this to be likely.

Assessment Framework

5.12 Since this is a thematic study of learning, we will not be following ICAI’s usual assessment framework, although the questions below draw upon the thinking behind it. As usual, we wish to consider DFID’s clarity of vision for learning (its objectives, captured under ‘incentives for learning’ below), how well it undertakes learning (effectiveness), the impact on DFID’s work and how well DFID develops its own learning approaches as a result of experience. We have shown the relationship between the assessment framework and how we will be ordering the evidence into our conceptual framework set out above. This should be seen as indicative only.

Coverage of evidence and limits to the methodology

5.13 This is a thematic study covering all of DFID. We are taking evidence from 2011, 2012 and 2013 and looking to draw conclusions about the organisation as a whole.

5.14 We will adhere to the standard ICAI principle of ensuring findings, relying on triangulated sources of evidence. Thus in order to enable an evaluative judgement to be made, there must usually be evidence from at least three different sources to provide a credible finding and evaluators must not be aware of credible contradictory evidence able to falsify or refute the finding. In some cases, where evidence is strong and cannot be contradicted, one or two sources of evidence may be sufficient.

5.15 It will be important that the staff sample sizes (e.g. from surveys and interviews) have validity. For instance, we are aware that DFID’s Use of Evidence survey (see paragraph 4.5 above) had responses from 461 middle and senior staff respondents on eight elements of the use of evidence (see Figure 6). This covered some 23% of all DFID’s staff (it had a response rate of 88% of those invited to respond). This survey provides a sample of sufficient size to provide a margin of error of 4% at a confidence level of 95%. In other words we know that, technically, it could be between 91% and 99% accurate in its representation of DFID staff’s views for each particular question it asked.

5.16 There is a specific need, however, to ensure that participation bias does not skew our findings. For instance, we will need to ensure that we have sufficient evidence from locally employed staff and from programme implementation level personnel.

5.17 For each of the four decision points (making better programme choices, creating theories of change, choosing delivery mechanisms and adapting and improving during implementation of activities we will identify three case study examples to draw upon, leading to a total of 12 examples.

5.18 We are sensitive to the need to avoid ‘survey fatigue’ in DFID and do not want to add unnecessarily to the burden of staff, hence we will only undertake our own evidence gathering, if insufficient evidence exists from current material.

5.19 Since we are covering the entire DFID organisation, any case studies will be our best attempt at finding examples that represent the whole. DFID is clearly a complex organisation and we will be careful not to simplify our findings in an unwarranted way.

5.20 The study of organisational learning has many different and reputable approaches. We have chosen a structure and set of learning elements, that we consider will provide the necessary support for assessing how effectively DFID and its staff learn in order to improve value for money and impact of aid programmes. We will, however, be attentive to any situations where the structure needs to be adjusted to accommodate new learning as the review progresses.

6. Roles and responsibilities

6.1 This review will be led and managed on a day-to-day basis by the Team Leader, who will be the primary point of contact with DFID.

6.2 Oversight of this review will be under the overall leadership of ICAI Project Director.

6.3 It is proposed that this assessment be undertaken by a core team of three with supplementary peer review if deemed necessary. While team members will have lead responsibility for answering sections of the framework, all will contribute to the analysis supporting the findings for each section.

Team Leader (Agulhas)

He is a Director of Agulhas Applied Knowledge. He specialises in aid effectiveness, governance and institutional development. He is on the core consortium team delivering ICAI’s reports. He has led ICAI reviews considering Bangladesh Climate Change, UNDP’s management of elections, DFID’s programmes of Health and Education in Bihar, its support for Rural Livelihoods in Western Orissa and Programme Partnership Arrangements. He has written on learning and knowledge management in development organisations, including in the past analysing how DFID compares with best practice in knowledge management. He will lead the team.

Learning Expert (Independent)

He will provide specialist learning expertise. As a consultant/coach for over eight years, he has worked with and supported senior management teams, government departments, small businesses and charities and other types of organisations such as laboratories. This work includes strategic realignments, restructuring, team and department development, system and process improvements and key relationship management. He is Steward with the Society of Organisational Learning in the UK (SoL-UK). For three years he was the part-time Director for a small international charity, Acid Survivors Trust International (ASTI). Working with the Chairman and Trustees, he developed the charity’s strategic goals and worked to ensure that they were successfully achieved. Currently, he is a part-time director of the small charity Working in Trust which is focussed on developing interdependence and trust in organisations. Prior to this, he spent over 25 years working in large corporations (up to 16,000 employees) with increasing responsibilities which included managing an overseas office.

Team Member (Agulhas)

She is a consultant with Agulhas Applied Knowledge and a former programme manager in the DFID Somalia team. She conducted much of the field research for ICAI’s reviews of DFID’s Climate Change Programme in Bangladesh²² and ICAI’s evaluations of DFID’s Electoral Support through UNDP²³ and DFID’s Bilateral Aid to Pakistan.²⁴ She also supported Agulhas’s work on ICAI’s review of DFID’s Support for Civil Society Organisations through Programme Partnership Arrangements.²⁵ She will provide analytical, research and administrative capability to the team.

Methodology Peer Review (University of Manchester)

He has more than 10 years’ experience teaching undergraduate and MA and MSc students development microeconomics (theory and applied) and supervising and teaching PhD students quantitative and qualitative research methods in British (University of East Anglia) and Indian higher education institutions. He has been in charge of extensive primary data collection exercises with a development economics angle in India (Western Uttar Pradesh, Andhra Pradesh, Coastal and Central Karnataka, Delhi, Mumbai), in Nepal and in East Africa. These included household surveys, tracking of in-depth interviews documenting the worklife histories of (young and other) migrants, interviews with local political leaders and behavioural experiments. He is the editor of the Journal of South Asian Development. His academic publication record features a wide range of poverty and development-related themes.

Researcher (KPMG)

He is a programme manager for the ICAI programme and assists in managing the overall programme. He is educated to Master’s level in Economics, with particular knowledge of economic growth and development and has previous experience in international financial institutions and non-profit organisations. He is familiar with ICAI, having worked on the recent ICAI reviews of DFID’s Support to Agricultural Research²⁶ and of DFID’s Support to Palestine Refugees through UNRWA.²⁷

He will support the gathering of information from DFID systems.

7. Management and reporting

7.1 We will produce a first draft report for review by the ICAI Secretariat and Commissioners by 10 January 2014, with time for subsequent revision and review prior to completion and sign off in w/c 24 February 2014.

8. Expected outputs and timeframe

8.1 The assessment will consist of the following elements

9. Risks and mitigation

9.1 The following sets out the key risks and mitigating actions for this assessment:

10. How this ICAI review will make a difference

10.1 This review seeks to synthesise ICAI and others’ observations that DFID is not learning adequately from its own and other’s experience. The review will seek to focus attention on corporate and individual behaviours and practices that could be changed. By improving its learning, we expect that the impact and value for money that DFID achieves will be improved.

Annex 1: Selected Bibliography

The following is a selected bibliography of material used in drafting this inception report.

• 3M Company, A Culture of Innovation, 2012.
• Ackermann, Edith K., Perspective-Taking and Object Construction: Two Keys to Learning, 1996.
• Aidemark, Jan, Knowledge Management Paradoxes, 2009.
• Argote, Linda, Organisational Learning: Creating, Retaining and Transferring Knowledge, 2013.
• Asian Development Bank, Knowledge Management Directions and Action Plan (2013-2015): Supporting ‘Finance ++’ at the Asian Development Bank, 2013.
• Bavarian Motor Works (BMW), Competitive Edge – Learning Organisation.
• Capra, Fritjof, The Hidden Connections, 2002.
• Cook, Scott D. N. and Brown, John Seely, Bridging Epistemologies: The Generative Dance Between Organisational Knowledge and Organisational Knowing, 1999.
• De Geus, Arie, The Living Company: Growth, Learning and Longevity in Business, 1999.
• Devane, Sinead and Wilson, Julian, Business Benefits of Non-Managed Knowledge, 2009.
• Dolcemascolo, Darren, Lean Leadership: A Tale of Two Companies – website article, August 2007.
• Dyer, Jeffrey H. and Nobeoka, Kentaro, Creating and Managing a High Performance Knowledge-Sharing Network: The Toyota Case, 2000.
• Erikson, Emily and Sampsa, Samila, Decentralisation, Social Networks and Organisational Learning, 2012.
• Harkema, S. J. M. and Browaeys, M. J., Managing innovation successfully: a complex process, 2002.
• Hovland, Ingie, Knowledge Management and Organisational Learning: An International Development Perspective, an annotated bibliography for Overseas Development Institute, August 2003.
• Ibarra, Herminia, Kilduff, Martin and Wenpin, Tsai, Zooming In and Out: Connecting Individuals and Collectiveities at the Frontiers of Organisational Network Research, July/August 2005.
• Kira, Mari and Frieling, Ekkehart, Collective Learning, Building on Individual Learning, September 2005.
• Kolb, D. A., Experiential learning: experience as the source of learning and development, 1984.
• Liker, Jeffery K., The Toyota Way: 14 Management Principles from the World’s Greatest Manufacturer, December 2003.
• March, James G., Exploration and Exploitation in Organisational Learning, February 1991.
• Matthews, Sarag and Thornton, Nigel, Doing the Knowledge: How DFID Compares with Best Practice in Knowledge Management, 2001, DFID.
• Jones, Harry and Mendizabel, Enrique, Strengthening learning from research and evaluation: going with the grain, Overseas Development Institute, 2010.
• Ramalingam, Ben, Learning how to learn, eight lessons for impact evaluations that make a difference – Background Note, Overseas Development Institute, April 2011.
• Roschelle, Jeremy and Teasley, Stephanie D., The Construction of Shared Knowledge in Collaborative Problem Solving,1995.
• Sanchez, Ron, Knowledge Management and Organisational Learning: Fundamental Concepts for Theory and Practice, 2005.
• Senge, Peter M., The Fifth Discipline, The Art and Practice of The Learning Organisation, 2006.
• Sharpanskykh, Alexei, Individual Decision Making in a Learning Organisation, 2009.
• Skerlavaj, Miha, Ji Hoon Song and Youngmin Lee, Organisational learning culture, innovative culture and innovations in South Korean firms, 2010.
• Tsoukas, Haridimos and Efi Vladimirou, What is Organisational Knowledge, November 2001.
• Unilever, Unilever Sustainable Living Plan: Progress Report 2012, 2013.
• World Bank: Knowledge for Development 2011, The State of World Bank Knowledge Services, 2011.

Footnotes

1. The team will update their analysis in the drafting phase to include newly published reports.
2. Independent Commission for Aid Impact: Annual Report to the House of Commons International Development Committee 2011-2012, ICAI, June 2012, Paragraph 54, http://icai.independent.gov.uk/wp-content/uploads/2011/11/ICAI-Annual-Report-2011-12-FINAL.pdf.
3. Terms of Reference: How DFID Learns, ICAI, http://icai.independent.gov.uk/.
4. These should not be taken as ICAI’s conclusions but emerging findings that will be tested.
5. See paragraph 4.2 above.
6. The number, range and detail of internal surveys during the last three years imply that we have access to sufficient primary data from DFID to mean we will not have to conduct our own survey. This clearly also reduces the burden on DFID for their participation in this ICAI study.
7. The 70:20:10 model suggests that any individual in the Civil Service gains 70% of their learning from personal experience 20% from other people and 10% from courses and reading.
8. Strengthening learning from research and evaluation: going with the grain, Overseas Development Institute, 2010, http://www.odi.org.uk/sites/odi.org.uk/files/odi-assets/publications-opinion-files/6327.pdf.
9. Quoted by Ingie Holvand in Knowledge management and organisational learning: an international development perspective, Overseas Development Institute, 2003.
10. Knowledge and Learning in Aid Organisations, Swedish Agency for Development Evaluation, 2007.
11. Knowledge Management is ‘the attempt to recognise what is essentially a human asset buried in the minds of individuals and leverage it into an organisational asset that can be used by a broader set of individuals on whose decisions the firm depends’ Larry Prusak, Head of Knowledge Management at IBM, cited in Sarah Matthews and Nigel Thornton, Doing the Knowledge: How DFID Compares with Best Practice in Knowledge Management, DFID, 2001.
12. Skerlavaj, Miha and Ji Hoon Song and Youngmin Lee, Organisational learning culture, innovative culture and innovations in South Korean firms, 2010.
13. Roschelle, Jeremy and Stephanie D. Teasley, The Construction of Shared Knowledge in Collaborative Problem Solving, 1995, Capra, Fritjof, The Hidden Connections, 2002.
14. Capra, Fritjof, The Hidden Connections, 2002; also Ibarra, Herminia and Martin Kilduff and Wenpin Tsai,Zooming In and Ou,: Connecting Individuals and Collectiveities at the Frontiers of Organisational Network Research, July/August 2005.
15. Kira, Mari and Ekkehart Frieling: Collective Learning, Building on Individual Learning, September 2005.
16. Cook, Scott D. N. and John Seely Brown: Bridging Epistemologies: The Generative Dance Between Organisational Knowledge and Organisational Knowing, July/August 1999.
17. Steve Denning, What is Knowledge Management?, paper presented to the World Bank Management Board 1998 quoted in Sarah Matthews and Nigel Thornton, Doing the Knowledge: How DFID Compares with Best Practice in Knowledge Management, DFID, 2001.
18. See Ackermann, Edith K., Perspective-Taking and Object Construction: Two Keys to Learning, 1996. See also Kolb, D. A., Experiential learning: experience as the source of learning and development, 1984 and Devane, Sinead and Wilson, Julian, Business Benefits of Non-Managed Knowledge, Electronic Journal of Knowledge Management, Volume 7, Issue 1, 2009, http://www.ejkm.com/volume7/issue1.
19. Tsoukas, Haridimos and Efi Vladimirou: What is Organisational Knowledge, 2001.
20. BOND is the UK membership body for organisations working in international development, see http://www.bond.org.uk.
21. DFID’s grading system: SCS = Senior Civil Service, A1 = G6, A2 = G7, A2 (L) = Senior Executive Officer, B1 (D) = Fast Stream, B1 = Higher Executive Officer, B2 = Executive Officer, C1 = Administrative Officer, C2 = Administrative Assistant, D = Drivers and Ancillary Staff.
22. DFID’s Climate Change Programme in Bangladesh, ICAI, 2011, http://icai.independent.gov.uk/wp-content/uploads/2011/11/ICAI-Report-DFID-Climate-Change-Programme-in-Bangladesh-FINAL1.pdf.
23. DFID’s Electoral Role through UNDP, ICAI, 2012, http://icai.independent.gov.uk/wp-content/uploads/2013/12/UNDP-report-FINAL.pdf.
24. Evaluation of DFID’s Bilateral Aid to Pakistan, ICAI, 2012, http://icai.independent.gov.uk/wp-content/uploads/2013/12/ICAI-Pakistan-Report_P1.pdf.
25. DFID’s Support for Civil Society Organisations through Programme Partnership Arrangements, ICAI, 2013, http://icai.independent.gov.uk/wp-content/uploads/2013/12/ICAI-REPORT-DFIDs-Support-for-CSOs-through-PPAs.pdf.
26. DFID’s Support to Agricultural Research, ICAI, 2012, http://icai.independent.gov.uk/wp-content/uploads/2013/12/ICAI-Agricultural-Research-report-FINAL.pdf.
27. DFID’s Support for Palestine Refugees through UNRWA, ICAI, 2013, http://icai.independent.gov.uk/wp-content/uploads/2013/12/ICAI-UNRWA-report-FINAL-110913.pdf.