Content uploaded by Boru Douthwaite
Author content
All content in this area was uploaded by Boru Douthwaite
Content may be subject to copyright.
Participatory Impact Pathways Analysis: a practical method
for project planning and evaluation1
Sophie Alvarez, Boru Douthwaite, Graham Thiele, Ronald Mackay, Diana Cordoba and
Katherine Tehelen
Paper prepared for: ‘Rethinking Impact: Understanding the Complexity of Poverty and
Change’ Workshop
March, 2008
Abstract
Participatory Impact Pathways Analysis (PIPA) is a practical planning, and monitoring
and evaluation approach developed for use with complex projects in the water and food
sectors. PIPA begins with a participatory workshop where stakeholders make explicit
their assumptions about how their project will make an impact. Participants construct
problem trees, carry out a visioning exercise and draw network maps to help them
clarify their ‘impact pathways’. These are then articulated in two logic models. The
outcomes logic model describes the project’s medium term objectives in the form of
hypotheses: which actors need to change, what are those changes and which strategies
are needed to realise these changes. The impact logic model describes how, by helping
to achieve the expected outcomes, the project will impact on people’s livelihoods.
Participants derive outcome targets and milestones which are regularly revisited and
revised as part of project monitoring and evaluation (M&E). PIPA goes beyond the
traditional use of logic models and logframes by engaging stakeholders in a structured
participatory process, promoting learning and providing a framework for ‘action
research’ on processes of change. The two logic models provide predictions of future
impact which can be used in priority setting. They also provide impact hypotheses
required for ex-post impact assessment.
Acknowledgements
We wish to particularly thank the support from the Challenge Program on Water and
Food, the EU-funded EULACIAS project and the DFID-funded Andean Change Project.
Author Bios
Sophie Alvarez is a monitoring and evaluation specialist working for the Challenge
Program on Water and Food (CPWF) and has worked as a consultant for other
organizations including CIAT, CIMMYT, CIP and Bioversity.
Dr. Boru Douthwaite is a technology policy analyst and evaluator working as the
Innovation and Impact Director in the Challenge Program on Water and Food (CPWF).
1 This is based partly on an ILAC Brief with the same title. See: http://boru.pbwiki.com/f/PIPA-ILAC-Brief-
pre-print.doc
1
His articles and an overview of his book “Enabling Innovation” are available at
http://boru.pbworks.com/Boru+Douthwaite
Introduction
People act on the basis of their understanding of how the world works – their ‘theories
of action’ (Argyris and Schön, 1974). We do X because we believe, based on past
experience or what we’ve read, that Y will happen. This applies to projects and
programs as well. So it follows that if you can improve a project’s theories of action you
may be able to improve how people implement it (in this paper we use project to mean
both project and program). This has long been recognized by a particular branch of
evaluation, called program theory evaluation, which describes projects’ theories of
action in a ‘logic model’ and then evaluates the project using the model as a framework
(see Chen, 2005 for example). Traditionally, logic models describe how project outputs
are developed with, and used by, others to achieve chains of outcomes that contribute
to eventual impact on social, environmental or economic conditions. In this paper we
describe the Participatory Impact Pathways Analysis (PIPA) approach, which allows
project staff and stakeholders to jointly describe the project’s theories of action, develop
logic models and use them for project planning and evaluation. The term ‘impact
pathways’ is synonymous with ‘theories of action’ and ‘program theory’. We use the
term because it is more widely understood in agricultural research. PIPA is similar in its
philosophy to Outcome Mapping (Earl et al. 2001). A main difference is that PIPA
stretches participants to predict how project outcomes can lead to social, economic and
environmental impacts.
Development and use of PIPA
The first PIPA workshop was held in January 2006 in Ghana, with seven projects
funded by the Challenge Program on Water and Food. To date, nine PIPA workshops
have been held for 46 projects in the CPWF, and 11 more for other projects.
Researchers from the International Center for Tropical Agriculture (CIAT - Spanish
acronym), WorldFish Center and the International Potato Center (CIP - Spanish
acronym) together with two evaluation specialists2 are developing PIPA. PIPA
developed from work at CIAT on innovation histories (see ILAC Brief no. 5 – Douthwaite
and Ashby, 2005) funded by the Institutional Learning and Change Initiative (ILAC). A
paper describing the approach was published in the Canadian Journal of Program
Evaluation (Douthwaite et al., 2007).
PIPA centers on a two- or three-day workshop in which ideally project implementers,
participating next users, end users and politically-important actors attend. Next users
are the people and organizations who will use what the project produces, while end
users are the people the next users serve. Clients and beneficiaries are synonyms for
next users and end users. Politically-important actors are those people and
organizations that can help create an enabling environment for the project, but with
which the project does not work directly.
2 Ronald Mackay and Rick Davies
2
The workshop process is designed to help participants surface, discuss and describe
their hypotheses for how project activities and outputs could eventually contribute to
desired goals such as poverty reduction. The description of these hypotheses is a
description of the project’s impact pathways.
PIPA has helped workshop participants to:
•Clarify, reach mutual understanding and communicate their project’s intervention
logic and its potential for achieving impact;
•Understand other projects working in the same program and identify areas for
collaboration;
•Generate a feeling of common purpose and better programmatic integration
(when more than one project is represented in the workshop);
•Produce a narrative describing the project's intervention logic and possible future
impacts (thus a form of ex-ante impact assessment);
•Produce a framework for subsequent monitoring and evaluation.
When the PIPA workshop works best
We have found the PIPA workshop is useful when two or more projects in the same
program wish to better integrate. At least two people for each project should attend,
preferably the project leader and some else who knows the project and has the time
and inclination to follow up on what comes out of the workshop. The PIPA workshop
also works well when one project wishes to build common understanding and
commitment with its stakeholders. In this case, two or more representatives from each
important stakeholder group should attend. The ideal stakeholder group size is four to
six and the ideal number of groups is three to six. We have facilitated workshops with
nine projects developing impact pathways but this leaves little time for individual
presentations and plenary, and participants tend to be overwhelmed by too much
information.
The PIPA process
We have used PIPA at the beginning, middle and end of projects. PIPA describes
project (or program) impact pathways in two ways:
(i) A problem or objectives tree that describes the linear logic, showing that if the
project helps solve certain problems, it will contribute to solving others and so
eventually achieve its goal; and,
(ii) Network maps showing how the actors involved work together, influence each
other and influence the general environment for the new knowledge or
technology being developed
The workshop process, shown in Figure 1, develops the two perspectives in turn and
then integrates them through developing an outcomes logic model that describes the
project strategies, outputs and outcomes necessary to achieve the project vision. The
table links the outcomes to the actor or group of actors that will bring them about, thus
making future evaluation easier. Somewhat similar logframes commonly used in the
CGIAR system often lack an actor focus and can end up containing narrative
statements in them without people, for example “rice yields increased by 25% in pilot
3
sites”. The impact logic model, when required, is developed after the workshop and
describes how the outcomes the project contributes to will scale-out and scale-up in
order to achieve social, economic and environmental impacts.
Figure 1: The PIPA Workshop
1 . P r o b l e m T r e e
2 . O u t p u t s
3 . V i s i o n
7 . O u t c o m e s l o g i c m o d e l
4 . " N o w "
n e t w o r k m a p
W h a t t h e p r o j e c t w i l l p r o d u c e
W h e r e p r o j e c t i s g o i n g - G o a l
N e c e s s a r y
r e l a t i o n s h i p s
i n p l a c e
t o p r o d u c e
t h e O U T P U T S
H e l p s u n d e r s t a n d p r o j e c t r a t i o n a l e
a n d w h a t n e e d s t o c h a n g e
5 . " F u t u r e "
n e t w o r k m a p
N e c e s s a r y
r e l a t i o n s h i p s
t o a c h i e v e
t h e V I S I O N
I d e n t i f y i n g a l i n e a r l o g i c l i n k i n g
p r o j e c t o u t p u t s t o p r o j e c t g o a l
I n t e g r a t i o n o f
b o t h v i e w s
6 . K e y
c h a n g e s
I d e n t i f y i n g t h e e v o l v i n g
n e t w o r k o f a c t o r s n e e d e d
t o a c h i e v e t h e v i s i o n
T h e o u t c o m e s t h e p r o j e c t w i l l h e l p a c h i e v e , h o w , a n d w i t h w h o m
8 . T i m e l i n e , t a r g e t s a n d m i l e s t o n e s
L i n k a c t i v i t i e s t o o u t c o m e s a n d s e t t a r g e t s a n d m i l e s t o n e s . T h e
b a s i s o f a n e v a l u a t i o n p l a n
Developing a cause-and-effect logic
The workshop begins with participants developing a problem tree that links the
problems the project is directly addressing with the social, environmental and/or
economic conditions it wishes to improve. The approach used for developing the
problem tree is based on work by Renger and Titcombe (2003). The branches of a
problem tree end when a problem that the project will directly address has been
identified. These ‘determinant’ problems help define the outputs (what the project
4
produces, used by others beyond project implementers) that the project needs to
develop. Sometimes, to avoid setting the linear logic in `negative´ terms (talking too
much about problems), it is necessary to further develop the problem tree into an
`outcomes´ tree. This is done by transforming the problems into their positive
counterpart, that is, the outcomes or impacts that the problem implies.
Figure 2: Presenting a problem tree in the Volta Basin Impact Pathways
Workshop
Developing a network perspective
Problem and objectives trees are seductively simple; they can lure people into thinking
that solving a limited set of discrete problems begins a domino-like cascade which
automatically achieves impact. Participants generally point this danger out themselves
on Day 1. Day 2, therefore, is about balancing cause-effect logic with a network
perspective, in which impact results from interactions between actors in an ‘innovation
system’. These interactions can be modelled by drawing network maps showing
important relationships between actors.
To connect the linear model with the network perspective, participants construct a vision
of success in which they imagine what the following classes of stakeholders will do
differently after the project:
1. The users of project outputs, or ‘next users’;
2. End user – the groups with whom the next users work;
3. Politically-important people and organizations who can help facilitate the project;
4. The project implementers themselves.
Next, participants draw a ‘now’ network map, showing current key relationships between
stakeholders, and a ‘future’ network map showing how stakeholders should link together
to achieve the vision. Participants then devise strategies to bring about the main
5
changes. The influence and attitude of actors is explicitly considered during these
exercises (see Figure 3 (ii)) based on work by Schiffer (2007)).
They then redraw the maps showing how the actors will need to be linked to achieve the
project’s vision. They record the most important changes in the networks and attitudes,
explaining why the changes are important and who needs to do what to make them
happen.
Figure 3: Drawing network maps in a PIPA workshop
(i) Drawing a network map (ii) Placement of influence towers and drawing of
‘smiley’ faces to indicate stakeholder attitude to
the project
Developing the outcomes logic model and an M&E plan
In the final part of the workshop, participants distil and integrate their cause-effect
descriptions from the problem tree with the network view of project impact pathways into
an outcomes logic model. This model describes in table format (see Error: Reference
source not found) how stakeholders (i.e. next users, end users, politically-important
actors and project implementers) should act differently if the project is to achieve its
vision. Each row describes changes in a particular actor’s knowledge, attitude, skills
(KAS) and practice, and proposes strategies to bring these changes about. The
strategies include developing project outputs with next users and end users who
subsequently employ them. The resulting changes are outcomes, hence the name of
the model, which borrows in part from Bennett’s hierarchy (Bennett and Rockwell, 2000;
Templeton, 2005).
6
Table 1: The outcomes logic model
Actor (or group of
actors who are
expected to change in
the same way)
Change in practice
required to achieve
the project’s vision
Change in KAS1
required to support
this change
Project strategies2 to
bring about these
changes in KAS and
practice?
1 Knowledge, Attitude and Skills
2 Project strategies include developing project outputs (knowledge, technology, etc.) with stakeholders,
capacity building, communication, political lobbying, etc.
The outcomes logic model is the foundation for monitoring and evaluation because it
provides the outcome hypotheses, in the form of predictions, which M&E sets out to
test. The predictions are that the envisaged project strategies will help bring about
desired changes in KAS and practice of respective actors.
M&E requires that the predictions made in the outcomes logic model be made SMART
(specific, measurable, attributable, realistic and time bound) so that project staff and
stakeholders can know whether or not predictions are being realized. The next step in
developing an M&E plan is to identify outcome targets, and milestones towards
achieving them. Outcome targets are ‘stretch’ targets in that they should be possible to
achieve but difficult, while the milestones are progress markers. Participants register
both in an Excel spreadsheet (Figure 4) that also shows the activities that make up the
strategies. We have found that without the link to activities, the strategies and the
changes envisioned can remain rather abstract and unrealistic. The Gantt chart itself is
a useful project management tool.
Figure 4: A screen-capture of the Excel spreadsheet used to link changes,
strategies, outcome targets and milestones
7
After the Workshop
(i) Monitoring and evaluation
After the workshop, participants complete their M&E plan and Gantt chart, ideally with
key staff and stakeholders who could not attend. If M&E is to contribute to project
learning, stakeholders should reflect on the validity of the impact hypotheses
periodically, not just at the end of the project. We suggest that projects hold a reflection
and adjustment workshop with their key stakeholders once a year with a smaller
meeting in between. In our experience these reflection sessions have worked better
when timed to coincide with another, technical or administrative meeting or training.
Ideally the first of these reflection sessions should have some facilitation, and help build
the capacity of the project team in charge of evaluation to carry out subsequent ones.
The first reflection session is also a good opportunity to provide training in facilitation of
meetings.
We use the graphic in Figure 5 to explain to participants how the reflection process
works. The numbers below relate to the graphic.
1. During the PIPA workshop, participants develop a shared view of where they want to
be in two years’ time, and describe impact pathways to achieve that vision. The
project then implements strategies, which lead to changes in KAS and practice of
the key stakeholders.
2. A workshop is held six months later to reflect on progress. The vision is changed to
some extent, based on what has been learnt, the outcome hypotheses are revised
when necessary and corresponding changes are made to project activities and
strategies. New milestones are set for the next workshop.
3. The process continues. The project never achieves its vision (visions are generally
used to motivate and stretch), but it does make real improvements.
8
Figure 5: Reflecting on progress along impact pathways (based on Flood, 1999)
1
F u t u r e w i t h o u t
i n t e r v e n t i o n
1 2
I m p r o v e m e n t
V i s i o n
2
1 2 3
I m p r o v e m e n t
0
R e f l e c t i o n
A d j u s t e d v i s i o n
3
1 2 3
I m p r o v e m e n t
0
P e r i o d i c R e f l e c t i o n s
A d j u s t e d v i s i o n s
A c t u a l I m p r o v e m e n t s
T i m e ( y e a r s ) T i m e ( y e a r s )
T i m e ( y e a r s )
I m p a c t P a t h w a y s
W o r k s h o p
0
I m p a c t p a t h w a y s
A d j u s t e d i m p a c t
p a t h w a y s
These reflection workshops can be seen as the culmination of one set of experiential
learning cycles and the beginning of others. If the reflections are well documented, they
can be analyzed at the end of the project to provide insights into how interventions do,
or do not, achieve developmental outcomes in different contexts. PIPA M&E thus
provides a framework for carrying out action research3. The quality of the research
depends on the facilitation of the reflections, the data used and the documentation of
the process. PIPA M&E is not prescriptive about the data used in the reflections, but
does encourage researchers to gather data using multiple methods. It also recommends
ways of introducing thematic and gender perspectives into the design of data-gathering
methods and reflection processes. One data-gathering method we have promoted in the
3 See Douthwaite et al (2007) for a published example of evaluation of a project’s progress along its
impact pathways
9
EULACIAS project is the ‘most significant change’ approach, in particular for picking up
unexpected consequences (see Davis and Dart, 2005).
(ii) Ex-ante and ex-post impact assessment
Ex-post impact assessment, which generally occurs several years after a project has
finished, seeks to 1) verify the direct benefits of the project and then 2) to trace how
further adoption and use of project outputs contributed to development impacts such as
poverty reduction, more sustainable livelihoods, etc. The changes listed in the
outcomes logic model are ones that are possible, at a stretch, to achieve within the
timeframe of the project. They generally describe the expected direct benefits of the
project and can be evaluated through the M&E described above. For CPWF projects
we have also constructed impact logic models that show, in flow-chart format, how
project activities lead to outputs that scale out and scale up to achieve eventual impact.
We constructed these models as a form of ex-ante impact assessment, but they also
provide the longer-term impact hypotheses required for ex-post impact assessment.
Where possible, the impact logic model is based on one or more published change
theories (see the LSC model below as the change model we normally use, and see
http://www.comminit.com/changetheories.html for others). We also help project staff
write an impact narrative4 because we have found that the discipline in writing an
explanation of causal mechanisms and influence strategies helps surface further
assumptions and improves clarity and understanding of what the project is trying to do.
An example of an impact logic model is shown in Figure 4, and the narrative describing
it can be found at http://boru.pbwiki.com/f/PN06%20Impact%20Narrative-4.DOC.
4 See Mayne (2004) for a description of performance stories upon which the idea of impact narratives
derives
10
Figure 4: Example of an impact logic model for the CPWF Strategic Innovations in Dryland Farming Project
11
Scaling-Up and Scaling-Out as the mechanisms that contribute to
impact
In the PIPA workshops we explain that the way a project will achieve both medium- and
longer-term impact is through two types of adoption of the knowledge and technologies
it produces – scaling-out and scaling-up. Scaling-out is the horizontal spread of project
outputs from farmer to farmer, community to community, within the same stakeholder
groups. Scaling-up is a vertical institutional expansion, based largely on a desire or
need to ‘change the rules of the game’. It can be driven by the influence of first-hand
experience, word-of-mouth and positive feed back from adopters and their grassroots
organizations on policy makers, donors, development institutions, and other
stakeholders who then have an interest in building a more enabling environment for the
scaling-out process.
We have developed the Learning Selection Change (LSC) model to explain how
scaling-out and scaling-up processes occur as a result of project activities in most of the
impact logic models we have constructed for CPWF projects. The model is based on
learning selection (Douthwaite, 2002), previous work on impact pathway evaluation
(Douthwaite et al. 2003), Bennett’s Hierarchy (used in Australia, see Bennett and
Rockwall, 2000; Templeton, 2005) and research that has found that information and
technology are more likely to be used when they are co-developed with the people who
will use them (Douthwaite, 2002; von Hippel, 1988). The LSC model works particularly
well for projects that co-develop technologies in pilot sites. We have found that it also
works for projects that produce models, toolkits and policy recommendations together
with the people who will use them.
A diagrammatic representation of the LSC model is shown in Figure 5 and can be
understood as follows (the numbers refer to the figure):
1. A project brings together people and resources to carry out activities in a particular
territory. A territory might be a pilot site, a rural community or an institutional
context.
2. As the result of project activities, people, represented in the model by participant i
and participant j, start going through experiential learning cycles. For example, they
decide to do something, like plant a new variety, then have an experience, make
sense of that experience, draw conclusions and then take further action, or not. In
this process, participant i interacts with participant j who is going through his own
experiential learning cycles. This interaction may change each others’ experience,
sense making, conclusions and subsequent action. This repeating process is called
‘learning selection’ (Douthwaite, 2002) because in it people are generating novelties,
making selection decisions and promulgating what works. In some ways this is
analogous to the algorithm ‘natural selection’ that drives evolution in the natural
world.
12
3. Project participants interact with other people as well, both inside and outside the
territory. The extent to which the good ideas and innovations they are generating
influence, and are influenced by, other actors depends upon how people are linked
to each other, the nature of those linkages, local norms and power relationships.
4. Through these interactions, changes begin to emerge. As a result of experiential
learning, participants undergo changes in their knowledge, attitudes and skills
(KAS). If participants see benefits in the novelties with which they are
experimenting, they will start adopting and adapting them and change their normal
way of doing things (their Practice). They will also start recommending the changes
to their peers (scaling out) and lobbying for a more supportive environment for the
changes (scaling up). Positive word of mouth builds a momentum that drives further
adoption that spreads beyond the territory. Further adoption leads to a series of
outcomes resulting from use that eventually contribute to broader level social,
economic and environmental impacts.
Figure 5: The Learning Selection Change (LSC) model that describes the causal
processes by which project interventions in a given territory bring about change
P a r t i c i p a n t ie n g a g e d i n
e x p e r i e n t i a l l e a r n i n g c y c l e s
A c t i o n
E x p e r i e n c e
M a k i n g s e n s e
D r a w i n g
c o n c l u s i o n s
A c t i o n
E x p e r i e n c e
M a k i n g s e n s e
D r a w i n g
c o n c l u s i o n s
P a r t i c i p a n t je n g a g e d i n
e x p e r i e n t i a l l e a r n i n g c y c l e s
A c t i o n
E x p e r i e n c e
M a k i n g s e n s e
D r a w i n g
c o n c l u s i o n s
A c t i o n
E x p e r i e n c e
M a k i n g s e n s e
D r a w i n g
c o n c l u s i o n s
O T H E R A C T O R S E m e r g e n t P r o p e r t i e s :
- C h a n g e s i n K A S
- C h a n g e s i n P r a c t i c e
- S c a l i n g o u t
- S c a l i n g u p
- C h a n g e s i n s o c i a l ,
e c o n o m i c a n d e n v i r o n m e n t a l
c o n d i t i o n s
1
2
4
3
2
B o u n d a r y o f t h e t e r r i t o r y
P r o j e c t A c t i v i t i e s c a r r i e d o u t i n
a g i v e n t e r r i t o r y w i t h r e s o u r c e s
t h a t o f t e n c o m e f o r m o u t s i d e
Conclusions
Participatory Impact Pathways Analysis (PIPA) is a relatively young and experimental
approach that draws from program theory evaluation, social network analysis and
research to understand and foster innovation. It has been developed to meet some of
13
the multiple evaluation and management needs of complex research-for-development
projects and programs. These requirements include:
-Carrying out an evaluation of likely project impacts and how they will occur (ex-ante
impact assessment);
-Helping projects better understand what each other are doing, identify common
interests and foster programmatic integration;
-Providing a framework and design for both compliance- and learning-based
monitoring and evaluation;
-Providing the impact hypotheses required for impact assessment after the project
has finished.
PIPA begins with a workshop which culminates in a project or program outcomes logic
model and the identification of outcome targets and milestones framework that is the
basis for monitoring and evaluation. If the project wishes to carry out ex-ante impact
assessment, or set the foundations for ex-post impact assessment, then the workshop
facilitators produce an impact logic model that shows how the outcomes described in
the outcomes logic model will likely play out to broader, higher order impacts. Both
logic models place greater emphasis on the actors involved in making change happen,
and how these actors themselves are expected to change, than traditional logic models.
Testing of impact hypotheses contained within the outcomes logic model through
regular reflection activities, as described in this paper, constitutes action research on
how to foster developmental outcomes based on the use of research outputs.
References
Argyris, C. and Schön, D. 1974. Theory in practice: Increasing professional
effectiveness, Jossey-Bass, San Francisco, USA.
Bennett, C.F. and Rockwell C. 2000. Targeting Outcomes of Program (TOP),
http://deal.unl,edu/TOP/synopsis.html.
Chen, H. T. 2005. Practical Program Evaluation: Assessing and Improving Planning,
Implementation, and Effectiveness. Sage Publications, California, USA.
Davies, R. and Dart, J. 2005. The Most Significant Change technique: a guide to its use.
Available from: http://www.mande.co.uk/docs/MSCGuide.htm
Douthwaite, B., Alvarez, B.S., Cook, S., Davies, R., George, P., Howell, J., Mackay, R.
and Rubiano, J. 2007. The Impact Pathways Approach: A Practical Application of
Program Theory in Research-for-Development. Submitted to the Canadian Journal of
Program Evaluation, Fall, 22 (2)
Douthwaite, B. and Ashby, J. 2005. Innovation Histories: A Method for Learning from
Experience. ILAC Brief 5. IPGRI, Rome. www.cgiar-
ilac.org/downloads/Briefs/Brief5Proof2.pdf
14
Douthwaite, B., Schulz, S., Olanrewaju, A., Ellis-Jones, J. 2007. Impact pathway
evaluation of an integrated Striga hermonthica control project in Northern Nigeria.
Agricultural Systems. 92 pp 201-222
Douthwaite, B., T. Kuby, E. van de Fliert and S. Schulz. 2003. Impact Pathway
Evaluation: An approach for achieving and attributing impact in complex systems.
Agricultural Systems 78 pp243-265
Douthwaite, B. 2002. Enabling Innovation: a practical approach to understanding and
fostering technological change. Zed Books, London.
Earl, S., Carden, F. and Smutylo, T., 2001. Outcome Mapping: Building Learning and
Reflection into Development Programs. Ottawa, Canada: International Development
Research Centre.
Flood, R. L. 1999. Rethinking the Fifth Discipline, Routledge, London and New York
Mayne, J. 2004. Reporting on outcomes: setting performance expectations and telling
performance stories. The Canadian Journal of Program Evaluation Vol. 19 (1) pp. 31-60
Renger, R. and Titcomb, A. 2002. A Three-Step Approach to Teaching Logic Models
American Journal of Evaluation. 23: 493-503
Schiffer, E. 2007. ‘The power mapping tool: a method for the empirical research of
power relations’, IFPRI Discussion Paper 00703
Templeton, D. 2005. Outcomes: Evaluating agricultural research projects to achieve
and to measure impact. In: Metcalfe, I, Holloway, B., McWilliam, J., and Inall, N. (eds)
Research Management in Agriculture: A Manual for the Twenty First Century. University
of New England.
von Hippel, Eric, 1988. The sources of innovation. Oxford University Press, New York
and Oxford.
15
