Battlefields of method: Evaluating Norwegian peace efforts in Sri Lanka

Full text

1
Goodhand, J, B. Klem, G. Sørbø (2014) ‘Battlefields of method: Evaluating Norwegian
peace efforts in Sri Lanka,’ in B. Bull, M. Kennedy-Chouane and O. Winckler (eds)
Evaluation Methodologies for Aid in Conflict, pp. 61-84. London: Routledge.
Chapter 4
Battlefields of method: Evaluating Norwegian peace
efforts in Sri Lanka:
Jonathan Goodhand, SOAS
Bart Klem, Zürich University
Gunnar M. Sørbø, Chr. Michelsen Institute, Bergen
1. Introduction
From 1999 to 2009, Norway was involved as ‘sole facilitator’ of the Sri Lankan peace process.
Negotiations ultimately broke down, leading to a return to full-scale war which ended with the
military victory by the Sri Lankan army over the Liberation Tigers of Tamil Tigers (LTTE) in
May 2009. An evaluation was commissioned by the Evaluation Department of the Norwegian
Agency for Development Co-operation (Norad) in 2010 to tell the story of Norway’s
engagement, assess the effects and identify broader implications and lessons. This was the first
Norwegian evaluation to be commissioned in the area of peace diplomacy which has become
one of the most distinctive aspects of Norwegian foreign policy.
Somewhat provocatively, our evaluation report, published in November 2011, was entitled
‘Pawns of Peace’ (Goodhand et al. 2011a). We borrowed and adapted the title of Harriet
Martin’s book on peacemaking, ‘Kings of Peace, Pawns of War’ (2006) in order to make the
point that peace mediators are often minor players in a much larger game.
Analysts who evaluate peace efforts, it might be added, are also part of ‘the game’; just as
mediators enter into intensely politicized and contested environments, so do evaluators.
Evaluating attempts to resolve conflict involves grappling with competing and partial
interpretations of complex processes. As with all forms of historical assessments that call for
explicit evaluation of events and policies, there is clearly a need for a systematic and robust

2
methodology, but peace evaluation can never be reduced to a technical exercise. Evaluators
inevitably get sucked into the politics surrounding conflict and intervention, and run the risk
of becoming ‘pawns’, just like peace mediators. In this chapter we aim to explore both the
methodological and political challenges associated with the evaluation of peace efforts in Sri
Lanka, and we will argue that the two sets of challenges are inseparable.
Unlike most other chapters in this book, the primary focus does not lie with development co-
operation in support of peace building. Rather, our evaluation encompassed a wider range of
international interventions, including diplomacy, ceasefire monitoring as well as aid. Although
aid provision – which included high profile donor pledging conferences and efforts to bring
about collaboration through joint aid mechanisms – was an important part of the international
strategy to support the peace process, development co-operation was not at the centre of our
evaluation. The peace process and the Norwegian role within it, was largely determined by
politics and not by the provision or withholding of aid. This explains the analytical focus of
the evaluation and also of this chapter.1
The chapter starts with a discussion of the research literature on peace processes and its
relevance to the design and practice of evaluating such efforts. We then provide a brief
overview of Norway’s peace efforts in Sri Lanka, followed by an exploration of the
methodological challenges and how we attempted to deal with them. We then tell the story of
how our evaluation evolved in practice, dividing its implementation into three chronological
phases. The concluding section draws on our experience in Sri Lanka to suggest some broader
implications.
2. The challenge of evaluating peace efforts
2.1 Social science challenges: structure, agency and uncertainty
The quantitative literature on mediation provides a rather mixed picture on the efficacy of
external mediation (Sarkees 2000; Dixon 2009; Licklider, 2009). Historically, most wars ended
through military victory rather than through a mediated settlement, and one body of research
has found that conflicts terminated through a negotiated settlement had a higher level of
recidivism (cf: Stedman 1997; Licklider 2009). Another, growing body of quantitative research
finds that peace settlements are becoming more ‘sticky’, identifying a positive correlation
between external intervention and sustainable peace (Doyle and Sambanis 2000; Human
Security Centre 2005; Fortna 2008; Human Security Centre 2010). This literature does not
1 International aid to Sri Lanka, and the politics around it, is a relatively well-studied area (Goodhand and Klem
2005; Bastian 2007; Goodhand and Walton 2009; Orjuela 2011; Rainford and Satkunanathan 2011).

3
provide a clear explanatory account about what works or does not work in relation to peace
processes, but it does have value in pointing to some of the key structural variables that shape
civil war termination, whilst also giving the evaluator (and peace mediator) a useful reminder
that peace processes frequently fail.
One has to turn to the more qualitative literature, including on case studies, in order to gain
insights about peace mediation in practice and capture the specificity and contingency of
individual contexts and interventions. This literature explores the ‘how to’ of mediation. Much
of this work is concerned with extracting lessons from analyses of particular peace processes.
The analytical focus tends to be on the design of peace negotiations, the question of timing
and ‘ripe’ moments, the necessary inclusion and sequencing of key conflict issues, dealing with
information asymmetries and security dilemmas, the role of incentives and disincentives, and
spoiler management strategies (Stedman 1997; Regan 2002; Zartman 2000, 2009). Much of
this literature is informed by rational choice models which assume that individual agents make
rational calculations and choices and that the decision-making of elite actors is a key
determinant of ‘success’ or ‘failure’. Mediation, in this sense, is about negotiating a complex
‘prisoners’ dilemma’. Protagonists can ‘choose’ peace over war, if mediators deploy the right
combination of (dis)incentives. Therefore a significant role is ascribed to individual agents,
including peace mediators.
One of the implications of this perspective is that the failure of peace negotiations can be
traced back to flawed decision-making processes, the incorrect deployment of carrots and
sticks, the failure to grasp opportunities and so forth. Also this approach is based upon a
teleological understanding of war to peace transitions – events, processes and actors are
judged in relation to the desired outcome (a negotiated settlement), leading to assessments
about whether they are for or against peace and more broadly whether the peace process is
moving ‘forwards’ or ‘backwards’ (Klem 2012). Whilst like the quantitative research, this
literature yields some important insights, it has some major weaknesses. First the rational-actor
model is largely blind to non-material, symbolic and collective factors that influence and shape
individual decision-making in peace processes. Second, by ascribing such an important role to
agency, the significance of wider structural factors is downplayed. This leads to unrealistic
expectations about what mediators can be expected to achieve and held accountable for. Just
because a peace process fails, for instance, does not mean that mediators necessarily made the
‘wrong’ decisions or failed to deploy their resources optimally.
The rational-actor perspective is particularly questionable in the case of non-coercive
mediators like Norway, with only limited means to persuade protagonists to change their
course of action. The stakes and interests vested in protracted warfare often outweigh the
instruments of international actors, who of course have their own sets of interests. Therefore,
though one should not deny the role of agency, particularly during peace processes when

4
institutions may be in a state of flux, the wider structures place considerable constraints on
both mediators and protagonists. Many after-the-event accounts of peace processes tend to
‘write agency back in’. Yet unless done in a balanced and critical way, this may obscure the
complex, messy reality of peace processes in which actors are always partially sighted; they are
responding to events rather than shaping them, in other words making it up as they go along;
and they are often severely constrained by wider power structures.
Therefore, whilst the research literature on peace processes and peace settlements provides
useful insights to the evaluator, there are also limitations that need to be born in mind. There
is no definitive theory in this field to guide action. Existing theory at best provides general
guidance, rather than clear prescriptions, for both practitioners and evaluators (Stern and
Druckman 2000:62). The most obvious yet important point, therefore, is to take contextual
diversity seriously. Peace processes take place in particular contexts at particular moments in
history and universal prescriptions and approaches have limited value. For the evaluator, this
means that the foundation for any credible assessment must start with a robust political
economy analysis of the context.
2.2 Political challenges: contested narratives, expressions of power
Like any other policy-oriented research, research on peace efforts is not detached from the
field it claims to study. Whilst it may be conventional wisdom that sound policies must be
‘evidence based’, it is also true that the relationship between research and policy may run in
the opposite direction, in the sense that policy priorities can shape the kinds of evidence and
findings generated by research. Evidence may be drawn upon selectively or manipulated to
support a particular policy narrative, a phenomenon that Cramer and Goodhand (2011) call
‘policy based evidence’. Policy priorities shape the kind of research that is done and determine
what questions are considered relevant. Policy makers may also be under pressure to make
selective use of scholarly literature to legitimise or de-politicise certain policies. This
phenomenon is reinforced when research findings are heterogeneous or contradictory, as is
often the case.
This is particularly true for evaluations, whether of peace efforts or other activities, which are
commissioned by intervening institutions. Of course, the formal rationale for evaluations is to
provide a critical, independent view. Evaluators are expected to serve a watchdog role,
assessing policy outcomes, and questioning policy dogmas. However, evaluations are shaped
by the mandate set by the commissioning agency. In order to be relevant, evaluators need to
adapt themselves to the policy logics and institutional practices they are studying. Evaluations
are to some extent political instruments that serve to legitimize policies and their impacts by
packaging and presenting them in particular ways. They capture messy realities in a
‘scientifically’ endorsed narrative. They make sense of institutions and interventions, even if

5
they criticise them. Evaluations are thus not simply instruments of empirical verification, they
‘do political work’, and as such are expressions of power.
This point is elaborated and developed in a much wider (broadly Foucauldian) literature on
development, state rule and neo-liberal assemblages (cf: Duffield 2007). For example, James
Ferguson’s seminal work on the ‘anti-politics machine’ (Ferguson 1990) argues that
development interventions tend to obfuscate the deeply political interests that they promote,
by breaking things down into bureaucratic units, projects, and categories. They render
technical what is in fact very political (Li 2007). Evaluations – be it on peace or development
interventions – may be seen in a similar light. They provide coherent narratives of policy and
practice, thus reducing messy realities to manageable proportions, and lend themselves to
political usage; to legitimise or de-legitimise particular courses of action. This does not mean
that evaluations necessarily become ‘scientific’ smokescreens for political actions – after all,
technical narratives often invoke counter-narratives – but it recognizes that evaluations
operate in political force fields that affect their methodological approaches and outcomes.
This applies to all varieties of policy-research2, yet peace evaluations are likely to be
particularly vulnerable to such pressures because it may be more difficult (though not
impossible) to ‘render peace technical’, given the fact that it so obviously touches upon critical
political issues related to security and sovereignty.
Evaluators of peace interventions thus unavoidably have to navigate a political arena
throughout their assignment: from the discussion of terms of reference to the tug of war over
contentious findings and minute rephrasing of the executive summary. Arguments about
methodology and evidence base are usually at the heart of these struggles, but the underlying
reasons these issues are contentious often have more to do with vested political interests,
public perceptions, and bureaucratic fears of possible political fall-out. When the stakes are
high, pressures are likely to increase. In this context, evaluators use similar arguments to
protect their product (and the voices and convictions manifest in it) as well as their reputation,
while also preserving their relations with possible future clients. Evaluation politics permeate
the entire evaluation process.
Interventions in war-torn areas generate their own information economies. Information is a
valuable resource which is hoarded, manipulated and disseminated, and can literally make the
difference between life and death. And the brokers operating in these information economies
have to navigate numerous issue to do with the nexus of power and knowledge: Who gets to
say what? Which voices are heard or suppressed? What data count as valid? What findings are
2 The controversies around the fourth assessment of the Inter-governmental Panel on Climate Change in
February 2010 reminds us that the supposedly less politicised natural sciences may fall prey to such dynamics as
well.

6
considered policy-relevant? Whose truth prevails in the echelons of power? If we are to study
peacebuilding evaluations as an empirical phenomenon, an acknowledgement of these forces
and interests is warranted.
3. Our object of study
According to the Terms of Reference, the main purpose of our evaluation was to “learn from
the unique Norwegian experience as a facilitator in the peace process in Sri Lanka”. More
specifically, evaluation questions required us to: (a) map the Norwegian peace engagement in
Sri Lanka from 1997 until 2009; (b) assess the role as facilitator between the parties on the one
hand, and the relationship to the international community on the other; (c) assess the
Norwegian facilitator role and the relationship to local parties and stakeholders; (d) assess the
ceasefire agreement and how the parties observed it; (e) assess Norway’s efforts in the last
phase of the war (January-May 2009); (f) assess results achieved through the Norwegian
facilitation of the peace process; and (g) draw lessons from the Norwegian engagement in the
Sri Lankan peace process.
These questions did not explicitly engage with the history, roots and dynamics of armed
conflict in Sri Lanka. However, given the importance of context, noted above, and to avoid an
overly teleological approach that places interventions at the heart of the analysis, we decided
the evaluation should not treat the rise and fall of Sri Lanka’s peace process as simply a
derivative of Norwegian intervention. Rather, the ambition was to review the turbulent fifteen
years of Sri Lankan history, and then place Norway’s efforts in that historical process, to
identify results and draw lessons.
Conceptually and methodologically, this is difficult. Incorporating a much wider set of issues
and factors into the makes it very difficult to demarcate the scope of the study. If an
understanding of Norwegian peace efforts in Sri Lanka requires a fine-grained analysis of the
evolution of the island’s ‘conflict system’ over fifteen years, including the involvement of the
Norwegian mediators and other international actors, there is clearly a risk that it all becomes
too much to handle, particularly within the time constraints of an evaluation. In our case this
was indeed a major challenge, even if the team could draw upon prior knowledge and
research.
For readers who are unfamiliar with Sri Lanka, we provide below a nutshell summary of the
conflict’s historical background and the last fifteen years of the war in particular.
Sri Lanka’s minority question has long historical roots, but became particularly pronounced in
the post-colonial era (from 1948 onwards). In our understanding, the clash between Sinhala

7
nationalism and Tamil separatism is rooted in the nature of the Sri Lankan state and the way
its democratic politics played out in a multi-ethnic environment. The contention between the
Sinhala (73%)3 majority and the Tamil minority (18%, of which at least one fourth are so-
called “Indian Tamils”) has always been most salient, but other groups – such as the Muslims
(9%) – have not escaped these dynamics (Moore 1985; Coomaraswamy 2003; De Silva 2005;
Spencer 2008; Uyangoda 2011). After several agreements on minority rights and regional
autonomy were not implemented, anti-state armed violence began in the early 1980s. The
LTTE subsequently emerged as the dominant Tamil faction fighting for an independent Tamil
homeland in the northeast of the island. India’s diplomatic and military intervention in the late
1980s resulted in renewed warfare and a further entrenchment of positions (Bullion 1995;
Loganathan 1996).
It was in the wake of Indian withdrawal, in 1991, that the possibilities of a Norwegian role
were first explored. But Norway was only formally invited as a “facilitator” in 1999, from
among several other candidates who had offered to play this role. Initial backchannel
negotiations proved unsuccessful and it was only with the election of a new Sri Lankan
government in late 2001 that a first major break-through was reached: the February 2002
ceasefire agreement between the government and the LTTE. The agreement was subsequently
monitored by an unarmed Nordic mission. Direct talks, backed by Sri Lanka’s aid donors
(most prominently Japan, the European Union and the United States), soon entered an
impasse. The subsequent “no-war-no-peace” phase was characterized by a fragmented
political landscape, continued violence and eroding popular support for the peace process,
despite an apparent window of opportunity after the December 2004 tsunami. Finally, full-
scale military offensives resumed in 2006. Sri Lanka’s replenished armed forces gradually
pushed back the LTTE and defeated the insurgents in May 2009, killing its entire leadership.
This military victory came at the cost of an estimated 30 to 40,000 civilian deaths and
bolstered a reform resistant government with little inclination to address the underlying ethno-
political issues of the war.
A basic starting point for our analysis was to define the war in Sri Lanka as a manifestation of
a deeper state crisis. This lead us to analyse the changing nature of the Sri Lankan state within
its global, regional and domestic settings, and to examine shifts in state-society relations over
time and the role of inter- and intra-elite competition in shaping political bargains, coalitions
and settlements. The rivalry between the United National Party (UNP) and the Sri Lankan
Freedom Party (SLFP) – both of which are predominantly Sinhala – concerns issues and
interests that are different from the country’s ethno-political crisis, but have often complicated
3 Ethnic statistics are clearly sensitive in Sri Lanka and there has not been a country-wide census since 1981
because of the war. The figures mentioned are a composite of the national census (2001), which did not cover
the war-torn north-east, and district-level data (2007) and thus provide a general indication only.

8
the conflict and impeded the search for its peaceful resolution (Venugopal 2009; Uyangoda
2011). Such dynamics help explain both the continuities and shifts in Sri Lanka’s political
economy during the course of peace negotiations. And they help place international
intervention, and specifically the role of Norway, in perspective. A political economy analysis
of the Sri Lankan peace process shows the relative autonomy of domestic political elite
decision-making, which limited the options for international actors. This was particularly the
case where international backers are divided and when the legitimacy of, and domestic
constituency for, such changes are limited (Goodhand et al., 2011b).
It is important to note that a conflict analysis also involves looking at the changing external
environment before and during the course of Norway’s engagement. Sri Lanka’s political elite
have had to balance and mediate different, sometimes competing external pressures. Other
domestic actors, including the LTTE, have likewise adapted to changes in the international
arena to generate political capital, gain access to resources and to build their legitimacy. It is
important to avoid a narrowly internal understanding of the civil war and to recognize the
inter-connectedness of domestic and external dimensions.
This was the analytical starting point for the evaluation. Our analysis placed Norway’s peace
efforts in Sri Lanka in a broad historical perspective and showed the magnitude of the
challenges the mediators faced. Drawing on this conflict analysis, we posited that a durable
peace settlement would require the following elements: 1) constitutional reform, 2) some level
of support for these reforms from the three largest ethnic communities (Sinhalese, Tamils,
Muslims), 3) a fundamental transformation of the LTTE into an organisation able to operate
in a context of democratic power-sharing, 4) addressing the rivalry between the two largest
Sinhala-dominated political parties (UNP and SLFP), and 5) ensuring that the settlement
would be acceptable to India. In parallel to these core political issues, there was an imperative
to address immediate humanitarian issues and preserve a level of respect for human rights,
both for moral reasons and to ensure the legitimacy of the settlement.
4. Methodological challenges and approach
Having provided an overview of our evaluative approach, we now turn to the core topic of
this book: the methodological challenges associated with evaluating peace building efforts in
conflict settings. As discussed in this volume, and elsewhere, these challenges are numerous
(cf: Stern and Druckman 2000; OECD/DAC 2012).

9
Firstly, there is the very basic question of what is the measure of ‘success’ or ‘failure’? Because
the Sri Lankan peace process broke down, does this mean that Norwegian efforts were a
failure, or should other accomplishments still be valued?
Secondly, there is the challenge of dealing with uncertainty. Uncertainty plays a bigger role in
questions of armed conflict (and peace) than is typically acknowledged by policy makers and
researchers. Particularly attempts at constructing measurable, predictive models tend to
inadequately approach conflict and peace processes as ‘closed systems’. Conversely if one
understands peace processes, like other domains of social relations, as an ‘open system’ this
means that the same causes can have different outcomes and similar outcomes can have
different causes. Furthermore, mediators are at a disadvantage compared to evaluators in the
sense that they cannot ‘postdict’ (Stern and Druckman, 2000:53) i.e. evaluators are able to
make ‘after the fact’ assessments based upon a knowledge of outcomes. Mediators are
operating with incomplete knowledge – and often information is deliberately withheld or
massaged by the conflicting parties – which makes planning and prediction extremely difficult.
In such a situation of uncertainty, a strategy that seemed logical at the start of a process may
prove to be erroneous or lead to fatal mistakes further down the line. And yet, the fact that a
particular decision had undesirable effects, does not mean that this was inevitable. While
evaluations usually have the benefit of hindsight4, they still have to grapple with the challenge
of open systems and uncertainty – for example they often take place too early to judge the
long-term effects of the intervention studied. With regard to Sri Lanka, the historical
significance of the collapsed peace process and the subsequent military victory depends to a
large extent on political developments in the years to come.
Thirdly, and related to the problem of uncertainty, is the problem of attribution: the challenge
of trying to separate out the effects of Norwegian interventions from the impacts of wider
international and domestic interventions. Impact on larger scale processes can normally just be
inferred rather than confirmed. Related to the attribution problem is the conceptual problem
of counterfactual history. Whether a different Norwegian strategy would have resulted in
different outcomes is impossible to prove. Whether the final war and military defeat of the
LTTE would have occurred without the peace process is similarly difficult to judge.
Fourthly, there are questions of access and secrecy. Peace efforts often require confidentiality
and actors may have information which they cannot or will not reveal. Key informants may be
dead, imprisoned or otherwise hard to reach.
4 One exception to this is ‘real time’ evaluation, which is carried out while an intervention is ongoing. Likewise
ex-ante or mid-term evaluations may not have the same benefits of hindsight.

10
Finally, as in all reconstruction of historical events, there is the challenge of dealing with
conflicting or unreliable accounts and discourses. The narratives of key protagonists are in a
sense “scripted” and aimed at particular audiences. Actors may inflate their own roles, present
a greater level of coherence and logic to decision making than was actually the case, and
smooth over the frequent gaps between declared intentions and actual behaviour.
Furthermore these narratives often clash with one another, reflecting the highly contested
nature of war to peace transitions.
We identified these difficulties at the very start of the evaluation and tried to develop strategies
for addressing them, while also being conscious of the fact that some of them are intractable.
Perhaps the most important strategy for dealing with several of these challenges is that of
triangulation – drawing upon different qualitative data sources (archival study, historical
analysis, individual interviews, and focus group discussions), and different levels of analysis
(macro, meso and micro) – to reduce possible biases and provide as a firm an empirical
foundation as possible.
In order to place Norwegian intervention in perspective, we combined an ‘inside-out’
approach (a detailed account of Norway’s involvement in the peace process) and an ‘outside-
in’ approach (analysis of the broader structural context, conflict and peace making dynamics)
and then sought to link the two. A careful ‘inside-out’ analysis of Norwegian involvement
provides an actor-oriented perspective that recognizes the spaces and opportunities for
individual agents to influence conflict dynamics and outcomes. This is not restricted to
Norwegian actors, but also explicitly recognizes the political agency of domestic elites. The
background to the Sri Lankan conflict provides an ‘outside-in’ perspective, giving an analytical
baseline of key trends and conflict dynamics, placing Norwegian interventions within a wider
structural context. This combination of approaches aims to capture the complex dynamics and
chains of causality that link individuals, institutions and structures in peace processes.
In order to connect the outside-in and inside-out perspective in a larger historical plot, we
identified key turning points (‘critical junctures’) in the period studied. It is at these moments
of change, we posited, that the outlook or composition of the LTTE and the government
changes or shifts take place in the political space and military options available to them.
Therefore, these events also had a profound – restraining or enabling – impact on Norway’s
room for manoeuvre as peace facilitator. Some of these turning points resulted from political-
military calculation – for example the LTTE withdrawing from the peace talks in 2003. Others
may be seen as manifestations of structural political trends (the election of Rajapaksa as
president in 2005) and others yet were entirely contingent (the tsunami in 2004). We used
these turning points to scrutinize Norwegian responses in relation to the knowledge and
opportunities that were available at the time. This close analysis of turning points seemed to
be the best way of dealing with problems of causation and attribution, structure and agency. It

11
enabled us to reconstruct critical events and engage in careful ‘process tracing’ to explore
causal mechanisms and the linkages between individual decision making and wider institutions
and structures.
In sum, we adopted a historical, political economy approach, where inside-out and outside-in
perspectives were linked through the use of key turning points. This approach aspires to
balance and connect structure and agency. It recognises long histories, the centrality of the
state and the structural constraints in which key actors operate, thus acknowledging a level of
path dependency. Mediators like all other actors in a conflict never have complete context
freedom – they inherit a particular situation, which places constraints on their choices and
actions. But on the other hand, our approach acknowledges that key actors have some room
for manoeuvre, and it does not dispense with contingency and uncertainty. We thus explicitly
attempted to ‘write key individuals back into the story’ without adopting a radically contingent,
actor-centred perspective. After all, the world does not follow Newtonian rules and principles
– it is more chaotic, random and uncertain, and individuals make a difference, even though
they do not act in contexts of their own choosing.5
5. Evaluation methodology and politics in practice
The sections below review the evaluation process in chronological order, including the
methodological challenges and strategies that came up at different points in time.
5.1 Overtures: contracting and designing the evaluation
The word overture has a double meaning: on the one hand it can mean a (musical)
introduction; on the other it can mean a performance of subtle self-promotion in order to
establish rapport (making overtures). The second meaning of this term resonates with the
process through which evaluators secure a contract. Prospective evaluators need to submit a
proposal that fulfills two basic requirements: it needs to be good (in order for the assignment
to be carried out successfully) and it needs to look good to the commissioning agent (in order
to get the contract). Often, of course, the two align, but they are not the same thing. For
example, reputed senior researchers may add gravity to a bid, even when their actual
involvement in the research may be minimal; tight time frames may look efficient, but may be
hard to enforce once a bidder gets into the assignment; limiting staff time may help make the
budget competitive, but will create the risk of massive unpaid overtime.
5 This approach bears some semblance to what Paul Richards has called causal process tracing in a stimulating
recent article (2011). For earlier discussion on process tracing, see for example George and McKeown (1985).

12
Team selection is one of the most crucial parts of the evaluation exercise. This particularly
applies to highly sensitive evaluations in the field of peace and conflict. To what extent are the
evaluators known and trusted in terms of the quality of their analysis? Do they have a track
record in this field? Is there a good combination of established and emerging talent in the
team? Do they have the necessary thematic, methodological and area-based expertise? Is there
any conflict of interest? To what extent is there a balance between ‘international’ and ‘local’
researchers? Is there an adequate gender balance? These were all important considerations for
us when putting together our evaluation team, which was composed of a core team of
international and Sri Lankan researchers, in addition to an advisory group of senior academics
and analysts.
Even after having won the Norad Evaluation Department bid (22 July 2010), the question of
team composition remained a contentious issue and raised a number of ethnical challenges.
The timing of the evaluation coincided with a number of developments in relation to Sri
Lanka, which heightened its sensitivity and politicization. After a brutal military victory in the
north-east in 2009, the Sri Lankan government had come under growing international pressure
to account for alleged war crimes. Therefore the regime, already suspicious of anything that
had to do with Norway and what it regarded as a flawed peace process, was especially sensitive
to any initiatives that attempted to look at the past. In parts of the Sri Lankan media, the
evaluation was lumped together with wider international efforts pushing for accountability in
relation to war crimes. As a result of tense political climate and the increasing visibility of the
evaluation, some of the Sri Lankan members of the team felt that they had to withdraw the
endeavour. Because selective withdrawal would have upset the balanced ethnic background of
the team, it was eventually decided that none of the Sri Lankan researchers would be included
on the team. This suggests that the principles of independence and transparency, considered
to be essential pillars of ‘good practice’ in evaluation, may be very difficult to abide by in
highly politicized and conflictual environments.
The initial phase of the research focused on interviews with key Norwegian actors and the
perusal of the archives of the Norwegian Ministry of Foreign Affairs (MFA) in Oslo. Archival
research led to the production of several lengthy annotated summaries, detailing Norwegian
efforts, perceptions and internal discussions throughout the study period. However, we were
aware that the archives did not provide us with a comprehensive database of historical facts.
One of the key Norwegian actors explained to us that the whole process had been relatively
informal. While the key decisions and statements were well-documented, many of the
preceding discussions, disagreements and considerations had only been discussed through
SMS communication or in private conversations. For example, it remained unclear how
exactly the 2002 ceasefire agreement had been negotiated. This made it very difficult to judge
Norway’s mediator role, especially whether or not some of the alleged flaws in the truce

13
(exclusion of the president, weak monitoring mission, unclear arrangements at sea) could have
been avoided.
Archival research yielded further deliberation about the identification and significance of
various turning points. We tried to deal with these key moments in their historical context. In
other words, rather than assessing decisions and actions with the benefit of hindsight, we tried
to construct a historical plot that faithfully captured the uncertainty that characterised
Norwegian involvement in Sri Lanka. It was also clear, however, that our selection of turning
points was nonetheless informed by knowledge of outcomes, something that the Norwegian
mediators did not have at the time. To evaluate the effectiveness of the Norwegian efforts, the
team needed to distinguish between processes and consequences that were clear at the time,
and might have been anticipated (and for which people could thus be held accountable), and
dynamics that could not have been anticipated (for example, the tsunami), and only became
clear afterwards.
Alongside the archival research and the initiation of our interviews with key actors, we
commissioned a number of sub-studies and data-gathering efforts to inform our analysis. This
included: 1) a review of the literature on mediation and peace building; 2) a study of prevalent
perspectives among Sri Lankan Tamil diaspora on the peace process; and 3) a (tri-lingual)
review of Sri Lankan media coverage of the peace process, and of our turning points in
particular.
5.2 “Fieldwork”: navigating an arena of conflicting and missing narratives
Partly overlapping with the previous phase, the main part of the evaluation comprised
‘fieldwork’, that is interviews with key informants, along with the review of written sources.
The crucial part of ‘the field’ however proved off-limits to us when Sri Lankan government
officials made it known to the Norwegian embassy in Colombo that it would be better not to
pursue our visa applications. Losing access to the country was a major setback. However, this
problem was partly mitigated by interviewing key people on Skype or telephone. The
impossibility of visiting Sri Lanka forced the team to focus our resources on accessing key
informants internationally: interviews in Norway (and Sweden), other European countries
(including Brussels, Geneva), the US (Washington DC, New York), India (Delhi, Chennai)
and – somewhat coincidentally – Singapore (where a significant Sri Lankan government
official happened to reside). Arguably this enhanced the complementary value of the
evaluation in relation to the existing scholarship on the Sri Lankan peace process, which is
often not as thorough when it comes to empirical research on Norway and other international
actors. Nevertheless, the team was concerned that its inability to visit Sri Lanka would
undercut the legitimacy of the exercise, particularly because our visa application had not

14
officially been rejected. Fortunately, this did not prove to be a problem when the evaluation as
published.
Among the key conflict protagonists, the main group missing from our list of interviewees was
the LTTE. The insurgents were clearly a vital player through the rise and fall of the peace
process and the dramatic end of the war, but the movement is much less well understood than
the political elite in the capital Colombo. Rather opaque and speculative interpretations about
the leadership and factions within the movement in the press or elsewhere tend obfuscate as
much as they elucidate. Both written and verbal accounts of the LTTE tend to reinforce rather
than debunk the movement’s somewhat mythical self-image.6 We had met with some of the
key LTTE figures in the past, including S.P. Tamilselvan (former leader of the political wing)
and Pulidevan (former head of the LTTE peace secretariat). We had access to people who
could provide credible accounts for other key cadres like Karuna (the break-away eastern
commander) and Anton Balasingham (the former LTTE negotiator). The latter moreover
wrote rather extensive memoirs (Balasingham 2004). But we never had access to the key
military figures (i.e. Pottu Amman, Soosai and the LTTE leader Prabhakaran), who were dead
when our evaluation started, and there was little reliable information that gave an insight into
their thinking and decision-making.
Crucial aspects of the evaluation were thus difficult to substantiate: What were the objectives
of the LTTE when they signed a ceasefire and started talks in 2002? Were the stated reasons
the real grounds for walking away in 2003 and what was their longer-term plan B? What was
the cause of the Karuna split? Why did they provoke the government into resuming hostilities
in late 2005 and mid-2006? Why did they not resort to guerrilla style warfare when they could
not sustain a state-like posture in late 2008 and 2009? We have tried to address all these
questions by working our way through the various theories and speculations by perusing the
LTTE’s own statements, accounts in public media and interviews with people who are well-
positioned to reflect on the LTTE. We discarded accounts that were clearly conspiratorial or
lacked any evidence. Beyond that, the evaluation report could only spell out plausible
interpretations but had no means to verify these conclusions.
Among the Colombo elite, the challenge was different. Most of the key people were still
accessible (either abroad or on the phone), most of them were quite willing to share their
perspectives and many of their actions and statements are well documented. However, many
6 There are a number of publications that claim authority in this regard, including journalistic accounts such as
‘Tigers of Lanka’ (Swamy 1995) and ‘Inside an Elusive Mind’ (Swamy 2006) the latter referring to LTTE leader
Prabhakaran. These provide interesting insights, but provide only a somewhat speculative basis for making
rigorous assessments with regard to some of the key considerations and decisions within the insurgent
movement. See also the rather hard-nosed scholarly confrontation between Sarvananthan (2007) and Stokke
(2006).

15
of them were politicians, with a strong vested interest in presenting their actions in a positive
light. The same, it might be noted, can be said about interviews with international policy
makers and (mainly Norwegian) politicians – many interviewees were prone to a level of
revisionism, and reframing of events with the benefit of hindsight. Many of them had strong
views on counterfactuals – if only the others had let them do things this way … – but these
could not be verified. Former President Chandrika Kumaratunga for example was very angry
about the fact that she had been left out of the ceasefire negotiations, because the newly
elected Prime Minister Wickremesinghe saw her as his primary rival. This, she correctly
argued, was unconstitutional, because the President is the commander-in-chief, but it was
impossible to say what would have happened if she had actually been consulted. Would she
not have blocked or delayed the process so much that it would have scuttled the deal
altogether? Would the LTTE have accepted her involvement in the process after her ‘war for
peace’ a few years earlier? It was hard to get a credible account of whether Kumaratunga’s
involvement would have strengthened or disabled the ceasefire agreement, which was
probably Norway’s most significant accomplishment in Sri Lanka. In our analysis, we
acknowledged that these were indeed difficult judgments, but also underlined that it was
‘problematic’ to exclude an actor with ‘a legitimate stake in the conflict’, and ‘sufficient
political power and support to destabilise the peace process’ (Goodhand et al. 2011a: 88).
On the Norwegian side, the evaluation had excellent access. The Norwegian government
provided full support to the evaluation, allowed unrestricted access to the archives and we
were able to do interviews with the key actors, such as government minister and former peace
envoy Erik Solheim, his successor as envoy Jon Hanssen-Bauer, former state secretary Vidar
Helgesen, ambassadors (to Colombo) Jon Westborg, Hans Brattskar and Tore Hattrem,
successive heads of mission of the ceasefire monitoring mission,7 as well as current and
former Norwegian ministers. As was the case with the Colombo elite, however, some of these
respondents had staked significant political and professional capital on the Sri Lankan peace
process. Although they supported the evaluative exercise and were open to critical reflection,
it did of course matter to them which version of the story would prevail in our report; or – to
be more precise – in the public perception of the report. We return to this below.
With regard to contextual knowledge, there was a widespread conviction among many of our
Indian and Sri Lankan respondents that the Norwegians were insufficiently savvy when it
came to the Sri Lankan polity, its layered rivalries and competing dynasties, the symbolic
meaning of its actions, its use of the vernacular media, and its relation to Buddhist leaders and
other prominent figures in society. We found some of this criticism credible, particularly
because the architects of Norway’s involvement had eschewed the creation of larger
institutional structures (often called ‘track 2 and 3’) and the involvement of specialised
7 Except the first one, who was no longer alive.

16
academics in support of peace efforts in Sri Lanka. However, it is rather difficult to provide
evidence for the lack of contextual awareness, though there were a few quotes indicating that
this was the case. It is hard to provide specific examples of how some people had
misunderstood something that happened several years ago.
Interviews with key Norwegian respondents were particularly important to construct what we
have called an inside-out perspective. The more structural outside-in perspective also drew on
interviews, but was largely based on the existing literature on Sri Lanka’s recent history and
available statistics (including opinion polls, electoral results, and economic growth figures).
This approach shifted the focus away from the minutia of day to day Sri Lankan politics, to
the longer-term structural changes that took place in our study period. This included shifts in
the international context, most saliently the changing attitudes towards insurgent movements
after 9/11 and the rising power of Asian states, including India and China. Other examples
were the increasing importance of diaspora networks, and the significance of globalised
circulation of politically vocal streams, which assumed significance within Sri Lankan
Buddhism. Some of these structural changes were themselves influenced by the peace process.
The split in the LTTE in 2004, for example, may have been influenced by several contingent
events, but it also reflected the difficulties for a rebel movement to preserve unity and
discipline under ceasefire conditions. On the Sinhala side of the political spectrum, the peace
process opened up major political space for ethno-nationalist, anti-elitist forces.
The way our evaluation dealt with these unintended side effects of the peace process caused
some debate, both among our Norwegian interlocutors and more widely. Paradoxically, we
found that the political dynamics around Norway’s involvement in the peace process became
centrally important in Sri Lanka, while Norway’s ability to influence these dynamics was in fact
very limited. The outside-in perspective brought into focus the powerful forces that
constrained Norwegian peace efforts. The inside-out perspective revealed that although some
of these structural forces were not immune to the effects of the internationalised peace
process, Norway did not have the power to steer these forces in intended directions.
5.3 Coda: contested conclusions
If we understand evaluations as attempts to understand and justify policy choices to particular
audiences, the last phase of writing, re-writing, deleting and rephrasing is clearly an interesting
one. Frequently the submission of a draft report provokes a ‘tug of war’ between evaluators
and the commissioners that ordered the evaluation. Though this is sometimes driven by
concerns over reputation or political fall-out, the legitimate way of criticising draft reports is to
take issue with methodological rigour and reliable evidence. Given that peace evaluations are
not a ‘hard science’, it can be quite challenging to reach consensus on these matters.
Moreover, the actors concerned know that once the report is made public, it is not just the

17
nuanced treatment of evidence that wins out: most readers will only read the title and the
executive summary. Different users may interpret (or frame) the main findings of the report
quite differently, irrespective of the evidence and analysis in the report.
In brief, our report argued that Norwegian peace efforts were heavily constrained by the
structural forces of Sri Lanka’s political economy. The demise of the peace process eventually
culminating in the government’s military victory was as much a story of the government
winning and the LTTE losing, as it was a story of Norway failing. In addition to this general
conclusion, we highlighted seven broader implications about international peace mediation in
a changing global context. Some of these points met with agreement; others were (sometimes
quite fiercely) contested. Our draft report underwent two cycles of elaborate feedback. Apart
from a long list of minor comments and corrections, the main issues of debate can be
summarized under three points.
First, some of the people involved felt that we had subjected Norwegian efforts to a “negative
framing”. The criticism was that we “downplayed” the positive results of Norwegian
intervention (such as the signing of the ceasefire), while painting a negative gloss to the overall
narrative. Also, some commentators felt we were providing a platform for politicians (such as
former President Chandrika Kumaratunga) who had an interest in disqualifying key parts of
Norway’s efforts while downplaying their own responsibility. Particular phrases were
considered to be problematic, specifically the title of the report (‘Pawns of Peace’) and parts of
the executive summary. The draft summary stated that the ‘Norwegian involvement in Sri
Lanka is a story of failure’, that Norway’s accomplishments were ephemeral, and that the
‘structural dimensions of the conflict’ had not been addressed.
These conclusions were considered too strong in view of the more nuanced narrative in the
report. In the view of our strongest critic: ‘The executive summary should be completely
revised’. We did rephrase parts of the summary, but the gist of it remained unchanged.
Rhetorical softening would undercut the strength of the report and the legitimacy of the whole
exercise particularly among non-Norwegian audiences, who might interpret our efforts as part
of a Norwegian whitewash campaign if we adopted overly diplomatic phrasing.
The second point of discussion was analytically more fundamental. Some commentators
argued that the report suffered from structural determinism; others took a more nuanced
position and said it was inconsistent in its positioning on structure and agency. In short, both
groups felt the analysis over-emphasized structural factors and longer-term trends, leaving
little analytical space for the way Norwegian decisions and efforts impacted (or could have
impacted) on these processes. While we firmly rejected the allegation of being determinists,
the more moderate critique of mixing structural perspectives with agency was not altogether
unfounded. As mentioned above, there was an apparent paradox in our analysis, which argued

18
on the one hand that Sri Lanka’s plight was driven by an entrenched conflict system in which
Norwegian peacemakers were only pawns, while on the other hand reviewing turning points,
which were subject to contingency and political decisions. Moreover, how could we criticise
Norwegian efforts if the conflict was path dependent anyway? One of the comments read:
“You cannot at the same time argue that outcomes were predetermined in the way the conflict
worked and at the same time attribute so much responsibility for failure (only) to Norway.”
We were convinced of the need to acknowledge structural factors as well as agency and
contingency, but to do so in a coherent and robust way is difficult. There are ample examples
of both: the significance of Sri Lanka’s post-colonial state institutions and the entrenched
rivalries in domestic politics are monumental; and yet there are clear moments of contingency
(the tsunami) and political choice (signing the ceasefire; the LTTE walking away from the
peace talks in 2003; the LTTE split in 2004; the government not signing the joint government-
LTTE aid mechanism in 2005; the LTTE boycotting the 2005 elections resulting in the victory
of the more hard-line candidate Rajapaksa). Our approach of inside-out and outside-in
perspectives woven together on the basis of key turning points was designed to provided a
measured (in our view) balance between structural and agency-based perspectives, but it does
not provide a waterproof defence against this sort of criticism.
The third set of criticism took issue with the adverse effects of Norwegian involvement in Sri
Lanka that we had identified. We argued that Norway’s involvement was not only the victim
of Sinhala ultra-nationalist forces gaining strength, but contributed to it as well. The
internationalized peace process fuelled well-established Sinhala anxieties about foreign
infringements on Sri Lankan sovereignty and left a lot of political space for the forces rallying
against Norwegian peace efforts. We also argued that Norway’s continued involvement after
the resumption of large-scale territorial offensives in 2006 posed an ethical problem: there was
no feasible space to work on peace talks of any sort, and the continued involvement of
Norwegian diplomats and Nordic ceasefire monitors lent itself to abuse by the parties. Neither
of the parties requested a Norwegian departure; the fiction of a ceasefire plausibly provided
tactical military advantages and served to limit the diplomatic fall-out of resuming a war that
resulted in large numbers of civilian casualties. We were critical of Norway’s role during this
phase of the conflict and argued for a consequentialist ethic (there comes a point where you
must withdraw when your involvement lends itself to such abuse), rather than the duty-based
ethic that some of our Norwegian respondents advocated (peace is a noble endeavour that
must always be tried).
This line of argument was met with strong criticism from some of our Norwegian
counterparts, who felt these were “unacceptable” and “unsubstantiated allegations”. The
continuation of Norwegian involvement after the resumption of war in 2006 had been
intended to limit the humanitarian damage, but our report questioned this rationale. We
argued that unintended and unforeseen consequences are not exceptional; they feature in

19
virtually every such intervention. War-making and peace-making are not neatly distinguishable
categories; they influence each other and they may overlap. Norwegian-supported peace
efforts were not a marginal political process in Sri Lanka; they were at the centre of the
island’s political dynamics. They were very controversial and played a dominant role in media
debates and successive elections. Though Norway’s room for manoeuvre and its ability to
effect change was very limited, its involvement produced a number of unintended effects, in
part because its efforts fuelled counter-forces or lent itself to strategic manipulation by
domestic actors. Though it was clearly impossible to prove the counterfactual, the
fragmentation of the LTTE (the split in 2004, which fundamentally shifted the military
balance) would not likely have happened without the peace process. And the internationalised
peace process opened up a large political space for more radical Sinhala nationalist politics to
enter the political mainstream. Not only were the Norwegians attacked by these forces; the
very notion of peace through political negotiations lost its legitimacy. We thus defended these
observations about unintended negative effects, while also making clear that Norway was not
solely responsible for these dynamics.
5.4 Public reception: divergent interpretations
Our argument was sustained with a detailed historical narrative, but this was not sufficient to
pre-empt or deflect the political forces that came into play when the final report was publicly
disseminated. Providing transparency and contributing to public debate were explicit
objectives of the evaluation and for that reason Norad’s Evaluation Department and the
Ministry of Foreign Affairs organised a high profile dissemination event. During the event,
Erik Solheim, who had been the most visible Norwegian player in the Sri Lankan peace
process and was now Minister of Environment and International Development, reacted to the
report and then it was discussed by a panel. His ministry had selected a small group of
panellists who were considered close to the minister. Solheim highlighted some challenges
they had faced on the way, reiterated some of the positive accomplishments, welcomed the
criticism and lessons learnt (without discussing them in much detail), and argued that the
report suffered from a structural determinist perspective and made criticisms that could only
be made with the benefit of hindsight.8
In parallel to the launch in Oslo, the report was published online on some of Sri Lanka’s
primary political debate platforms (Groundviews; Transcurrents). It was important for us to
make sure ‘our version’ of the story and the entire report were directly accessible, because we
were concerned that some of Sri Lankan media reporting on the evaluation would be skewed
and politicized. The Island in particular, had been a source of rather speculative, even
8 A full video of the launch is available at: http://www.regjeringen.no/nb/dep/ud/lyd_bilde/nett-tv-
2/fred_srilanka.html?id=662709

20
conspiratorial coverage of our evaluation efforts. At the beginning of the evaluation, the paper
underlined that CMI (the lead agency) had long been “a major recipient of Norwegian funds”
(The Island, 19 December 2011), while also suggesting we would investigate some of the
assassinations and shed light on what happened at the end of the war (the allegations of war
crimes). It explicitly connected the evaluation to other on-going international efforts: the
inquiry by the UN into allegations of war crimes (the Secretary General’s Panel of Experts),
the Lessons Learnt and Reconciliation Committee (LLRC, the domestic response to
international pressure to investigate) and the rather inflammatory documentary of the UK TV
station Channel 4 (‘Sri Lanka’s Killing Fields”). Particularly the UN panel and the Channel 4
film became symbols of what was perceived as foreign infringement on Sri Lankan
sovereignty driven by supposedly pro-Tamil and/or neo-colonial international actors. Our
evaluation had neither the explicit mandate nor the means to investigate war crimes or gather
forensic evidence on killings; it’s aim was to reflect on Norway’s role over a twelve-year
period. But, The Island presented as part of a larger international effort to undermine the
government’s victory over the LTTE and white wash Norway’s errors (if not malign
intentions). For instance, when the release of our report was delayed, the Island wrote:
‘Norway seems to be reluctant to make public a costly evaluation of its unsuccessful peace
efforts in Sri Lanka. Although Norway initially planned to unveil the final report in the first
week of April 2011, ahead of UNSG Ban Ki-Moon’s unsubstantiated ‘war crimes’ report, an
influential section in the Norwegian government is concerned about the outcome. Sources
told The Island that in view of a spate of revelations made by Wikileaks with regard to the
Norwegian-led peace process since February 2002 the interested parties would not be able to
manipulate the Norwegian evaluation.’
(The Island 27 July 2011)
Coverage of the release of the report followed suit: “There is an unmistakable tendency to
apologize on behalf of and whitewash the Norwegian effort – not surprising considering that
this is a Norwegian government commissioned study and the evaluation team would not have
got the job if they were not going to write something to absolve their paymasters of blame.”
(The Island, 12 November 2011) Meanwhile the Ministry of Defence selectively highlighted a
quote in the report to commend Defence Secretary Gotabhaya Rajapaksa (who is also the
president’s brother) as the only person with the foresight to predict the LTTE could be
defeated militarily.
An article in the internet newspaper Asian Tribune, by contrast, observed that through our
report, Norway “ finally admitted” that its peace efforts were a “political disaster, despite the
carefully-calculated spin applied over the catalogue of significant failures.” The author
acknowledged our point about unintended and unforeseen consequences of peace efforts, but
underlined that “rubbishing the sentiments of the majority community as irrelevant, belongs

21
to neither” and closed with the argument that the report ought to have “apologized for using
20 million inhabitants of Sri Lanka as guinea pigs for the botched geo-political experiment.”
(Asian Tribune, 12 November 2011).
A more astute, pro-Sri Lankan government response came from political scientist and
journalist Dayan Jayatilleka, who was the ambassador representing Sri Lanka in the UN
Human Rights Council at the end of the war. He complimented the report as “useful and
good”, but “empty at its very core”, because it failed to confront the question: “how does one
make peace with a non-state (therefore unconstrained) actor that is fanatical, politico-
ideologically fundamentalist and totalitarian?” In other words, we should have accepted that
the insurgents were not amenable to a negotiated settlement. From a realist perspective, he
argued, the LTTE’s track record of violence and recalcitrant positions made a “final war” the
only feasible option. The Norwegian effort “was foredoomed” and “the war had to be fought
to win”.
On the opposite side of the political spectrum, the criticism was quite different, but no less
fierce. Tamilnet – an established Eelamist website, associated with what used to be the LTTE –
underlined that our report “subconsciously sees ‘victory’ in the war”, while failing to
acknowledge that international peace efforts had “abetted genocide”. In sum: “Norway
washes its hand of its responsibilities to victims” (Tamilnet, 11 November 2011).
We discuss these reviews at some length because they illustrate the political arena into which
the evaluation entered. The political work of the evaluation was not simply the drawing of
lessons on the basis of scientific findings. Instead, the evaluation had to create as firm a
methodological ground as possible to navigate divergent political pressures and withstand
critical attacks from various quarters. We produced a text that explicated underlying
assumptions, conceptual foundations, contradictory interpretations and their varied evidence
bases and sought a reasonable middle ground between competing versions of the story,
attempting to preserve nuances and tensions in a productive way. But this nuanced openness
made the report more vulnerable to knowledge brokers from across the political landscape
drawing out simplistic – and diametrically opposed – punch lines to serve their own agendas.
This section has highlighted the range of opinionated responses to our report. This provides
insights into the polarised political landscape in which our efforts were set, but it may wrongly
create the impression that the evaluation created an enormous reaction. That was not the case.
Some media – including much of the Norwegian media – in fact had rather minimal coverage
of the report. And the heated debate that did take place on some platforms died away within a
matter of days or weeks. The launch of our report provided an occasion to contest and
confirm established positions; it did not necessarily influence those positions. This sobering
fact also became clear from the follow-up to our report in Oslo. Norad’s Evaluation

22
Department received written comments from the Ministry of Foreign Affairs which repeated
some of Solheim’s main points, but also recognised that the report “largely reflects main
challenges and, partly, constraints that Norway faced in Sri Lanka”, including the difficulties of
trying to combine different roles (facilitator, monitor and aid donor), and the importance for
mediators to have a strategy towards media (Norad 2012). There was, however, no willingness
to discuss whether the Norwegian decision to continue as mediator rather than pull out in
2006 might have been morally questionable, and most of the general lessons that we tried to
draw from Norway’s engagement in Sri Lanka were ignored in the official response to the
evaluation.
6. Conclusions and broader implications
The methodological, analytical and political insights from this evaluation cannot simply be
carried over to other contexts. Norwegian peace efforts in Sri Lanka involved a lot of ‘high
politics’. Our evaluation assessed the performance of relatively powerful and politically savvy
actors, and there was a significant amount of media attention to both the Norwegian
involvement and our evaluation. Given that this was the first comprehensive evaluation of
Norwegian peace efforts and that a government minister (Erik Solheim) had played a major
role, it meant that the stakes were high on the Norwegian side. That is not always the case for
such evaluations. The experience of other evaluators, particularly in terms of the political
reactions to the release, is likely to be somewhat different. And yet the argument of this
chapter has purchase beyond Sri Lanka. Other high profile evaluations have clearly been
subject to similar dynamics (cf Waage, 2008) and we believe lessons from our evaluation can
be useful to others.
First, such evaluations are an ineluctably political exercise. Whilst there are strong pressures to
present them as technical, scientific and dispassionate analyses, this is never an accurate
representation of reality. As we have highlighted here, good evaluation is partly about
improved methodology, but also (and perhaps more so) about the ability to navigate and
engage with politics. To some extent this is the case with all forms of evaluation. However,
evaluation in conflict-affected environments tends to bring out in much sharper relief the
political stakes of the evaluated interventions. We can understand peace processes as
charismatic moments in politics when the stakes are extremely high and the protagonists have
a strong interest in selling particular narratives and versions of events and limiting what can
and cannot be said and what is considered legitimate and illegitimate. Much in line with
Ferguson’s (1990) and Li’s (2007) work on development, peace interventions and evaluations
may be prone to ‘rendering’ technical. People’s careers and even their lives may in fact depend
upon which version of events prevails.

23
Some very practical and concrete implications for evaluators and those who commission
evaluations flow from this analysis. For example, it is clear that evaluation teams should be
composed primarily of individuals who have had a long engagement with the context(s) in
question and who can navigate the politics and interpret and decode the information
economy. This suggests that those funding conflict related evaluations should think very
carefully about the balance between such context knowledge and general “technical”
knowledge on conflict/peacebuilding or evaluation methods. There has been a trend in recent
years for knowledge on conflict issues to be commoditized and universalized – with the
growth of Conflict Advisors and Strategic Conflict Assessment experts who have general
knowledge of the field outside of particular conflict settings [?]. We feel that there should be a
stronger commitment to context-based expertise and robust political economy analysis.
Another implication of acknowledging the politics of evaluation is to recognise that evaluators
do not simply look down objectively from the commanding heights and pass neutral
judgments on the messy reality below them. They are part of the ‘mess’ they are evaluating,
and this needs to be explicitly acknowledged to avoid doing harm, i.e. not making the ‘mess’
even worse for those who have to live with it on a daily basis. Therefore, just as there has been
some significant work done in recent years on the ethical and methodological challenges of
doing research in areas of conflict (Nordstrom and Robben 1995; Finnström 2001; Smith and
Robinson 2001; Gould 2010), there is a need to develop ethical guidelines for evaluators
operating in conflict settings, which set out clearly how they should endeavour to ‘do no
harm’.
Second, the chapter has discussed the methodological challenges and our way of dealing with
them in the evaluation of Norwegian peace efforts in Sri Lanka. Our approach involved
‘systematic eclecticism’, using mixed methods in a systematic way. We were transparent about
underlying assumptions, causal inferences and associated methods, to enable readers to make
their own judgments about the accuracy and validity of our findings and conclusions. Whilst
conflict and peacebuilding can never be reduced to universal laws and dictums, comparative
and quantitative research on peace processes can certainly add insight. Evaluators should aim
for more systematic and innovative mixes of quantitative and qualitative methods. Our
contribution to that end has been on the qualitative end of the spectrum with an exploration
of a historical political economy approach combining an inside-out perspective with an
outside-in perspective and using turning points, to deal with the perennial problems of
uncertainty, attribution, conflicting accounts and the balance between structure and agency.
The overarching argument of this chapter has been that the political arena of peace
evaluations and the methodological challenges they face are related. It is clearly important to
improve the rigour and transparency of the methodologies used in evaluating peace efforts,

24
but these scholarly advances will never provide a water-tight, “neutral” scientific vantage point
for evaluating deeply political interventions. In fact, evaluations that claim to have neutralised
or circumvented the political content of the interventions they study may in fact be missing
critical aspects of these interventions. Our chapter has sought to ask some uncomfortable
questions about the links between the worlds of research and policy. The evaluator often sits
in a twilight zone bridging these two worlds, acting as a broker and translator between them.
In the development world, critical evaluation is presented as a learning tool, leading to smarter
and more effective policies. Yet, it is also clear that evaluation, often the result of crises of
legitimacy or political attacks on development policies or projects, has the effect of re-
legitimizing the world of development (e.g. Ferguson 1990; Duffield 2007). In this context,
commissioning evaluations shows openness to learning, without forcing significant structural
changes. The findings of research projects are used selectively by policy makers to provide
ammunition for existing policies or new policy departures that have already been decided.
While there has been a remarkable professionalization of evaluative practice over recent years,
there remains a significant rift between the elevated aspirations of scrutinizing and improving
policy-practice and the political work that evaluations actually do.
Bibliography
Balasingham, A. (2004) War and Peace: Armed Struggle and Peace Efforts of the Liberation Tigers.
Mitcham: Fairmax Publishing.
Bastian, S. (2007) The Politics of Foreign Aid in Sri Lanka: Promoting and Supporting Peace. Colombo:
ICES.
Bullion, A. (1995) India, Sri Lanka and the Tamil crisis, 1976-1994: An international perspective.
London/New York: Pinter.
Coomaraswamy, R. (2003) ‘The Politics of Institutional Design: An Overview of the Case of
Sri Lanka,’ in S. Bastian, and R. Luckham (eds.) Can democracy be designed? The politics of
institutional choice in conflict-torn societies: pp. 145-169. London: Zed Books.
Cramer, C & Goodhand, J (2011) ‘Hard science or waffly crap? Evidence-based policy versus
policy-based evidence in the field of violent conflict’ in Baylss, K, Fine, B & Waeyenberge, E
(2011) The Political Economy of Development. The World Bank, Neo-Liberalism and Development
Reseearch Pluto Press, London, UK, pp. 215 – 238.
De Silva, K. (2005) A History of Sri Lanka. Colombo: Vijitha Yapa Publications.

25
Dixon, J. (2009) ‘Emerging consensus: Results from the second wave of statistical studies on
civil war termination’, Civil Wars, 11(2): 121-136.
Doyle, M. and N. Sambanis (2006) Making War and Building Peace, Princeton: Princeton
University Press.
Duffield, M. (2007) Development, Security and Unending War: Governing the World of Peoples.
Cambridge and Malden: Polity Press.
Ferguson, J. (1990) The Anti-Politics Machine: Development, Depoliticization, and Bureaucratic Power in
Lesotho. Cambridge: Cambridge University Press.
Finnström, S. (2001) ‘In and out of culture: Fieldwork in war-torn Uganda’, Critique of
Anthropology, 21(3): 247-258.
Fortna, P. (2004) ‘Does Peacekeeping Keep Peace? International Intervention and the
Duration of Peace After Civil War’, International Studies Quarterly, 48 (2): 269-292.
George, A. and McKeown, T. (1985) ‘Case Studies and Theories of Organizational Decision
Making’, Advances in Information Processing in Organizations, 2: 21-58.
Goodhand, J. and B. Klem, with D. Fonseka, S.I. Keethaponcalan, D. and S. Sardesai (2005)
Conflict, Aid and Peacebuilding in Sri Lanka, 2000 – 2005. Washington DC: the Asia Foundation.
Goodhand, J., B. Klem and G. Sørbø (2011a) Pawns of Peace: Evaluation of Norwegian
Peace Efforts in Sri Lanka, 1997-2009. Norad evaluation 5/2011.
Goodhand, J., D. Rampton, R. Venugopal and N. de Mel (2011b) Strategic Policy Assessment.
Study commissioned by DFID.
Goodhand, J. and O. Walton (2009) ‘The limits of liberal peacebuilding: international
engagement in the Sri Lankan peace process,’ Journal of Intervention and Statebuilding 3 (3): 303-
323.
Gould, K. (2010) ‘Anxiety, epistemology, and policy research ‘behind enemy lines’’, Geoforum,
41: 15-18.

26
Human Security Centre (2005) Human Security Report 2005: War and Peace in the 21st Century.
Vancouver: University of British Columbia.
Human Security Project Report (2010) Human Security report 2009/2010: The Causes of Peace and
the Shrinking Costs of War. Vancouver: Simon Fraser University.
Klem, B. (2012) In the Wake of War: The Political Geography of Transition in Eastern Sri Lanka. PhD
Dissertation at the University of Zurich, Switzerland.
Li, T. (2007) The Will to Improve: Governmentality, Development, and the Practice of Politics. Durham:
Duke University Press.
Licklider, R. (2009) ‘Civil War Outcomes,’ Saltzman Working Paper, No. 11, November 2009
<http://www.siwps.com/programs/SWP.attachment/no-11--/No%2011%20-
%20Licklider.pdf> Accessed 10 January, 2011.
Loganathan, K. (1996) Sri Lanka, Lost Opportunities: Past Attempts at Resolving Ethnic Conflict.
Colombo: University of Colombo.
Martin, H. (2006) Kings of Peace, Pawns of War: The untold story of peace-making. Centre for
Humanitarian Dialogue, Continuum, London.
Moore, M. (1985) The state and peasant politics in Sri Lanka. London: Cambridge University
Press.
Norad (2012) ‘Evaluering av norsk fredsinnsats på Sri Lanka, 1997-2009’. Oslo: Norad.
Nordstrom, C. and A. Robben (1995) (eds) Fieldwork Under Fire: Contemporary Studies of Violence
and Survival. Berkeley: University of California Press.
OECD/DAC (2012) ‘Guidance on evaluating peacebuilding activities in settings of conflict
and fragility – Improving learning for results’ DAC Guidelines and Reference Series,
DCD/DAC(2012)40/FINAL.
Orjuela, C. (2011) ’Buying Peace? Politics of Reconstruction and the Peace Dividend
Argument,’ in K. Stokke and J. Uyangoda (eds.): Liberal Peace in Question: Politics of State and
Market Reform in Sri Lanka: pp. 121-140. London: Anthem Press.

27
Rainford, C. and A. Satkunanathan (2011) ‘From SIHRN to Post-War North and East: The
Limits of the “Peace through Development” Paradigm in Sri Lanka’, in K. Stokke and J.
Uyangoda (eds.) Liberal Peace in Question – politics of state and market reform in Sri Lanka: pp. 121-
141. London: Anthem Press.
Regan, P (2002) ‘Third Party Interventions and the Duration of Intra-State Conflicts’, Journal of
Conflict Resolution, 46(1): 55-73
Richards, P. (2011) ‘A systematic approach to cultural explanations of war: Tracing causal
processes in two West African insurgencies’, World Development, 39(2): 212-220.
Sarkees, M. (2000) ‘The correlates of war data on war: An update to 1997’, Conflict Management
and Peace Science, 18(1): 123-144.
Sarvananthan, M. (2007) ‘In Pursuit of a Mythical State of Tamil Eelam: A Rejoinder to
Kristian Stokke,’ Third World Quarterly, 28(6): 1185-1195.
Smith, M. and G. Robinson (2001) (eds) Researching Violently Divided Societies: Ethical and
Methodological Issues. London: Pluto.
Spencer, J. (2008) ‘A Nationalism without Politics? The Illiberal Consequences of Liberal
Institutions in Sri Lanka,’ Third World Quarterly, 29(3): 611-629.
Stedman, S. (1997) ‘Spoiler Problems in Peace Processes,’ International Security, 22(2): 5-53.
Stern, P. and D. Druckman (2000) ‘Evaluating Interventions in History: The Case of
International Conflict Resolution’, International Studies Review, 2(1): 33-63.
Stokke, K. (2006) ‘Building the Tamil Eelam State: Emerging State Institutions and Forms of
Governance in LTTE-Controlled Areas in Sri Lanka,’ Third World Quarterly, 27(6): 1021-1040.
Swamy, N. (1995) Tigers of Lanka: From Boys to Guerrillas. Colombo: Vijitha Yapa.
Swamy, N. (2006) Inside an Elusive Mind. Colombo: Vijitha Yapa.
Uyangoda, J. (2011) ‘Travails of State reform in the Context of Protracted Civil War in Sri
Lanka,’ in K. Stokke and J. Uyangoda (eds.) Liberal Peace in Question: Politics of State and Market
Reform in Sri Lanka: pp. 46-75. London: Anthem Press.

28
Venugopal, R. (2009) Cosmopolitan Capitalism and Sectarian Socialism: Conflict, Development, and the
Liberal Peace in Sri Lanka. PhD dissertation, Oxford University, Oxford.
Waage, H. H (2008) ‘Postscript to Oslo: the Mystery of Norway's Missing Files’
Journal of Palestine Studies, Vol. 38, No. 1 (Autumn 2008), pp. 54-65
Zartman, W. (2000) ‘Ripeness: The Hurting Stalemate and Beyond,’ in P. Stern and D.
Druckman (eds.) International Conflict Resolution after the Cold War: pp. 225-250. Washington DC:
National Academy Press.
Zartman, W. (2009) ‘Interest, Leverage and Public Opinion in Mediation,’ International
Negotiation 14: 1-5.