Socrates Back on the Street: Wikipedia's Citing of the Stanford Encyclopedia of Philosophy

Full text

International Journal of Communication 2 (2008), 1269-1288 1932-8036/20081269
Copyright © 2008 (John Willinsky). Licensed under the Creative Commons Attribution Non-commercial No
Derivatives (by-nc-nd). Available at http://ijoc.org.
Socrates Back on the Street:
Wikipedia’s Citing of the Stanford Encyclopedia of Philosophy
JOHN WILLINSKY
Stanford University
Introduction
In bemoaning the Internet's “cult of the amateur,” Andrew Keen holds up as a prime instance
the Web's number-one information destination: “And then there is Wikipedia, an online encyclopedia
where anyone with opposable thumbs and a fifth-grade education can publish anything on any topic from
AC/DC to Zoroastrianism,” which is for Keen “the blind leading the blind — infinite monkeys providing
infinite information for infinite readers, perpetuating the cycle of misinformation and ignorance” (2007, p.
4). To substantiate what he sees as Wikipedia’s abysmal record, he cites a Forbes’ article on corporations
caught tinkering with their articles in the online encyclopedia, using the example of a Wal-Mart employee
deleting, as Keen misquotes the story, “the line about underpaid employees making less than 20 percent
of the competition” (ibid.).
1
However wrong Keen gets it in this case, Wikipedia’s reliability remains an open question in many
people’s minds. Instructors warn students not to cite Wikipedia in their papers, as if any student would be
advised, after fifth grade, shall we say, to quote an encyclopedia as an authoritative source (Jaschik,
2007). Still, others have pointed out how Wikipedia continues to be beset by factual mistakes and
awkward writing, even if the errors are corrected shortly after being noted (Read, 2006).
2
Wikipedians, as
the eight million registered writers and editors contributing to this work would be known, have responded
John Willinsky: john.willinksky@stanford.edu
Date submitted: 2008-10-31
1
Keen was citing Evan Hessel in Forbes who wrote that “an employee, also identified by a Wal-Mart IP
address, cut a line stating the mega retailer paid its employees 20 percent less than its competitors did”
(Hessel, 2006). It is only because Wikipedia makes its edits public that such deletions were detectable
by tools others have developed (see wikdgames http://wired.reddit.com/wikidgame/?s=top). Keen
does not note that in 2005, a “Criticism of Wal-Mart” article was initiated in Wikipedia and now runs to
9,000 words.
2
For a summary of the various measures of Wikipedia accuracy and verifiability, see Fallis (2008).
Wikipedia formalized and posted its policy on the verifiability of claims in its articles in 2003.

1270 John Willinsky International Journal of Communication 2(2008)
to these concerns by increasing the focus on accuracy and verification by, for example, placing a series of
admonishing tags at the top of entries. One reads, “This article is a stub” (“You can help Wikipedia by
expanding it”), while another is “The examples and perspective in this article or section may not represent
a worldwide view of the subject” (“Please improve this article or discuss the issue on the talk page”). Then
there are the tiny superscripted annotations, placed discreetly at the end of an unsubstantiated claim, that
read “citation needed.”
Still, it strikes me as shortsighted to view a massive social phenomenon of this scale solely in
terms of the resulting encyclopedia’s accuracy.
3
That so many people are working together out of an
interest in helping other people learn, whether about Wal-Mart or Wittgenstein, should be assessed on a
number of grounds. Wikipedia represents yet another Internet-spawned manifestation of what has been
called an “impossible public good” (Kollock, 1999, p. 230). The term was first used to describe open
source software, such as Linux and MediaWiki (which is the system used to run Wikipedia). This software
is at once freely distributed, while being open to tinkering and contributions. What is impossible about it is
how this piece of intellectual property, in which people have invested a good deal of work, be can have
been made public in a non-proprietary, loosely governed way. The spirit of open source software has
infused two related Internet developments, namely, the open access movement, which is directed toward
providing free online access to university-based research publications (Harnad et al., 2008; Willinsky,
2006), and the open educational resources initiative, which makes freely available learning materials and
tools, from course syllabi to big bang simulations (Caswell, Henson, Jensen, & Wiley, 2008).
4
While most assessments of Wikipedia understandably focus on its reliability as a reference work,
it might also be regarded as the largest, most globally situated of open educational resources. Unlike the
traditional encyclopedia, Wikipedia is an open construction site for learning attended to by readers and
Wikipedians, and readers becoming Wikipedians. It is also an open educational resource that takes
advantage of and contributes to open source software in the form of MediaWiki. And it draws, as this
study will demonstrate, on the new sources of open access to research and scholarship to increase its
documentation and enrich the educational experience it provides for readers and Wikipedians. Through
this convergence of open elements and (impossible) public goods, Wikipedia is arguably changing the
place of learning in the world at large. Not only is Wikipedia open in the sense of the being read and
edited, Wikipedia also opens the ways and means of assembling this knowledge. The openness of
Wikipedia’s edit, discussion, and history pages make public the remarkable investment that people make
3
In 2007, the Pew Internet and American Life Project found that a third of American adults who go online
consult Wikipedia (Rainie & Tancer, 2007). It also found that nothing else comes close in terms of
“educational and reference Web sites” traffic, as Wikipedia receives 24% of the traffic (followed by
Yahoo! Answers at 4%).
4
These three movements, open source software, open access and open educational resources, do not
exhaust the new openness, as it applies as well to the Creative Commons, open data, open notebook
science, open knowledge, and others.

International Journal of Communication 2 (2008) Socrates Back on the Street 1271
in getting things right, finding (increasingly) warrants and backing, and organizing comprehensive articles
and support systems to raise the quality of the work.
5
This study examines one aspect of the open knowledge-building process and one point of the
convergence among open initiatives, as Wikipedians and Wikipedia readers draw on open access research
and scholarship. It considers how Wikipedians are using the open access and peer-reviewed Stanford
Encyclopedia of Philosophy (SEP) to strengthen and enrich English language Wikipedia articles both within
and outside of traditional philosophy topics. A high proportion of SEP’s entries are being cited by
Wikipedians, both in writing and discussing articles — with some of them providing pivotal points in the
learning that goes into the articles — and the majority of those citations were used by Wikipedia readers
over a two-week period. Finally, as a further indication of how Wikipedians are responding to questions on
verification and reliability, the links in Wikipedia to SEP and a range of other academic resources, both
open and closed, are presented to indicate how SEP represents part of the larger public and educational
benefit of open access to research and scholarship.
The Stanford Encyclopedia of Philosophy was conceived in 1995 by John Perry and Edward N.
Zalta at Stanford’s Center for the Study of Language and Information (Perry & Zalta, 1997). It represents
a new breed of scholarly communication. As its principal editor, Zalta set out to create a “dynamic”
encyclopedia that combined peer review with ongoing updating and revision. Entries offer internal and
external links to related materials. SEP is intended to be “useful both to professional scholars and the
general public” (ibid.), an important part of which is how it has continued to be free to read, based on a
variety of grants, with a current program to have it funded on an ongoing basis by an endowment, with
support from research libraries and philosophy departments. SEP is a new sort of knowledge resource. It
is not only free and peer-reviewed, but periodic with entries added and updated, and older editions
archived.
The SEP entries represent the work of individual philosophers, with their e-mail address and a
link to home page at the bottom of the entry. While some entries are as heavily footnoted as any scholarly
work in the humanities, others do not directly cite secondary literature, much like Encyclopædia
Britannica.
6
SEP is not published by a press or publisher, per se, but comes out of the Metaphysics
Research Lab in the Center for the Study of Language and Information at Stanford University, which holds
the copyright to everything but the entries which are copyrighted by the authors. At the time of this study,
5
Wikipedia refers to its encyclopedia entries as “articles” (at least for the most part), while SEP uses
“entries,” and this convention is followed in this paper.
6
To take two of the SEP entries discussed in this paper, Aristotle has 27 footnotes and Philoponus none,
even when referring to the secondary literature: “Nowadays, Philoponus is often celebrated for having
been one of the first thinkers to reconcile Aristotelian philosophy with Christianity.” The difference
between these two entries speaks to how SEP moves between journal article and encyclopedia, in a
scholarly tradition that dates back to origins of the Philosophical Transactions which when launched in
1665 as a monthly was repackaged as something of an annual encyclopedic volume, with an index as a
guide to its entries.

1272 John Willinsky International Journal of Communication 2(2008)
SEP had 1,026 entries on philosophic figures and concepts (while adding a half-dozen or so entries a
month); the Web site was receiving roughly half a million hits a week.
In considering how Wikipedians are using SEP, this study examines all of the links in Wikipedia
articles that lead to SEP and the extent to which those links were used by Wikipedia readers over a two-
week period. An earlier study I conducted on Wikipedia established that a very small proportion (2%) of
its articles had links to research that readers could open and read, without having membership in a
research library (2007). The study also demonstrated that, with a little bit of an effort, relevant open
access research could be found for 60 percent of the Wikipedia entries in the study’s sample.
7
This
subsequent study examines in some detail how one particular open access resource, namely SEP, is
contributing to the educational quality of Wikipedia. It is intended to, among other things, encourage
Wikipedians to make greater use of open access, and the academic community to make a greater
proportion of their work open access.
Method
This study of Wikipedia’s use of SEP was made possible by the close cooperation of the editors at
the Stanford Encyclopedia of Philosophy. After discussing the intent, scope, and design of the study with
Principal Editor Edward N. Zalta and Senior Editor Uri Nodelman, both provided helpful suggestions to
improve the study and the Weblogs for SEP over a two-week period (in which the IP addresses had been
scrambled to protect the identity of users). The information provided to us on the Weblogs identified that
set of users who had arrived at SEP by clicking on a link in Wikipedia. The Weblogs, for the two-week
period between June 22 and July 5, 2008, revealed, after some cleaning up of the data, where in
Wikipedia people clicked on a link leading to SEP, and where that link led to in SEP.
8
This data did not tell us how long readers spent in SEP nor, of course, what sense they were
making of the SEP entries they encountered. But it did tell us that readers of the articles Aristotle and
Politics in Wikipedia were clicking on the links in those articles that led to SEP. At the very least, we might
assume that clicking on a SEP link indicated an interest in seeing what more specialized knowledge was
available on the topic, if not an interest in learning more about the topic. Most of the links indicated that it
was the Stanford Encyclopedia of Philosophy, suggesting that readers were at least learning about this
resource as a more specialized, open work.
7
Björk, Roos, and Lauri, for example, have calculated that 20% of the scholarly literature’s yearly output
is freely available online through open access journals, archives and personal Web sites (2008).
8
The SEP weblogs originally contained 17,724 records of users coming from Wikipedia to SEP. The logs
were then rid, for purposes of this research, of (a) computer-generated traffic from bots and crawlers
(836 records), (b) doubled records, which occurred through a redirect that took users from one URL to
another in SEP (3,297 records) and (c) records without an identifiable source and/or target in the two
target works (188 records). Allowing for some overlap among these categories, this left a total of
13,363 records of users moving from Wikipedia to SEP.

International Journal of Communication 2 (2008) Socrates Back on the Street 1273
Before analyzing Wikipedia readers’ use of SEP links, an analysis was conducted of all references
to SEP in Wikipedia. This was done using Wikipedia’s advanced search capacities to search for SEP’s URL
(e.g., http://plato.stanford.edu/) for “external links” within the encyclopedia.
9
These searches revealed
which pages in Wikipedia had links to which SEP pages, indicating how Wikipedians were citing SEP in
articles, discussions, and user profile pages. A similar search was conducted with nine other academic
resources, from arXiv.org to ScienceDirect to establish the degree to which Wikipedia is drawing on
research and scholarship more generally.
Results
SEP
R
EFERENCES IN
W
IKIPEDIA
At the time of this study, 1,741 Wikipedia articles contained one or more links to 942 SEP entries,
for a total of 2,263 links leading from Wikipedia to SEP. Wikipedia has links to slightly more than 80% of
the 1,026 entries that made up SEP at that time (with additional links to earlier, and now archived editions
of SEP).
10
The vast majority of the links were to SEP entries about ideas such as Truth and Causality, as
well as Pleasure and The Meaning of Life, rather than entries devoted to philosophers. That said, the
Wikipedia article with the largest number of SEP links was Aristotle, with 14 references leading to SEP,
followed by Truth with 10 links to SEP (Table 1).
11
The SEP entry most often cited by Wikipedia was
Libertarianism, with 15 links leading to it, followed by Atheism and Agnosticism and Classical Logic (Table
2). In what follows, I will examine how SEP has served the construction of Wikipedia’s Aristotle article, as
well as the discussion surrounding its Atheism article, before turning to Wikipedia readers’ use of the SEP
links during the test period.
9
http://en.Wikipedia.org/wiki/Special:LinkSearch
10
While this is a study of how readers of Wikipedia come to SEP, it can be pointed out that SEP contains
17 entries with linked references to Wikipedia articles, largely under “Internet Resources,” with a link,
for example, in the SEP entry for Time leading to Wikipedia articles for Eternalism, Philosophy of Space
and Time, and Presentism.
11
Although not counted as a Wikipedia article, the Wikipedia user page for Simfish had 13 links to SEP
embedded among his lists of things that interest this contributor, who has done some 300 edits to
Wikipedia since 2004.

1274 John Willinsky International Journal of Communication 2(2008)
Table 1. Wikipedia pages with the most links to SEP.
Wikipedia Article SEP Links
Aristotle 14
Truth 10
Immanuel Kant 9
Causality 8
Computational Epistemology 8
Philosophy of Physics 8
Politics 8
Rene Descartes 8
Epistemology 7
Physicalism 7

International Journal of Communication 2 (2008) Socrates Back on the Street 1275
Table 2. Stanford Encyclopedia of Philosophy entries most often cited in Wikipedia.
SEP Entry Wikipedia citations
Libertarianism 15
Atheism and Agnosticism 14
Classical Logic 14
Copernicus 13
David Hume 13
Ontological Arguments 13
William Godwin 12
Karl Marx 12
Friedrich Nietzsche 12
Zeno’s Paradoxes 12
Karl Popper 12
Note: The URL for SEP itself, although not counted as an SEP entry for
purposes of this table is included 72 times.
ARISTOTLE
Wikipedia’s Aristotle article is close to 9,000 words long, and it was edited on a daily basis during
the course of this study. The Wikipedia Version 1.0 Editorial Team rates the article to be B-class, which
means that “no reader should be left wanting, although the content may not be complete enough to
satisfy a serious student or researcher.” The article itself dates back to April 21, 2003 when, at 3,000
words, it was largely devoted to the philosopher’s biography, with the text largely lifted without attribution
from the Catholic Encyclopedia (Turner, 1907). A small amount of space was given to Aristotle’s method
and a summary of three criticisms of his work. The bibliography back then did contain links to what was
already a rich set of Aristotle’s works freely available online, principally through Virginia Tech.
The Wikipedia article on Aristotle, a little more than five years later, now includes well-referenced
summaries of his work in physics, metaphysics, biology and medicine, as well as practical philosophy.
12
12
One still feels that the entry could use the eye of a sharp editor, with its use of expressions such as
“Aristotle is one of the most important founding figures in Western philosophy” (to call him a founding
figure surely makes “one of the most important” redundant) and to say that “he was the first to create a

1276 John Willinsky International Journal of Communication 2(2008)
The top of the article features a baseball-card-like sidebar with his picture, birth and death dates, along
with his “school” and “notable ideas.” At the end of the entry, after the List of Aristotle’s Works,
References, Further Reading, and See Also, there is a list of External Links made up of “Collections of
Aristotle’s Works,” which are available in English and the original Greek from five Web sites, and “Articles
on Aristotle,” which includes links to the Aristotle entries in the Internet Encyclopedia of Philosophy and
the Catholic Encyclopedia.
13
Finally, there is a set of 13 links to as many SEP entries that bear on
Aristotle’s work:
Scholarly surveys of focused topics from the Stanford Encyclopedia of Philosophy:
articles on Aristotle in the Renaissance, Biology, Causality, Commentators on Aristotle,
Ethics, Logic, Mathematics, Metaphysics, Natural philosophy, Non-contradiction, Political
theory, Psychology, Rhetoric
This set of SEP links in Wikipedia did not include the main entry for Aristotle in SEP himself, for it was not
published until two months after the study, but they otherwise sought to extend Wikipedia’s reach by
providing links to the various SEP Entries devoted to different aspects of this peripatetic philosopher’s
work.
SEP also turns up in one of the 54 footnotes for the Aristotle article.
14
The footnote simply reads
Stanford Encyclopedia of Philosophy, and serves as a reference to the following statement in the article:
“In a similar vein, John Philoponus, and later Galileo, showed by simple experiments that Aristotle's theory
that the more massive object falls faster than a less massive object is incorrect.” The footnote is
hyperlinked directly to the “theory of impetus” section of the John Philoponus entry in SEP by Christian
Wildberg. While Philoponus’ theory is based on a misguided sense of a kinetic force being imparted to
falling or thrown objects, Wildberg points out, it did lead Philoponus to experimentally test and disprove
Aristotle’s conclusion about the differing speeds of falling bodies, much as Galileo did centuries later.
Wildberg’s entry for Philoponus provides a substantial list of primary and secondary sources, although
under “Other Internet Resources,” there is only the request to “please contact the author with
suggestions.”
Wikipedia co-founder Jimmy Wales, has recently spoken of how Wikipedia articles have been
improved as they are “more detailed, more accurate, hopefully better written, fleshed out more, with . . .
comprehensive system of Western philosophy” suggests anachronistically that he fashioned something
that did not come into existence, even as an idea, until well over a millennia had passed.
13
The Aristotle entry for the Internet Encyclopedia of Philosophy has as its concluding line: “The author of
this article is anonymous. The IEP is actively seeking an author who will write a replacement article”
(2006), while the Catholic Encyclopedia entry starts out, “The greatest of heathen Philosophers . . .”
and was originally published in the 1907 edition of the encyclopedia.
14
These items are listed in the entry as “Notes,” but are referred to as “footnotes” in the Wikipedia style
guideline Citing Sources, and will be called such in this article: “These [references, footnotes,
parenthetical reference] are the most common methods of making articles verifiable. A Wikipedia editor
is free to use any of these methods or to develop new methods; no method is preferred.”

International Journal of Communication 2 (2008) Socrates Back on the Street 1277
two or three footnotes to tell you where to go and check it” (Young, 2008). In May 2005, for example, the
Wikipedia entry on Aristotle, which was by then more than 5,000 words, was without footnotes or any
links to SEP. The 51 footnotes that accompanied the Aristotle entry may tell you where to go and check,
but only three of those footnotes led Wikipedia readers to the sources, to SEP’s discussion of how
Aristotle’s theory of falling bodies was experimentally refuted, to Cicero’s Academica in the original Latin
(courtesy of Project Gutenberg), and to a discussion of Aristotle’s last will and testament.
15
This pattern of including the relevant SEP entries under External Links and including relatively
few references to SEP in the footnotes to the Wikipedia article proved common enough.
16
While it does not
make sense to recommend a single source for Wikipedia articles, it is worth noting that such citing speaks
to two points. The first is SEP’s dual nature, as both a scholarly work (and as such an excellent source)
and a reference work (and thus at a remove from excellent sources). The second has to do with how SEP
stands out among the footnoted sources as the one that readers can readily and consistently check, and
thus speaks to the educational advantages of open access footnotes. Google Book Search is being used
with links that embody prepared searches that take one to a precise page, with the key term highlighted.
As well, a good number of freely available primary sources, from Aristotle to Cicero, also contribute to this
idea of the Wikipedia article opening the gate to a larger world of learning.
T
HE
R
OLE OF
SEP
IN
W
IKIPEDIA
D
ISCUSSION
P
AGES
SEP also proved useful to Wikipedians in the discussion pages that accompany each Wikipedia
article. In these pages, they included 216 links to SEP (representing roughly 10 percent of the links from
Wikipedia to SEP). Foremost among the discussions in which SEP figured was the one that accompanied
the Wikipedia Atheism entry. SEP was cited 18 times over the course of an extensive discussion that
overshadows the 10,000-word entry itself. The discussion runs to dozens of archived pages some of which
were well over 20,000 words of back and forth among Wikipedians. In the course of all this “talk” about
the article, J.J.C. Smart’s SEP entry Atheism and Agnosticism came up repeatedly.
The first of these comes in 2004, within a year of the publication of Smart’s SEP entry. It followed
references that had been made in Wikipedia’s Atheism discussion page to Encyclopaedia Britannica on the
topic by one discussant only to have them dismissed by another (“[EB’s] watered down and not
intellectually as rigorous as actual scholarly primary writings”). At that point, Adraeus, who worked with
the WikiProject Philosophy group, introduced a definition of atheism from the Oxford English Dictionary,
complete with the accompanying citation from 1587, to which Adraeus adds how he or she “spoke with
84-year-old Dr. John Smart of Philosophy who wrote the Atheism-Agnosticism entry in the Stanford
Encyclopedia of Philosophy” (as Smart’s entry is accompanied, after all, by links to his e-mail and
telephone number on his home page at Monash University). Adraeus spoke to Smart, who is himself the
15
The third footnote speaks to the ongoing challenge of online access, for when I reached the final draft of
this paper, the Google Book Search link in the Aristotle footnote no longer worked, as the book had
been removed from the web, most likely for reasons of copyright.
16
To take the next three Wikipedia articles, after Aristotle, in Table 1, Truth had four of its 71 footnotes
leading to SEP, Kant had one out of 78 leading to SEP, and Causality had none out of 21 footnotes.

1278 John Willinsky International Journal of Communication 2(2008)
subject of a Wikipedia entry, on whether “the weak definition of atheism is the product of the freethought
movement.” Smart responded, Adraeus reports, “that at the time of writing he was unaware of the
distinctions” (presumably between weak and strong definitions).
17
Six months later, another discussant, in objecting to someone deleting what was said to be a
poorly written and clumsy section, responds emphatically that “the removed text was A DIRECT QUOTE
from Stanford Encyclopedia of Philosophy” (with a link to Smart’s SEP entry Atheism and Agnosticism).
Another Wikipedian responds that the SEP cite was a copyright violation (having perhaps yet to learn
about “fair use”), while yet another points out that “SEP entries are written from the perspective of the
authoring philosopher” while “our job as Wikipedia editors is to use whatever resources available to us to
provide objective contributions to our encyclopedia.”
18
This discussant does allow that “you can, however,
quote SEP entries if you provide citation.”
19
Two years later in 2007, as the discussion on Atheism continued, Wikipedian Brian0918 was led
to exclaim in exasperation, “This is getting ridiculous, repeating the same arguments over and over.” To
settle things, Brian0918 reintroduces “one of the most well-known atheist philosophers, and his entry in
the Stanford Encyclopedia of Philosophy (comprehensive as expected).” Smart is then cited for his
statement that “atheism means the negation of theism, the denial of the existence of God.”
20
The point is
quickly disputed by Rrhain, who declares Smart wrong in the face of what atheists actually hold “based on
surveys,” namely that they do not believe in God, rather than that God does not exist. It is a critical
intellectual point in the philosophy of language, as Rrhain goes on to put it: “It really doesn't matter what
philosophers think atheists think. Only atheists can tell you what their philosophy is.” And, in fact, as an
earlier discussant noted, Smart opens Atheism and Agnosticism in SEP with a reference to Wittgenstein
(on family resemblances between atheism and agnosticism), and Wittgenstein would have been likely to
agree with Rrhain’s point, given his belief that the meaning of words arises from their use.
21
It is not,
then, that referencing SEP settles a debated point, but that it brings a point of reference that further
focuses the discussion, making the point of disagreement that much clearer, which from an educational
perspective is to do a lot.
17
http://en.Wikipedia.org/wiki/Talk:Atheism/Archive_14. In reviewing this paper, SEP editor Edward Zalta
commented: “On our view, the so-called "weak" definition of atheism ("not believing that God exists")
doesn't distinguish an atheist from an agnostic (who both fails to believe that God exists and fails to
believe that God doesn't exist). The so-called "strong" definition of atheism ("believing that God does
not exists") is really the only proper definition. There may be controversy about this in circles outside of
philosophy, but most philosophers appreciate this point” (Personal communication).
18
SEP’s policy: “Entries should be objective/neutral analyses/surveys that offer a broad perspective of the
topic rather than advocate a point of view . . . Encyclopedia entries should . . . strive for balance by
presenting the important arguments that have been put forward on both sides of an issue.
http://plato.stanford.edu/guidelines.html
19
http://en.Wikipedia.org/wiki/Talk:Atheism/Archive_22
20
http://en.Wikipedia.org/wiki/Talk:Atheism/Archive_33
21
http://en.Wikipedia.org/wiki/Talk:Atheism/Archive_27. Also see Wittgenstein (2001).

International Journal of Communication 2 (2008) Socrates Back on the Street 1279
As a final example of SEP’s role on Wikipedia discussion pages, it figured in the intense
philosophical discussion around the Wikipedia entry for Bobby Jindal, current governor of the state of
Louisiana. The question is how best to represent and reference Jindal’s stance on abortions. Jindal is, in
his own words, “100 percent against abortion, no exceptions,” and the question for Wikipedians working
on his entry is how to represent this stance in relation to its consequences for women’s health. The
discussion deals at some length with the teachings of the Catholic Church, of which Jindal is a member.
22
At one point in this discussion, Daniel Zimmerman states that Jindal could support an abortion if
it was the result of a “double effect,” in which the abortion took place as a result of another vital, life-
saving procedure: “Here is another source,” he goes on to write, “that describes the double effect and
shows that under Catholic law, direct abortions are prohibited even if it is to save the life of the woman.”
23
He then offers as a “verifiable source,” a quotation from Alison McIntyre’s SEP entry Doctrine of Double
Effect that constitutes a hypothetical example of the doctrine in which a doctor who opposes abortion
nonetheless operates on a pregnant woman with cancer, which he knows will result in the loss of the
fetus.
24
Ferrylodge then objects, based on his own reading of the McIntyre entry. Ferrylodge notes how
McIntyre observes that hypothetical examples have limits in their applicability: “It is hard to find a
principled ground” McIntyre writes, “for drawing this distinction [between intended and unintended
abortion in the case of the doctor and woman with cancer] that could serve as a guide to moral
judgment.”
25
SEP becomes, in this case, a guide to the doctrine’s moral distinctions as well as to the
challenges that can be raised in response to how the doctrine is presented. Still, the discussion continues
between the two for another month, before it is finally resolved with the article’s careful documentation of
Jindal’s stance on abortion-without-exception: “Thanks for working with me,” Daniel Zimmerman adds on
the discussion page in response to Ferrylodge’s final suggestions about providing more bibliographic
information in the footnotes, “to find a good solution to make this a better article.”
26
R
EADERS
’
U
SE OF
W
IKIPEDIA
’
S
SEP
L
INKS
22
http://en.Wikipedia.org/wiki/Talk:Bobby_Jindal. Biographical entries of the living are a particularly
sensitive area for Wikipedia and its policy is “that editors must take particular care about adding
biographical material about a living person” and that there is to be “no original research” (reiterated in
the “biographies of living persons policy”), which becomes an issue when Jindal’s well-known stance is
cited without backing by, to further quote the policy, “reliable, third-party, published sources.”
http://en.Wikipedia.org/wiki/Wikipedia:Biographies_of_living_persons. On the critical incident behind
this policy, see Wikipedia Tightens the Reins (2005).
23
http://en.Wikipedia.org/wiki/Talk:Bobby_Jindal
24
http://plato.stanford.edu/entries/double-effect/
25
http://plato.stanford.edu/entries/double-effect/
26
The Jindal entry in Wikipedia on this point currently reads: “He opposes abortion without exception.
However, he does not condemn medical procedures meant to save the life of a pregnant woman that
would indirectly cause the termination of the pregnancy.” Each sentence is backed by two footnotes that
include quotations from the local media.

1280 John Willinsky International Journal of Communication 2(2008)
To begin by summarizing how Wikipedia’s readers (including Wikipedians) used Wikipedia’s links
to SEP, it appears that 1,248 of the 2,262 SEP links in Wikipedia were clicked at least once, with 20 of the
links being used more than 50 times, adding up to 13,363 clicks on SEP links over the two weeks. All of
this clicking led readers to visit 780 SEP entries, which was 83 percent of the 942 SEP entries that
Wikipedians had cited from the complete set of 1,026 entries that had been completed in SEP at that time.
Apart from SEP’s homepage, which was the most frequently cited and visited SEP page, the entry
in SEP where Wikipedia readers ended up most often was Machiavelli, which had 246 visits in the course
of this brief study (Table 3). Yet even among 216 SEP hyperlinked references to SEP that were buried
within Wikipedia’s discussion pages, 16 were clicked on by readers in the course of this study, suggesting
that Wikipedians and other readers return on occasion to this aspect of Wikipedia, a few months or years
after the point was originally made. However you look at it, Wikipedia readers took advantage of the SEP
links, doing so, one can only speculate, to see for themselves what authority Wikipedia rested on and
possibly to discover what else could be learnt about a topic, whether they chose to pursue that knowledge
or not.
Table 3. SEP pages most frequently visited from Wikipedia over a two-week period,
with number of links to the SEP pages found in Wikipedia.
SEP Page Visits Links
Stanford Encyclopedia of Philosophy 496 72
Machiavelli 243 4
Definition of Art 103 1
Ethics 100 1
Philosophy of Economics 99 4
Critical Theory 98 5
Propositions 88 1
Respect 87 5
Principia Mathematica 85 7
Existentialism 80 4

International Journal of Communication 2 (2008) Socrates Back on the Street 1281
SEP’s Machiavelli entry is cited in Wikipedia’s Machiavelli article twice and once each in its articles
Politics and Virtù.
27
In the case of Virtù, a short 80-word Wikipedia article describing the concept that
Machiavelli used to cover those qualities needed to advance the state (not always virtuous by any means),
the link to SEP leads directly to the relevant section of the SEP Machiavelli entry, where Carey Nederman
explains the concept in relation to Machiavelli’s concepts of power and fortune. The Wikipedia article
Politics, in addition to citing the SEP Machiavelli entry, also cites SEP entries on authority, Confucius,
Plato, Aristotle, Locke, Mill, and Marx. When it comes to discussing Machiavelli, the Wikipedia article
Politics includes a quotation from the opening of Nederman’s SEP entry on a clearly intriguing point about
the man’s stance and reputation: “Machiavelli did not invent ‘Machiavellianism’ and may not even have
been a ‘Machiavellian’ in the sense often ascribed to him.” Six readers clicked on the footnote for this
quotation taking them to Nederman’s SEP Machiavelli entry. The use of SEP in the footnotes offers readers
what is still a relatively rare Wikipedia experience of being able to immediately follow up on an obviously
controversial point with a current scholarly consideration of the issue.
28
As to why the links to the
Machiavelli entry in SEP tops the list of links that readers pursued, it may seem obvious enough in these
desperate political times when state interests seem to overrule public concerns. Not the least of it was the
June 10, 2008 release of Paul Alexander’s book Machiavelli’s Shadow: The Rise and Fall of Karl Rove
(2008).
The third most frequently visited SEP page by Wikipedia readers in this study was Thomas
Adajian’s entry, The Definition of Art. All of the readers who came to this SEP entry came to it from an
External Link in the Art article that read “The Definition of Art
entry in the Stanford Encyclopedia of
Philosophy by Thomas Adajian.” This model of reference clarity was not the only source provided for
readers interested in the definition of art. Under Further Reading, Wikipedia provided a small boxed-off
section that reads “Look up Art in the Wiktionary, the free dictionary.” The Wiktionary definition of art,
begins with “Human effort to imitate, supplement, alter, or counteract the work of nature.” The Wikipedia
article Art also seeks to define art, which it follows with a disclaimer around definitions: “Generally art is a
(product of) human activity, made with the intention of stimulating the human senses as well as the
human mind; by transmitting emotions and/or ideas. Beyond this description, there is no general agreed-
upon definition of art.” Still, readers have on reaching the end of the Wikipedia reached for further insight
into what defines art. The other entries under External Links seem to lack the focus of the SEP one: “Art
and Play from the Dictionary of the History of ideas” seems narrow by comparison, and it is not
immediately apparent what an “In-depth directory of art” is, to take two other of the seven entries.
27
At least one reader expressed appreciation for the SEP link in the discussion page for the Wikipedia
Machiavelli article, after much discussion of the poor quality of writing in the entry: “I am glad the
Stanford text/link is here, at the bottom — the best I've seen in a while . . . Now, that's a well-written
text on M.!!! ;) :)” http://en.Wikipedia.org/wiki/Talk:Niccol%C3%B2_Machiavelli
28
It should be noted that readers did not follow up on a second quotation from Nederman’s SEP entry on
Machiavelli in the Wikipedia Politics article: “For Machiavelli, there is no moral basis on which to judge
the difference between legitimate and illegitimate uses of power.”

1282 John Willinsky International Journal of Communication 2(2008)
In the SEP entry The Definition of Art, Adajian does not offer up an easy answer for art collectors
or students. It begins, “the definition of art is controversial in contemporary philosophy” and concludes
with a quotation by Kendall Walton stating that “it is not at all clear that these words —‘What is art?’ —
expresses anything like a single question.” Of course, we do not know how far readers went with this
6,000-word SEP entry, after arriving there via Wikipedia. But if they only read the first sentence and
looked away, they would have had a taste of what philosophy makes of the world, and realized that what
it makes is not some definitive and certain form of knowledge to which all should then adhere. The lesson
here is not that philosophers hold to a number of definitions, but that the very act of defining art is
controversial within philosophy. It is the briefest but no less important glimpse into the nature of
knowledge in an academic sense. And to suggest that 50 people a week, somewhere in the world, might
just catch hold of that glimpse in the course of their day seems promising from an educational
perspective.
A final instance of relatively frequent SEP use by Wikipedia readers comes with Wikipedia’s René
Descartes article, which led to 229 visits to SEP. One thing that distinguishes the SEP links in this entry is
how well organized and laid out they are, forming a bridge from the Wikipedia entry to the wealth of
materials on Descartes in SEP (Figure 1). The Wikipedia’s External Links for Descartes begin with a set of
“General” entries, which include Descartes’ works online, followed by a second set grouped under Stanford
Encyclopedia of Philosophy. Each link leads to a substantial essay by a different philosopher, not least of
which is the entry Descartes and the Pineal Gland (the seat of the soul and the organ of thought for
Descartes) by Gert-Jan Lokhorst. However, the most popular for Wikipedia readers was the SEP entry
Descartes’ Epistemology (97 visits), followed by Descartes’ Theory of Ideas (62 visits). It does make for a
tidy demonstration of Wikipedia readers making their way to the end of the article and pursuing the next
level of more specialized and scholarly analysis, at whatever level, from a glance through to a careful
read.

International Journal of Communication 2 (2008) Socrates Back on the Street 1283
Figure 1. SEP links in Wikipedia External Links for René Descartes.
Discussion
What I am referring to as the educational quality of Wikipedia has much to do with the
opportunity it provides to go beyond its own article in learning more about a topic and the way that
knowledge about that topic has been assembled. This learning about knowledge may take place on a
peripheral basis, that is, out of the corner of the reader’s eye, and may only be only noticed after reading
about Wikipedia in another setting. Controversies about Wikipedia’s reliability and the increasing emphasis
among Wikipedians on verifiability and documentation only further this thinking about the nature of
knowledge and how this reliability is established through stringing together sources and connections.
29
What is exposed with Wikipedia is the degree to which knowledge is the product of learning in the sense of
the editing, discussion, history that surrounds every article.
This article has tried to make it clear that Wikipedia entries are not some sort of unconscious
expression of what is collectively known, of the sort that “prediction markets” use to accurately second-
guess election results (Fallis 2008, p. 1670; Sunstein, 2006). Rather, the Wikipedia entries considered
29
Wikipedia has Policy and Style Guidelines on Verifiability, Verification Methods, Citing Sources,
Referencing for Beginners, WikiProject for Citation Cleanup, and a number more.

1284 John Willinsky International Journal of Communication 2(2008)
here, at least, represent hard and concerted work, especially from the 70 or so members of the
WikiProject Philosophy group, who, while still operating anonymously, have set out for themselves goals
and tasks.
30
Still, the Wikipedian openness about their knowledge work is not without its risks. People may
still come to assume that Wikipedia represents this open and always-formative, often-contested approach
to knowledge in direct contrast to the firm and fixed knowledge produced by academics and other experts.
Thus, I find that even the favorable judgments of Wikipedia — as when Fallis finds it “sufficiently reliable”
and “more verifiable than most other information sources” (2008, p. 1667) — can miss the work’s
parallels to the tentative and formative nature of learning within both public and academic realms.
Fortunately, the increasing openness of scholarly work may make the parallels more apparent.
While Wikipedia’s use of SEP has been the focus of this study, Wikipedians are providing links to
scholarly works in far more areas than philosophy (Table 4). In the case of arXiv.org, an online repository
of over 500,000 papers in particle physics and other sciences recorded 10,000 visits from Wikipedia
readers in August of 2008, led by an interest among readers in super symmetry and the new Large
Hadron Collider.
31
Like SEP, arXIv.org provides open access to all of its content. Other research databases
being used by Wikipedia editors, such as PubMed, operate as indexes for research articles, a growing
number of which are freely available.
32
Still, the vast majority of references to scholarly materials in
Wikipedia lead to subscription or pay-per-view journals and database services.
33
Such references do
bolster the reliability and verifiability of Wikipedia articles, in principle, while doing less to extend its
educational quality.
30
http://en.wikipedia.org/wiki/Wikipedia:WikiProject_Philosophy
31
Simeon Warner, personal communication, September 8, 2008.
32
Take as an example the WikiProject devoted to Molecular and Cellular Biology whose 200 members seek
to “create and perfect articles . . . on subjects that are discussed at the primary and secondary school
level,” with the idea that “the perfect article is complete, but accessible to a secondary school student.”
In the process, one of their members has built a ProteinBoxBot, an automated script that has already
created 9,000 Wikipedia articles for mammalian genes, populating those pages not only with the
appropriate symbol, description, function, genomic location, and structure, but with a handful of
“Further Reading” references (contributing to the high number of CrossRef links in Table 4). If the bot
was able to select open access articles for the Wikipedia articles, then from my perspective it would
increase the degree to which the entry is “accessible to a secondary school student.”
33
In terms of restricted access to these resources, a discussant objects to a statement in the Wikipedia
article Machiavelli by saying “without reference to a reliable academic source, I don't think this kind of
statement should be included” which was followed by an incomplete recommendation and a request that
“someone with access to JSTOR or the like should pad this out.”
http://en.Wikipedia.org/wiki/Talk:Niccol%C3%B2_Machiavelli

International Journal of Communication 2 (2008) Socrates Back on the Street 1285
Table 4. Scholarly resources used by Wikipedia editors, with approximate number of items associated
with resource and number of links in Wikipedia to the resource.
* CrossRef’s Digital Object Identifiers (DOI) are used by journals in PubMed and ScienceDirect, leading to
overlap in these counts, as well as by Highwire Press journals, which make 2 million articles free, largely
some period after publication.
That said, efforts are under way to increase public access to research and scholarship on a
number of fronts, from federal legislation affecting government-sponsored research to university policies
mandating that faculty members archive an open access version of their work (Harnad et al., 2008). As
well, a growing number of journals are finding ways of offering open access to their content. This means
that what Wikipedians have demonstrated through their active use of SEP, in providing readers with a no-
cost means of going well beyond the Wikipedia article in learning more about a topic, will be on the
increase in the years ahead. I would encourage Wikipedians to turn to and cite open access research,
particularly in annotated footnotes that guide readers to not only to the source for the claims made but to
SCHOLARLY
SITE TYPE OF RESOURCE TOTAL
ITEMS WP LINKS
OPEN ACCESS SCHOLARLY ARTICLES
SEP Peer-reviewed philosophy encyclopedia 1,026 1,300
SSRN Repository of social science working and published
papers. 164,000 1,000
arXiv.org Repository of working and published papers in high
energy physics and other sciences 500,000 5,000
Citeseer Index of scientific papers, principally in computer
sciences 768,000 1,200
ABSTRACTS WITH SOME OPEN ACCESS
ARTICLES
RePEc Repository and index to economics published and
unpublished papers 510,000 500
PubMed Life sciences index to published literature 6,000,000 5,000
SAO/NASA ASD Smithsonian Astrophysical Observatory data
system and digital library of articles 7,200,000 7,000
SUBSCRIPTION / PAY-PER-VIEW ACCESS
JSTOR Digitized back issues of journals from across the
disciplines 2,000,000 4,500
ScienceDirect Journal articles and books from Elsevier, the
largest scientific publisher 9,000,000 2,400
CrossRef * Publishers’ identifier system for journal articles 32,000,000 181,600

1286 John Willinsky International Journal of Communication 2(2008)
what more can be explored by the reader about particular aspects or issues raised within a Wikipedia
article. To assist in this process of using open access research in Wikipedia, I set out in the previous study
I published (in the open access journal First Monday) step-by-step instructions for Wikipedians and others
to both find relevant open access research and to add a link to that research in the Wikipedia article that
makes it clear to readers that with a click there is more to be learned (2007).
34
In the case of the Stanford Encyclopedia of Philosophy, Wikipedia editors have managed to draw
on 80% of the entries in this scholarly enterprise, and readers appear to be using the these links in
considering particular aspects of a philosopher’s work, as we saw with Aristotle, as well as timeless ideas
such as what is art. If reference-book editors tend to eschew the citing of other reference works, given the
circularity of such practices, this may be a case in which it makes perfect sense.
35
SEP’s comprehensive
entries balance the substantive review article with the encyclopedia’s comprehensive entry. Wikipedia's
use of SEP increases the public’s opportunities to learn about both the nature of what we know and how
we come to know it. As such, SEP opens new doors on learning for interested Wikipedia readers, letting
them in on how philosophers talk to each other and their students, as well as affording philosophers a
chance to get back on the streets that once were so diligently haunted by Socrates.
◊◊◊
This study was ably assisted by Ahava Shira, with helpful support and advice from Anne White, Edward
Zalta, and Uri Nodelman. Funding was provided by the Pacific Press endowment at the University of British
Columbia.
34
Recently, Google Scholar made finding an open access version all the easier by indicating in its search
results which links lead directly, without subscription or credit card, to a PDF or HTML file.
35
The step avoids a common self-referential issue when reference works cite each other, which reduces
their claim to being a guide to some aspect of the world; on the Oxford English Dictionary and this
matter, see Willinsky (1994).

International Journal of Communication 2 (2008) Socrates Back on the Street 1287
References
Alexander, P. (2008). Machiavelli’s shadow: The rise and fall of Karl Rove. Modern New York:
Times/Rodale.
Aristotle. (384-322 BCE): Overview (2006). Internet encyclopedia of philosophy. Retrieved October 22,
2008, from http://www.utm.edu/research/iep/a/aristotl.htm
Aristotle. (1907). Catholic encyclopedia. New Advent. Retrieved October 22, 2008, from
http://www.newadvent.org/cathen/01713a.htm
Björk, B., Roos, A., & Lauri, M. (2008). Global annual volume of peer reviewed scholarly articles and the
share available via different Open Access options. Proceedings ELPUB2008 Conference on
Electronic Publishing, Toronto, Canada. Retrieved October 22, 2008, from
http://www.oacs.shh.fi/publications/elpub-2008.pdf
Caswell, T., Henson, S., Jensen, M., & Wiley, D. (2008). Open Educational Resources: Enabling universal
education. The International Review of Research in Open and Distance Learning, Vol. 9, No. 1
(2008). Retrieved October 22, 2008, from
http://www.irrodl.org/index.php/irrodl/article/viewArticle/469/1001
Cicero, M. T. (2005). Academica. Ed. James S. Reid. London: Macmillan, 1874. Gutenberg Project.
Retrieved October 22, 2008, from http://www2.cddc.vt.edu/gutenberg/1/4/9/7/14970/14970-
h/14970-h.htm#BkII_119.
Fallis, D. (2008). Toward an epistemology of Wikipedia. Journal of the American Society for Information
and Science Technology, 59(10), 1662-1674.
Harnad, S., Brody, T., Vallieres, F., Carr, L., Hitchcock, S., Gingras, Y., Oppenheim, C., Hajjem, C., & Hilf, E.
(2008). The access/impact problem and the green and gold roads to open access: An update.
Serials review, 34(1), 36-40. Retrieved October 22, 2008, from
http://eprints.ecs.soton.ac.uk/15852/
Hessel, E. (2006, September 19). Shillipedia. Forbes.com. Retrieved October 22, 2008, from
http://www.forbes.com/business/forbes/2006/0619/056.html
Jaschik, S. (2007, January 26). A stand against Wikipedia. Inside Higher Ed. Retrieved October 22, 2008,
from http://www.insidehighered.com/news/2007/01/26/wiki
Keen, A. (2007). The cult of the amateur: How blogs, MySpace, YouTube, and the rest of today's user-
generated media are destroying our economy, our culture, and our values. New York: Doubleday.

1288 John Willinsky International Journal of Communication 2(2008)
Kollock, P. (1999). The economics of online cooperation: Gifts and public goods in cyberspace. In Eds. M.
A. Smith & P, Kollock, Communities in Cyberspace (pp. 220- 239). London: Routledge.
Perry, J., & Zalta, E.N. (1997). Why philosophy needs a “dynamic” encyclopedia. Unpublished paper.
Center for the Study of Language and Information, Stanford University. Retrieved October 22,
2008, from http://plato.stanford.edu/pubs/why.html
Rainie, L., & Tancer, B. (2007). Wikipedia users. Unpublished Data Memo. Washington: Pew Internet &
American Life Project. Retrieved October 22, 2008, from
http://www.pewinternet.org/PPF/r/212/report_display.asp
Read, B. (2006). “Can Wikipedia ever make the grade?” Chronicle of Higher Education, volume 53,
number 1, A31-A35.
Sunstein, C.R. (2006). Infotopia: How many minds produce knowledge. New York: Oxford University
Press.
Turner, W. (1907). Aristotle. In The Catholic encyclopedia. New York: Robert Appleton.
Wikipedia Tightens the Reins. (2005, December 5). Wired. Retrieved October 22, 2008, from
http://www.wired.com/science/discoveries/news/2005/12/69759
Willinsky, J. (2007). What Open Access research can do for Wikipedia. First Monday, 12(3).
http://www.uic.edu/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/1624/1539
Willinsky, J. (2006). The access principle: The case for open access to research and scholarship.
Cambridge, MA: MIT Press. Open access version retrieved October 22, 2008
http://mitpress.mit.edu/9780262232425
Willinsky, J. (1994). Empire of words: The reign of the OED. Princeton, NJ: Princeton University Press.
Wittgenstein, L. (2001). Philosophical investigations (trans. G.E.M. Anscombe). Oxford: Blackwell.
Young, J.R. (2008, June 13). Wikipedia’s co-founder wants to make it more useful to academe, Chronicle
of Higher Education, 54(40), A18.