Content uploaded by Lisa Larrimore Ouellette
Author content
All content in this area was uploaded by Lisa Larrimore Ouellette on Dec 26, 2023
Content may be subject to copyright.
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
65
PATENT EXPERIMENTALISM
Lisa Larrimore Ouellette*
ANY scholars have wrestled with what I call the “first-order
question” in patent law: What policies should we adopt to promote
innovation? This Article grapples with the second-order question: What
policies should we adopt to promote innovation about promoting
innovation? I argue that empirical progress in patent law depends on
greater policy diversity (rather than the current emphasis on uniformity),
but unconstrained “laboratories of experimentation” are suboptimal due to
the spillovers from local policies. Instead, patent policy makers should
adopt a third way between uniformity and local control: centralized
promotion of policy variation. The optimal approach to such policy
experimentation depends on the context. Randomized policy experiments
should be used more often, both in the field (for example, testing prizes in a
random selection of pharmaceutical classes) and in the lab (for example,
testing how varying disclosure affects performance in implementing
software patents). But many nuanced, dynamic issues—such as the patent-
eligibility of new technologies in heterogeneous jurisdictions—are better
approached not through fixed experiments, but rather through an adaptive
“experimentalist” governance regime. Local actors—patent examiners,
judges, or even individual countries—should be granted broad discretion to
meet centrally defined framework goals, with the requirement of defending
their decisions through robust peer review. Even where controlled
experiments are infeasible, experimentalist policies could elicit local
knowledge, generate varied observational data, and encourage more robust
theory development about the mechanisms by which innovation policies
work. This pluralistic, evidence-based approach to patent policy can be
*Assistant Professor, Stanford Law School. For valuable suggestions as this project has
evolved, thanks to David Abrams, BJ Ard, Ian Ayres, Jack Balkin, Jim Bessen, Sarah
Burstein, TJ Chiang, Rochelle Dreyfuss, Erin File, Roger Ford, Michael Frakes, Jeanne
Fromer, Jonah Gelbach, Heather Gerken, John Golden, Brad Greenberg, Paul Gugliuzza,
Daniel Hemel, Cynthia Ho, Dan Ho, Dan Kahan, Margot Kaminski, Amy Kapczynski, Jeff
Larrimore, Mark Lemley, Jim Liebman, Oskar Liivak, Yair Listokin, Greg Mandel, Daniel
Markovits, Jonathan Masur, Luke Norris, Nick Ouellette, Laura Pedraza-Fariña, David
Pozen, Arti Rai, Michael Risch, Chuck Sabel, Lea Shaver, Chris Sprigman, Melissa
Wasserman, Heidi Williams, Maggie Wittlin, Tim Wu, and participants at the 2013 IP
Scholars Conference, the Junior Scholars in IP Workshop, and workshops at Duke
University School of Law, Columbia Law School, Northwestern University School of Law,
Fordham University School of Law, and the George Washington University Law School.
M
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
66 Virginia Law Review [Vol. 101:65
guided by recent trends in personalized and evidence-based medicine, and
the resulting framework for legal experimentation has implications for
policy learning beyond patent law.
INTRODUCTION ........................................................................................ 66
I. EMPIRICAL UNCERTAINTY IN PATENT LAW ....................................... 75
A. Direct Measurements of Costs and Benefits .............................. 77
B. Case Studies, Interviews, and Surveys ....................................... 78
C. Econometric Approaches ........................................................... 80
D. The Need for Greater Policy Variation ..................................... 84
II. RANDOMIZED PATENT LAW .............................................................. 87
A. Laboratory Experiments ............................................................ 87
B. Policy Experiments .................................................................... 91
1. Randomizing Across Similar Technologies ......................... 92
2. Randomizing Across Patents ............................................... 95
3. Randomizing Across Examiners .......................................... 96
C. The Limitations of Randomization ............................................. 99
1. Randomizing Across Countries ......................................... 100
2. Randomizing Across Universities ...................................... 103
III. REGULATED PATENT FEDERALISM .................................................. 104
A. Patent Examination ................................................................. 106
B. Patent Adjudication ................................................................. 109
C. Innovation Policy ..................................................................... 113
IV. TOWARD EXPERIMENTALIST PATENT POLICY ................................. 118
A. Domestic Institutional Choice ................................................. 119
B. International Policy and TRIPS Constraints ........................... 121
C. Learning from Personalized, Evidence-Based Medicine......... 125
CONCLUSION ......................................................................................... 127
INTRODUCTION
Despite well over a century of intense interest, we lack answers to
fundamental empirical questions in patent law.1 Do patents provide a net
innovation incentive? Are other incentives—such as prizes, grants, or
1See Adam B. Jaffe, The U.S. Patent System in Transition: Policy Innovation and the
Innovation Process, 29 Res. Pol’y 531, 531 (2000); Fritz Machlup & Edith Penrose, The
Patent Controversy in the Nineteenth Century, 10 J. Econ. Hist. 1, 1–2 (1950).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 67
research tax incentives—superior?2 If innovation is funded with these
alternative mechanisms, should patents also be allowed on the resulting
inventions?3 If patents are allowed, which inventions deserve patents,
how broad should the scope of those patents be, how much disclosure
should be required, and how should those patents be examined? And
how should policies evolve with changing technologies?4
The dominant push in patent law, however, has been toward
uniformity. Interpretation of U.S. patent law has been centralized in the
U.S. Court of Appeals for the Federal Circuit,5 patent cases have begun
to be funneled to certain district court judges,6 and state law innovation
regimes are limited by preemption doctrine.7 The discretion of other
countries to experiment with innovation policy has been limited by the
Agreement on Trade-Related Aspects of Intellectual Property Rights
(“TRIPS”) and other international agreements,8 and the U.S. patent
system has been informally exported by sending Federal Circuit judges
2See generally Daniel J. Hemel & Lisa Larrimore Ouellette, Beyond the Patents-Prizes
Debate, 92 Tex. L. Rev. 303, 303 (2013) (developing a new taxonomy of innovation
incentives).
3Cf. Mark A. Lemley, Are Universities Patent Trolls?, 18 Fordham Intell. Prop. Media &
Ent. L.J. 611, 619–25 (2008) (describing the debate over university patenting).
4Cf. Yochai Benkler, The Wealth of Networks: How Social Production Transforms
Markets and Freedom 461–62 (2006) (arguing that the shift to a networked global
information economy increased the inefficiency of IP rights).
5See Rochelle Cooper Dreyfuss, In Search of Institutional Identity: The Federal Circuit
Comes of Age, 23 Berkeley Tech. L.J. 787, 788 (2008).
6See Robert Gunther & Omar Khan, Patent Pilot Program, One Year Later, N.Y. L.J., Jan.
7, 2013, at S6.
7See Bonito Boats, Inc. v. Thunder Craft Boats, Inc., 489 U.S. 141, 156–57 (1989);
Jeanne C. Fromer, The Intellectual Property Clause’s Preemptive Effect, in Intellectual
Property and the Common Law 265, 265–66 (Shyamkrishna Balganesh ed., 2013).
8Agreement on Trade-Related Aspects of Intellectual Property Rights, Apr. 15, 1994,
Marrakesh Agreement Establishing the World Trade Organization, Annex 1C, 1869
U.N.T.S. 299 [hereinafter TRIPS]; see Rotec Indus. v. Mitsubishi Corp., 215 F.3d 1246,
1253 (Fed. Cir. 2000) (“[W]e must recognize one of [TRIPS’s] declared purposes:
harmonizing worldwide patent law.”); Peter K. Yu, The International Enclosure Movement,
82 Ind. L.J. 827, 832 (2007) (“[T]he one-size-fits-all templates [in TRIPS and other]
agreements have drastically reduced the policy space available to less developed
countries.”). Administrative harmonization in the Patent Cooperation Treaty also drives
substantive harmonization. See Amy Kapczynski, Harmonization and Its Discontents: A
Case Study of TRIPS Implementation in India’s Pharmaceutical Sector, 97 Calif. L. Rev.
1571, 1625–26 (2009). And the European Union is creating a unified patent system. Unitary
Patent & Unified Patent Court, European Patent Office, http://www.epo.org/law-
practice/unitary.html (last visited July 12, 2013).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
68 Virginia Law Review [Vol. 101:65
to teach foreign patent judges9 and by training foreign patent examiners
through the U.S. Patent and Trademark Office (“PTO”).10 Over the past
thirty years, U.S. policy makers have generally strengthened and
harmonized patent protection.
This emphasis on uniformity is unfortunate in light of the vast
uncertainties about patents. Locking the world into uniformly strong
patent protection simply makes it more difficult to address these
questions because empirical progress depends on policy variation.11
Furthermore, optimal innovation policy likely varies across
heterogeneous jurisdictions.12 U.S. patent policy should focus not on
uniformity, but rather on improving innovation incentives through an
evidence-based approach that depends on policy diversity. This Article
addresses this second-order question of how to address patent law’s
uncertainties through policy experiments: How do we promote
innovation in promoting innovation?
9See Randall R. Rader, The State of Patent Litigation, 21 Fed. Cir. B.J. 331, 331 (2011)
(stating that the author was asked “to encourage the Chinese judiciary” to strengthen IP
enforcement); Lisa Larrimore Ouellette, The Federal Circuit & International Patent Law,
Written Description (Feb. 13, 2013, 1:59 PM), http://writtendescription.blogspot.com/2013/
02/international-patents.html.
10 See U.S. Patent & Trademark Office, U.S. Dep’t of Commerce, Performance and
Accountability Report: Fiscal Year 2011, at 32 (2011), http://www.uspto.gov/
about/stratplan/ar/2011/USPTOFY2011PAR.pdf (stating that the PTO’s Global
Intellectual Property Academy trained over 5500 attendees from 138 countries in the 2011
fiscal year); Kapczynski, supra note 8, at 1622–27; see also Tom Fox, Talking Leadership
with the Patent and Trademark Office’s Teresa Stanek Rea, Wash. Post (July 11, 2013),
http://www.washingtonpost.com/blogs/on-leadership/wp/2013/07/11/talking-leadership-
with-the-patent-and-trademark-offices-teresa-stanek-rea (“[W]e’re hoping that other
countries will also modify their substantive patent laws to more closely align them
with . . . our best practices.”).
11 Cf. David S. Abrams, Did TRIPS Spur Innovation? An Analysis of Patent Duration and
Incentives to Innovate, 157 U. Pa. L. Rev. 1613, 1624 (2009) (“Empirical research on the
incentive effects of patent duration is quite difficult due to the scarcity of policy variation.”).
12 For example, corruption levels vary between countries, see Corruption Perceptions
Index 2011, Transparency Int’l, http://www.transparency.org/cpi2012/results (last visited
July 12, 2013), and incentives that depend on government decision makers acting in the
public interest (such as grants and prizes) seem less likely to be effective in high-corruption
countries. Optimal policy also likely varies across technologies, as numerous scholars have
argued. See Dan L. Burk & Mark A. Lemley, Policy Levers in Patent Law, 89 Va. L. Rev.
1575, 1577 (2003); Michael W. Carroll, One for All: The Problem of Uniformity Cost in
Intellectual Property Law, 55 Am. U. L. Rev. 845, 846–47 (2006); Benjamin N. Roin, The
Case for Tailoring Patent Awards Based on Time-to-Market, 61 UCLA L. Rev. 672, 676
(2014).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 69
The role of so-called laboratories of experimentation has long been
viewed as a benefit of federalism,13 and yet central coordination of
patent law is valued due to business costs and the externalities of
innovation—jurisdictions do not internalize all the benefits of their
innovation laws. But the debate between centralized uniformity and
local control overlooks a third option: promotion of policy variation by
central planners. Recent scholarship has articulated clearer visions for
these systematic experiments, ranging from policy randomization14 to
“experimentalist” systems in which local experimentation is guided by
central goals and robust feedback.15 And yet there has been little
discussion of when each approach to policy variation should be used.16
The patent system is a useful lens for analyzing these different
approaches because the special challenges of patent experimentation
magnify their strengths and weaknesses. First, measuring success is
hard. The welfare benefit of innovation is difficult to quantify, leading to
the substitution of imperfect proxies.17 Second, policy uncertainty is
particularly salient in patent law, where the relevant decisions are often
made by multinational companies with long time horizons.18 Where the
costs of nonuniformity are high, experimentation may not be cost-
justified. And third, patents have pronounced transjurisdictional effects.
Local patent laws have global effects on both knowledge users (who
benefit from the resulting knowledge even if they do not share the costs
of supra-competitive pricing) and knowledge producers (who benefit
13 Cf. Akhil Reed Amar, Five Views of Federalism: “Converse-1983” in Context, 47
Vand. L. Rev. 1229, 1233–35 (1994) (dating the metaphor to New State Ice Co. v. Liebmann,
285 U.S. 262, 311 (1932) (Brandeis, J., dissenting)).
14 See Michael Abramowicz et al., Randomizing Law, 159 U. Pa. L. Rev. 929, 933 (2011);
infra notes 114–16.
15 See, e.g., Michael C. Dorf & Charles F. Sabel, A Constitution of Democratic
Experimentalism, 98 Colum. L. Rev. 267, 287–88 (1998); Charles F. Sabel & William H.
Simon, Minimalism and Experimentalism in the Administrative State, 100 Geo. L.J. 53, 55
(2011); see also infra notes 164–70.
16 See Abramowicz et al., supra note 14, at 931 n.2 (criticizing “the most prominent”
account of policy experimentation, Dorf & Sabel, supra note 15, for “mentioning
randomization . . . only once,” but also not comparing the two approaches themselves).
17 Metrics become even more challenging when we consider non-efficiency values, and
quantitative studies will need to be complemented by qualitative and theoretical work.
18 But note that research is always unpredictable; for example, about eighty percent of
drugs that begin clinical trials will fail. See F.M. Scherer, The Pharmaceutical Industry—
Prices and Progress, 351 New Eng. J. Med. 927, 928 (2004). Also, disuniformity is different
from uncertainty.
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
70 Virginia Law Review [Vol. 101:65
from foreign patents under the entrenched norm of nondiscrimination19).
These effects will dampen the impact of a policy change based on the
size of that jurisdiction’s market, although the problem is not
insurmountable because innovators still focus more on their home
market even in today’s global economy.20
I will analyze how existing empirical work has addressed these
challenges in Part I. To illustrate the diversity of methodological
approaches and the extent of empirical uncertainty, I will focus on one
of the most important (and, to outsiders, surprising) open questions:
Do patents even promote innovation? Although various studies, both
qualitative and quantitative, have made progress on this question, none
provides a clear answer. But I will also argue that the heated debate
over this question is misdirected: The key question is not whether the
current patent system promotes innovation on average, holding all
other innovation-related policies constant. Rather, the question for
patent scholars should be: How can we improve patent laws in light of
the complex array of innovation laws and institutions with which they
interact (including research and development (“R&D”) tax credits,
government grants and prizes, regulations, contract law, immigration,
education, antitrust, social norms, and more) to increase social welfare
(including not only innovation and efficiency, but also non-efficiency
values such as equity, autonomy, and privacy)?21 Progress on this
question can be made through both large, substantive changes (such as
altering the strength of patent protection) and changes at the margins
(such as improving the efficiency of patent examination and
adjudication), and this Article suggests both kinds of experiments. But
we will make more empirical progress by studying not whether patents
19 Since the nineteenth century, many countries have had to offer foreigners the same
patent protections as their own nationals. Paris Convention for the Protection of Industrial
Property of March 20, 1883 art. 2, July 14, 1967, 21 U.S.T. 1583, 1631, 828 U.N.T.S. 305,
313. TRIPS incorporated and reiterated this requirement. TRIPS, supra note 8, arts. 2–4.
20 See infra notes 73–77 and accompanying text.
21 Despite the dominant focus on how patents can best incentivize innovation, a number of
scholars have argued for a broader conception of welfare. See, e.g., Madhavi Sunder, From
Goods to the Good Life: Intellectual Property and Global Justice 31–32 (2012). Other
scholars have recognized that innovation is affected by a host of laws and institutions. See,
e.g., Kauffman Task Force on Law, Innovation & Growth, Rules for Growth: Promoting
Innovation and Growth Through Legal Reform 4–6 (2011). But the interaction of patents
with other innovation policies has been insufficiently explored. See, e.g., Hemel & Ouellette,
supra note 2, at 306–07 (describing the scant literature comparing R&D tax credits and
patents as innovation incentives).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 71
currently work on average, but why they do (or do not) work in more
specific contexts, as well as what it actually means for patent laws to
increase welfare.
Before turning to this Article’s core arguments about how to make
empirical progress with different issues in patent law in Parts II and III, I
note that most of this analysis brackets questions of political feasibility.
Before worrying about how political economy constraints might impede
optimal innovation policy, it is first necessary to determine an optimal
innovation policy. But I take as a given the basic global legal structure,
including national sovereignty and U.S. constitutional law. The intended
audience for and implementer of most of these ideas is a well-motivated
member of Congress. Some of my suggestions may be more likely than
others, but they are at least all policies that Congress could realistically
pursue.
Part II will describe how randomized experiments should be used to
address certain empirical questions in patent law. Section II.A will argue
that laboratory experiments should be more widely used for initial tests
of radical ideas. Section II.B will then propose that randomization of
actual patent policies in the field should be performed to compare well-
defined policy options whose adoption depends on their average effect
on something observable, as long as the policies can be randomly
assigned to different units such that there are minimal spillovers between
groups. For example, one could randomly select a sample of
pharmaceutical categories to be eligible for a trial of the opt-in Health
Impact Fund medical prize scheme, and then compare R&D spending,
health impacts, and costs.22 And proposals for new rights, such as
“commercialization patents,”23 could initially be tested on a randomized
basis. One could also randomly assign patent examiners to different
incentive structures and then audit a sample of their cases.
However, as Section II.C will explain, randomization would be
ineffective for many nuanced, dynamic questions of patent law. For
example, by the time a random trial of the patentability of human genes
across countries24 reached a significant conclusion about the average
22 See infra note 127 and accompanying text.
23 Ted Sichelman, Commercializing Patents, 62 Stan. L. Rev. 341, 345 (2010).
24 This assumes that a sensible category of unpatentable genes could be defined. Compare
Ass’n for Molecular Pathology v. Myriad Genetics, Inc., 133 S. Ct. 2107, 2111 (2013)
(distinguishing between “naturally occurring” DNA and “synthetically created” cDNA), with
Noam Prywes, The Supreme Court’s Sketchy Science, Slate (June 14, 2013, 12:15 PM),
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
72 Virginia Law Review [Vol. 101:65
effect on genetic research (assuming transjurisdictional effects could be
accounted for), the state of genetic research would have changed so that
the result would have low “external validity,” that is, it could not be
generalized to conditions outside the experiment. Innovation conditions
may also be sufficiently heterogeneous (both in terms of inherent
differences and differences in non-patent innovation policies) that the
average treatment effect has little bearing on an individual country’s
optimal policy. The required trial duration would also make such
experiments very costly, both financially and politically.
Part III will then propose that the regulated local autonomy of
“experimentalist” governance—whether exercised by patent
examiners, judges, or individual states or countries—may be a more
effective framework for some of these more nuanced issues. Unlike
randomized field or laboratory experiments, which are discrete projects
to identify the best of a few static policy options in a particular context,
experimentalism is an adaptive, evidence-based method of governance.
Actors at all levels of the patent system should make choices based on
a framework goal of promoting innovation (and with awareness of
non-patent innovation policies), but local actors’ autonomy should be
constrained through robust peer review, evaluation by collaboratively
defined metrics, and a duty to explain their choices based on existing
theory and evidence.
While the externalities of innovation make international coordination
desirable (to prevent freeriding), there is no good reason to coordinate
around patent law rather than R&D tax credits, grants, prizes, or some
combination of incentives. Allowing countries to experiment beyond the
constraints of TRIPS—and requiring deviations to be justified by pro-
innovation rationales supported by theory and available data—may lead
to further improvements in our understanding of innovation policy.25
The European Commission has taken steps in this experimentalist
direction with its Open Method of Coordination, which combines an
objective of R&D investment of three percent of GDP with peer reviews
http://www.slate.com/articles/health_and_science/science/2013/06/supreme_court_patent_case
_science_the_justices_misunderstand_molecular_biology.html (criticizing this distinction).
25 Cf. Graeme B. Dinwoodie & Rochelle C. Dreyfuss, A Neofederalist Vision of TRIPS:
The Resilience of the International Intellectual Property Regime 14–16 (2012) (arguing that
TRIPS should not be viewed as a supranational code); Kapczynski, supra note 8, at 1574–75
(describing strategies to counteract the harmonizing effect of TRIPS).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 73
and country comparisons along various metrics.26 Although imperfect in
implementation, this example shows that international experimentalism
may be feasible. Experimentalism might also be valuable at smaller
scales: Patent litigation might be improved by encouraging district
courts to use their discretion to experiment and by dividing appellate
patent jurisdiction in a way that avoids forum shopping (along with
congressional encouragement to create circuit splits where an alternative
rule appears promising),27 and applying experimentalist principles to
patent examination might lead to improved methods of locating “prior
art” (the earlier references against which an application is compared,
which are critical for evaluating patentability).
To be sure, when randomized field experiments are feasible, their
evidence is more credible than that resulting from experimentalism, and
in some cases the best approach may be to apply experimentalist
techniques, such as peer review, in a randomized fashion. But where
randomization is infeasible, experimentalism alone can still lead to
empirical progress. It will produce varied observational data to test
theoretical and structural models, it may be possible to analyze some
policies as so-called natural experiments,28 and the experimentalist
structure will encourage richer development of qualitative evidence such
as case studies and interviews.29 And experimentalism’s most important
strengths may be its ability to improve the inputs (what policies should
be tested?) and the metrics (what does it mean for a policy to “work”?)
for innovating with innovation policy. Given the complexity of
innovation policy space, the limited existing data from varied policies,
and the differing views on what increases welfare, the best approach
26 See infra notes 198–210 and accompanying text.
27 Cf. Jeanne C. Fromer, District Courts as Patent Laboratories, 1 U.C. Irvine L. Rev. 307,
322 (2011) (arguing that district courts “can be improved so as to turn them into reliable
patent laboratories”); Craig Allen Nard & John F. Duffy, Rethinking Patent Law’s
Uniformity Principle, 101 Nw. U. L. Rev. 1619, 1625 (2007) (“[I]n addition to the Federal
Circuit, at least one extant circuit court should be allowed to hear district court appeals
relating to patent law.”); Xuan-Thao Nguyen, Dynamic Federalism and Patent Law Reform,
85 Ind. L.J. 449, 451–52 (2010) (describing “local patent reform” through the district
courts).
28 See, e.g., Abrams, supra note 11, at 1614; Josh Lerner, The Empirical Impact of
Intellectual Property Rights on Innovation: Puzzles and Clues, 99 Am. Econ. Rev. 343, 343
(2009).
29 For an overview of how rigorous qualitative work can be conducted using the same
logic of causal inference that underlies quantitative methods, see Gary King et al., Designing
Social Inquiry: Scientific Inference in Qualitative Research 76, 82–84 (1994).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
74 Virginia Law Review [Vol. 101:65
when randomization is infeasible may be to encourage better theory
development and greater policy variation through an experimentalist
method that generates local buy-in, elicits local information, and
promotes local values. Sometimes the best “experiment” is simply to
explore what policy options are even possible.30
In sum, I argue for a pluralistic approach, in which the optimal
approach to policy variation depends on the context. Randomized
laboratory experiments are best for inexpensive initial tests of new ideas.
Randomized field experiments should be used when evaluating well-
defined policy options with measurable outcomes that are expected to be
relatively stable across time and across jurisdictions, and when
spillovers between assignments can be minimized. And experimentalism
is valuable for generating buy-in from local units, for promoting local
innovation with policy design, for eliciting local knowledge about the
applicability of different policies to heterogeneous conditions, and for
encouraging more robust theory development about the mechanisms by
which innovation policies work.
Of course, implementing this approach to policy experimentation will
not be easy, and Part IV will explore considerations such as institutional
choice and international political economy. Some of this Article’s
suggestions would be relatively straightforward to implement; for
example, the PTO has been open to testing new policies, so PTO
leadership might be interested in more systematically reducing the
administrative and error costs of examination through the approach
advocated here. But for many of the more substantive policy changes, it
is less clear which institutions should coordinate these experiments, or
how policy makers’ natural risk aversion could be overcome. I do not
claim to have answers to these problems, but I suggest that
experimentalism itself may be a promising structure for choosing among
experimental approaches to different issues. And policy makers
implementing a more evidence-based patent regime might also borrow
from the trend toward personalized and evidence-based medicine, which
blends “graded” evidence with local clinical expertise and a focus on
30 Cf. Chris Ansell, What Is a “Democratic Experiment”?, 9 Contemp. Pragmatism 159,
164, 172 (2012) (arguing that “if we want to treat experimentalism as a particular political
and epistemic methodology, but do not want to be held to the gold standard of randomized
controlled experiments, then one alternative is the framework offered by design science,”
under which “the goal of experimentation is to identify the range of variables affecting
outcomes of interest”).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 75
understanding the mechanisms by which different treatments affect
diverse patients.
Many questions in patent law are hard to answer, but that does not
mean that we cannot make progress through this pluralistic and
pragmatic approach. Although some of the biggest questions will be
difficult to resolve even with optimally designed experiments, global
policy standardization will make it impossible to resolve even the
smallest uncertainties. Policy makers should thus embrace a new focus
on patent experimentalism.
I. EMPIRICAL UNCERTAINTY IN PATENT LAW
The benefits and costs of the patent system cannot be resolved
theoretically. Knowledge is non-rivalrous and generally non-excludable,
so it will be undersupplied absent state action such as increasing
excludability through patents.31 And it is clear that innovation responds
to incentives,32 and patents are one type of incentive. But patents have
costs: In addition to creating deadweight loss and restricting access,33
patents might overinduce innovation,34 discourage subsequent
innovation,35 raise transaction costs,36 or direct R&D away from
beneficial but unpatentable inventions.37 Other theoretical benefits of
patents are similarly ambiguous: Patents encourage disclosure, but this
disclosure may not be worth the cost;38 patents facilitate firm entry, but
they also lock in monopolies;39 patents induce investment in later-stage
31 See Joseph Stiglitz, Knowledge as a Global Public Good, in International Intellectual
Property in an Integrated World Economy 93, 93 (Frederick M. Abbott et al. eds., 2007).
32 See Liam Brunt et al., Inducement Prizes and Innovation, 60 J. Indus. Econ. 657, 657
(2012); David Popp, Induced Innovation and Energy Prices, 92 Am. Econ. Rev. 160, 161
(2002).
33 See F. Scott Kieff et al., Principles of Patent Law 57–65 (4th ed. 2008).
34 See Yoram Barzel, Optimal Timing of Innovations, 50 Rev. Econ. & Stat. 348, 348
(1968).
35 See Suzanne Scotchmer, Standing on the Shoulders of Giants: Cumulative Research and
the Patent Law, J. Econ. Persp., Winter 1991, at 29, 37–38.
36 See Michael A. Heller & Rebecca S. Eisenberg, Can Patents Deter Innovation? The
Anticommons in Biomedical Research, 280 Science 698, 698 (1998).
37 See Amy Kapczynski & Talha Syed, The Continuum of Excludability and the Limits of
Patents, 122 Yale L.J. 1900, 1942 (2013); Benjamin N. Roin, Unpatentable Drugs and the
Standards of Patentability, 87 Tex. L. Rev. 503, 504–05 (2009).
38 See Lisa Larrimore Ouellette, Do Patents Disclose Useful Information?, 25 Harv. J.L. &
Tech. 545, 587 (2012).
39 See Bronwyn H. Hall, Patents and Patent Policy, 23 Oxford Rev. Econ. Pol’y 568, 572
tbl.1 (2007).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
76 Virginia Law Review [Vol. 101:65
commercialization, but many inventions are commercialized absent
exclusive rights.40 For developing countries, patents might increase
foreign direct investment, but they might also harm domestic imitators
and decrease consumer welfare.41 The extent of these benefits and costs
cannot be resolved without empirical inquiry.42
In the 1950s, economists Edith Penrose and Fritz Machlup concluded
that the existing empirical evidence was insufficient to establish whether
patent laws enhanced social welfare.43 Professor George Priest and
Judge Richard Posner have offered similar laments.44 Professor Mark
Lemley has stated that these concerns “ring[] false today” due to an
“outpouring of empirical economic work.”45 But none of these studies
resolves whether patents have a net positive effect on innovation, much
less their net welfare effect, or whether alternative innovation incentives
such as grants, prizes, and tax credits are inferior.46 This Part briefly
40 See generally Roberto Mazzoleni & Richard R. Nelson, Economic Theories About the
Benefits and Costs of Patents, 32 J. Econ. Issues 1031, 1041 (1998) (discussing how “[a]
simple head start on commercialization” can yield high enough profits to induce
development without patents).
41 See Lee Branstetter & Kamal Saggi, Intellectual Property Rights, Foreign Direct
Investment and Industrial Development, 121 Econ. J. 1161, 1188 (2011).
42 Of course, for those who believe patents are a natural right, these utilitarian concerns are
irrelevant. See generally Adam Mossoff, Who Cares What Thomas Jefferson Thought About
Patents? Reevaluating the Patent “Privilege” in Historical Context, 92 Cornell L. Rev. 953,
982–83 (2007) (arguing that early American courts and commentators justified patents
within natural rights theory). But while “rights-based theories do influence debates about
intellectual property theory in general, the consensus among those studying the American
patent system is to focus on utilitarian approaches.” F. Scott Kieff, Property Rights and
Property Rules for Commercializing Inventions, 85 Minn. L. Rev. 697, 697–98 (2001).
43 Staff of Subcomm. on Patents, Trademarks & Copyrights of the S. Comm. on the
Judiciary, 85th Cong., An Economic Review of the Patent System, Study No. 15, at 79–80
(Comm. Print 1958) (prepared by Professor Fritz Machlup); Edith Tilton Penrose, The
Economics of the International Patent System 40 (1951) (“If national patent laws did not
exist, it would be difficult to make a conclusive case for introducing them; but the fact that
they do exist shifts the burden of proof and it is equally difficult to make a really conclusive
case for abolishing them.”).
44 Richard A. Posner, Intellectual Property: The Law and Economics Approach, 19
J. Econ. Persp. 57, 59 (2005); George L. Priest, What Economists Can Tell Lawyers About
Intellectual Property: Comment on Cheung, 8 Res. L. & Econ. 19, 21 (1986).
45 Mark A. Lemley, Property, Intellectual Property, and Free Riding, 83 Tex. L. Rev. 1031,
1065 n.137 (2005); see Dan L. Burk & Mark A. Lemley, The Patent Crisis and How the
Courts Can Solve It 38 (2009).
46 Some argue that there is “weak or no evidence that strengthening patent regimes
increases innovation.” Michele Boldrin & David K. Levine, Against Intellectual Monopoly
192 (2008); see Michele Boldrin & David K. Levine, The Case Against Patents, 27 J. Econ.
Persp. 3, 3 (2013). Others counter that research “has amply established a causal link between
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 77
reviews what has been learned about the extent to which patents
promote innovation. My goal is not to comprehensively review all
studies of patents’ welfare effects, but rather to illustrate the diverse
sources of evidence and the difficulty of reaching a clear answer on
this one question.
A. Direct Measurements of Costs and Benefits
The conceptually easiest approach to evaluating the patent system is
to directly estimate its costs and benefits. For example, Professor Josh
Lerner estimated patent legal costs in 1991 to be $1 billion (in 1991
dollars), which he called “a substantial amount relative to the $3.7
billion spent by U.S. firms on basic research in 1991.”47 Annual patent
legal costs today are closer to $5 billion for prosecution and $2.5 billion
for litigation.48 But comparing these figures with basic research
spending is misleading, both because there is no reason to exclude
applied R&D—which brings U.S. R&D spending to over $400 billion
per year49—and because it is unclear how R&D spending compares to
the value of patents. The private value of patented technology will
generally be greater than its R&D cost (or else the research would not
have been pursued), but patents are not the only means of appropriating
that private value, so the added value of a patent must be distinguished
from the value of the underlying technology.50
James Bessen and colleagues address these problems by estimating
patents’ value from renewal rates and stock-market value, and
estimating costs based on event studies of lost market value in patent
the development of intellectual property and the growth of our national economy.” Kieff,
supra note 42, at 699 n.4. Most reviews fall in the middle. See, e.g., James Bessen &
Michael J. Meurer, Patent Failure: How Judges, Bureaucrats, and Lawyers Put Innovators at
Risk 93 (2008) (stating that “evidence is inconclusive” on whether patents’ “total net
effect . . . is positive”); Nat’l Research Council of the Nat’l Acads., A Patent System for the
21st Century 2 (Stephen A. Merrill et al. eds., 2004); Bronwyn H. Hall & Dietmar Harhoff,
Recent Research on the Economics of Patents, 4 Ann. Rev. Econ. 541, 546–49 (2012).
47 Josh Lerner, Patenting in the Shadow of Competitors, 38 J.L. & Econ. 463, 470 (1995).
48 Hemel & Ouellette, supra note 2, at 365.
49 See Mark Boroush, Nat’l Sci. Found., InfoBrief: U.S. R&D Spending Resumes Growth
in 2010 and 2011 but Still Lags Behind the Pace of Expansion of the National Economy 2
tbl.1 (2013).
50 See Ashish Arora et al., R&D and the Patent Premium, 26 Int’l J. Indus. Org. 1153,
1173 (2008) (concluding from firm-level surveys and R&D data that patented inventions
earn a fifty percent premium).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
78 Virginia Law Review [Vol. 101:65
disputes.51 They conclude that outside pharmaceuticals and chemicals,
total litigation costs exceed total patent rents for U.S. public firms—
probably by a factor of two by the late 2000s.52 While these kinds of
advances in measurement are critical to empirical progress, and evidence
of rapidly increasing litigation costs should give patent proponents
pause, Bessen’s results do not prove that patents do not promote
innovation: The most innovative firms might have significant benefits,
even if the net effect for public firms is negative.53 These studies also do
not show that patents are not welfare enhancing: The social costs and
benefits of patents are larger than their effects on public firms. And most
importantly, studying the effects of patents alone does not reveal
whether we would be better off without patents.
B. Case Studies, Interviews, and Surveys
If weighing all the costs and benefits is too difficult, perhaps one can
examine how patents actually worked in specific cases. As exemplified
by This American Life’s much-discussed When Patents Attack,54 patent
debates abound with narrative evidence: cursory anecdotes or detailed
case studies in which patents seemed to lead to a good or bad result.55 Of
course, careful qualitative case studies can produce valuable data to
maximize leverage in testing and refining a theory or to generate new
51 Bessen & Meurer, supra note 46, at 95–146; James Bessen et al., The Costs and Benefits
of United States Patents 3 (Bos. Univ. Sch. of Law, Working Paper No. 13-24, 2014),
available at http://ssrn.com/abstract=2278255; see also Ariel Pakes & Mark Schankerman,
The Rate of Obsolescence of Patents, Research Gestation Lags, and the Private Rate of
Return to Research Resources, in R&D, Patents, and Productivity 73 (Zvi Griliches ed.,
1984) (pioneering the renewal approach). Of course, renewal data does not allow one to
directly measure the most valuable patents, and stock market approaches omit private firms.
See Glynn S. Lunney, Jr., On the Continuing Misuse of Event Studies: The Example of
Bessen and Meurer, 16 J. Intell. Prop. L. 35, 50–51 (2008) (criticizing the use of event
studies).
52 Bessen & Meurer, supra note 46, at 141; Bessen et al., supra note 51, at 5 fig.1.
53 But see Bessen & Meurer, supra note 46, at 124 (“[T]he more a firm spends on R&D, all
else being equal, the more likely it is to be sued for infringement.”).
54 This American Life: When Patents Attack . . . Part Two!, Chicago Public Radio (May
31, 2013), available at http://www.thisamericanlife.org/radio-archives/episode/496/when-
patents-attack-part-two.
55 For example, Adam B. Jaffe & Josh Lerner, Innovation and Its Discontents: How Our
Broken Patent System Is Endangering Innovation and Progress, and What to Do About It
(2004), primarily relies on anecdotes of what it views as abusive practices.
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 79
hypotheses.56 But the complexity of innovation makes patent case
studies challenging.57 Case studies have shown that patents are not
always necessary for innovation58 but also that private-ordering
solutions can prevent patents from becoming a problem.59 And none of
these studies prove that having more or fewer patents would have caused
more or less innovation.
Larger surveys can help overcome concerns about generalizing from
small numbers of interviewees. In the 1980s and 1990s, U.S.
manufacturers reported that patents were generally not essential to the
introduction of products other than pharmaceuticals and chemicals, and
were less effective than other appropriation means such as secrecy and
first-mover advantage.60 But these surveys show that patents still play a
role in appropriation, and other surveys reveal that patents do affect
56 See King et al., supra note 29, at 45; Gary King & Eleanor Neff Powell, How Not to Lie
Without Statistics 4–5 (Aug. 22, 2008) (unpublished manuscript), available at
http://gking.harvard.edu/files/gking/files/nolie.pdf).
57 See, e.g., Joel Mokyr, Editor’s Introduction: The New Economic History and the
Industrial Revolution, in The British Industrial Revolution 1, 1 (Joel Mokyr ed., 2d ed.
1999); George Selgin & John L. Turner, Strong Steam, Weak Patents, or the Myth of Watt’s
Innovation-Blocking Monopoly, Exploded, 54 J.L. & Econ. 841, 843 (2011). For a
thoughtful discussion of the potential and limitations of innovation case studies, see Lea
Shaver, Illuminating Innovation: From Patent Racing to Patent War, 69 Wash. & Lee L.
Rev. 1891 (2012).
58 See, e.g., Kal Raustiala & Christopher Sprigman, The Knockoff Economy: How
Imitation Sparks Innovation 14 (2012) (describing how innovation has flourished without IP
in fields such as fashion, cuisine, open-source software, computer databases, and the
financial industry); Sec’y’s Advisory Comm. on Genetics, Health & Soc’y, U.S. Dep’t of
Health & Human Servs., Gene Patents and Licensing Practices and Their Impact on Patient
Access to Genetic Tests 2 (2010) (finding no case in which a patented genetic test was the first
to market).
59 Adam Mossoff, The Rise and Fall of the First American Patent Thicket: The Sewing
Machine War of the 1850s, 53 Ariz. L. Rev. 165, 209 (2011). But see Ryan L. Lampe &
Petra Moser, Do Patent Pools Encourage Innovation? Evidence from the 19th-Century
Sewing Machine Industry 6 (Nat’l Bureau of Econ. Research, Working Paper No. 15061,
2009), available at http://www.nber.org/papers/w15061 (suggesting that these patents
discouraged innovation).
60 See Richard C. Levin et al., Appropriating the Returns from Industrial Research and
Development, 1987 Brookings Papers on Econ. Activity 783, 816; Edwin Mansfield, Patents
and Innovation: An Empirical Study, 32 Mgmt. Sci. 173, 180 (1986); Wesley M. Cohen et
al., Protecting Their Intellectual Assets: Appropriability Conditions and Why U.S.
Manufacturing Firms Patent (or Not) 28 (Nat’l Bureau of Econ. Research, Working Paper
No. 7552, 2000), available at http://www.nber.org/papers/w7552.
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
80 Virginia Law Review [Vol. 101:65
firms’ investment decisions,61 and that patents help small firms raise
capital and compete against larger firms.62
Surveys and interviews are valuable—it is important to know whether
the people whom patents are supposed to be helping think the system is
working. But while people are somewhat reliable at reporting what they
have done in the past, they are less good at explaining why they made
past choices, or at predicting future choices.63 Economists thus prefer
data about how people and firms have actually behaved under different
policy regimes.
C. Econometric Approaches
To evaluate an innovation policy, one ideally would compare our
world to a counterfactual parallel universe. But given real-world
constraints, econometric approaches are generally the best tool for
teasing out causal relationships from complex innovation data. Because
the welfare increase due to innovation is hard to quantify,
econometricians use proxies such as R&D spending, patent counts
(sometimes weighted by citations), trademark registrations, or
surveys.64 Regressions on cross-country panel data have generally
found that stronger patent laws (based on various metrics) are
statistically significantly correlated with innovation proxies such as
private R&D spending.65 These studies face a potential endogeneity
61 See Robert P. Merges, Uncertainty and the Standard of Patentability, 7 High Tech. L.J.
1, 11 & n.30 (1992) (collecting sources); Roin, supra note 37, at 545–47.
62 See Stuart J.H. Graham et al., High Technology Entrepreneurs and the Patent System:
Results of the 2008 Berkeley Patent Survey, 24 Berkeley Tech. L.J. 1255, 1262 (2009);
Bronwyn H. Hall & Rosemarie Ham Ziedonis, The Patent Paradox Revisited: An Empirical
Study of Patenting in the U.S. Semiconductor Industry, 1979–1995, 32 RAND J. Econ. 101,
104 (2001); Ronald J. Mann, Do Patents Facilitate Financing in the Software Industry?, 83
Tex. L. Rev. 961, 985–86 (2005).
63 See Marianne Bertrand & Sendhil Mullainathan, Do People Mean What They Say?
Implications for Subjective Survey Data, 91 Am. Econ. Rev. 67, 68 (2001); John A. List, Do
Explicit Warnings Eliminate the Hypothetical Bias in Elicitation Procedures? Evidence from
Field Auctions for Sportscards, 91 Am. Econ. Rev. 1498, 1504 (2001).
64 See Paul H. Jensen & Elizabeth Webster, Another Look at the Relationship Between
Innovation Proxies, 48 Austl. Econ. Papers 252, 253–54 (2009).
65 See, e.g., Yongmin Chen & Thitima Puttitanun, Intellectual Property Rights and
Innovation in Developing Countries, 78 J. Dev. Econ. 474, 475–76 (2005); David M. Gould
& William C. Gruben, The Role of Intellectual Property Rights in Economic Growth, 48 J.
Econ. Dev. 323, 328 (1996); Sunil Kanwar & Robert Evenson, Does Intellectual Property
Protection Spur Technological Change?, 55 Oxford Econ. Papers 235, 238–40 (2003); Keith
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 81
problem66—increased R&D might cause stronger patent laws (such as
through lobbying by R&D-intensive industries), rather than vice
versa67—although some studies attempt to control for this effect.68
Endogeneity problems can be reduced by seeking changes that appear
not to stem from domestic lobbying, which can be analyzed as so-called
natural experiments.69 In one of the most comprehensive studies, Josh
Lerner examined policy changes in sixty nations over 150 years and
found that pro-patent changes tended to increase patenting by foreigners,
but decrease domestic patenting, both in the country undergoing the
change and in Great Britain (which had relatively stable policy).70
But these cross-country studies suffer from another problem, which is
less remarked upon but more significant: Patent law has
transjurisdictional effects. Increasing patent protection in one small
country will not noticeably increase innovation in that country if
domestic firms were already innovating for the global market71—rather,
it could decrease welfare by increasing domestic deadweight loss
E. Maskus & Mohan Penubarti, How Trade-Related Are Intellectual Property Rights?, 39
J. Int’l Econ. 227, 228–29 (1995).
66 Endogeneity problems, such as this reverse causation issue, occur when an independent
variable (here, patent strength) is not determined outside the system (that is, it is
endogenous, not exogenous), so that it is correlated with the error term. Endogeneity
problems can also result from omitted variables, measurement error, or simultaneity (when
an independent and dependent variable are jointly determined). See Jeffrey M. Wooldridge,
Econometric Analysis of Cross Section and Panel Data 54–55 (2d ed. 2010).
67 Indeed, patent protection in high-income countries is correlated with R&D spending
from five years earlier. Juan C. Ginarte & Walter G. Park, Determinants of Patent Rights: A
Cross-National Study, 26 Res. Pol’y 283, 298–99 (1997). And one study found the
correlation of patent rights with R&D intensity “disappears when lagged R&D intensity is
included in the regression.” Martin Falk, What Drives Business Research and Development
(R&D) Intensity Across Organisation for Economic Co-operation and Development (OECD)
Countries?, 38 Applied Econ. 533, 545 (2006).
68 See Chen & Puttitanun, supra note 65, at 477; Maskus & Penubarti, supra note 65, at
232. Other studies argue that endogeneity is unlikely to be a concern. See Gould & Gruben,
supra note 65, at 338; Kanwar & Evenson, supra note 65, at 242 n.17.
69 E.g., Mariko Sakakibara & Lee Branstetter, Do Stronger Patents Induce More
Innovation? Evidence from the 1988 Japanese Patent Law Reforms, 32 RAND J. Econ. 77,
80 & n.8 (2001); F.M. Scherer & Sandy Weisburst, Economic Effects of Strengthening
Pharmaceutical Patent Protection in Italy, 26 Int’l. Rev. Indus. Prop. & Copyright L. 1009,
1023 (1995).
70 Lerner, supra note 28, at 343, 347–48.
71 See Daniel Nepelski & Giuditta De Prato, Does the Patent Cooperation Treaty Work? A
Global Analysis of Patent Applications by Non-Residents 20 (Joint Research Ctr. of the
European Comm’n, Working Paper No. JRC79541, 2013), available at
http://ipts.jrc.ec.europa.eu/publications/pub.cfm?id=6222.
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
82 Virginia Law Review [Vol. 101:65
without any gain in innovation. Such studies thus reveal little about the
effects of similar policy changes in large markets such as the United
States—in other words, they measure so-called partial equilibrium
effects, not general equilibrium effects. These spillovers will mute the
effect of a patent policy change in a single jurisdiction based on the size
of that jurisdiction’s market.72 This problem is lessened because there
are barriers to trade: Inventors are more likely to exploit inventions in
their home country.73 But studies of a single country’s patent policy
change will understate the policy’s impact. In contrast, studies of non-
patent policies (such as R&D tax credits, which typically benefit
inventors only within a jurisdiction) will overstate impact if they cause
firms to move to that jurisdiction—meaning that empirical comparisons
of innovation policies may be biased against patents.
It is thus noteworthy that some studies—including the cross-country
panel data studies described earlier—have found significant effects from
increasing a country’s patent protection. In 1986, Taiwan extended the
scope of patent protection and improved patent enforcement, and “[a]n
abrupt increase [in R&D intensity] took place in 1987 and it stayed at a
high level afterwards.”74 And another study found that the introduction
of pharmaceutical patents “in nations with high levels of development,
education, and economic freedom do stimulate innovation.”75
The United States, as the world’s largest market, is least prone to this
problem of transjurisdictional spillovers, although there have been
relatively few policy changes to form the basis of a natural experiment.76
72 For example, if there were no trade barriers or economy-of-scale effects, a change that
would have caused a twenty percent R&D increase in an isolated market would, if enacted in
a country with ten percent of the global market, only cause a two percent increase in that
country (and in every other country).
73 See Jonathan Eaton & Samuel Kortum, International Technology Diffusion: Theory and
Measurement, 40 Int’l Econ. Rev. 537, 539 (1999); Phillip McCalman, Reaping What You
Sow: An Empirical Analysis of International Patent Harmonization, 55 J. Int’l Econ. 161,
173–77 (2001).
74 Shih-tse Lo, Strengthening Intellectual Property Rights: Experience from the 1986
Taiwanese Patent Reforms, 29 Int’l J. Indus. Org. 524, 528 (2011).
75 Yi Qian, Do National Patent Laws Stimulate Domestic Innovation in a Global Patenting
Environment? A Cross-Country Analysis of Pharmaceutical Patent Protection, 1978–2002,
89 Rev. Econ. & Stat. 436, 436 (2007); see also Jean O. Lanjouw & Iain M. Cockburn, New
Pills for Poor People? Empirical Evidence After GATT, 29 World Dev. 265, 287 (2001)
(finding limited evidence of an increase in Indian R&D after India was required to
strengthen pharmaceutical patent protection).
76 The creation of the Federal Circuit is sometimes viewed as a pro-patent event, but its
effect on R&D is ambiguous. See Hall & Ziedonis, supra note 62, at 101–02, 106; Samuel
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 83
One such change was the 1994 shift in patent terms from seventeen
years from grant date to twenty years from application date, and
Professor David Abrams found the length of term extension for different
patent classes (which varied based on PTO processing time) to be
positively correlated with patent count.77 As Abrams notes, “research on
the incentive effects of patent duration is quite difficult due to the
scarcity of policy variation,”78 although a recent study takes advantage
of the effective patent term decrease for cancer drugs that require long
clinical trials to estimate that the net present value of the life-years at
stake due to decreased innovation is $2.2 trillion.79 And in another study
of de facto variation across inventions, Professor Heidi Williams
demonstrated that IP-like contractual restrictions on certain genes led to
a twenty to thirty percent decrease in subsequent innovation.80 These are
important results in the contexts of drug-development and gene patents,
but lack of similar variation (either actual or effective) makes it difficult
to conduct similar studies in other contexts.
In sum, econometric studies show that patent laws affect inventor
behavior, and there is some evidence that longer patent terms can
promote more investment. But does the R&D caused by patents create
value for society, and does that value outweigh the patent system’s
significant costs? In some cases, the evidence suggests the answer is yes;
for example, the estimated $2.2 trillion in U.S. life-years at stake due to
the decreased effective patent term for some cancer drugs81 seems
unlikely to be offset by the corresponding costs, suggesting that longer
exclusivity (such as through FDA regulations) for some drugs would be
welfare-enhancing compared to the status quo. In most cases, however,
the evidence is more ambiguous on whether patents even provide a net
incentive for innovation, much less whether their total welfare effect is
positive. And this ambiguity is underscored by the lack of contemporary
Kortum & Josh Lerner, Stronger Protection or Technological Revolution: What Is Behind
the Recent Surge in Patenting?, 48 Carnegie-Rochester Conf. Series on Pub. Pol’y 247, 248,
289–90 (1998).
77 Abrams, supra note 11, at 1614.
78 Id. at 1624.
79 Eric Budish, Benhamin N. Roin & Heidi Williams, Do Fixed Patent Terms Distort
Innovation? Evidence from Cancer Clinical Trials 34 (Nat’l Bureau of Econ. Research,
Working Paper No. 19430, 2013), available at http://www.nber.org/papers/w19430.
80 Heidi L. Williams, Intellectual Property Rights and Innovation: Evidence from the
Human Genome, 121 J. Pol. Econ. 1, 24 (2013).
81 See Budish et al., supra note 79, at 1, 34.
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
84 Virginia Law Review [Vol. 101:65
measures of innovation outside the patent system.82 If you have strong
pro-patent or anti-patent priors, this review of the existing evidence is
unlikely to change your mind.83
D. The Need for Greater Policy Variation
Whether patents promote innovation thus remains unanswered (and
hotly disputed), but in some sense this is also the wrong question. What
patent scholars should really care about is not whether patents (as
implemented) promote innovation on average, holding all other policies
constant. Rather, the question should be: How can policy makers
improve (or replace) patent laws—in light of the complex array of other
innovation policies with which they interact—to increase social welfare
(including not only innovation, but also non-efficiency values such as
equity, autonomy, and privacy)? Figuring out whether patent laws
promote innovation might help answer this question, but even more
important is understanding why patent laws work (or not), as well as
what it means for them to “work.”
The evidence analyzed thus far generally confirms that patents
provide some R&D incentive, but also that patents are not the only (or
often the most effective) means of appropriating value. This is not
surprising: It was already evident that there are many non-patent
incentives, including tax-funded transfers (such as grants and tax credits)
and other incentives (such as first-mover advantage, contractual
restrictions, and nonmonetary incentives). Patent laws also interact with
an array of laws and legal institutions, including immigration, education,
contract, tort, antitrust, and the administrative state. But I am not aware
of empirical attempts to determine which mix of innovation laws is most
effective.84
82 The studies in this section measured innovation inputs (for example, R&D spending) or
outputs as measured by patent counts; there is little current data about unpatented inventions.
Petra Moser has studied unpatented nineteenth-century inventions from World’s Fair
catalogs. E.g., Petra Moser, Patents and Innovation: Evidence from Economic History, 27
J. Econ. Persp. 23, 24–27 (2013). On the difficulty of measuring technological progress, see
John M. Golden, Innovation Dynamics, Patents, and Dynamic-Elasticity Tests for the
Promotion of Progress, 24 Harv. J.L. & Tech. 47, 54–59 (2010).
83 Cf. Lisa Larrimore Ouellette, Cultural Cognition of Patents, 4 IP Theory 28 (2014)
(examining motivated reasoning in patent policy debates).
84 One of the few empirical studies to compare tax incentives, grants, and patents is Falk,
supra note 67, although it does not show which generates the most private R&D spending.
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 85
While policy makers will probably never locate the optimal mix of
innovation law, patents could be improved along various dimensions.
But policy uniformity impedes this progress. The empirical work
reviewed in this Part shows that different sources of data can be used to
test different aspects of our understanding of patents, but also that policy
evaluation depends on variation—across jurisdiction, time, or
technology—in the independent policy choice of interest. The causal
effect of a patent policy on a given firm at a given time is the difference
between (a) that firm’s behavior under the policy and (b) the way the
same firm would have behaved at the same time without the policy (that
is, the counterfactual). We cannot observe the counterfactual, so the
causal effect must always be estimated, such as by observing how that
firm behaves at a different time without the policy, or how a different
(but similar) firm behaves at the same time without the policy. But if
there are no observations without the policy—that is, under complete
policy uniformity—one cannot even make these estimates of the
policy’s causal effect.
Yet the dominant move in patent law has been toward uniformity.85
Congress has singled out patents as an area where uniformity is
especially desirable: its “central purpose” in creating the Federal Circuit
was to promote national uniformity in patent law;86 the patent term was
adjusted in 1994 to comply with TRIPS’s goal of “uniformity
of . . . protection for intellectual property around the world”;87 and the
2011 America Invents Act moved to a first-to-file regime to “promote
greater international uniformity.”88
85 See supra notes 5–10 and accompanying text.
86 H.R. Rep. No. 97-312, at 23 (1981); see also id. at 20 (stating, in a section titled “The
Need for Increased Uniformity in Patent Law,” that “the Federal Circuit will provide
nationwide uniformity in patent law”); S. Rep. No. 97-275, at 2 (1981) (“Congress
determines there is a special need for nationwide uniformity [in patent law.]”). There is some
question as to whether the Federal Circuit has in fact achieved this goal. See Jason Rantanen
& Lee Petherbridge, Disuniformity, 66 Fla. L. Rev. 2007 (2014).
87 General Agreement on Tariffs and Trade (GATT): Intellectual Property Provisions:
Joint Hearing on H.R. 4894 and S. 2368 Before the Subcomm. on Intellectual Prop. &
Judicial Admin. of the H. Comm. on the Judiciary and the Subcomm. on Patents, Copyrights
& Trademarks of the S. Comm. on the Judiciary, 103d Cong. 163 (1994) (statement of Bruce
A. Lehman, Assistant Secretary of Commerce and Comm’r of Patents & Trademarks, U.S.
Dep’t of Commerce); see Uruguay Round Agreements Act, Pub. L. No. 103-465, § 532, 108
Stat. 4809, 4983–88 (1994).
88 Leahy-Smith America Invents Act, Pub. L. No. 112-29, § 3(p), 125 Stat. 284, 293
(2011). Also, the 1952 codification of the obviousness standard was “added to the statute for
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
86 Virginia Law Review [Vol. 101:65
Uniformity has benefits: It can create administrative savings for both
multinational companies89 and patent offices.90 And minimum
international patent standards prevent countries from freeriding on
innovation financed by foreign consumers—a dynamic that, for now,
benefits the IP-exporting United States.91 But the increased uniformity
provided by the Federal Circuit and TRIPS might come at the cost of
increasingly locking the world into a globally suboptimal innovation
system. Given the persistent empirical uncertainty in patent law, this
move toward uniformity seems misguided.
The move toward uniformity in patent law is not without critics.
Professor John Duffy has argued that the benefits of uniformity must be
balanced with the benefits of diversity, and (with Professor Craig Nard)
that appellate patent jurisdiction should be spread to an additional circuit
(or two) to foster doctrinal innovation.92 Professors Graeme Dinwoodie
and Rochelle Dreyfuss argue for a flexible approach to TRIPS that
allows regulatory competition.93 Professor Paul Gugliuzza questions
whether exclusive federal jurisdiction for patent cases is desirable.94 And
some scholars argue that U.S. states should be allowed to offer patent
incentives.95 More unconstrained “laboratories of experimentation” in
patent law could provide useful information; for example, even where
people can self-select into different policy options, we can learn
something from seeing the policies into which they opt. But
unconstrained “laboratories” may under-innovate due to the externalities
uniformity and definiteness.” H.R. Rep. No. 82-1923, at 7 (1952); S. Rep. No. 82-1979, at 6
(1952).
89 See H.R. Rep. No. 97-312, at 23 (“[U]niformity in [patent] law . . . will be a significant
improvement from the standpoint of . . . businesses . . . . This can have important positive
ramifications for the nation’s economy.”). But it is possible that patent laws generate the
greatest social welfare through their effect on a few small startups that produce disruptive
innovations.
90 See John M. Golden, Proliferating Patents and Patent Law’s “Cost Disease,” 51 Hous.
L. Rev. 455, 489–92 (2013) (describing the growth in such work-sharing programs).
91 For a discussion of how this is changing, see infra notes 244–45 and accompanying text.
92 John F. Duffy, Harmony and Diversity in Global Patent Law, 17 Berkeley Tech. L.J.
685, 703 (2002); Nard & Duffy, supra note 27; see also Amanda Frost, Overvaluing
Uniformity, 94 Va. L. Rev. 1567, 1597 (2008) (arguing that courts overvalue legal
uniformity, even for multi-state actors).
93 Dinwoodie & Dreyfuss, supra note 25, at 5–6.
94 Paul R. Gugliuzza, Patent Law Federalism, 2014 Wis. L. Rev. 11, 14.
95 Camilla A. Hrdy, State Patents as a Solution to Underinvestment in Innovation, 62
U. Kan. L. Rev. 487, 488 (2013); Douglas Gary Lichtman, Note, The Economics of
Innovation: Protecting Unpatentable Goods, 81 Minn. L. Rev. 693, 694 (1997).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 87
of both innovation itself (jurisdictions do not internalize all the benefits
of innovation policy) and innovation about innovation (jurisdictions also
do not internalize the benefits of policy experiments96).
This debate between centralized uniformity and local control
overlooks a third option, which is comparatively new in the legal
academic literature: centralized promotion of the policy variation that is
critical for empirical progress. The following Parts turn to the growing
literature on policy experimentation and explore how these approaches
might be used to make progress on some open questions about the patent
system.
II. RANDOMIZED PATENT LAW
As explained in Part I, the challenge to resolving many empirical
questions is constructing the counterfactual: Observations about how
well (or poorly) patents work do not prove that a different system would
be worse (or better). Econometric approaches can compare the effects of
different policies (in different jurisdictions or times, or for different
technologies) across similar groups, but differences between the groups
must be controlled for and will not always be observable. The gold
standard for minimizing differences between comparison groups is
random assignment of the policy treatment of interest.97 This Part
describes how patent law would benefit from much greater use of
randomized experiments, both in the laboratory (Section II.A) and in the
field (Section II.B). However, as explained in Section II.C,
randomization is not a panacea for patent law’s persistent empirical
uncertainty.
A. Laboratory Experiments
Laboratory experiments allow precise control of the environment,
enabling clean tests of theories under idealized conditions.98 Of course,
the artificial environment is also the main drawback of lab experiments,
96 See Yair Listokin, Learning Through Policy Variation, 118 Yale L.J. 480, 546 (2008).
97 See Joshua D. Angrist & Jörn-Steffen Pischke, Mostly Harmless Econometrics: An
Empiricist’s Companion 11 (2009); David Greenberg et al., Social Experimentation and
Public Policymaking 18 (2003).
98 See Elizabeth Hoffman & Matthew L. Spitzer, Experimental Law and Economics: An
Introduction, 85 Colum. L. Rev. 991, 993–94 (1985).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
88 Virginia Law Review [Vol. 101:65
as it can be difficult to extrapolate to the real world.99 But because lab
experiments are generally much cheaper than field experiments, they
should be used much more widely for initial tests of new patent-related
hypotheses.
The recent growth in lab experiments involving IP shows the promise
of this technique and suggests paths for new experiments in the patent
context. For example, studies of distortions in bargaining over creative
works have raised questions about the efficiency of IP licensing
markets.100 Running similar experiments with varied subjects and
conditions—including in more complex simulations of IP exchanges,101
or with repeat players or participants who negotiate through agents—
will help test theories of how IP markets function and how to improve
their efficiency. Other experiments have attempted to study when
subjects decide to “invest” or “patent” in controlled environments.102
None of these stylized environments maps reality or is alone sufficient
to support policy change, but identifying how real people deviate from
rational actor models may stimulate the production of new theories.
99 This concern extends not only to the environment, but also to the subjects, who are often
undergraduate volunteers or Amazon Mechanical Turk workers. But see Jesse Chandler et
al., Nonnaïveté Among Amazon Mechanical Turk Workers: Consequences and Solutions for
Behavioral Researchers, 46 Behav. Res. Methods 112, 113 (2014); Filippos Exadaktylos et
al., Experimental Subjects Are Not Different, 3 Sci. Rep. 1, 4 (2013).
100 See Christopher Buccafusco & Christopher Sprigman, Valuing Intellectual Property:
An Experiment, 96 Cornell L. Rev. 1, 4 (2010); Christopher Buccafusco & Christopher Jon
Sprigman, The Creativity Effect, 78 U. Chi. L. Rev. 31, 32 (2011); Christoph Engel &
Michael Kurschilgen, Fairness Ex Ante and Ex Post: Experimentally Testing Ex Post
Judicial Intervention into Blockbuster Deals, 8 J. Empirical Legal Stud. 682, 699 (2011);
Andreas Glöckner et al., The Endowment Effect in Groups With and Without Strategic
Incentives (Max Planck Inst. for Research on Collective Goods, Bonn, Ger.), Oct. 2009, at
15, available at http://www.coll.mpg.de/pdf_dat/2009_35online.pdf; Christopher Jon
Sprigman et al., What’s a Name Worth?: Experimental Tests of the Value of Attribution in
Intellectual Property, 93 B.U. L. Rev. 1389, 1434–35 (2013).
101 For a study of a complex simulated patent trading system, see Eskil Ullberg, Trade in
Ideas: Performance and Behavioral Properties of Markets in Patents 35–59 (2012).
102 See Uwe Cantner et al., Competition in Product Design: An Experiment Exploring
Innovation Behavior, 60 Metroeconomica 724, 727 (2009); Christoph Engel & Marco
Kleine, Who Is Afraid of Pirates? An Experiment on the Deterrence of Innovation by
Imitation (Max Planck Inst. for Research on Collective Goods, Bonn, Ger.), Nov. 2013, at
28, available at https://www.coll.mpg.de/pdf_dat/2013_07online.pdf; Patrizia Sbriglia &
John D. Hey, Experiments in Multi-Stage R&D Competition, 19 Empirical Econ. 291, 291
(1994); Andrew W. Torrance & Bill Tomlinson, Patents and the Regress of Useful Arts, 10
Colum. Sci. & Tech. L. Rev. 130, 135 (2009); Daniel John Zizzo, Racing with Uncertainty:
A Patent Race Experiment, 20 Int’l J. Indus. Org. 877, 896 (2002).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 89
The most challenging experiments are those that study not the
business decision to invest in innovation, but the innovative process
itself. Recent experiments by Professors Chris Buccafusco, Zachary
Burns, Jeanne Fromer, and Chris Sprigman measured performance on
problems of mathematical and verbal inventiveness under a low
copyright-like threshold (such that minimal success was sufficient for
entry into a lottery) as compared with higher patent-like thresholds (such
that high performance was necessary for entry), and they found
statistically significant differences between groups.103 Similar tests of
incentive structure could be performed in more realistic innovation
environments. For example, one could offer engineering students
different rewards for a design project.104
Experimenters should also look for real short-term innovation
contests where the environment can be controlled. For example, in one
ten-day contest between elite programmers (with real-world reputation
implications), experimenters found that performance was nearly doubled
by either offering a $1000 prize or allowing programmers to self-sort
into those who preferred a competitive environment over team problem-
solving.105 Such small-scale, highly controllable field experiments are a
very promising avenue for studying innovation and incentives at
relatively low cost.
Laboratory experiments can be used to bring more scientific rigor to
many aspects of the patent system beyond innovation or transactions.
For example, Professor Greg Mandel has found that subjects are more
likely to judge the solution to a problem to be obvious if they are told
what the solution is,106 raising questions about our current method of
103 Christopher Buccafusco et al., Experimental Tests of Intellectual Property Laws’
Creativity Thresholds, 92 Tex. L. Rev. 1921, 1978 (2014).
104 While it is unlikely that engineering undergraduates would create truly novel and
nonobvious inventions in a laboratory setting, they might create inventions that are novel and
nonobvious over a closed set of prior art to which they are given access.
105 Kevin J. Boudreau & Karim R. Lakhani, “Fit”: Field Experimental Evidence on
Sorting, Incentives and Creative Worker Performance 28 (Harvard Bus. Sch., Working Paper
No. 11-107, 2011), available at http://www.hbs.edu/faculty/Publication%20Files/11-107.pdf.
106 Gregory N. Mandel, Patently Non-Obvious: Empirical Demonstration that the
Hindsight Bias Renders Patent Decisions Irrational, 67 Ohio St. L.J. 1391, 1451 (2006);
Gregory Mandel, Patently Non-Obvious II: Experimental Study on the Hindsight Issue
Before the Supreme Court in KSR v. Teleflex, 9 Yale J.L. & Tech. 1, 39 (2007). It is not
clear, however, that subjects who were unaware of the solution reached the normatively
“right” decision—though if a cognitive approach to nonobviousness succeeds only as a
proxy for an economic approach, this provides stronger support for a test based explicitly on
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
90 Virginia Law Review [Vol. 101:65
assessing nonobviousness, and it seems worth replicating this
experiment for more complicated technologies. Professors David
Schwartz and Chris Seaman used mock jurors to study the effect of
different instructions on the presumption of patent validity;107 a similar
experiment could study whether the recently heightened standard for
willful infringement matters.108 To study whether juries and judges
differ in their construction of patent claims109—the oft-contested legal
descriptions of inventions that set the boundaries of the right to
exclude—one could present the same evidence to lay subjects and to
legally trained subjects (perhaps using third-year law students or
underemployed patent lawyers as a proxy for judges) and see how their
constructions differ. Or to study whether claim construction turns on
linguistic indeterminacy or policy judgments,110 one could see how
subjects vary their constructions (given the same allegedly infringing
product) when evidence varies on (a) the patentee’s technical and
economic contribution or (b) linguistic facts.
Laboratory experiments could also improve our understanding of the
technical value of patent disclosures. Contrary to popular wisdom that
scientists never read patents, my survey of nanotechnology researchers
economic considerations. See Michael Abramowicz & John F. Duffy, The Inducement
Standard of Patentability, 120 Yale L.J. 1590, 1612 (2011); Tun-Jen Chiang, A Cost-Benefit
Approach to Patent Obviousness, 82 St. John’s L. Rev. 39, 105 (2008).
107 David L. Schwartz & Christopher B. Seaman, Standards of Proof in Civil Litigation:
An Experiment from Patent Law, 26 Harv. J.L. & Tech. 429, 459 (2013); see also Microsoft
Corp. v. i4i Ltd. P’ship, 131 S. Ct. 2238, 2251 (2011) (“[A]lthough we have no occasion to
endorse any particular formulation, . . . the jury may be instructed to consider that it has
heard evidence that the PTO had no opportunity to evaluate before granting the patent.”).
108 The Federal Circuit raised the standard for willful infringement (which can result in
treble damages) in In re Seagate Technology, LLC, 497 F.3d 1360, 1371 (Fed. Cir. 2007) (en
banc), but willful infringement was found in only ten percent fewer cases post-Seagate. See
Christopher B. Seaman, Willful Patent Infringement and Enhanced Damages After In re
Seagate: An Empirical Study, 97 Iowa L. Rev. 417, 441 (2012). Researchers could
investigate whether there is also little effect in the lab, and if so, whether there is some other
treatment that causes a more significant change.
109 Cf. Teva Pharm. USA, Inc. v. Sandoz, Inc., 135 S. Ct. 831 (2015) (holding that
underlying factual disputes in patent claim construction are reviewed for clear error);
Markman v. Westview Instruments, Inc., 517 U.S. 370, 388–91 (1996) (holding that claim
construction is an issue for the judge, not the jury).
110 Cf. Tun-Jen Chiang & Lawrence B. Solum, The Interpretation-Construction Distinction
in Patent Law, 123 Yale L.J. 530, 613 (2013) (arguing that claim construction turns on
policy judgments and not linguistic indeterminacy).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 91
found that many do read patents for their technical content.111 I argued
that stronger disclosure requirements would likely have a net benefit, but
acknowledged the challenge of quantifying the value of better
disclosure. This problem could be tackled in the lab. For example,
fictitious software patents could be prepared with varying levels of
disclosure. Programmers could then be asked to implement the
algorithm, and their programs could be compared based on the time
needed to produce a functioning program and the program’s quality
based on metrics such as runtime.112
There are few immediate barriers to increased use of laboratory
experiments to inform patent policy. As already noted, these
experiments are cheap compared with field experiments, but policy
makers should recognize the benefits of funding multi-day studies of
actual innovation.113 Legal scholars should continue pursuing these
experiments (working with psychologists and others with training in
social science research) and using the results to refine theories and
formulate new hypotheses.
B. Policy Experiments
Although laboratory experiments are useful for inexpensive initial
tests of policy ideas, it is difficult to generalize from the lab environment
to the real world. The best way to evaluate a new policy is to conduct a
randomized experiment in the field. A number of government-funded
randomized policy experiments have been conducted since the 1960s,
primarily in social services.114 Thus far, “the [U.S.] government has
111 Ouellette, supra note 38, at 570. Respondents also suggested that most patents are not
enabled and that patent disclosures could be greatly improved.
112 The algorithmic challenge should be tailored to the skill of the subjects and the time
available for the experiment—different problems would be appropriate for undergraduate
students completing a homework assignment than for expert programmers. In the ten-day
contest described earlier, the problem was to optimize the contents of a space flight medical
kit, and the winning algorithm is now used for all NASA missions. Boudreau & Lakhani,
supra note 105, at 10.
113 Some funding is available from the National Science Foundation (“NSF”). Nat’l Sci.
Found., FY 2013 Budget Request to Congress, at SBE-1 to -2 (2012) (noting that the NSF
provides sixty-two percent of federal funding for academic research in the social, behavioral,
and economic sciences, with appropriations around $250 million per year).
114 See Greenberg et al., supra note 97; Jerry A. Hausman & Davis A. Wise, Introduction
to Social Experimentation 1, 1 (Jerry A. Hausman & Davis A. Wise eds., 1985); see, e.g.,
Katherine Baicker et al., The Oregon Experiment—Effects of Medicaid on Clinical
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
92 Virginia Law Review [Vol. 101:65
almost never conducted randomized tests of policies affecting middle- or
upper-class individuals, corporations, or the structure of government
itself.”115 But in a recent article, Professors Michael Abramowicz, Ian
Ayres, and Yair Listokin have laid out a convincing case for increased
use of randomized policy experiments.116 They argue that, in addition to
providing the best solution to omitted variable bias in natural
experiments, randomized field experiments are also easier to explain and
thus more likely to have real-world impact.117
As I will explain in Section II.C, innovation policy presents a hard
case for randomization, as illustrated by hypothetical experiments that
would randomize across jurisdictions or across patentees. But first, this
Section describes how patent policy could benefit by randomizing across
technologies, across new patent rights, or across patent examiners.
1. Randomizing Across Similar Technologies
Different patent policies could be randomly applied to different
technological classes in cases where it is possible to foresee
developments and draw boundaries between similar technologies.
Although this is not generally possible,118 it seems feasible for many
Outcomes, 368 New Eng. J. Med. 1713, 1714 (2013); James J. Heckman et al., The Rate of
Return to the HighScope Perry Preschool Program, 94 J. Pub. Econ. 114, 114 (2010).
115 John O. McGinnis, Accelerating Democracy: Transforming Governance Through
Technology 54 (2013). But see Abramowicz et al., supra note 14, at 991 (describing the
“nearly randomized elimination of short-sale restrictions for one-third of firms in the Russell
3000”); Dennis J. Aigner, The Residential Electricity Time-of-Use Pricing Experiments:
What Have We Learned?, in Social Experimentation, supra note 114, at 11. Wealthy
individuals are frequently affected by random treatment, such as in assignment to
government decision makers. See Adam M. Samaha, Randomization in Adjudication, 51
Wm. & Mary L. Rev. 1, 74 (2009).
116 Abramowicz et al., supra note 14, at 937. Others have also advocated more randomized
policy experiments. See Jim Manzi, Uncontrolled: The Surprising Payoff of Trial-and-Error
for Business, Politics, and Society, at xvi–xvii (2012); McGinnis, supra note 115, at 54–56;
Cass R. Sunstein, Nonsectarian Welfare Statements 12 (Harvard Law Sch. Pub. Law & Legal
Theory, Working Paper No. 13-33, 2013), available at http://ssrn.com/abstract=2317909.
117 Abramowicz et al., supra note 14, at 941–42.
118 Randomization across technologies will not work when optimal policy varies across
those technologies. Cf. Burk & Lemley, supra note 12 (citing work on technology-specific
tailoring of patent law); Carroll, supra note 12 (same); Roin, supra note 12 (same). And it
will not work when researchers can easily game the system by shifting an invention’s
technology classification. Cf. John R. Allison & Starling D. Hunter, On the Feasibility of
Improving Patent Quality One Technology at a Time: The Case of Business Methods, 21
Berkeley Tech. L.J. 729, 786 (2006) (finding that when the PTO implemented more stringent
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 93
medical technologies where the problem to be solved is well defined:
Society wants better treatments for known diseases. There is strong
evidence that patents provide an incentive for pharmaceutical
development,119 but policy makers have little empirical information on
the relative effectiveness of different policies that may be more cost
effective or lead to improved health outcomes.
Thus far, new drug incentives have not been introduced using
randomization, making evaluation difficult. For example, the 1983
Orphan Drug Act provided three new incentives to develop drugs for
rare diseases: seven years of market exclusivity, a fifty percent tax credit
for clinical trials, and increased grants.120 The number of new orphan
drugs per year increased thirteen-fold, but empirical analyses have not
disentangled the effects of the different incentives.121 Daniel Hemel and
I have argued that the combination of exclusivity and tax credits is likely
synergistic, but theory cannot tell us whether incentives should have
been weighted more toward exclusivity or toward tax credits.122 And
perhaps the incentives are too strong: Steven Rattner recently criticized
the Act as creating a “jackpot” for companies.123
Randomization should be used to improve orphan drug incentives. An
amendment to the Orphan Drug Act could randomize over all rare
diseases (of which there are about 6800124) such that, for example, half
receive the status quo while half receive the most promising
alternative.125 Researchers should then measure the number of new
clinical trials and new drugs under each system, as well as each system’s
review of business methods, some applicants shifted their filings to another technology
class).
119 Even critics of the patent system agree that patents provide a net positive incentive to
pharmaceutical firms. See, e.g., Bessen & Meurer, supra note 46, at 139–40.
120 Orphan Drug Act, Pub. L. No. 97-414, §§ 2, 4–5, 96 Stat. 2049, 2050–51, 2053–57
(1983).
121 See M. Miles Braun et al., Emergence of Orphan Drugs in the United States: A
Quantitative Assessment of the First 25 Years, 9 Nature Rev. Drug Discovery 519, 522
(2010); Wesley Yin, Market Incentives and Pharmaceutical Innovation, 27 J. Health Econ.
1060, 1062 (2008).
122 See Hemel & Ouellette, supra note 2, at 378–80.
123 Steven Rattner, An Orphan Jackpot, N.Y. Times Opinionator (June 30, 2013, 9:06 PM),
http://opinionator.blogs.nytimes.com/2013/06/30/the-orphan-jackpot.
124 Rare Disease Information, Nat’l Org. for Rare Disorders, http://www.rarediseases.org/
rare-disease-information (last visited July 12, 2013).
125 This alternative could be a minor variation, such as nine years of exclusivity and a
thirty percent tax credit, or it could be a more radical variation, such as government-run trials
or a prize system.
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
94 Virginia Law Review [Vol. 101:65
cost (including the cost of supra-competitive prices caused by
exclusivity, which are effectively a “shadow” sales tax on these
drugs126). While any randomized policy experiment should be sensitive
to ethical concerns, especially when human lives are at stake, I think that
it is, if anything, unethical not to pursue such experiments. Given the
state of uncertainty about the best incentives for drug development,
anyone with an orphan disease should want the experiment to go
forward ex ante. The experiment should continue until one system
emerges as a winner—at which point orphan diseases could be re-
randomized using the winning system and whatever then seems to be the
most promising alternative. In this manner, progress could slowly be
made toward an improved incentive system.
Drug development for non-orphan diseases could also benefit from
randomization. One promising drug development policy is the Health
Impact Fund (“HIF”), which would give pharmaceutical companies the
choice between (1) selling a new drug at high prices and receiving the
usual patent reward or (2) selling the drug at marginal cost and receiving
prizes based on its assessed global health impact in terms of Quality
Adjusted Life Years (“QALYs”).127 As the world’s largest market, the
United States could test the HIF even without support from other
countries.128 But policy makers would learn more about the HIF’s
effectiveness if instead of introducing it for all new drugs, it were
initially offered for only a random subset of diseases or therapeutic
classes. Researchers should then measure relevant outcomes for each
group, including R&D spending, new clinical trials and marketing
approvals, health impact (including for drugs not registered with the
HIF), and total cost.
Although the United States is not currently offering prizes for drug
development, funding agencies including the Department of Defense,
Department of Energy, and NASA have increasingly offered fixed prizes
for other specific technologies, such as $250,000 for an astronaut glove
and $1.75 million for a wearable power pack.129 It is unclear whether
126 See Hemel & Ouellette, supra note 2, at 312–14, 371–73.
127 Aidan Hollis & Thomas Pogge, Incentives for Global Health, The Health Impact Fund:
Making New Medicines Accessible for All 92–93 (2008).
128 Id. at 47.
129 See Deborah D. Stine, Cong. Research Serv., R40677, Federally Funded Innovation
Inducement Prizes 3–5 tbl.1 (2009). President Obama has urged agencies to increase their
use of prizes. See Nat’l Econ. Council et al., A Strategy for American Innovation: Securing
Our Economic Growth and Prosperity 12 (2011).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 95
these prizes are most effective as alternatives or complements to
patents.130 To study this, a random subset of prizes could require winners
to dedicate the invention to the public domain as a condition of receiving
the prize,131 and another random subset could offer twice the award that
the agency originally chose as the optimal prize size. One could then
measure whether either the ability to maintain patent rights or the larger
prize has a significant effect on the rate at which prizes are claimed.
2. Randomizing Across Patents
Randomly changing the legal rights accompanying some patents
(such as shortening their term) would not reveal anything about the
causal effect of patents on invention because invention occurs before a
patent application is filed (and thus before the randomization). But
patents may also affect behavior after the patent is granted; namely, they
might encourage inventors to commercialize their inventions.
Randomized field experiments could shed light on these
commercialization theories.
To incentivize commercialization, Professor Michael Abramowicz
has proposed patent extension auctions,132 and Professor Ted Sichelman
has advocated “commercialization patents”—short-term affirmative
rights to commercialize undeveloped invention patents.133 In contrast,
Mark Lemley has argued that these “ex post justifications” for patents
are unpersuasive.134 Under his reasoning, granting patent extensions or
130 Patents may not be a useful addition to fixed prizes because (1) their ability to elicit
market information is less valuable when the government has already identified a specific
goal, and (2) like prizes, they are awarded ex post and thus are of little additional help to
researchers facing capital constraints. See Hemel & Ouellette, supra note 2, at 376.
131 Agencies are prohibited from “gain[ing] an interest in intellectual property developed
by a participant in a [prize] competition without the written consent of the participant,” 15
U.S.C. § 3719(j)(1) (2012), but they could require relinquishment of patent rights as a prize
condition.
132 Michael Abramowicz, The Danger of Underdeveloped Patent Prospects, 92 Cornell L.
Rev. 1065, 1071–72 (2007).
133 Sichelman, supra note 23. Sichelman is not the first to propose patent rights based on
commercialization. See, e.g., William Kingston, The Innovation Warrant, in Direct
Protection of Innovation 59, 59 (William Kingston ed., 1987). Edmund Kitch noted this
commercialization benefit of patents, but his “prospect theory” focused more on patents’ role
in coordinating follow-on research. Edmund W. Kitch, The Nature and Function of the
Patent System, 20 J.L. & Econ. 265, 276–79 (1977).
134 Mark A. Lemley, Ex Ante Versus Ex Post Justifications for Intellectual Property, 71
U. Chi. L. Rev. 129, 130–32 (2004).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
96 Virginia Law Review [Vol. 101:65
new commercialization patents would (at least in most industries) lead to
decreased competition, not increased commercialization.135
These competing theories could be tested by subjecting
Abramowicz’s or Sichelman’s proposal to a randomized trial. After
firms or individuals have qualified for a patent extension, the
government could randomly grant only some of these new rights (ideally
on discrete products, not components). Researchers could then track
whether each invention is available in the marketplace and, if so, the
approximate number of firms from which it is available.136 It may be
most fruitful to initially test these ideas only for those areas of
technology where we expect commercialization theory to have the most
purchase. Analogizing these new rights to lottery tickets would make
them more politically palatable, and this experiment would be unlikely
to raise constitutional problems.137
3. Randomizing Across Examiners
A final example of how randomized field experiments could improve
the patent system is in the process of patent examination at the PTO and
its foreign counterparts.138 The PTO is frequently criticized for granting
135 Id. at 148–49.
136 A similar experiment could be conducted in countries that already offer short patents
for commercialization by randomly denying a subset of such patents. See generally
Protecting Innovations by Utility Models, World Intellectual Prop. Org.,
http://www.wipo.int/sme/en/ip_business/utility_models/utility_models.htm (last visited July
12, 2013). But see Sichelman, supra note 23, at 398–99 (arguing that these patents have
limited effectiveness because patentees do not commit to commercialization). But the
random denial of an established right is likely to be less politically palatable than the random
grant of a new right.
137 As Judge Friendly explained in rejecting a constitutional challenge to a welfare
experiment, “the Equal Protection clause should not be held to prevent a state from
conducting an experiment designed for the good of all, including the participants, on less
than a statewide basis” and objections to allowing officials to arbitrarily select recipients of
benefits “are inapposite to the selection, on a random but rational basis, of certain areas of
the state to try out a program for the very purpose of determining whether it, or some
variation of it should be made applicable to all.” Aguayo v. Richardson, 473 F.2d 1090, 1109
(2d Cir. 1973). For a thorough discussion of ethical and equality concerns with randomizing
law, see Abramowicz et al., supra note 14, at 963–74.
138 Patents are examined by national offices or by regional offices such as the European
Patent Office (“EPO”) (which operates concurrently with national patent offices throughout
Europe). See Frequently Asked Questions: Patents, World Intellectual Prop. Org.,
http://www.wipo.int/patents/en/faq_patents.html (last visited July 12, 2013). Patents granted
by regional offices are country specific, although the European Union is creating a unified
patent system. See Unitary Patent & Unified Patent Court, supra note 8.
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 97
too many invalid patents, in part because of the limited time examiners
have to examine the average patent.139 Invalid patents can create
significant social costs when they become the subject of expensive
patent litigation. It is not clear, however, that spending more resources
on patent examination would be efficient. Most patents never become
important or disputed, so it probably makes sense to thoroughly consider
only the patents that eventually matter.140
While determining the optimal amount of time to spend on patent
examination is difficult, randomized experiments could improve patent
examination within the current time constraints. The PTO employs
about 8000 examiners,141 who are grouped by technology within “art
units” and are managed in groups of thirteen to twenty by supervisory
patent examiners (“SPEs”).142 Applications are assigned to SPEs based
on technology, and SPEs assign applications to examiners in a process
that is essentially random (and that could be made actually random).143
Proposals for improving examination could be tested by randomizing
over either individual examiners or SPE groups, depending on the extent
to which the treatment could be isolated. The PTO already randomly
audits examiners: The Office of Patent Quality Assurance reviews a
random sample of all examiner actions (which will catch both false
139 See, e.g., Jaffe & Lerner, supra note 55, at 34; Joseph Farrell & Robert P. Merges,
Incentives to Challenge and Defend Patents: Why Litigation Won’t Reliably Fix Patent
Office Errors and Why Administrative Patent Review Might Help, 19 Berkeley Tech. L.J.
943, 944 (2004). The time allotted to examination of each patent varies by technology and
the examiner’s experience but is typically around twenty hours. See Michael D. Frakes &
Melissa F. Wasserman, Does Agency Funding Affect Decisionmaking?: An Empirical
Assessment of the PTO’s Granting Patterns, 66 Vand. L. Rev. 67, 135–36 (2013).
140 See Mark A. Lemley, Rational Ignorance at the Patent Office, 95 Nw. U. L. Rev. 1495,
1497 (2001); see also Hemel & Ouellette, supra note 2, at 365 (estimating that the costs of
litigation are only about half the costs of prosecution). But see Jonathan S. Masur, Costly
Screens and Patent Examination, 2 J. Legal Analysis 687, 688 (2010) (arguing that costly
examination has a screening value distinct from improving quality).
141 U.S. Patent & Trademark Office, U.S. Dep’t of Commerce, Performance and
Accountability Report: Fiscal Year 2012, at 10 (2012).
142 See David S. Kim & Glenn M. Kubota, Behind the Scenes at the USPTO: Accounting
for the Supervisory Patent Examiner, Morrison & Foerster IP Q. Newsl., Summer 2011, at 2.
143 Mark A. Lemley & Bhaven Sampat, Examiner Characteristics and Patent Office
Outcomes, 94 Rev. Econ. & Stat. 817, 822 (2012); see U.S. Patent & Trademark Office, U.S.
Dep’t of Commerce, Manual of Patent Examining Procedure § 903.08(b) (8th ed., rev. 9
2012). A primary examiner I spoke with saw no logistical reason that assignment could not
be randomized.
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
98 Virginia Law Review [Vol. 101:65
positives and false negatives).144 Experimenters could thus compare
audit results of examiners operating under different policies.
Note that while some aspects of examination—such as whether an
examiner located a clearly novelty-defeating piece of prior art—can be
evaluated more objectively than others, this experimental approach does
not require judgments about whether granting more or fewer patents
increases innovation. Whatever the threshold for more contested
doctrines such as obviousness and enablement,145 examination should at
least be reliable, in the sense of different examiners giving the same
result for the same patent application. And it seems likely that
examination is more reliable with more intense scrutiny (although this
should be verified). So while examination results cannot be compared to
some objectively correct result, they can at least be compared to the
results of closer scrutiny of the same patents.
The PTO has already experimented on a non-random basis with
numerous pilot programs.146 These are promising steps, but the PTO
would learn even more from these pilots by initially testing them on a
randomized basis. Similarly, the PTO could measure the effect of
randomly varying the “count” system (which determines how much
credit examiners receive for different activities) across examiners, while
adjusting quotas to keep the net resources devoted to examination
constant.147 As another example, Congress could test the proposal of
144 U.S. Patent & Trademark Office, U.S. Dep’t of Commerce, Adoption of Metrics for the
Enhancement of Patent Quality: Fiscal Year 2011, at 4 (2010).
145 For the basic requirements for patentability, see 35 U.S.C. §§ 101–103, 112 (2012).
146 See, e.g., Full First Action Interview Pilot Program, U.S. Patent & Trademark Office,
http://www.uspto.gov/patents/init_events/faipp_full.jsp (last visited July 12, 2013); Peer
Review Pilot FY2011, U.S. Patent & Trademark Office, http://www.uspto.gov/patents/
init_events/peerpriorartpilotindex.jsp (last visited July 12, 2013); Press Release, No. 07-19,
U.S. Patent & Trademark Office, USPTO Celebrates a Decade of Telework (June 4, 2007),
available at http://www.uspto.gov/news/pr/2007/07-19.jsp.
147 From the 1970s to 2010, the PTO used the same count structure. U.S. Gov’t
Accountability Office, GAO-07-1102, U.S. Patent and Trademark Office: Hiring Efforts Are
Not Sufficient to Reduce the Patent Application Backlog 7 (2007). This system may have
created incentives for examiners to encourage more requests for continued examination
(“RCEs”). See Arti K. Rai, Growing Pains in the Administrative State: The Patent Office’s
Troubled Quest for Managerial Control, 157 U. Pa. L. Rev. 2051, 2064–65 (2009). There
were “significant changes” in 2010. Press Release, No. 10-08, U.S. Patent & Trademark
Office, Recently Announced Changes to USPTO’s Examiner Count System Go into Effect
(Feb. 18, 2010), available at http://www.uspto.gov/news/pr/2010/10_08.jsp. Because the
count system was changed for all examiners at once, measuring the effect of the change is
difficult, although the PTO can gain some information from measuring effects across time. It
may be possible to make further improvements; for example, one examiner I spoke with
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 99
Professors Michael Frakes and Melissa Wasserman to restrict the
renewal fees that the PTO may retain (to avoid biases) by requiring
renewal fees resulting from patents granted under randomly selected
SPEs (who are aware of their assignment) to be diverted to general
treasury funds, and then comparing grant rates under different SPEs.148
C. The Limitations of Randomization
The previous Section described ways in which the patent system
could benefit from randomized field experiments, but randomization is
not a panacea for the empirical uncertainty in patent law. Some
economists have criticized over-reliance on randomized trials for social
policies and development, noting both technical problems such as
“randomization bias” (where experimental subjects differ from those
normally subject to the policy) and the concern that even the best
experiments result only in a mean treatment effect for a specific
situation, with no indication of why a policy worked or whether it would
succeed in other contexts.149 Professor Angus Deaton argues that
“heterogeneity is not a technical problem calling for an econometric
solution but a reflection of the fact that we have not started on our
proper business, which is trying to understand what is going on.”150
These limitations are particularly salient for innovation. For example,
while it might be appealing to think that randomizing across
jurisdictions or patenting entities could lead to important empirical
gains, this Section argues that such experiments would not be worth
their costs.
suggested a creative system that would enable the amount of credit an examiner receives for
an individual case to track the difficulty of the application.
148 Frakes & Wasserman, supra note 139, at 121. PTO resistance could be reduced by a
commitment to provide a fixed lump sum in exchange for the renewal fee diversion. Of
course, it is possible that the mechanism behind the effect measured by Frakes and
Wasserman is so subtle that examiner awareness of the experiment would eliminate the
effect.
149 See, e.g., Angus Deaton, Instruments, Randomization, and Learning About
Development, 48 J. Econ. Literature 424, 426, 445 (2010); James J. Heckman & Jeffrey A.
Smith, Assessing the Case for Social Experiments, 9 J. Econ. Persp. 85, 86 (1995). But see
Abhijit V. Banerjee & Esther Duflo, The Experimental Approach to Development
Economics, 1 Ann. Rev. Econ. 151, 152 (2009); Guido W. Imbens, Better LATE Than
Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009), 48 J. Econ.
Literature 399, 399–400 (2010).
150 Deaton, supra note 149, at 430.
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
100 Virginia Law Review [Vol. 101:65
1. Randomizing Across Countries
An experiment in which patent policies (for example, the status quo
under TRIPS versus no patents) were randomly assigned by country
(with the hope of observing the effect of patents on R&D spending or
economic growth) would avoid problems with endogeneity and omitted
variables that plagued the natural experiments in Section I.C. But such
an experiment would encounter at least five significant problems. (1) As
Section I.C describes, patents create transjurisdictional spillovers for
both users (who benefit from the resulting knowledge without sharing its
cost) and innovators (who are able to obtain patents in foreign markets),
so one might see better outcomes in no-patent countries even if this is
not the general equilibrium outcome. (2) Countries would have to be
allowed to opt in to the experiment (due to national sovereignty),151 and
the results might not apply to countries that did not volunteer. (3) The
measured average treatment effect will depend on factors that vary
between countries (such as whether they have effective alternatives to
patents, the availability of capital, and the composition of industries) and
thus might be of little use for determining the optimal policy for a
specific country.152 (4) The nature of innovation can change: An
experiment fifty years ago might have little relevance in today’s
networked information economy where user-centered innovation is
increasingly rivaling a manufacturer-centered approach.153 (5) Although
there are many proposals for improving innovation policy, the high
political and financial cost of international policy randomization and the
151 Cf. John Goering, Comments on Future Research and Housing Policy, in Choosing a
Better Life? Evaluating the Moving to Opportunity Social Experiment 383, 383–401 (John
Goering & Judith D. Feins eds., 2003) (describing an opt-in field experiment in which poor
families who received vouchers to move to low-poverty areas were compared with others
who wanted to move but were not chosen by the random lottery).
152 See Deaton, supra note 149, at 441 (“The mean treatment effect . . . may be of limited
value for a . . . policymaker contemplating specific patients or policies.”).
153 See Benkler, supra note 4, at 460–73; Eric von Hippel, Democratizing Innovation 112–
17 (2005); Deven R. Desai & Gerard N. Magliocca, Patents, Meet Napster: 3D Printing and
the Digitization of Things, 102 Geo. L.J. 1691, 1719–20 (2014). The point is not that we
know that the effectiveness of patents will vary over time—it is that we do not know that
their effectiveness will not vary over time, and there are plausible reasons that optimal
innovation policy today might look different from optimal innovation policy a few decades
ago, and we may see further changes in the future. This point applies broadly: All evidence
involves past events, and policy makers should evaluate whether the evidence’s applicability
has changed over time. Cf. Ouellette, supra note 38, at 565–66 (arguing that old surveys
about how researchers use patents may be outdated in an era where patents are readily
searchable online).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 101
lengthy time required for effective experiments154 would make it
possible to test only a small number of alternatives.
Experiments on more specific aspects of patentability would have
similar problems. For example, even if the patentability of genes or
software or other controversial subject matter areas could be randomized
across countries, innovation in these fields changes rapidly, so the
results might have little applicability by the time the experiment is
complete.
Slightly more progress could be made by focusing on outcomes
specific to an individual country. For developing countries with small
markets, the added incentive of domestic patent laws is small compared
with the incentive of foreign protection in larger markets; the main
argument for why patent laws benefit developing countries is that they
increase foreign direct investment (“FDI”).155 But whether patent laws in
fact increase FDI for developing countries is an open empirical
question.156 To address this question, the World Trade Organization
(“WTO”) could allow patent laws to be suspended for a random
selection of small developing countries that would prefer not to be
bound by TRIPS, and then FDI in those countries could be compared
with FDI in the control group of countries that volunteered for the
experiment but were not allowed to relax their patent laws. There are
potential problems with randomization bias (if countries that volunteer
for the experiment are significantly different from countries that do not)
and with measuring only a partial equilibrium effect (such as if there is a
fixed amount of FDI that goes to patent-protective countries in the
experiment but would have gone to countries without patents if none of
the developing countries had patents). But the result could still provide
154 The time-to-market for new innovations can be under a year for software or insurance
products but is typically well over a decade in fields such as oil and gas drilling,
pharmaceuticals, and fuel cells. Roin, supra note 12, at 719 tbl.1.
155 See Branstetter & Saggi, supra note 41 (laying out these arguments).
156 Compare Walter G. Park & Douglas C. Lippoldt, Technology Transfer and the
Economic Implications of the Strengthening of Intellectual Property Rights in Developing
Countries 28 (OECD Trade Policy, Working Paper No. 62, 2008), available at
http://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?doclanguage=en&cote=t
ad/tc/wp%282007%2919/final (concluding that stronger patent protection stimulates FDI),
with Etienne Pfister & Bruno Deffains, Patent Protection, Strategic FDI and Location
Choices: Empirical Evidence from French Subsidiaries’ Location Choices in Emerging
Economies, 12 Int’l J. Econ. Bus. 329, 344 (2005) (“[S]tronger patent protection in countries
with a high GDP or with a low R&D intensity seems to reduce the attractiveness to FDI.”).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
102 Virginia Law Review [Vol. 101:65
some evidence to rebut or support the argument that patent laws benefit
developing countries by increasing FDI.
This FDI experiment would thus be more successful than an attempt
to measure the net innovation incentive caused by patents. It could
provide a qualified answer to a disputed policy question: “Does patent
protection for developing countries increase FDI on average?” However,
the answer to this question does not seem sufficiently valuable to justify
the experiment for at least three reasons. First, the answer—the average
treatment effect—reveals little about whether patent laws increase FDI
in any given context. FDI seems likely to be influenced by many legal
policy levers beyond patent laws, including the state of contract and
trade secrecy law, workforce skill (which is related to education and
immigration laws), labor laws, investment laws, and trade barriers.
Perhaps eliminating patents increases FDI in countries with certain
contract laws, but in countries without those laws there is a bigger drop
in FDI, so the average effect is negative. But such contract laws should
not be treated as a heterogeneity problem to be averaged away. They are
a critically important variable that should be part of our theory of how
patent laws influence FDI.
Second, FDI is also likely to be influenced by more nuanced factors
that are less easily observable. It seems plausible that countries whose
leaders use diplomatic skill and effort to attract FDI will have more
FDI—variables that are difficult to observe and impossible to randomly
assign (one cannot tell some countries to be more skillful or to put in
more effort). And third, FDI is not the ultimate policy outcome of
interest—it is simply an argument about how patents might benefit
developing countries. But if the argument is that developing countries
should have patents because it benefits them, should they not decide
what they view as a “benefit”? An individual country would seem to
have a comparative advantage in both defining its own success and
understanding all the complicated policy levers for achieving that
success. Perhaps a better approach would be to incentivize developing
countries to use their local knowledge and experiment with their own
policies in ways that promote global innovation and development
goals—a different approach to policy experimentation, which I will
return to in Part III.
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 103
2. Randomizing Across Universities
Randomization’s limitations are not confined to the international
context. For example, suppose patenting policies were randomized over
universities to test whether the Bayh-Dole Act is the best legal
framework for promoting the commercialization of university
research.157 Some universities could remain under the status quo (in
which patents and exclusive licenses for federally funded research are
allowed158), and others could be required to use a “market test” of
offering research under a nonexclusive license for a nominal fee before
an exclusive license is granted. While mandatory participation would be
more feasible than in the international context, an opt-in system (in
exchange for a monetary incentive) would still be more likely, leading to
significant randomization bias concerns. This experiment would also
measure only partial equilibrium effects: It might lead to re-sorting of
professors across universities that would not occur if all universities
adopted the same policy.159 Schools might also not behave as they would
if the changes were permanent, such as by delaying licenses or acting
differently because they are being watched.160 And a final problem is
that measurements would be difficult and costly. Significant funding
would be required to measure technology transfer that occurs outside the
patent system and to measure qualitative changes in research (including
changes in scientific norms).161
It might be possible to overcome these problems with sufficient
political and financial resources, but it is not clear that the outcome
157 See Bhaven N. Sampat, Patenting and US Academic Research in the 20th Century: The
World Before and After Bayh-Dole, 35 Res. Pol’y 772, 778–86 (2006).
158 See 35 U.S.C. §§ 200–212 (2012).
159 There is some evidence that varying patent royalties causes such sorting. See Saul Lach
& Mark Schankerman, Incentives and Invention in Universities, 39 RAND J. Econ. 403, 427
(2008). To minimize this problem, the experiment could use matched samples (so that if one
professor leaves, her corresponding match is also not counted) or an “intent to treat”
methodology (counting professors who move with the university to which they were
originally assigned). See Abramowicz et al., supra note 14, at 958–59 (providing an
overview of these techniques).
160 See John G. Adair, The Hawthorne Effect: A Reconsideration of the Methodological
Artifact, 69 J. Applied Psychol. 334, 334–36 (1984).
161 A recent study of exogenous variation in the availability of venture capital funding for
life-science researchers at top U.S. universities found that greater funding leads to increased
patenting but also “changes the type of innovation being pursued: it leads to shorter-horizon,
less-cited, and more narrowly-focused projects.” Julian Kolev, Credit Constraints and Their
Impact on Innovation: Evidence from Venture Capital Exits (Nov. 21, 2012) (unpublished
manuscript) (on file with author).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
104 Virginia Law Review [Vol. 101:65
would be worth it. Even though “assessment of the social welfare effects
of Bayh-Dole . . . remains an open empirical question,”162 an empirical
study is not needed to determine whether universities should grant
exclusive patent licenses when firms are eager to commercialize the
inventions without exclusive rights. There is no compelling theoretical
justification for Bayh-Dole besides the idea that exclusive rights are
sometimes necessary for commercialization.163 The obstacles to
amending Bayh-Dole are (1) showing that there is an effective way to
sort out which inventions require exclusive rights and (2) overcoming
political constraints. A randomized study would probably be no more
likely to lead to adoption of the “market test” approach than encouraging
a few universities to adopt the policy to show that it is (or is not)
feasible.
In conclusion, while innovation policy would benefit from more
randomized experiments, innovation is a hard case for field
randomization. Even if a global autocrat could control spillovers and
randomization bias, she could not design a randomized trial that would
reveal the “best” innovation policy mix because the effectiveness of
policies (as well as the definition of “best”) will vary with many time-
and jurisdiction-dependent factors. A policy designed by a central
planner cannot account for these case-specific, dynamic conditions.
Policy makers should use randomized experiments only when
(1) adoption of a well-defined policy depends on its effect on something
observable (and not on buy-in from regulated parties); (2) the policy can
be randomly (and ideally mandatorily) assigned to different units (such
as different technologies) such that there are minimal spillovers; and
(3) the policy’s effect is expected to be sufficiently constant across units
and time that the mean treatment effect will be relevant for future policy
decisions.
III. REGULATED PATENT FEDERALISM
Over the past three decades, while patent law has become
increasingly centralized with uniform rules, Professor Charles Sabel and
a variety of coauthors have extolled increased policy variation through a
162 Sampat, supra note 157, at 773.
163 See Lemley, supra note 3, at 621–22; Lisa Larrimore Ouellette, Comment, Addressing
the Green Patent Global Deadlock Through Bayh-Dole Reform, 119 Yale L.J. 1727, 1731
(2010).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 105
governance regime they term “experimentalism.”164 Experimentalism is
characterized by a “center” and “local units” that “set and revise
goals . . . in an iterative process with four basic elements”:
(1) collaboratively defined “framework goals”; (2) “broad discretion”
for local units to meet the goals; (3) “peer review” that
“require[s] . . . local units to describe and explain their efforts”; and
(4) revision of the goals.165 This work is part of a broader “New
Governance” literature, which encompasses issues ranging from
management to democratic legitimacy,166 but I am most interested in the
ability of this structure to promote learning.167 Experimentalism’s
“governing norm” is “the capacity for learning and adaptation”168—a
valuable feature in the dynamic field of innovation.
As an example of the experimentalist structure, child welfare systems
in Alabama and Utah give substantial discretion to local caseworkers to
promote the “best interests” of the child, but this discretion is cabined by
requiring such choices to be “collaborative and explicit,” and by
subjecting a subset of cases to intensive peer review.169 The “local” units
also can be larger than individual actors: The European Union has used
experimentalist governance—in which countries’ implementation of
164 Sabel’s early work focused on experimentalism in corporate governance. See, e.g.,
Michael J. Piore & Charles F. Sabel, The Second Industrial Divide: Possibilities for
Prosperity, 300–01 (1984) (describing the corporate use of “flexible specialization” in
response to market disorder and uncertainty). He later expanded these ideas to government
regulation. See, e.g., Dorf & Sabel, supra note 15, at 315, 345.
165 Sabel & Simon, supra note 15, at 79.
166 See generally Orly Lobel, New Governance as Regulatory Governance, in The Oxford
Handbook of Governance 65 (David Levi-Faur ed., 2012) (analyzing the emerging field of
new governance). This literature is also tied to dialogic visions of federalism, including
accounts of the power of local units to shape debate by being uncooperative. See Heather K.
Gerken, The Supreme Court, 2009 Term—Foreword: Federalism All the Way Down, 124
Harv. L. Rev. 4, 61–69 (2010) (describing the role of states experimenting with different
approaches to problems).
167 Other scholars besides Sabel and his coauthors have also explored how a decentralized
regulatory structure can lead to policy learning. See, e.g., Daniel A. Farber, Environmental
Protection as a Learning Experience, 27 Loy. L.A. L. Rev. 791, 798 (1994); Jody Freeman,
Collaborative Governance in the Administrative State, 45 UCLA L. Rev. 1, 28–29 (1997).
168 Sabel & Simon, supra note 15.
169 Kathleen G. Noonan, Charles F. Sabel & William H. Simon, Legal Accountability in
the Service-Based Welfare State: Lessons from Child Welfare Reform, 34 Law & Soc.
Inquiry 523, 541, 556 (2009).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
106 Virginia Law Review [Vol. 101:65
broad framework goals is improved through peer review—for issues
ranging from telecommunications to occupational health and safety.170
Experimentalism and randomized field experiments both involve
promotion of local policy variation by central planners, which I have
argued is critical to empirical progress in patent law. But these two
approaches are otherwise radically different. Randomized experiments
are comparatively short term, with the goal of identifying (and then
mandating) the “right” answer based on quantifiable (and often
efficiency-based) metrics; experimentalism is a process of dynamic
governance that can more easily accommodate diverse values.
Randomization tests external constraints on local units who are subjects
of, not participants in, the experiments; experimentalism depends upon
the participation and buy-in from the local units. Randomized
experiments are controlled by central planners who precisely specify the
relevant policy choices; experimentalism elicits policy suggestions,
metrics, and goals from actors throughout the system.
Experimentalism appears promising for promoting policy variation
and learning for many innovation issues. It can increase local buy-in,
promote local experiments, elicit local knowledge about heterogeneous
conditions, generate better measures of whether policies are succeeding
in light of local values, and increase focus on the mechanisms through
which innovation laws act. To be sure, without some use of chance it is
difficult to assess whether a policy—including experimentalism itself—
is working. In some cases it may be possible to solve this problem by
introducing experimentalist techniques on a randomized basis. But even
where this is infeasible, experimentalism may be a better approach than
the current focus on patent uniformity.
A. Patent Examination
The experimentalism literature has focused most heavily on the
administrative state, so its most obvious application in patent policy is in
the administrative process of patent examination at the PTO and its
foreign counterparts. The problem of improving examination within
limited time constraints also provides a good illustration of the
distinction between randomized experiments and an experimentalist
approach.
170 See Charles F. Sabel & Jonathan Zeitlin, Learning from Difference: The New
Architecture of Experimentalist Governance in the EU, 14 Eur. L.J. 271, 273–86 (2008).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 107
Subsection II.B.3 proposed that randomized experiments be used to
evaluate examination policies, such as changing the “counts” assigned to
different activities, restricting the renewal fees that the PTO may retain,
and introducing pilot programs. These top-down randomized
experiments can be used to adjust examiners’ external constraints. But
random assignment is ineffective for telling examiners how to spend
their time within those external constraints, or for improving their skills
and intrinsic motivation, even though improving the nuanced process of
examination and the norms within the PTO may be even more important
for addressing the PTO’s problems than getting the right external
constraints.171
As explained in Subsection II.B.3, examiners work in groups of
thirteen to twenty under an SPE. Examiners begin as assistant examiners
whose decisions must be reviewed; after a rigorous review process they
can become primary examiners with “signatory authority” to issue office
actions without review.172 Primary examiners thus receive little
substantive supervision, although all decisions are subject to random
audits by the Office of Patent Quality Assurance.173 Primary examiners
review the work of some assistant examiners, but assistant examiners do
not review anyone’s work.
Examiners are given training on various databases that they can
search for prior art, as well as somewhat detailed rules about
patentability standards in the Manual of Patent Examining Procedure,174
but they are given little formal instruction on the process of
examination.175 Which databases should be searched? How should
search queries be constructed? How do you know when to stop
171 See Iain M. Cockburn, Samuel Kortum & Scott Stern, Are All Patent Examiners Equal?
Examiners, Patent Characteristics, and Litigation Outcomes, in Patents in the Knowledge-
Based Economy 19, 52–53 (Wesley M. Cohen & Stephen A. Merrill eds., 2003)
(emphasizing the importance of norms for improving the non-mechanistic process of patent
examination); Mark Lemley, Fixing the Patent Office 7, 10 (Nat’l Bureau of Econ. Research,
Working Paper No. 18081, 2012) (citing Lemley & Sampat, supra note 143) (arguing that
human resource policies such as “training examiners to search better” could improve
examination).
172 See Kim & Kubota, supra note 142; Lemley & Sampat, supra note 143, at 818–19.
Starting in 2000, an additional “second pair of eyes” review was required for business
method patents, but this program has since been shut down. See Lemley, supra note 171, at
8–9.
173 See supra note 144 and accompanying text.
174 See U.S. Patent & Trademark Office, supra note 143.
175 Of course, the best SPEs might provide successful informal instruction.
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
108 Virginia Law Review [Vol. 101:65
searching? Given the limited time available, should more energy be
devoted to prior art searches or to investigating whether the invention’s
description is adequate for replication? These process questions seem at
least as important as the structure questions examined in Subsection
II.B.3, but the non-mechanical process of examination cannot be
specified in a precise way that could then be randomly varied. And the
optimal process will vary based on the technology,176 the specific patent,
and the examiner’s skills (for example, different search strategies might
work better for a non-native English speaker).
These process questions could be tackled with an experimentalist
approach. The PTO already has the necessary elements for
experimentalism: The “center” is the PTO leadership (or the SPEs), the
“local units” are individual examiners, the “framework goal” is to
improve examination efficiency within time and budgetary constraints,
and examiners have (by necessity) broad discretion in how to meet that
goal. The PTO could institute a peer review system modeled after
experimentalist systems such as child welfare administration in Alabama
and Utah. The optimal process of patent examination—how examiners
can most efficiently use the limited time available to make the best
judgment of whether an application should be granted—cannot be
specified with any more precision than the optimal process of serving
the “best interests of the child” in child welfare administration. But as in
the child welfare context, an experimentalist approach could improve
examination by creating a process for examiners to share tacit
knowledge and diffuse positive norms.
For example, the PTO could set aside a few days each year for
extensive group discussions of a few applications. The examiner
responsible for each application could explain how she did her prior art
search, why she made the rejections she did, why she did or did not ask
for claims to be amended or words to be defined, etc. And her peers
could describe how they might have done things differently.177 It would
176 The PTO is transitioning from a classification system with 150,000 subdivisions to a
new Cooperative Patent Classification system with 260,000 subdivisions. See Kristin
Whitman, Ready or Not, the Cooperative Patent Classification Has Arrived!, Intellogist Blog
(July 12, 2012), http://intellogist.wordpress.com/2012/07/12/ready-or-not-the-cooperative-
patent-classification-has-arrived.
177 One examiner informed me that the PTO has previously conducted a few training
sessions in a roundtable format, where examiners had intense discussions about other ways
to conduct prior art searches for sample applications, and that these sessions were very
useful.
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 109
not be possible to engage in such extensive review of a statistically
significant number of each examiner’s applications, but the goal would
be qualitative learning, not quantitative quality control. Another
possibility would be to sometimes assign two examiners to one case,
especially as a way of training new examiners. Given the time it can take
to understand a single application, this would enable even more in-depth
discussion than a qualitative group review, as it would force two
examiners to agree on how the examination process should proceed.
While these interventions would likely improve examination, peer
review is also costly. To address the difficult question of how much peer
review is cost-justified, the experimentalist literature could learn from
the literature on randomization: Even though the examination process
itself cannot be randomized, the approach to improving it could be. For
example, various experimentalist interventions (including group peer
reviews or team examination) could be tested on a random selection of
SPE groups, and as with the interventions in Subsection II.B.3, results of
Office of Patent Quality Assurance audits could be compared across
groups.
As I have noted, the PTO’s leadership has already demonstrated great
willingness to experiment with pilot programs, and they (or
experimentally inclined leaders at any foreign patent office) could also
implement the experiments proposed in this article.178 Encouragingly,
one examiner told me that the PTO fosters innovation about examination
through an intranet site for examiners to suggest ideas, and that the
management reviews these ideas for feasibility. Some of these
suggestions, coming from those immersed in the examination process,
may create more significant gains than the ideas sketched out above. My
goal is not to have the PTO conduct specific policy experiments—it is to
have the PTO (and other patent offices) consider the benefits of
randomization and experimentalism when testing new policies.
B. Patent Adjudication
Although the experimentalism literature has not discussed the courts,
there are many parallels between experimentalism and the U.S. judicial
178 One obstacle to implementing randomized PTO experiments may be the examiners’
union, the Patent Office Professional Association. But the union might not object if the PTO
makes clear that the average benefits and bonuses for each random group will be the same,
and that the experiments will ultimately improve conditions for all examiners.
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
110 Virginia Law Review [Vol. 101:65
system. The federal courts (which have exclusive jurisdiction over the
patent laws179) involve a series of “centers” and “local units” (with the
courts of appeals as centers to the district courts and local units to the
U.S. Supreme Court). Judges likely agree on framework goals—finding
the “best” interpretation of statutes, upholding the Constitution,
resolving disputes fairly and efficiently—although the meaning of these
goals is the subject of frequent debate. Judges have broad discretion in
how to meet these goals: District judges often receive formal deference,
and courts of appeals have de facto discretion because the Supreme
Court reviews only a fraction of their decisions. But this discretion is
constrained because courts must justify their decisions, and they are
subject to feedback and peer review. For example, a panel of judges on a
court of appeals might have its decision vacated by its en banc court,
reversed by the Supreme Court, or rejected by a sister circuit that finds
the decision unpersuasive.
Courts are not well positioned to measure the incentive effects of their
rulings, though they could encourage submission of more rigorous
empirical evidence.180 But they are very well positioned to improve
procedural aspects of patent litigation. Does a more deferential standard
of review for claim construction make it more predictable?181 How do
juries differ from judges in patent cases?182 Could trials be more
efficient? Would routine fee shifting reduce litigation?183 Could the
asymmetry that causes defendants to focus on non-infringement rather
than invalidity be reduced?184
179 See 28 U.S.C. § 1338 (2012). But see Gunn v. Minton, 133 S. Ct. 1059, 1068 (2013).
180 In some cases, courts could also mandate experimental solutions, perhaps including
randomized trials. Public law litigation has already resulted in experimentalist remedies,
such as rules that “incorporate a process of reassessment and revision with continuing
stakeholder participation.” Charles F. Sabel & William H. Simon, Destabilization Rights:
How Public Law Litigation Succeeds, 117 Harv. L. Rev. 1016, 1069 (2004).
181 See Jonas Anderson & Peter S. Menell, Informal Deference: A Historical, Empirical,
and Normative Analysis of Patent Claim Construction, 108 Nw. U. L. Rev. 1, 76–77 (2014).
182 See Kimberly A. Moore, Judges, Juries, and Patent Cases—An Empirical Peek Inside
the Black Box, 99 Mich. L. Rev. 365, 409 (2000) (suggesting that “there are some
differences in judge and jury resolution of patent cases” that are “impossible to quantify”
based on existing data).
183 See Thomas F. Cotter, Comparative Patent Remedies: A Legal and Economic Analysis
148–49 (2013).
184 See Roger Allan Ford, Patent Invalidity Versus Noninfringement, 99 Cornell L. Rev.
71, 102–12 (2013).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 111
Courts could also improve substantive patent law when the open
question is about the administrability of a standard, rather than its
theoretical soundness. For example, it seems clear from a theoretical
perspective that a more economic approach to assessing nonobviousness
(including examining evidence of independent invention) would be more
efficient,185 although it is unclear whether such a standard is
administrable in practice. Doctrinal innovation could help answer this
question.
District courts’ role as laboratories of experimentation for patent
litigation has been recognized: Professor Xuan-Thao Nguyen has
highlighted their procedural innovations with local patent rules,186 and
Professor Jeanne Fromer has argued that they could improve
technology-specific tailoring of patent law.187 These efforts may be
aided by the pilot program to allow certain district judges to hear more
patent cases.188 But the usual experimentalist appellate structure has
been eliminated through the centralization of patent appeals in the
Federal Circuit.189 As discussed previously, Nard and Duffy have argued
that patent appeals should be spread to one or two additional courts,
which would require the Federal Circuit to explain its decisions—and its
rejection of competing arguments—in a way that is persuasive to its
peers.190 They also illustrate another important lesson that
experimentalism can learn from randomization: To prevent the results of
doctrinal experiments from being tainted by self-selection, appellate
jurisdiction of district court cases could be randomly assigned after the
185 See Abramowicz & Duffy, supra note 106, at 1676–77; Chiang, supra note 106.
186 Nguyen, supra note 27, at 477–83.
187 Fromer, supra note 27, at 321–22; Jeanne C. Fromer, Patentography, 85 N.Y.U. L. Rev.
1444, 1507 (2010). But see Megan M. La Belle, The Local Rules of Patent Procedure, 46
Ariz. St. L.J. (forthcoming 2015) (manuscript at 40) (on file with the Virginia Law Review
Association) (advocating uniform rules to govern patent procedure).
188 See Rochelle C. Dreyfuss, Percolation, Uniformity, and Coherent Adjudication: The
Federal Circuit Experience, 66 SMU L. Rev. 505, 535–36 (2013); supra note 6 and
accompanying text.
189 See 28 U.S.C. § 1295(a) (2012) (giving the Federal Circuit exclusive appellate patent
jurisdiction).
190 Nard & Duffy, supra note 27, at 1668; cf. Rochelle Cooper Dreyfuss, Lecture, What the
Federal Circuit Can Learn from the Supreme Court—and Vice Versa, 59 Am. U. L. Rev.
787, 803 (2010) (noting that the Federal Circuit “rarely provides insight into the policy
rationale for its own decisions”). For an argument that the Supreme Court can facilitate
experimentation under the current appellate structure, see John M. Golden, The Supreme
Court as “Prime Percolator”: A Prescription for Appellate Review of Questions in Patent
Law, 56 UCLA L. Rev. 657, 720 (2009).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
112 Virginia Law Review [Vol. 101:65
district court filing.191 Experimentalist systems should be aware of how
self-selection into local units might affect results.192
As Nard and Duffy explain, the value of sharing appellate patent
jurisdiction will be enhanced if the additional courts start “fresh” with
patent precedent, rather than adopting Federal Circuit precedent or
following their own pre-Federal Circuit case law.193 I would go further:
Congress should specify that it is spreading appellate patent jurisdiction
because it values the innovation and learning that disuniformity allows
(in contrast to the uniformity-focused legislative history of the Federal
Circuit’s creation), and that when given a choice between adopting the
same rule as the Federal Circuit or a different rule that could plausibly
be better (or at least is not demonstrably worse), the different rule should
be presumptively favored. An explicit congressional statement to this
effect might help overcome the natural bias toward harmonization that is
caused by judicial review. And the Supreme Court should recognize the
value of leaving a circuit split unresolved when the disuniformity causes
little harm and there are insufficient data to determine which is the better
rule.
Although jurisdictional reform seems unlikely, centralization of
patent appeals in a single appellate court may have seemed equally
unlikely before it happened (barely over thirty years ago). The Federal
Circuit continues to be subject to frequent Supreme Court rebukes194 and
191 Nard & Duffy, supra note 27, at 1668; cf. 28 U.S.C. § 2112(a)(3) (2012) (stating that
challenges to agency action filed in multiple districts shall have the court of appeals
determined by “random selection”); Samaha, supra note 115, at 5 (noting other random
aspects of adjudication).
192 As noted previously, patent examination experiments would benefit from ensuring that
patents are randomly assigned to examiners. See supra note 143 and accompanying text.
193 Nard & Duffy, supra note 27, at 1672–74.
194 For rebukes on patent issues since 2006, see Ass’n for Molecular Pathology v. Myriad
Genetics, Inc., 133 S. Ct. 2107, 2111, 2114–15, 2120 (2013); Gunn v. Minton, 133 S. Ct.
1059, 1062–63, 1068–69 (2013) (reversing a Texas Supreme Court decision that had
followed a Federal Circuit rule); Caraco Pharmaceutical Laboratories, Ltd. v. Novo Nordisk
A/S, 132 S. Ct. 1670, 1679–80, 1688 (2012); Mayo Collaborative Services v. Prometheus
Laboratories, Inc., 132 S. Ct. 1289, 1305 (2012); Bilski v. Kappos, 130 S. Ct. 3218, 3223–
24, 3226, 3231 (2010) (affirming the judgment but stating that the Federal Circuit was wrong
to hold that the “machine-or-transformation test” was the sole test for the patentability of
processes under 35 U.S.C. § 101); Quanta Computer, Inc. v. LG Electronics, Inc., 553 U.S.
617, 621, 638 (2008); KSR International Co. v. Teleflex Inc., 550 U.S. 398, 407, 415, 426–28
(2007); MedImmune, Inc. v. Genentech, Inc., 549 U.S. 118, 132, 137 (2007); Microsoft
Corp. v. AT&T Corp., 550 U.S. 437, 447, 457–59 (2007); eBay Inc. v. MercExchange,
L.L.C., 547 U.S. 388, 390–94 (2006). Cf. Ill. Tool Works Inc. v. Indep. Ink, Inc., 547 U.S.
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 113
scholarly criticism,195 and it might be possible to channel this criticism
into appellate reform.
C. Innovation Policy
The examples of patent examination and adjudication illustrate how
experimentalism could improve patent policy at the edges, such as by
improving the quality of issued patents, making litigation more efficient,
and testing the administrability of substantive patent standards. But the
hardest case for patent experimentation is varying substantive policy
with the goal of testing not administrability, but broader welfare effects.
As described in Part II, randomized experiments should be used when
choosing between well-defined policies with stable, measurable
outcomes and when minimizing spillovers is possible (such as by
randomizing over technologies), but the number of contexts where these
conditions will be met is limited.
What should be done for the majority of issues where it is unclear
which of many innovation policy options is optimal, and where testing
policies through a controlled experiment is infeasible? One could
advocate for (arbitrarily) choosing one policy to at least gain the benefits
of harmonization, but there is not even evidence that a uniform policy is
more efficient than a diverse mix of innovation policies.196 Furthermore,
as I have argued, policy uniformity makes it much more difficult to
make any empirical progress. And I have seen no justification for the
current approach of harmonizing across patent law but not across other
innovation laws such as R&D tax incentives, grants, or prizes.197 In
28, 32–33, 46 (2006) (vacating the Federal Circuit’s opinion, but noting that “respondent
reasonably relied on our prior opinions”).
195 See Dreyfuss, supra note 5, at 789 (“[O]bservers of the patent system have voiced
increasingly vociferous complaints about the state of patent jurisprudence, and by extension
about the Federal Circuit.”); Golden, supra note 190, at 718 (arguing that the Federal Circuit
should have recognized problems with its nonobviousness and patentable subject matter
doctrines before Supreme Court review because “[s]eparate reports by the Federal Trade
Commission and the National Academy of Sciences had highlighted problems with existing
doctrine” and “[s]cholarly and professional criticism had often been severe”).
196 For example, diversity could allow researchers with socially beneficial projects that are
not incentivized under the innovation policy in one jurisdiction to pursue their projects in a
different jurisdiction that does incentivize their projects.
197 While there is some limited coordination across research grants, see, e.g., Donors, Int’l
Fin. Facility for Immunisation, http://www.iffim.org/donors (last visited July 12, 2013), most
non-patent coordination efforts have been unsuccessful, see, e.g., Next Step: A Pilot, Health
Impact Fund, http://healthimpactfund.org/next-steps (last visited July 12, 2013).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
114 Virginia Law Review [Vol. 101:65
these contexts, the experimentalist approach appears promising for
promoting policy diversity while maintaining the benefits of central
coordination to limit the problems of externalities.
Since the 2000s, the European Commission (“EC”) has taken steps
toward experimentalist innovation policy with its Open Method of
Coordination (“OMC”) and other mechanisms for transnational
learning.198 Innovation-related framework goals include the objective of
increasing R&D intensity (R&D spending as a percentage of GDP) to
3%, with public R&D spending of 1% of GDP,199 as well as an “action
plan” with intermediate goals.200 Member states have discretion in
setting their own innovation policies, but they are subject to monitoring
and reporting requirements. States are compared quantitatively along
many dimensions of innovation, including R&D intensity, education,
scientific publications, the percentage of small- and medium-sized
enterprises innovating in-house, patents and trademarks, and
employment in knowledge-intensive industries.201 States are also subject
to detailed and prescriptive (albeit voluntary) peer reviews. For example,
Slovenia was told that its “poorly coordinated system and non-
transparent support schemes” should be addressed with an “inter-
Ministerial Cabinet Committee approach”; that grants should be
awarded based on “research excellence” rather than the “hardly
competitive” system of funding to particular research groups (with a
“success rate of . . . 95% (!)”); and that language requirements should be
relaxed “at least to enable postgraduate courses and programs in
English” to attract foreign students and academics.202 Recommendations
for Belgium included increasing coordination between its “complex and
fragmented” regional innovation programs (including by opening
198 See generally Nina McGuinness & Conor O’Carroll, Benchmarking Europe’s Lab
Benches: How Successful Has the OMC Been in Research Policy?, 48 J. Common Market
Stud. 293 (2010) (describing the progress of the OMC).
199 See Barcelona European Council, Presidency Conclusions 20 (Mar. 15–16, 2002).
200 Comm’n of the European Cmtys., Investing in Research: An Action Plan for Europe 3
(2003).
201 European Comm’n, Innovation Union Scoreboard 2013, at 4–5 (2013). The
Organisation for Economic Co-operation and Development (“OECD”) provides similar
benchmarking, which “focuses more on learning for improving performance in general
without common political objectives.” Marianne Paasi, Collective Benchmarking of Policies:
An Instrument for Policy Learning in Adaptive Research and Innovation Policy, 32 Sci. &
Pub. Pol’y 17, 18 (2005).
202 Frits von Meijenfeldt et al., European Research Area & Innovation Comm., Policy Mix
Peer Reviews: Country Report—Slovenia 14, 21–22, 29 (2010).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 115
regional funding programs to teams from other regions), focusing on
science education at younger ages to address a future “skills shortage,”
and increasing public R&D expenditures from 0.85% of GDP to the 1%
target.203
The main problem with the 3% OMC is that unlike in an ideal
experimentalist structure, there is no “hard” central control. The 3% goal
is non-binding, peer reviews are voluntary, and there are no incentives
for success or penalties for failure, creating “a clear mismatch between
the scale of the task and the resources deployed to meet this goal.”204
(The value of harder mechanisms is illustrated by another OMC on
researcher mobility that was far more successful in achieving specific
policy outcomes by using specific requirements in an area of EC
competence (immigration) and “the carrot of co-financing.”205) Still, the
3% target may have “succeeded in focusing attention”206 and serving as
a “mobilizing factor” that led to “a common cognitive framework in
R&D” and “dramatically increased focus on science and technology in
national political agendas.”207 A comprehensive assessment found some
specific evidence of policy learning.208 For example, the Netherlands
introduced an “innovation voucher” mechanism of small grants to
facilitate technology transfer between public institutions and small
businesses, and after the vouchers’ success was demonstrated through a
randomized study,209 the policy spread to many other member states.210
203 Joaquín Serrano Agejas et al., European Research Area & Innovation Comm., Policy
Mix Peer Reviews: Country Report—Belgium 3, 5, 7 (2011).
204 McGuinness & O’Carroll, supra note 198, at 311; see also Esko Aho et al., Indep.
Expert Grp. on R&D & Innovation, Creating an Innovative Europe 13 (2006) (arguing that
the 3% target is “a necessary but not sufficient condition for an innovative Europe”).
205 McGuinness & O’Carroll, supra note 198, at 305–06, 309.
206 European Comm’n, Europe 2020: A Strategy for Smart, Sustainable and Inclusive Growth
10 (2010); see R&D Expenditure, European Comm’n Eurostat, http://epp.eurostat.ec.europa.eu/
statistics_explained/index.php/R_%26_D_expenditure (last updated Sept. 25, 2014) (reporting
progress).
207 McGuinness & O’Carroll, supra note 198, at 307; see also Susana Borrás, Policy
Learning and Organizational Capacities in Innovation Policies, 38 Sci. & Pub. Pol’y 725,
730 (2011) (reviewing the literature on the benefits of the OMC).
208 Expert Grp. for the Follow-up of the Research Aspects of the Revised Lisbon Strategy,
European Comm’n, The Open Method of Coordination in Research Policy: Assessment and
Recommendations 25–26, 29 (2009).
209 Maarten Cornet, Björn Vroomen & Marc van der Steeg, Do Innovation Vouchers Help
SMEs to Cross the Bridge Towards Science? 3 (CPB Neth. Bureau for Econ. Policy
Analysis, Discussion Paper No. 58, 2006).
210 See Borrás, supra note 207, at 732.
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
116 Virginia Law Review [Vol. 101:65
My point is not that the EC’s innovation policy is ideal—it is simply
that a transjurisdiction experimentalist approach to innovation is
possible, and that it can be integrated with randomized field experiments
as appropriate. By requiring a certain amount of R&D spending (which
ideally would include the “shadow” reward to patentees211) rather than
requiring a specific innovation policy such as twenty-year patents,
countries could overcome the problem with the externalities of
innovation while providing greater room for policy experimentation and
removing the asymmetry that privileges patents above other mechanisms
for rewarding innovators.
Of course, as I have already explained, experimentalism does not
solve the difficulties with gathering meaningful evidence of the average
effects of new policies where spillovers, heterogeneities, and dynamic
effects make randomized experiments infeasible or not useful. And
unlike in the administrative context, it would be difficult to test
international experimentalism on a randomized basis. But the
experimentalist approach alone can still promote learning, as the E.U.
experience demonstrates. For example, it will produce varied
observational data to test theoretical and structural models,212 and it may
be possible to analyze some policy changes as “natural experiments.”213
The experimentalist structure—particularly the requirements of
articulation and peer review—will also directly encourage more robust
theory development, as well as the development of more qualitative
evidence such as case studies and interviews. These approaches may be
211 See Hemel & Ouellette, supra note 2, at 311–12.
212 Professor Chris Ansell has suggested that experimentalism could be analyzed under the
framework of design science, which “moves away from hypothesis testing under controlled
conditions and instead focuses on generating richer observations about the wider set of
interactions between the design and its context.” Ansell, supra note 30, at 172.
213 As described in Section I.C, although studies are limited by the lack of policy variation,
some empirical progress has already been made through these techniques. While the
experiments described there focus on whether patents promote innovation, a number of other
studies have taken advantage of natural variation to study other aspects of the patent system.
See, e.g., David S. Abrams & R. Polk Wagner, Poisoning the Next Apple? The America
Invents Act and Individual Inventors, 65 Stan. L. Rev. 517, 562 (2013); Joshua S. Gans,
David H. Hsu & Scott Stern, The Impact of Uncertain Intellectual Property Rights on the
Market for Ideas: Evidence from Patent Grant Delays, 54 Mgmt. Sci. 982, 982 (2008); Josh
Lerner, Patenting in the Shadow of Competitors, 38 J.L. & Econ. 463, 463 (1995); Michael
Noel & Mark Schankerman, Strategic Patenting and Software Innovation, 61 J. Indus. Econ.
481, 481 (2013); Stuart Graham & Deepak Hegde, Do Inventors Value Secrecy in Patenting?
Evidence from the American Inventor’s Protection Act of 1999, at 3 (Nov. 2, 2012)
(unpublished manuscript), available at http://ssrn.com/abstract=2170555.
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 117
better suited to making empirical progress on why patent policies work.
As Angus Deaton explains,
Without a prior theory and within its own evidentiary standards, [a
randomized controlled trial] targeted at “finding out what works” is
not informative about mechanisms, if only because there are always
multiple mechanisms at work. . . . Learning about theory, or
mechanisms, requires that the investigation be targeted toward that
theory, toward why something works, not whether it works. Projects
can rarely be replicated, though the mechanisms underlying success or
failure will often be replicable and transportable. . . .
. . .
In the end, there is no substitute for careful evaluation of the chain of
evidence and reasoning by people who have the experience and
expertise in the field.214
Experimentalism also directly encourages innovation in innovation
policy—the development of creative policy inputs for testing, such as
the Netherlands’ innovation vouchers. Just as experimentalism is likely
more effective than top-down randomization for improving the nuanced
process of prior art searching, it will also be superior for nuanced
policies whose implementation depends on buy-in from the local units,
as seems likely to be the case for policies such as awarding research
grants or attracting FDI.
Finally, experimentalism might be able to improve the metrics by
which policies are judged. I have argued that the goal of an evidence-
based patent regime should be to increase social welfare, and yet I have
not defined what this means. Increasing efficiency is an important
component of welfare, but the choice between policies also necessarily
implicates important value judgments, such as the extent to which
innovation should be subsidized by everyone or only by users of the
resulting products.215 Indeed, some scholars have argued that the
persistent empirical uncertainty over what is “efficient” in IP law means
policy makers should focus much more heavily on other values.216 These
214 Deaton, supra note 149, at 441–42, 450.
215 See Hemel & Ouellette, supra note 2, at 345–52.
216 See Robert P. Merges, Justifying Intellectual Property 2–4 (2011); Amy Kapczynski,
The Cost of Price: Why and How to Get Beyond Intellectual Property Internalism, 59 UCLA
L. Rev. 970, 970 (2012); Oskar Liivak, Maturing Patent Theory from Industrial Policy to
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
118 Virginia Law Review [Vol. 101:65
non-efficiency values are easier to measure when central planners
encourage local units to take an active role in defining success.
In sum, experimentalism maintains a Hayekian recognition of the
value of local knowledge.217 Given the extent of empirical uncertainty
about innovation policy and the lack of consensus about what increases
social welfare, the best approach for many issues may be to encourage
greater policy variation through an experimentalist method that
generates local investment in improving innovation policy, promotes
local values, and allows greater exploration of what policy options are
even possible.
IV. TOWARD EXPERIMENTALIST PATENT POLICY
Thus far, I have described a variety of approaches to patent policy
experimentation that can help make progress on the key question for
patent scholars: How can patent policy be improved, in light of the array
of other innovation policies, to increase social welfare? I have argued for
a pluralistic approach, in which some issues are tested through
randomized laboratory or field experiments and others are explored
through experimentalism’s blend of bottom-up learning and robust
feedback. As I have explained, these different approaches will be most
valuable in different contexts: Randomized laboratory experiments are
optimal for inexpensive initial tests of new ideas. Randomized field
experiments should be used to evaluate well-defined policy options with
measurable outcomes that are expected to be relatively stable across
time and across jurisdictions, and when spillovers between assignments
can be minimized. And experimentalism is valuable for generating buy-
in, promoting local policy innovations, eliciting local knowledge about
the applicability of different policies to heterogeneous conditions, and
generating better measures of whether policies are succeeding in light of
local values.
But who should decide when to use which approach, and how should
different methods of policy experimentation be blended? These are
difficult questions, worthy of articles of their own, but I will note that
Intellectual Property, 86 Tul. L. Rev. 1163, 1165 (2012); David McGowan, Copyright
Nonconsequentialism, 69 Mo. L. Rev. 1, 1–2 (2004); Alfred C. Yen, The Interdisciplinary
Future of Copyright Theory, 10 Cardozo Arts & Ent. L.J. 423, 424 (1992).
217 Friedrich A. Hayek, The Use of Knowledge in Society, 35 Am. Econ. Rev. 519, 519–
20 (1945).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 119
experimentalism itself may be a promising institutional structure for
choosing whether policy randomization or the experimentalist approach
of regulated local autonomy is more appropriate for a given issue.
Forcing local units to justify their policy choices based on existing
evidence may highlight the gaps in our empirical knowledge and may
encourage local units to consent to having their policy set randomly
where that is the most promising way to make progress. But there are
still complex questions of which institutions should be the “centers” of
these experimentalist systems. This Part lays out a few considerations
for implementation of patent experimentalism in the domestic and
international arenas and concludes by suggesting that these institutions
could learn from trends in medicine toward a personalized, evidence-
based approach.
A. Domestic Institutional Choice
As noted above, laboratory experiments, district court experiments,
and PTO experiments are all already occurring to varying extents. This
should be encouraged, especially in ways that are more transparent and
reproducible. As Professors Gary King and Eleanor Neff Powell explain,
knowledge develops by “iterating between theory development and
building empirical evidence,”218 and researchers should seek “new
observable implications of the same theory, collect those data, and see
whether they are consistent with the theory.”219 Laboratory experiments
can complement the kinds of evidence described in Part I, including
qualitative case studies and econometric studies. Based on this iteration
between theory and experiment, scholars should suggest policy changes
that seem ready for broader testing220—and we should also be clear
about the limitations of existing theories and data.221
The main challenge for an evidence-based patent regime will be
creating variation in substantive innovation policies. I have described
two approaches to central promotion of policy variation: field
218 King & Powell, supra note 56, at 2.
219 Id. at 4.
220 For example, it seems more likely than not that longer exclusivity or increased funding
for drugs requiring long clinical trials would be an improvement. See Budish et al., supra
note 79, at 1–2.
221 Cf. Lee Epstein & Gary King, The Rules of Inference, 69 U. Chi. L. Rev. 1, 15–17
(2002) (finding that every one of hundreds of law review articles reviewed violated at least
one basic rule of inference).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
120 Virginia Law Review [Vol. 101:65
experiments in which specific policies are randomly assigned to local
units, and experimentalist regimes in which local units have a voice in
setting goals and metrics and discretion in choosing policies to meet
these goals. In Parts II and III, I gave examples of when each of these
approaches might be promising (including blended examples in which
the use of experimentalism itself is randomly assigned). Some of these
experiments would initially require acts of Congress, but who should run
these experiments and interpret the data?222
One possibility is the PTO’s Office of the Chief Economist (“OCE”),
which was created in March 2010 with research goals including
“[u]ncovering how IP relates to economic growth, performance and
employment.”223 Alternatively, Professors Stuart Benjamin and Arti Rai
have called for a new innovation agency, which could be appropriate for
these second-order questions of innovation about innovation policy.224
Abramowicz, Ayres, and Listokin have argued that the Office of
Management and Budget (“OMB”) or a new agency should be tasked
with running and interpreting the results of all policy experiments.225
And Professor John McGinnis has suggested that a congressional
agency, modeled on the Congressional Budget Office, should
recommend experiments to be inserted into new legislation.226
These are all plausible options, and the decision raises standard issues
of administrative law and institutional choice. But I note here one less
obvious consideration: Policy makers should be attentive to evidence on
how to design and communicate experiments so that the resulting
knowledge has the policy impact it should, rather than being explained
away through motivated reasoning by those who do not find it
222 This Article focuses on government-driven innovation policy, so the “who” is typically
a government actor. It is worth noting, however, that private companies offer diverse
innovation incentives to their employees, and that this may be a fertile ground for future
empirical study.
223 Press Release, No. 10-63, U.S. Patent & Trademark Office, USPTO Chief Economist
Unveils Agency’s New Economic Research Agenda During Conference on Intellectual
Property and the Innovation Economy (Dec. 9, 2010), available at http://www.uspto.gov/
news/pr/2010/10_63.jsp.
224 Stuart Minor Benjamin & Arti K. Rai, Fixing Innovation Policy: A Structural
Perspective, 77 Geo. Wash. L. Rev. 1, 1, 6 (2008).
225 Abramowicz et al., supra note 14, at 982–84. A possible location within the OMB is the
Office of Information and Regulatory Affairs (“OIRA”). See generally Cass R. Sunstein,
The Office of Information and Regulatory Affairs: Myths and Realities, 126 Harv. L. Rev.
1838 (2013) (describing OIRA).
226 McGinnis, supra note 115, at 55.
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 121
cognitively congenial. In a separate essay, I argue that patent discourse
seems to have a similar pathology to other polarized debates over facts
(such as over climate change), with cultural values shaping priors and
affecting the weight given to new evidence.227 Before investing in an
expensive randomized experiment, it may be prudent to fund smaller
experiments on whether acceptance of new evidence on patents depends
on how it is communicated or who is doing the experiments.
From this perspective, the PTO’s OCE might be suboptimal as a
center for patent experimentation due to the possibility of perceived
bias. The OCE’s first report did not inquire into how IP relates to
growth: It simply quantified the economic contribution (not the IP-
specific contribution) of an extremely broad list of “IP-intensive”
industries as five trillion dollars.228 The PTO website advertised this as
“IP Contributes $5 Trillion and 40 Million Jobs to US Economy,”
leading to praise from pro-patent commentators229 and criticism from
patent reformers.230 Even if the OCE later authors a report that does
quantify the economic contribution of IP, it might do little to lessen the
polarization over patents because of the OCE’s position within the PTO.
An independent agency might thus be preferable for coordinating U.S.
experiments with substantive innovation policy.
B. International Policy and TRIPS Constraints
The biggest hurdle to experimentation with substantive patent law is
the international TRIPS agreement, which requires WTO members (that
is, almost all countries231) to make twenty-year patents “available for
227 Ouellette, supra note 83, at 28; see also Dan M. Kahan, Ellen Peters, Maggie Wittlin,
Paul Slovic, Lisa Larrimore Ouellette, Donald Braman & Gregory Mandel, The Polarizing
Impact of Science Literacy and Numeracy on Perceived Climate Change Risks, 2 Nature
Climate Change 732, 732 (2012) (finding that increased science literacy increases
polarization over climate change).
228 Econ. & Statistics Admin. & U.S. Patent & Trademark Office, U.S. Dep’t of
Commerce, Intellectual Property and the U.S. Economy: Industries in Focus 45 (2012).
229 See, e.g., Renee C. Quinn, IP Contributes $5 Trillion and 40 Million Jobs to US
Economy, IP Watchdog (Apr. 11, 2012, 6:24 PM), http://www.ipwatchdog.com/2012/
04/11/ip-contributes-5-trillion-and-40-million-jobs-to-us-economy.
230 See, e.g., Stephan Kinsella, USPTO/Commerce Dept. Distortions: “IP Contributes $5
Trillion and 40 Million Jobs to Economy,” Ctr. for the Study of Innovative Freedom (Apr.
20, 2012), http://c4sif.org/2012/04/usptocommerce-dept-distortions-ip-contributes-5-trillion-
and-40-million-jobs-to-economy.
231 See Members and Observers, World Trade Org., http://www.wto.org/english/thewto_e/
whatis_e/tif_e/org6_e.htm (last visited July 12, 2013).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
122 Virginia Law Review [Vol. 101:65
any inventions . . . in all fields of technology.”232 But there is still some
room for diversity. TRIPS explicitly recognizes the value of local
implementation,233 and patents are not required for medical methods,
“animals other than micro-organisms,” or inventions that do not meet
the open-textured requirements of being “new,” involving an “inventive
step,” and being “capable of industrial application.”234 Countries may
grant “limited exceptions” to patent rights or, in some cases, compulsory
licenses.235
Numerous scholars have advocated an expansive interpretation of
TRIPS flexibilities,236 and some countries have been pushing against
patent harmonization. For example, India has made use of TRIPS
flexibilities in the pharmaceutical sector,237 and the Indian Supreme
Court recently affirmed the high bar for obtaining “evergreening”
patents.238 Similarly, Canada has raised the utility bar for pharmaceutical
patents based on the benefit promised in the patent application.239 The
United States or U.S. companies are challenging these decisions,240 but
232 TRIPS, supra note 8, arts. 27, 33.
233 Id. art. 1 (“Members shall be free to determine the appropriate method of implementing
the provisions of this Agreement within their own legal system and practice.”).
234 Id. art. 27.
235 Id. arts. 30–31; see also World Trade Organization, Declaration on the TRIPS
Agreement and Public Health ¶ 5b, Nov. 14, 2001, 41 I.L.M. 755, 755 (2002) (expressly
confirming the right of each country to determine exhaustion rules and permit parallel
imports).
236 See, e.g., Dinwoodie & Dreyfuss, supra note 25, at 201; Cynthia M. Ho, Access to
Medicine in the Global Economy: International Agreements on Patents and Related Rights
223–51 (2011); Kapczynski, supra note 8, at 1574; Katherine J. Strandburg, Evolving
Innovation Paradigms and the Global Intellectual Property Regime, 41 Conn. L. Rev. 861,
896–97 (2009). Professor Sarah Rajec argues that tailored laws should be evaluated based on
their “harm to concerns of certainty, fairness, economy, and public choice.” Sarah R.
Wasserman Rajec, Evaluating Flexibility in International Patent Law, 65 Hastings L.J. 153,
208 (2013).
237 See Ho, supra note 236, at 91–124; Kapczynski, supra note 8, at 1573–75.
238 Novartis AG v. Union of India, A.I.R. 2013 S.C. 1311, 1367–68 (India) (concluding
that a patent on a cancer drug with enhanced stability and bioavailability is invalid under
§ 3(d) of the Indian Patents Act, which requires enhanced “therapeutic efficacy”).
239 See Eli Lilly Can. Inc. v. Novopharm Ltd. (2010), [2012] 1 F.C.R. 349, para. (76 Can.
Fed. Ct. App.) (“Where the specification does not promise a specific result, no particular
level of utility is required; a ‘mere scintilla’ of utility will suffice. However, where the
specification sets out an explicit ‘promise’, utility will be measured against that promise.”).
240 See Notice of Intent to Submit a Claim to Arbitration Under NAFTA Chapter Eleven
¶ 55, Eli Lilly & Co. v. Canada (Nov. 7, 2012), available at http://www.international.gc.ca/
trade-agreements-accords-commerciaux/assets/pdfs/disp-diff/eli-01.pdf (challenging Canada’s
“promise” doctrine); Demetrios Marantis, Office of the U.S. Trade Representative, Exec.
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 6:40 PM
2015] Patent Experimentalism 123
should instead welcome these innovations because of the potential for
policy learning.
While TRIPS allows minor experiments with patent policy as well as
experiments with using alternative incentives as supplements to the
current patent regime (such as conditioning enhanced R&D tax credits
on relinquishment of patent protection or agreement to shorter patent
terms241), more radical patent experimentation would require a TRIPS
amendment. As I have argued, there is no good reason to require
international coordination on patents but not on other rewards for
innovators such as R&D tax credits, grants, or prizes. Of course, I
recognize that the political economy of patent reform makes amending
TRIPS highly unlikely.242 An attraction of TRIPS for the United States
is that it prevents U.S. consumers from subsidizing technology for the
world by ensuring that some of the costs of U.S. R&D are borne by
consumers overseas who pay higher prices for U.S.-patented products.
But the global IP political landscape is shifting. Many advanced
economies—including France, Germany, Spain, and Japan—are now net
IP importers.243 And although the IP-exporting United States has likely
been a net beneficiary under TRIPS,244 this too may be changing. The
number of patents granted to foreign inventors has been growing faster
than the number of patents granted to U.S. inventors, and since 2008, the
PTO has issued more patents to foreigners than to U.S. residents.245
Perhaps as the United States joins other advanced economies as no
Office of the President, 2013 Special 301 Report, at 38 (2013), available at
http://www.ustr.gov/sites/default/files/05012013%202013%20Special%20301%20Report.pdf
(“The United States is concerned that the recent decision by India’s Supreme Court with
respect to India’s prohibition on patents for certain chemical forms absent a showing of
‘enhanced efficacy’ may have the effect of limiting the patentability of potentially beneficial
innovations.”).
241 See Hemel & Ouellette, supra note 2, at 370.
242 See generally F.M. Scherer, The Political Economy of Patent Policy Reform in the
United States, 7 J. Telecomm. & High Tech. L. 167, 201–07 (2009) (describing the opposing
interests in patent reform debates).
243 See Keith E. Maskus, Intellectual Property Rights in the Global Economy 81 (2000).
244 See Nat’l Sci. Bd., 1 Science and Engineering Indicators 2008 ch. 6, at 6 (2008) (noting
that the United States continues to be a net IP exporter); Keith E. Maskus, Intellectual
Property Rights and Economic Development, 32 Case W. Res. J. Int’l L. 471, 493 (2000)
(estimating that TRIPS would lead to net inflow of $5.8 billion into the United States per
year).
245 See Dennis Crouch, Does the Shift in IP Ownership Predict a Political Shift in the IP
Debate?, Patently-O (Mar. 11, 2013), http://www.patentlyo.com/patent/2013/03/does-the-
shift-in-ip-ownership-predict-a-political-shift-in-the-ip-debate.html.
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
124 Virginia Law Review [Vol. 101:65
longer being a clear winner under TRIPS, there will be some momentum
for reform.
If the WTO or another international organization emerges as the
center of global patent experimentalism, it can learn from the European
Union’s OMC experience. As discussed in Section III.C, by requiring a
policy output such as public R&D spending of one percent of GDP,
rather than a specific policy input such as twenty-year patents, countries
can avoid the problems with externalities in innovation policy while
allowing room for experimentation.246 But at least three improvements
should be made if the OMC approach is transferred to the broader
international context. (1) Financial incentives should be used to enforce
objectives, the way the WTO’s threat of trade sanctions has given teeth
to the TRIPS regime. (2) Although the OMC has focused on mutual
learning with the ultimate goal of harmonization of innovation policy,
one of the framework goals of international patent experimentalism
should be policy variation itself—countries should be rewarded for
trying a different policy when the outcome is ambiguous based on
existing theory and data. (3) Countries should be encouraged to use their
increased autonomy to experiment with incentives in ways that are
susceptible to measurement; for example, even if a new incentive is not
tested with a randomized trial, it may still be introduced in ways
susceptible to counterfactual analysis.247
Even if the constraints of TRIPS prove too rigid for significant patent
policy variation in the short term, countries can still experiment with
non-patent innovation policies. Countries are free to offer different
rewards to innovators as optional substitutes for patents. And as
previously discussed, innovation is also influenced by a host of other
laws and institutions beyond these financial rewards, and it may be
easier to experiment with these non-patent policies. The difficulties of
learning from uniform patent policy should provide a cautionary tale for
efforts to increase uniformity in these other innovation policies. For
example, variation in the enforceability of noncompete agreements has
suggested that they discourage employee mobility and slow economic
246 See supra notes 198–203 and accompanying text.
247 See Heidi Williams, Innovation Inducement Prizes: Connecting Research to Policy, 31
J. Pol’y Analysis & Mgmt. 752, 770 (2012) (describing an ex post counterfactual analysis
for a recent trial advance market commitment for a pneumococcal vaccine).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 125
growth,248 and the possibility of learning more from this and other
variations in trade secrecy protection weighs against enacting a federal
trade secrets act.249
C. Learning from Personalized, Evidence-Based Medicine
Finally, I would urge any institution that is implementing a patent
experimentalist regime (and the scholars who study such institutions)
not only to adopt a pluralistic approach to policy experiments, but also
to continue to learn—including from outside the legal field—about how
to improve this experimental process. In particular, patent
experimentalists might borrow from the medical literature, where
researchers face a similar problem of isolating the causal effects of
particular treatments in complex systems.
The evidentiary gold standard of randomized controlled trials evolved
in the context of medical treatments.250 But randomized trials of medical
treatments are not perfect: They raise the same concerns as those
discussed above for randomized policy experiments, in that they reveal
only the average treatment effect for the trial group. Randomized
controlled trials of medical interventions “often lack external validity,”
and regulators have been urged to pay more attention to assessing the
generalizability of trial results.251 And the criticism of focusing on “what
works” at the expense of “why” has also been leveled in the medical
context.252
248 Ronald J. Gilson, The Legal Infrastructure of High Technology Industrial Districts:
Silicon Valley, Route 128, and Covenants Not to Compete, 74 N.Y.U. L. Rev. 575, 578
(1999).
249 Cf. Protecting American Trade Secrets and Innovation Act of 2012, S. 3389, 112th
Cong. § 2 (2012) (providing federal jurisdiction for theft of trade secrets).
250 See Marcia L. Meldrum, A Brief History of the Randomized Controlled Trial: From
Oranges and Lemons to the Gold Standard, 14 Hematology/Oncology Clinics N. Am. 745,
745–46 (2000); see also Upjohn Co. v. Finch, 422 F.2d 944, 946 (6th Cir. 1970) (upholding
the FDA’s decision to forbid the sale of a drug that did not satisfy its new regulations
requiring well-controlled trials); 21 C.F.R. § 314.126 (2012) (laying out FDA requirements
for well-controlled studies).
251 Peter M. Rothwell, External Validity of Randomised Controlled Trials: “To Whom Do
the Results of This Trial Apply?,” 365 Lancet 82, 82 (2005).
252 See, e.g., Robert B. Nussenblatt et al., Translational Medicine—Doing It Backwards, 8
J. Translational Med., Feb. 2010, at 3 (criticizing the “[l]ack of sufficient rigor in
conceptualizing clinical studies aimed not only at validation of therapies but also of learning
from all results so as to better design subsequent trials.”).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
126 Virginia Law Review [Vol. 101:65
The past few decades have seen trends in medical research toward
“personalized medicine,” which focuses on understanding the
mechanisms by which different treatments operate so that they can be
applied to the right patients at the right time.253 There has also been
evidence that well-designed observational studies can be as effective as
randomized trials,254 coupled with optimism about the role of electronic
health records in speeding the “progression of knowledge.”255 The
medical community’s incorporation of diverse sources of evidence to
optimize health outcomes for heterogeneous patients may be instructive
for patent experimentalists’ efforts to optimize innovation outcomes
over heterogeneous jurisdictions and technologies.
Additionally, to focus attention on where the gaps in its knowledge
are, it might be useful for an innovation agency (or other center of patent
experimentalism) to follow the lead of the evidence-based medicine
movement in grading the available evidence on different issues.256 Just
as “some questions about therapy do not require randomised trials . . . or
cannot wait for the trials to be conducted,”257 some patent policy
decisions need to be made with lower confidence in available evidence.
Like experimentalism, evidence-based medicine seeks to blend the best
external evidence with the local clinical experience to solve complex
problems,258 and there are probably additional lessons that patent
experimentalism—and legal experimentation more generally—could
learn from this field.
253 See, e.g., Lynda Chin et al., Cancer Genomics: From Discovery Science to
Personalized Medicine, 17 Nature Med. 297, 297 (2011) (stating that “a solid mechanism-
based understanding” of how cancer treatments function is necessary for progress); Geoffrey
S. Ginsburg & Huntington F. Willard, Genomic and Personalized Medicine: Foundations
and Applications, 154 Translational Res. 277, 286 (2009) (“Personalized medicine seeks to
use advances in knowledge about genetic factors and biological mechanisms of disease
coupled with unique considerations of an individual’s patient care needs . . . .” (quoting
Nomination of Kathleen G. Sebelius: Hearing Before the S. Comm. on Fin., 111th Cong. 190
(2009) (responses to questions for the record by Hon. Kathleen Sebelius))).
254 See John Concato et al., Randomized, Controlled Trials, Observational Studies, and the
Hierarchy of Research Designs, 342 New Eng. J. Med. 1887, 1890 (2000).
255 Charles P. Friedman et al., Achieving a Nationwide Learning Health System, 2 Sci.
Translational Med., Nov. 2010, at 1.
256 See, e.g., Gordon H. Guyatt et al., GRADE: An Emerging Consensus on Rating Quality
of Evidence and Strength of Recommendations, 336 Brit. Med. J. 924, 924 (2008).
257 David L. Sackett et al., Evidence Based Medicine: What It Is and What It Isn’t, 312
Brit. Med. J. 71, 72 (1996).
258 See id. at 71; R. Brian Haynes et al., Clinical Expertise in the Era of Evidence-Based
Medicine and Patient Choice, 7 Evidence-Based Med. 36, 38 (2002).
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
2015] Patent Experimentalism 127
CONCLUSION
I make no claim that the approach to promoting innovation in
innovation policy laid out above will be easy to achieve. But I think it is
valuable to set aside perceived political constraints and ask: “If public-
interest-minded scholars had complete freedom to design patent policy,
is uniformity really a desirable goal? Would it be possible to learn more
from nonuniform patent policy? And if so, what kind of nonuniformity
would we want?” This Article has attempted to answer these questions.
I have argued that empirical progress depends critically on policy
variation, but also on the right kind of policy variation. Because
jurisdictions do not internalize either all the benefits of their innovation
policies or all the benefits of their innovations with innovation policies,
and because these spillovers complicate empirical evaluation, complete
local autonomy is not the right answer. Rather, policy experiments
should be deliberately promoted by central planners. Randomization
should be used when choosing between well-defined policy options with
measurable outcomes that are expected to be relatively stable, and when
the policies can be randomly assigned such that there are minimal
spillovers between experimental groups. When these conditions are not
met, experimentalism’s regulated local autonomy may be the best way
to make progress. In particular, experimentalism is valuable for
generating buy-in from local units, for promoting local innovation with
policy design, for eliciting local knowledge about the applicability of
different policies to heterogeneous conditions, and for generating better
measures of whether policies are succeeding in light of local values.
Patents and other innovation laws involve government interference to
correct market failures. But the government is not omniscient; patents
themselves represent a somewhat Hayekian recognition of the
distribution of knowledge, allowing central planners to elicit privately
held information about potential innovations.259 Experimentalism might
allow central planners to make a similar move with innovation law itself,
eliciting local information about policy improvements. But just as
patents are suboptimal when the government does have good
information about potential R&D projects,260 regulated local autonomy
is not always the best choice for promoting patent policy variation.
259 Hayek himself was skeptical that the patent system actually induces innovation. F.A.
Hayek, The Fatal Conceit: The Errors of Socialism 37 (W.W. Bartley III ed., 1989).
260 See Hemel & Ouellette, supra note 2, at 330.
OUELLETTE_BOOK (DO NOT DELETE) 2/20/2015 5:55 PM
128 Virginia Law Review [Vol. 101:65
In 2005, Mark Lemley described the difficulty of optimizing IP
protection:
We may not know exactly how to calibrate the right level of
intellectual property protection, but we can be reasonably certain that
neither “no protection” nor “absolute control over externalities” is the
right answer. Hard as it is to get the balance right, we will never do it
if we simply stop trying.261
Given the ambiguity of the evidence discussed in Part I and the array of
non-patent innovation incentives, I do not think we are even certain that
“no protection” is not the right answer. (In other words, I think the
existing evidence would receive a low “grade” on an evidence-based-
medicine-like grading scale.262) But I strongly agree that we should not
stop trying, which is what we are doing by pushing the world toward a
uniform system with ambiguous welfare consequences.
Finally, while this Article has developed a framework for policy
experimentation in the innovation context, the pros and cons of different
approaches to experimentation that this Article has explored are trans-
substantive, so this framework applies to policy learning beyond patents.
As I have explained, patent law presents a hard case for experimentation
because of measurement difficulties, spillovers, and its impact on
multinational actors with long time horizons, but these problems are not
unique to patents. And designing useful policy experiments will be even
easier in fields that do not have all of these difficulties, and that have not
already been pushed so strongly toward uniformity. I doubt there is a
legal field without important open empirical questions, so most fields
will likely benefit from increased attention to how policy diversity can
be promoted in the way that is most conducive to evidence-based
learning.
261 Lemley, supra note 45, at 1067.
262 See supra notes 256–58 and accompanying text (proposing such a scale).
