Performance assessment in science and academia: effects of the RAE/REF on academic life

Full text

! !
!
!
Centre for Global Higher Education working paper series
Performance assessment in
science and academia:
effects of the RAE/REF on
academic life
Len Ole Schäfer
Working paper no. 7
September 2016
!

Published by the Centre for Global Higher Education,
UCL Institute of Education, London WC1H 0AL
www.researchcghe.org
© Centre for Global Higher Education 2016
ISSN 2398-564X
The Centre for Global Higher Education (CGHE) is the largest research
centre in the world specifically focused on higher education and its future
development. Its research integrates local, national and global perspectives
and aims to inform and improve higher education policy and practice.
CGHE is funded by the Economic and Social Research Council (ESRC)
and the Higher Education Funding Council of England (HEFCE), and is a
partnership led by the UCL Institute of Education with Lancaster University,
the University of Sheffield and international universities Australian National
University (Australia), Dublin Institute of Technology (Ireland), Hiroshima
University (Japan), Leiden University (Netherlands), Lingnan University
(Hong Kong), Shanghai Jiao Tong University (China), the University of Cape
Town (South Africa) and the University of Michigan (US).
The support of the Economic and Social Research Council (ESRC) and the
Higher Education Funding Council of England (HEFCE) is gratefully
acknowledged.

Performance assessment in science and academia:
effects of the RAE/REF on academic life
Len Ole Schäfer
Contents
List of tables .................................................................................................................. 3!
List of figures ................................................................................................................ 3!
List of abbreviations .................................................................................................... 4!
Abstract .......................................................................................................................... 1!
1. Introduction ............................................................................................................... 3!
2. Theoretical framework ............................................................................................ 4!
2.1 Academic life ............................................................................................................................ 4!
2.2 State of research ..................................................................................................................... 6!
2.3 Higher educational alterations .............................................................................................. 7!
2.4 Functional model of performance assessment in science and academia ............... 10!
2.5 Conflict theoretical model of performance assessment in science and academia 12!
2.6 Hypotheses ............................................................................................................................ 13!
3. Empirical procedure ............................................................................................ 15!
3.1 Qualitative interviews and sample structures ................................................................ 15!
3.2 Questionnaires ...................................................................................................................... 17!
3.3 Analytical strategy ................................................................................................................ 18!
4. Empirical results ................................................................................................... 19!
4.1 The academic and scientific sphere ................................................................................ 19!
4.1.1 Research performance ................................................................................................ 19!
4.1.2 Inequality ........................................................................................................................ 20!
4.1.3 Diversity .......................................................................................................................... 21!

4.1.4 Academic freedom ....................................................................................................... 22!
4.1.5 The relationship between teaching and research ................................................. 23!
4.1.6 Recruitment policies .................................................................................................... 24!
4.1.7 Research motivation and pressure ........................................................................... 26!
4.1.8 Power in performance assessment in science and academia ........................... 26!
4.2 Connections between the academic, scientific and political sphere ....................... 27!
4.2.1 Research performance ................................................................................................ 27!
4.2.2 Inequality ........................................................................................................................ 27!
4.2.3 Diversity .......................................................................................................................... 28!
4.2.4 Academic freedom ....................................................................................................... 28!
4.2.5 The relationship between teaching and research ................................................. 29!
4.2.6 Recruitment policies .................................................................................................... 30!
4.2.7 Research motivation and pressure ........................................................................... 30!
4.2.8 Power .............................................................................................................................. 30!
5. Summary ................................................................................................................ 31!
Empirical results ........................................................................................................................... 31!
Scientists and academics ..................................................................................................... 31!
Connections between scientists, academics and politics ............................................. 32!
Research results in the light of the state of research .......................................................... 33!
Future research ............................................................................................................................ 35!
Literature ..................................................................................................................... 36!
List of tables
Table 1: Sample 1 ........................................................................................................................... 16
Table 2: Sample 2 ........................................................................................................................... 16
List of figures
Figure 1: RAE/REF and academic life ........................................................................................... 6
Figure 2: Higher educational alterations ....................................................................................... 8
Figure 3: Functional model of performance assessment in science and academia ......... 11
Figure 4: Conflict-theoretical model of performance assessment in science and
academia ............................................................................................................................................ 13
Figure 5: The effects of the RAE/REF in context ..................................................................... 19
Figure 6: Research interest, panel criteria and academic freedom ..................................... 23

Figure 7: Strategic decisions triggered by the RAE/REF ....................................................... 25
Figure 8: Motivational effects of the RAE/REF ......................................................................... 26
List of abbreviations
Research Assessment Exercise (RAE)
Research Excellence Framework (REF)

www.researchcghe.org
1
Performance assessment in science and academia:
effects of the RAE/REF on academic life
Len Ole Schäfer
Len Ole Schäfer is a PhD student in Sociology at the University of Bamberg,
Germany. Email: len-ole.schaefer@uni-bamberg.de
Abstract
This working paper extends Burton R. Clark's notion of academic life and applies, in
the first step, theories of higher educational alterations on performance assessment
in science and academia. There are six different functions of performance
assessment in science and academia. First, there is a belief in numbers. Second,
there is reactivity due to indicators and judgement criteria. Third, there are market
models effecting performance assessment in science and academia. Fourth, there is
business control of scientific and academic entities. Fifth, there is discipline and
control of universities. Sixth, these functions go hand in hand with global quality
seals.
In the second step, functional and conflict-theoretical models are developed to
explain performance assessment in science and academia. The first model explains
the production of knowledge in universities as a stratification of meritocratic
performance entities, such as performance, quality, talent and achievement on the
basis of equality of opportunity. The second model explains the production of
knowledge as being stratified by power, capital and status hierarchies on the basis of
inequality of opportunity. These two theoretical models are combined to explain
performance assessment in science and academia in the UK.
In the next step, the paper analyses the effect of the Research Assessment Exercise
(RAE) and Research Excellence Framework (REF) on academic life. Research
results show that the RAE/REF affects a) research performance; b) inequality; c)
diversity; d) academic freedom; e) the relationship between teaching and research; f)
recruitment policies; g) research motivation; and i) power. One of the main research
results reveals that it is not clear whether the performance of UK science and

www.researchcghe.org
2
academia has grown in the wake of the mechanisms of the RAE/REF. The RAE/REF
increases inequality in the treatment of staff, gender and minorities due to its rules
and implementation practices in the university sector. Nevertheless, there are
initiatives to counteract this rising inequality. Research diversity looks set to be
reduced due to a lesser likelihood of change in fundamental research tracks. The
RAE/REF is likely to inhibit diversity in research topics due to scientists and
academics conforming to the RAE/REF time cycle. Academic freedom in terms of
contribution to the academic and scientific community is likely to be reduced, and the
choice of research topics is limited in view of the strong emphasis on high impact
journals and the introduction of impact as a measure of economic and social benefits
in the real world. Both the recruitment process and the academic choice of research
topics are affected as the impact agenda favours short-term relationships over long-
term relationships alongside speculative, creative, innovative, risky and blue sky
research. Research intensive universities show a decrease in their motivation to
produce advanced, globally leading research due to the pressure exerted by the
RAE/REF to yield ongoing research results within a certain time frame. The lower
ranked institutions are under pressure and control due to universities' quality
management aimed at performing better in the RAE/REF. There is a wide use of
performance reviews, computer models and information systems to raise RAE/REF
scores in the long run. Scientists, academics and the micro-management are putting
more emphasis on the way in which they can gain more effective case studies.
There is also evidence that performance assessment in science and academia is
based on power relationships in the context of the RAE/REF.

www.researchcghe.org
3
1. Introduction
This working paper is designed to provide an introduction of the subject matter and
will present my research questions. I will then offer an insight into my theoretical
framework and explain my theoretical hypotheses for the qualitative study. Finally, I
will introduce my empirical procedure, referring to sample procedures and
assessment methods, and will present my empirical research results. At the end of
the paper I will give a summary and an outlook to further research.
We are currently experiencing a global research competition for an increase in
scientific performance and the recruitment of research stars (cf. Münch 2011;
Schäfer 2012; Münch 2014; Münch and Schäfer 2014; Wieczorek and Schäfer
2016). This development is accompanied by a search for institutional models
designed to raise research performance (cf. Münch 2014) as well as by reforms to
enhance economy, efficiency and effectiveness of higher education systems (cf. B.
Clark 1998:2). This competition is rooted in the redefinition of the academic field into
markets on both a national and global level (cf. Marginson 2006). This leads to a
stratification of universities (cf. Marginson and Wende 2007; Hazelkorn 2008, 2009;
Hazelkorn and Ryan 2013; Hazelkorn 2014; Münch 2011, 2014) as a result of the
audit culture (cf. Strathern 1997; Power 1999) and shows a concentration of
research resources in the top institutions (cf. Henkel 1999; Marginson, Murphy, and
Peters 2010:157).
The instruments used to increase scientific and academic performance in the UK are
the Research Assessment Exercise (RAE) and Research Excellence Framework
(REF). These have an impact on academic life in the UK. Nicholas Stern recently
undertook a review of the Research Excellence Framework (REF) and there is
currently much debate in the academic, media and political sphere as to the effects
of the RAE/REF on academic life (cf. Matthews 2015a, 2015b, 2016a, 2016b, 2016c,
2016d; Oancea 2016; Bentley 2016; Cuthbertson 2016; Locke 2016).
With this context in mind, I analyse the effects of the Research Assessment Exercise
(RAE) and Research Excellence Framework (REF) on academic life, since there is
clearly a lack of knowledge about the mechanisms underlying these effects. Hence,
my research questions concern the RAE's effects a) on research performance; b) on
inequality; c) on diversity; d) on academic freedom; e) on research and teaching; f)
on recruitment policies; g) on motivation; and h) on power. I will start by clarifying 1)
the notion of academic life, because I assume that the RAE/REF has an impact on
academic life. 2) I will present a literature review on the identified dimensions of
academic life. 3) I will develop a model of higher educational alterations affecting
performance assessment in science and academia and the RAE/REF. 4) I will
develop a functional and conflict-theoretical model of performance assessment in
science and academia and will derive corresponding hypotheses. 5) I will present my
empirical procedure and will analyse the effects of the RAE/REF on the relevant
dimensions of academic life. To answer these questions I refer to a set of theories

www.researchcghe.org
4
ranging from reactivity by Espeland and Sauder (cf. Espeland and Sauder 2007),
ruling by numbers by Theodore M. Porter (cf. Porter 1996), audit society by Michael
Power (cf. Power 1999), surveillance and governmentality by Michel Foucault (cf.
Foucault 1976, 2000), entrepreneurial university by Burton R. Clark (cf. B. R. Clark
1998), academic capitalism by Sheila Slaughter, Gary Rhoades and Richard Münch
(Rhoades and Slaughter 1997; Münch 2011, 2014) and the Matthew effect in
science by Robert K. Merton (cf. Merton 1968, 1988), to the functionalist model of
science and academia by Robert K. Merton and Davis and Moore (cf. Davis and
Moore 1945; Merton 1995:17) and the conflict theoretical model of science and
academia by Pierre Bourdieu (Bourdieu 1979, 1980, 1983, 1987, 1992a, 1992b,
1996, 1998, 2006). To analyse the hypothesis I conducted a qualitative study at
three different universities and in three disciplines in the UK. I also carried out
interviews with policy advisers from a funding body, a higher education agency, a
leading adviser to the government and two research experts in the field of the effects
of the RAE/REF.
2. Theoretical framework
2.1 Academic life
Burton R. Clark understands academic life as academic community, disciplinary
differentiation and academic freedom, with its goals being the increase in quality of
research and teaching (cf. Clark 1989). This model will be extended with the
theoretical framework of Richard Münch (cf. Münch 2007, 2011, 2014) and my own
considerations. First, I will distinguish between research performance vs. research
power, equality vs. inequality, diversity and uniformity, academic freedom and
unfreedom, research and teaching vs. research and recruitment policies. The first
dimension refers to the measurement of daily academic and scientific research with
regard to publication output in quantitative terms while, in qualitative terms, it focuses
on the advancement of knowledge, and leads to new ideas, theories and methods.
We can therefore distinguish between groundbreaking, long-term, speculative
research, which is innovative and creative, and normal science.
Research performance is purely based on the performance of individual,
organisational and macrological actors. It takes into account their merits in the
school systems and is further rewarded in the higher education system right through
to their research careers. Research power means that actors refer to certain power
networks: institutional communication procedures to establish their research ideas in
an institutional context and in the wider academic and scientific community. It
stretches to the political and media sphere and is based mainly on social interactions
between relevant influential individuals, organisations and macrological spheres.
The next dimension is equality vs. inequality. This assumes that there are certain
policies which increase or decrease inequality and equality. Such policies are
intended to neutralise the effects of an individual’s social background. The

www.researchcghe.org
5
discussions reveal positive and negative discrimination, and exclusion from the
possibility of equal performance. Another aspect of inequality involves research
allocation mechanisms and the distribution of funds. In theoretical terms inequality
can be understood as access to different social resources and positions – such as
access to social networks – and in empirical terms as the unequal or equal
distribution of grants, publications, citation rates and personal factors.
The next dimension is diversity vs. uniformity. We can distinguish diversity in
research topics, gender and ethnic diversity. Diversity in the former enables a wide
variety of research topics, methods and theories, research questions, problems and
academic interests. Diversity in gender and minorities refers to the social fact that
there are merit-based proportions of gender and minorities in the academic and
scientific system.
The next dimension refers to academic freedom and academic unfreedom.
Academic freedom means that there is an absence of censorship in daily academic
practices and the freedom to undertake and choose research methods and theories
freely. An academic has the ability to work on the issues he/she wants to research
and to question the issues he/she would like to answer. It further relates to freedom
of speech, writing and autonomy. There are no constraints that limit one’s behaviour.
It is purely the research interest which fuels the discovery of new theories and
methods in the field of the academic and scientific sphere.
The dimension of research and teaching vs. research refers to the dimension of
practices related to the practice of undertaking research and teaching. This includes,
for instance, the chance to publish research, write papers, attend conferences, the
teaching curriculum and so on. It is closely connected with the crucial skill of
distributing knowledge among generations of students. Whereas the former is based
on a strong connection between research and teaching, the latter rests on a
concentration on research.
The final aspects are recruitment policies, the way academics are appointed and
what criteria are relevant in this decision-making process. The dimension of
recruitment policies considers the manner in which scientists are appointed and what
criteria are crucial for initiating an appointment. These above mentioned dimensions
are stratified by individual, organisational and institutional actors. There is, of course,
a difference in these dimensions between individual, organisational and macrological
actors (See Fig. 1).

www.researchcghe.org
6
Figure 1: RAE/REF and academic life
2.2 State of research
The RAE/REF's goal is to increase research performance and research quality and it
is used as an efficient allocation of research grants to ‘excellent’ universities (cf.
Frølich 2008:5): ‘Hand in hand with the rise of the New Public Management and
expanding global techno-economic competition, an increasing prominence has been
given to the idea that university systems employing output incentives and
competition mechanisms are more efficient and productive than systems in which
such incentives and mechanisms are employed less or not at all.’ (cf. Auranen and
Nieminen 2010:822). While citation rates are rising (cf. King 2004; Adams 2011;
Bornmann and Leydesdorff 2013), the proportion of papers in the Web of Science is
decreasing (cf. Adams 2011:7; Münch 2014:112). UK science is having a decreasing
impact (cf. King 2004; Adams 1998:615) and a decreasing efficiency in an
international comparison (cf. Auranen und Nieminen 2010:830) and on the
disciplinary level (cf. Münch 2014:227–228; 242).
One goal of the RAE/REF is to identify excellence in research and to selectively
support high performers in the scientific and academic world (cf. Science and
Technology Committee 2002a, 2002b, 2004a, 2004b). But inequality is rising in a
whole series of STJU indicators (cf. Münch und Schäfer 2014:64). There is inequality
in the field of staff as a result of the administrative workload for teachers and limits
on research freedom for researchers. Inequality also occurs in terms of publications
in top journals (cf. Harley 2000:565) and funding of established institutions instead of
new universities (cf. Orr 2003:45–46). Finally, there is an enforcement of gaming
strategies of informed institutions (cf. Harley 2000:555).

www.researchcghe.org
7
As regards diversity, short-term publication work is encouraged by the RAE/REF,
while innovative and interdisciplinary contributions are inhibited (cf. McNay 1997:70;
Oancea 2010a:260, 2014:91). In the field of economics, there is a list of economic
core journals determining recruitment decisions and driving scientific and academic
behaviour towards the hierarchy of high impact journals (cf. Harley and Lee
1997:1431; Lee, Pham, and Gu 2013); in addition, research topics must meet the
preferences of panel members. McNay and Talib argue that speculative and long-
term scientific work is inhibited (cf. McNay 1997; Talib 2002:264). The literature
points to decreasing academic freedom as a result of an increase of managerial
control. But there is also increasing managerial autonomy (cf. Melo, Sarrico, and
Radnor 2010:251; Teelken 2012:282). UK academics feel free (cf. Pritchard
2005:445), yet the RAE/REF generates an administrative burden so that academics
would prefer to spend more time on research than they are currently able (cf.
Pritchard 2005:445).
Furthermore, contributions to a vital academic community are inhibited (cf. Talib
2002:359). All in all, the entrepreneurial university might reduce academic freedom.
(cf. Palfreyman 2006:17–19): ‘Again, it can be seen that the content of autonomy is
all important. Universities are increasing their autonomy and academics are
increasing their freedom, but in market economic terms.’ (Marginson 1997:366).
Also, a division of the UK university landscape into research and teaching institutions
is observable (cf. Meier and Schimank 2009:55). With regards to teaching, a drop in
teaching quality becomes obvious (cf. Jenkins 1995). The knowledge transfer
between research and teaching is decreasing (cf. Broadhead and Howard 1998a:10)
and the status of lecturers is also decreasing (cf. Pritchard 2005:447). In addition,
there is discrimination against new universities (cf. Kreckel 2008:186–87). With
reference to recruitment policies, there is a recruitment of scientists who publish in
top journals and a recruitment of scientists with a vast research potential: research
has gained considerable weight over teaching (cf. McNay 1997:65–67).
2.3 Higher educational alterations
Starting from theories of general educational alterations in the higher education
system, six elements describing performance assessment in science and academia
can be identified. First, there is a belief in numbers. Second, we come across
reactivity in the form of a change of behaviour towards the goal structure of
RAE/REF indicators. Third, we can identify a sort of commercialisation underlining
the new relations between university and students in terms of a market model
placing students as customers, and universities as suppliers. Fourth, the scope of
economic control of research through university management is highlighted. Fifth, we
establish control and discipline in the context of the RAE/REF indicators. Sixth, there
is evidence reinforcing the assumption of an audit culture in universities: we identify
the function of a global quality seal.

www.researchcghe.org
8
Figure 2: Higher educational alterations
We can understand the RAE/REF's effects on academic life when we differentiate
and connect it to general alterations in the higher educational landscape. Numbers,
like efficency indicators (publication and citation rates), are increasingly used in the
political sphere to measure the output of higher education systems in reports of
reviews and for the House of Commons. As Porter describes it, there is a use of
numbers in the political and bureaucratic state machinery. This leads to an
enforcement of economic criteria, and a management of research proposals is
observable. We might say that there is impersonal measurement of scientists
through indicators, leading to an authority of numerical values and a belief in
numbers.
Experts play a crucial role in this context, because it is their construction and
interpretation of performance that matters. Nevertheless, it is a process of
construction and interpretation (cf. Porter 1996). Furthermore, one can identify a
behaviour of reactivity. We can observe a behavioural change in the direction of
numerical values and an optimising academic and scientific behaviour towards
numerical values. These are automated procedures that can be described as
governance through numerical models. This enables a career through impact factors
(cf. Espeland und Sauder 2007).
From the literature of academic capitalism we know that increased financing takes
place through tuition fees alongside the privatisation and commercialisation of
universities. This development is accompanied by managerialism of resource
allocation. In addition, profit and rationalisation techniques are implemented and
monopoly positions and increasing inequality is observed. The result is a competition
among students for academic products (cf. Rhoades und Slaughter 1997; Münch
2011, 2014). Burton R. Clark's study of the entrepreneurial university draws the
focus onto the business control of university entities that are increasingly regulated

www.researchcghe.org
9
by central management. The result is a global economic and symbolic competition,
while, at the same time, cooperation with industry is increasing. Economic entities
are implemented in the university system and appointment structures bring about a
global competition for research ‘stars’. There is a disentanglement from
governmental income resources and a pressure to meet investors' expectations (cf.
B. R. Clark 1998).
The effects of the RAE/REF become further understandable when we survey and
control research performance through indicators. There is a huge data collection and
input into mathematical functions available. Political actors build material and
performance-related classification systems and allocate resources to functional
sections. We can speak of a power of mathematical regularity and a data template of
individual performances (cf. Foucault 1976, 2000). In addition to these higher
educational alterations there is an implementation of economic, efficient and
effective structures. Quality assurance and internal control procedures are used and
motivational structures and conformity with the procedures are created. Formal
inspections define the standards of the auditors, and the auditors themselves can be
understood as judgemental entities that have an impact on university management.
Governmental audits are increasingly being replaced by private audits. This trend is
accompanied by an evidence-based interpretation of audit seals. They are
interwoven in political processes and are exploited for political programmes and in
brand management (cf. Power 1999).
The theoretical elements – a belief in numbers, reactivity, business control,
disciplining and quality seals – are core elements in understanding the effects of
performance assessment in science and academia and in understanding academia
and science in general. We see an increasing belief in statistical models, rankings
and league tables. The actors internalise vertical ranking values and integrate them
into their judgement criteria. Hence, there are also life course and generation effects.
In methodological terms, we can combine life course analysis with event history
analysis and multilevel approaches over time. The individual judgement criteria are
stratified by the institutional hierarchy of the universities. Scientific and academic
actors choose and predict their future on the basis of university ranking positions.
This restricts academic freedom in the sense that research interest is combined with
strategic behaviour. Therefore, science and academia are based more and more on
statistical evidence. In this context, creative and innovative elements of scientific and
academic life are less likely. The element of reactivity changes the behaviour of
scientists and academics so that they anticipate changes in indicators and
consequently modify their behaviour to conform with the changing definitions of
ranking systems. They hypothesise about certain indicators and start to change their
behaviour in these directions. As a result, there is a shift from academic and
scientific interests towards strategic interests.
The element of business control means that management structures are very
important in the implementation and interpretation of ranking structures in

www.researchcghe.org
10
universities. They are increasingly used to improve the individual performance of
scientists and academics. This development is accompanied by peer review
processes designed to stimulate the improvement of scores. The disciplining
element changes the behaviour of individual and organisational actors, whose
actions are influenced by the perceived judgement criteria of peers and ranking
systems. They tailor their research to certain topics to fit with the criteria of certain
journals. This process starts right from the selection of the title of the publication and
stretches to the choice of sources and so on. It is closely entwined with processes of
economic rationality of scientists and academics. The element of the quality seal
means that the overall ranking judgement over a certain university – for example, for
an individual working at a certain university – is a seal for both the academic and the
scientific audience. It affects recruitment decisions and career chances.
It will be interesting to see how far national quality insurance systems like the
RAE/REF have a global charisma in science and academia. At the moment they are
still not popular and comparable enough to have a stronger effect than media
rankings of universities. This points further to the ways in which the academic and
scientific, economic, cultural, and media fields are interrelated. At the core is the
individual, with emotional and rational procedures and mind and body. Furthermore,
the fields are connected via network positions that are vertically stratified to
individual, organisational and macrological spheres. The networks are labeled as
performance and power-related networks with high ability and strong power
connections. Hence, the field is divided in elite power performance-related power
structures and a non-elite segment.
Against this backdrop, I have developed a theoretical model designed to explain the
effects of the RAE/REF's performance assessment on science and academia in the
UK. Using this model, I differentiate between functionalist and conflict theoretical
explanation strategies. In functionalist terms, the constitution of performance
assessment in science and academia is functional for the entire university system,
because it allows a selection of knowledge and researchers according to
performance, achievement, qualifications and talents. The conflict theoretical model
of performance assessment in science and academia argues that there are
institutions possessing a high number of material and symbolic resources that are
reinforced by central actors holding monopolies in the distribution of material and
symbolic resources.
2.4 Functional model of performance assessment in science and
academia
It is possible to distinguish two theoretical models as explanation strategies. On the
one hand, there is the functional model of performance assessment and, on the
other hand, there is the conflict theoretical model. In the first case, UK performance
assessment can be understood as a measure to strengthen the top performers in the

www.researchcghe.org
11
UK university system. In the second case, historical structures determine capital
accumulation of UK performance assessment.
In accordance with the functional model of performance assessment (see Fig. 3)
society consists of functional entities. These entities are in a positive relationship to
the entire organism and there are specific behavioural patterns, which are essential
for the system to survive. It is assumed that the RAE/REF system – perceived as a
functional system – ensures the functionality of the UK university system. There is a
selective distribution of resources to the high performers of the university system.
This enables the recruitment of talent, excellent students and star researchers. This
selection process has a positive impact on knowledge evolution and can be
understood as producing inequality on the basis of merit (Fig. 3).
However, there are also dysfunctional effects. For example, there are informal
relationships that infiltrate the academic system and capital concentration around the
high performers. Increasing inequality is used as a tool to increase productivity (cf.
Davis und Moore 1945; Merton 1968, 1988). From this starting point, Tumin argues
that elite circles reduce the survival probability and selection of talent. An
accumulation of advantages takes place for more privileged children through
monetary funds, and elite circles gain psychological rewards with positive effects on
self-realisation and leisure behaviour. It is assumed that the RAE/REF strengthens
extrinsic work motivation and creates positive self-images for elite universities and
negative self-images for the wider mass of universities. This leads to decreasing
creativity and integrative power in science (cf. Tumin 1953). In this context,
Marginson argues that performance data entries in league tables are limited in
identifying the actual university performance (cf. Marginson 2015).
Figure 3: Functional model of performance assessment in science and academia

www.researchcghe.org
12
2.5 Conflict theoretical model of performance assessment in
science and academia
In accordance with the conflict-theoretical model of performance assessment in
science and academia there is a concentration of attention on elite universities. This
goes hand in hand with the definition of performance criteria through monopoly
positions. The scientific and academic discussions and publications are restricted to
lighthouses: there is a concentration of attention to well-funded universities. Social
exchange takes place between elite institutions in the same way as the accumulation
of capital and the establishment of ranking positions. Profit interests determine
knowledge acquisition and scientific and academic action. One can speak of a
screening of the science system with indicators, and an implementation of an elite
habitus.
The RAE/REF rates actors due to capital distribution and there is an accumulation of
economic and symbolic capital at top institutions. Scientists and academics with high
values of symbolic capital are recruited. This leads to the assignment of an elite
habitus to institutions, scientists and academics. Inequality can be understood on the
basis of the unequal distribution of capital, power and status (Fig. 4). This is in
accordance with Pierre Bourdieu’s field and capital theory (cf. Bourdieu 1979, 1980,
1983, 1987, 1992a, 1992b, 1996, 1998, 2006). He refers to an elite habitus and
classifications and assessments of actors based on excellence criteria in a social
space. The field is comparable to an elitist game: the cards in the game correspond
to capital, whereby competitions are held and social positions are taken. The field's
boundaries are determined through entrance rules and standards and the state
bundles resources and exercises power over fields. Actors try to accumulate state
capital to strengthen their power positions. The habitus of the actors is a disposition
system, and concentrates on experiences and historical conditions. The academic
elites transmit material chances to the next generation.
Inequality is thus maintained throughout generations and enables career paths.
Elitist language is loaded with symbolic power and linguistic entities are recognised.
The RAE/REF's evaluation activities favour top publishers and exercise symbolic
power. Power structures are obscured and negated by symbolic violence and are
legitimised by the state (cf. Bourdieu 1979, 1980, 1983, 1987, 1992a, 1992b, 1996,
1998, 2006). In this context, Marginson argues there is a stratification by status and
achievement (cf. Marginson 2011).
It is very likely that the functional and conflict-theoretical models of performance
assessment in science and academia are both relevant to explaining effects in the
university sector. There are reproductive elements of power, capital and status and
also performance-driven aspects such as individual, organisational and macrological
abilities to perform.

www.researchcghe.org
13
Figure 4: Conflict-theoretical model of performance assessment in science and
academia
2.6 Hypotheses
Several methods are very well suited to developing hypotheses for this study. I will
consider 1) the state of research, 2) higher educational alterations, and 3) the
performance and power-based model of performance assessment in science and
academia. Furthermore, I have gathered information during my qualitative interviews
enabling me to develop hypotheses in an iterative way.
In regards to the state of research I will formulate hypotheses 1 a) – f).
Hypothesis 1a) assumes that the research performance is increasing or decreasing
due to a concentration of resources. This can be accompanied by a higher or lower
probability for elite institutions to publish in high impact journals. The concentration of
resources can be understood as Gini coefficients in financial, personal, citation and
publication resources.
Hypothesis 1b) assumes an increase or decrease of inequality. This inequality can
be measured through different indicators and in qualitative terms with the dimensions
of gender, the promotion of early career researchers and the progression and status
of different positions.
Hypothesis 1c) assumes that there are different dimensions of diversity. Diversity
can be enhanced by political procedures or can be reduced due to administrative

www.researchcghe.org
14
workload and an orientation towards panel members' preferences for certain
research topics and the expectations of leading journals. Furthermore, it is assumed
that there are certain favourable output types in the RAE/REF process. Another
crucial feature is that the RAE/REF does not enable scientists and academics to
change their research track fundamentally, and they must conform to the RAE/REF
time cycle.
Hypothesis 1d) assumes that academic freedom is reduced due to an orientation
towards impact measures, high impact culture, administrative burden, quality
management and time pressures. This orientation is increased through managerial
autonomy and resource competition.
Hypothesis 1e) assumes that there is a separation of research and teaching or a
stronger connection between the two. It is assumed that the latter involves an
administrative burden for researchers marked as ‘non research active’ and who
therefore experience a low level of recognition in the academic field. It also implies
that teaching is undervalued compared to research and there is a loss of knowledge
transfer between student generations or a transmission of certain research
knowledge.
Hypothesis 1f) assumes that there are selective criteria in the appointment practices
or general criteria. The effect on recruitment policies means that we will witness a
visible change of recruitment criteria in favour of mainstream research and an
emphasis on symbolic, instead of functional, values.
In relation to higher educational alterations I hypothesise that 2a) numbers are
gaining more and more significance in academic processes. They stimulate
academic and scientific behaviour and are relevant as regards different institutional
ranks and disciplines. For example, an orientation towards impact factors and
strategies to raise ranking positions are indicative of the impact which the belief in
numbers has on daily academic practices.
2b) I assume that there is a behavioural change towards reactivity to the
performance indicators. It is possible that academics and scientists are more
strongly focused on certain output types and general quantitative indicators.
2c) I assume that there is a shift towards a commercialisation of academic practices.
2d) I assume that business control plays an ever more important role in academic
practices.
2e) I expect that control and discipline are relevant and are differentiated towards
certain institutional ranks. For example, lower and middle ranked institutions are
more strongly controlled by management practices than higher academic and
scientific ranked institutions.

www.researchcghe.org
15
2f) I suspect that quality seals are relevant in the process of scientific and academic
discovery.
3) I assume that the functional and conflict-theoretical model of performance
assessment in science and academia is essential.
In the following empirical inquiry I will try to corroborate some of the hypotheses with
qualitative data material. Not all hypotheses are analysable with qualitative data.
Hence, we still have to refer to quantitative data and discussion analysis. These
analyses will form part of my dissertation and will be used at further conferences and
in presentations of my work.
3. Empirical procedure
3.1 Qualitative interviews and sample structures
The study is ethically confirmed by the Federal Ministry of Education and Research
in Germany and its main funding line ‘performance assessment in science’ as well as
by the University of Bamberg and its related research project ‘unintended effects of
performance assessment in science and academia’ headed by Professor Münch.
The interview data is handled strictly confidentially and the interview responses were
carried out anonymously due to the sensitive nature of this research project. In the
first step I established a random sample of interviewees: I contacted a number of
scientists and academics, selected according to their discipline and their institutional
affiliation. The crucial point was for academics and scientists to be employed at the
university and to hold a position as head, reader, lecturer or professor. The response
rate was 10 per cent. Next, I arranged an interview appointment, a time and a
location. The interviews were structured by a questionnaire I had prepared
beforehand. To start with, I undertook a pretest with three different scientists and
academics from three different disciplines to find out whether my questionnaire was
accurate enough to provide the answers required for my research project. This was
crucial in order to obtain information with regard to problems arising over the
understanding of the questions and possible duplication of the knowledge content.
In the second step I constituted a sample of the universities in accordance with their
institutional affiliation and their 2008 RAE ranking. This decision was made to control
changes in the RAE/REF criteria, to ensure stability and to open up the possibility of
interviewing academic and scientific staff who are climbing the scientific and
academic ladder. I interviewed scientists and academics from three different
institutions (low, middle and high ranked) and three different disciplines (chemistry,
sociology and history).
As you can see (Tab. 1), it was problematic to get three interviewees from research
intensive universities for all disciplines. It also proved difficult to get three sociologists
from the lower ranked institutions. Over the next few months I will try to conduct
some more interviews via Skype; however, this possibility is restricted by limited staff

www.researchcghe.org
16
numbers and a lacking readiness to participate in the study. One possibility would be
to reduce the sample size to two scientists and academics per discipline and
institution. However, this would involve a reduction of potential information.
The next step is designed to deepen understanding of the academic, scientific and
political discussions and debates. In the discourse structure, certain institutions can
be identified as main actors. To comprehend positions and strategies of the main
actors in political, scientific and academic discourse, I will conduct interviews with
representatives of central institutions. The interviewees will be selected on the basis
of their hierarchical level within the institution, their relevance within the institution,
and because of previous findings they have gathered with regard to the discourse.
This procedure will supply a deep insight into the effects of the performance
assessments perceived by the scientists and academics themselves in their role as
professors, heads, lecturers and readers in chemistry, sociology and history.
Furthermore, valuable insights will come from the scientific, academic and political
actors in the mediated sphere to the political field.
Discipline
Institution
Total
1
2
3
1
3
3
3
9
2
2
3
1
6
3
2
3
3
8
Total
7
9
7
23
Table 1: Sample 1
In the second step I tried to contact central actors in the field of science and
academic policy. The process of getting in touch with funding bodies, higher
education agencies and the government is not an easy one. I was able to conduct
five interviews with policy advisers from a funding body, one interview with a policy
analyst and adviser from a higher education agency. Also, I interviewed a leading
scientific adviser to the government. Furthermore, I carried out interviews with two
leading scholars in the field of the RAE/REF. Members of Parliament were not
available for an interview due to time restrictions.
Institution
Position
Number
Funding Body
Policy Adviser
5
Higher Education Agency
Policy Analyst
1
Research experts
Leading researcher in the
field of the RAE/REF
2
Scientist
Leading adviser to the
government
1
Σ
9
Table 2: Sample 2

www.researchcghe.org
17
3.2 Questionnaires
First of all, I introduce the questionnaire with a general question concerning the
interviewees' work experiences. Next, I ask them about general changes in the
university landscape in the wake of the RAE/REF. I then enquire about the RAE's
performance goals and the perceptions of the interviewees as to whether these
goals have been achieved. In the RAE's functionalist view, it is supposed to have a
positive impact on research efficiency and on performance measured in terms of
publication and citation rates. In a conflict-theoretical argumentation mode, it is
perceived as a symbolic struggle for recognition which could lead to a drop in
research efficiency.
The following set of questions relates to the effect the RAE has on inequality. In this
context, I am interested in a process of increasing or decreasing inequality and its
effects on the perceptions of the interviewees. The subsequent questions are related
to the effects of the RAE/REF on diversity. To deepen the understanding of diversity,
there will be a question as to the understanding of the term and the effects it has on
everyday scientific and academic work. Furthermore, I tried to address the diversity
issue with questions relating to its conformity to the RAE/REF’s time cycle and the
change of research focus.
The following group of questions is related to academic freedom. I would like to find
out whether academic freedom is promoted by the RAE/REF and whether the
interviewees have the opportunity to choose research themes freely. It starts with a
question concerning the interviewees' understanding of the term ‘academic freedom’.
I am especially interested in the RAE/REF's effect on freedom of research, teaching
freedom, learning freedom and contibution to the academic community. The
following questions are related to the RAE/REF's effect on research and teaching. In
this context, I am interested in finding out whether the interviewee is involved in
teaching and how much of his/her workload is allocated to teaching activities. I am
also interested in the teaching load of the entire department and the university.
Next, I focus on the functionalism of research and teaching. I will conclude this group
of questions with a look at the division of labour in an attempt to discover whether a
concentration of the best qualified brains in either teaching or research is desirable.
Subsequently, I address the subject of recruitment policies. In this context, I am
interested in identifying the crucial criteria in the recruitment process of scientists.
The change of recruitment criteria in the wake of the RAE/REF is of special
relevance here. I will then have a look at the RAE/REF's effects on research
motivation and power. In the following step, I will focus on the institutional effect and
ranking position of the academic institution.
The final questions refer to standardisation: the influence of the university
management and quality management in the context of the RAE/REF. First, I am
going to consider the effects the RAE/REF has on standardisation. Second, I am
interested in the way the university management affects decisions of academics and

www.researchcghe.org
18
scientists. Third, I would like to consider the effect of quality management of
academic life in the context of the RAE/REF. Ultimately, I am referring to power
relationships in science and academia to test the conflict-theoretical model of
performance assessment in science and academia.
3.3 Analytical strategy
This analysis is a first interpretation of the data on the basis of my notes taken during
the interviews. I will analyse the transcripts of the interviews. In this paper I will
concentrate on the similarities rather than the institutional and disciplinary
differences. Nevertheless, I will take institutional and disciplinary effects into account
whenever this is possible. First, I will have a close look at the effects on a) research
performance, b) inequality, c) diversity, d) academic freedom, e) relationship of
teaching and research, and f) on recruitment policies, g) research motivation, and h)
power.
It should be mentioned that the RAE/REF's effects are embedded in a frame of
specific actors and historical changes (Fig. 5). This is grounded on a research result
identified by, for example, Oancea: ‘The influences of RAE 2008 were not perceived
by the respondents as homogenous, but as multi-directional and depending on a
range of factors.’ (Oancea 2010b:8). So even if I asked the interviewees about the
effects of the RAE/REF on academic life, it is still possible that the interviewees were
relating their knowledge to processes other than the RAE/REF. This means, for
example, that when an interviewee described the RAE/REF as encouraging a high
impact journal culture, this could reflect a worldwide trend in the scientific and
academic sphere. A further example is that university management interprets the
rules of the RAE/REF, and, furthermore, implements decisions in their own way.
Another example is panel judgement. Sometimes interviewees said that the panel
judges performance in a biased way. However, they have no knowledge about
certain practices within the panels, not being panel members themselves. So even if
the interviewees responded to the best of their knowledge, it is still possible that truth
is distorted in their messages. We have to keep in mind that both the researcher and
the interviewees did their best to ask and respond to the questions in an objective
way. But it is nevertheless a process of constructing the social world within which we
live. To take this into account I have enriched my analysis with quantitative
measurable data.

www.researchcghe.org
19
Figure 5: The effects of the RAE/REF in context
4. Empirical results
4.1 The academic and scientific sphere
4.1.1 Research performance
What effects does the RAE/REF have on research performance? 20 of the
interviewees said that UK science and academia are doing well in the world (01_01;
01_02; 01_03; 01_04; 01_05; 01_06; 02_01; 02_03; 02_04; 02_05; 02_06; 02_08;
02_09; 03_01; 03_02; 03_03; 03_04; 03_05; 03_06; 03_07). UK science and
academia are placed second in the world and are leading worldwide in research in
terms of publication and citation rates. This finding corroborates Oancea's
hypothesis that the RAE/REF is perceived as increasing the quality of publications
(cf. Oancea 2014:89).
This underlines the theoretical assumption that numbers are increasingly crucial for
decision-making processes and that there is a belief that performance in science and
academia has increased. This, in turn, underlines hypothesis 2a) claiming that
numbers are increasingly influencing the decision-making process. Furthermore, it
underlines the functional model of performance assessment in science and
academia, which assumes that science and academia are perceived as functional in
terms of the performance of individual, organisational and macrorelevant indicators.
One interviewee said that if one accounts for the investment of GDP, which is below
average, the UK does even better than the US (03_05). The analysis shows that
Academic and
scientific
communifty
Government
Peer review
and panel
members
Policy
advisers
Funding
bodies
University
management
Historical
changes

www.researchcghe.org
20
people are looking more and more at league tables when they refer to the worldwide
leading position of the UK’s science and academia.
To further deepen the question, I required the interviewees to refer to the RAE/REF's
effect on their academic and scientific performance. First of all, there is concern
about the long-term perspective of the RAE/REF. 10 academics and scientists said
that it inhibits long-term speculative and risky research due to its short-termism
(01_02; 01_03; 01_04; 02_05; 02_07; 02_09; 03_02; 03_03; 03_05; 03_06).
Interviewees indicated that there is an advancement of English speaking countries
(01_05; 02_02; 02_07). As regards the effect on academic performance, the
interviewees said that impact reduces academic freedom and there is a drive
towards high impact journals. Yet only four interviewees stated that the RAE/REF
encourages the researchers to plan and organise their research (01_01; 02_2;
02_03; 02_04). 18 scientists and academics did not mention the effect at all. This
research result supports hypothesis 1a) as it observes a belief in performance
increase and also a performance decrease due to a focus on short-term publication
strategies.
4.1.2 Inequality
Five interviewees said that the main funding goes to the leading, established
universities (01_04; 01_05; 02_01; 03_02; 03_03). There are head hunting
strategies to recruit the best researchers (01_05). One interviewee claimed that it is
easier to get Research Council grants with very high RAE/REF scores (02_01).
There is also concern about the devaluing of theoretical work (02_06). But still, there
is a chance for new universities to climb up the academic ladder. They use the
RAE/REF’s research funding income to compete with the Russell Group and other
institutional actors.
In terms of gender inequality, the interviewees reported a dominance of white
middle-class men who protect their elitist position. There is concern about the
relative academic chances of women (02_02; 02_03; 02_05; 02_06; 02_07; 02_08)
who often take career breaks due to child caring responsibilities, which proves to be
a real concern for their RAE/REF research profile. One interviewee said that women
are less likely to be nominated for the REF cycle. Furthermore, women are likely to
be assigned administrative tasks rather than actively engaging in research to boost
their research profile (03_03). The following literature provides more insight and
ideas with regard to the gender and minority issue (cf. Bagilhole 1993; Blake and La
Valle 2000; Knights and Richards 2003; Probert 2005; Barbezat and Hughes 2005;
Doherty and Manfredi 2006; Ackers 2007; Hey et al. 2011; Shepherd 2011; Grove
2015).
One interviewee mentioned that theoretical work is devalued in the sense that
empirical work gets better RAE scores (02_06). Furthermore, there are concerns
about the chances of early career researchers who are overloaded with work and

www.researchcghe.org
21
need REF-able results for career progression (01_03; 01_04; 01_06; 02_02; 02_03;
02_05; 02_06; 02_07; 02_08; 03_05; 03_06; 03_07). This is supported by the
research results of Oancea (cf. Oancea 2010b:6–7, 2014:99). See furthermore (cf.
Manfredi and Vickers 2009). These research results indicate that the RAE/REF has
created a level of inequality, particularly in the fields of resources, gender, minority,
and early career progression and promotion. Oancea also identifies perceived
effects of the RAE/REF on promotion prospects (cf. Oancea 2014:89). These
research results are in line with the argument that the wake of the RAE/REF and its
implementation in universities has resulted in an increase in inequality. The
RAE/REF fuels inequality in gender, early career promotion and progression, and
confirms hypothesis 1b).
4.1.3 Diversity
Overall, the interviewees understand diversity and inequality as synonyms. So there
are diversity issues relating to the chances of women, minorities and diversity in
research topics. For further research results see (cf. Oancea 2010b:7). With regard
to diversity, one interviewee said that it changes people's behaviour towards thinking
about where to publish as opposed to changing the topics they are researching. To
deepen this analysis one has to take into account the responses regarding the
change of research focus (see below).
Change of research focus
22 interviewees said that they did not change their research focus as a result of the
RAE/REF (01_01; 01_02; 01_03; 01_04; 01_05; 01_06; 02_01; 02_02; 02_03;
02_04; 02_05; 02_06; 02_07; 02_08; 02_09; 03_01; 03_02; 03_04; 03_05; 03_06;
03_07). This can be interpreted in three ways. First, it can be assumed that the
RAE/REF did not make it necessary for researchers to change their research focus,
and it is still therefore the researcher's choice to focus on specific topics. The second
interpretation is that the RAE/REF does not enable people to undertake a big shift in
their research focus. Some interviews suggest that both interpretations are very
likely. So the RAE/REF inhibits a major shift in the research focus and research
career of the scientists and academics. But it is also true that the academics and
scientists are not forced to change their research focus due to the RAE/REF.
If you consider this topic in more detail you will see that 11 interviewees identified
conformity with the RAE/REF time cycle (01_04; 01_05; 02_06; 02_07; 02_08;
02_09; 03_01; 03_03; 03_04; 03_06; 03_07). This conformity could lead to a lesser
likelihood of a researcher taking a fundamentally new research path and trying to
find new directions in a research area. A third interpretation is purely academic and
scientific: academics and scientists are in this way driven by research interest and
success in their research field. They don’t change their focus because they have
research interest in the topics they are researching. This supports hypothesis 1c)
claiming that the RAE/REF does not enable the scientists and academics to change
their research track fundamentally.

www.researchcghe.org
22
4.1.4 Academic freedom
Choice of research themes
My first question in relation to academic freedom asked the interviewees what kind of
effect the RAE/REF has on their choice of research themes. Eight interviewees
replied that the RAE/REF had affected their choice of research themes (01_02;
02_01; 02_04; 02_08; 02_09; 03_01; 03_05; 03_06). The aspects mentioned
included impact (01_02; 01_04; 02_02; 02_03; 02_05; 02_06; 02_08; 02_09; 02_09;
03_01; 03_02; 03_04; 03_06), time problems (02_01) and creating high impact
journal research (01_04; 02_01; 02_02; 02_03; 02_04; 02_06; 02_08; 03_04; 03_05;
03_07).
During the interviews it became more and more obvious that the academics and
scientists have a very strong interest in their research work. Hence, we cannot
conclude that the RAE/REF influences and more or less prescribes research topics.
But there is serious concern on the part of academics and scientists that the
introduction of measuring impact might drive them to focus on more practically
orientated topics (01_02; 01_04; 02_02; 02_03; 02_05; 02_06; 02_08; 02_09;
02_09; 03_01; 03_02; 03_04; 03_06). Furthermore, the time cycle of the RAE/REF
inhibits long-term and risky research and encourages short-term knowledge
production (01_02; 01_03; 01_04; 02_05; 02_07; 02_09; 03_02; 03_03; 03_05;
03_06). In this context, an effect on academic freedom is observable because
academics and scientists are not free to study every topic in which they have a
research interest.
The interviewees also said that there is a bias towards the ‘trendy’ and ‘glamorous’
research topics often featured in high impact journals. From this perspective, a drive
towards metrics will limit the academic freedom of scientists and academics through
a concentration on high impact journals (Fig. 6). Some interviewees indicated that
they were now striving to get into these journals because they are valued more
highly in the RAE/REF panels. The academics and scientists would not normally
have published in these journals but it is increasingly an implicit norm of how the
panel members are judging their quality of work. Whether or not it is true that panel
members favour high impact journals, academics believe that they do and are thus
changing their research behaviour accordingly. This supports hypothesis 2b) which
assumes that there is a change of behaviour towards measured aspects of the
scientific and academic world such as the high impact factor of publications. There is
a clear shift in research topics due to the perceived judgment criteria of the
RAE/REF (Fig. 6).
With all these findings in mind, the question arises as to how we should handle this
social fact? A drive towards more metrics (a more governmental approach of
regulating science and academia) would encourage a rush towards high impact
journals and restrict academic freedom to publish in the journals where a type of
research best fits. Furthermore, most of the interviewees said that the RAE/REF had

www.researchcghe.org
23
little or no effects on their teaching freedom (01_01; 01_02; 01_03; 01_06; 02_02;
02_03; 02_04; 02_05; 02_06; 02_07; 02_09; 03_01; 03_02; 03_03; 03_04).
In relation to freedom of learning there are only small effects (01_02; 01_06; 02_01;
02_06; 02_08; 02_09; 03_02; 03_04). The curricula appear not to be affected, but
there could be an indirect effect due to the emphasis on research. Some
interviewees indicated that there are restrictions to student learning, and time
pressures on research and recruitment processes to fit with REF-able criteria. It is
really important to point out that it is not clear whether the drive towards high impact
journals is favoured by the RAE/REF. Policy advisers claimed that high impact
journals are not a judgement criterion of the panel members (04_01). One policy
adviser from a funding body said that impact only accounts for 10 per cent (04_01).
This underlines hypothesis 1d) suggesting that the choice of research themes is
limited due to an orientation towards high impact journal culture and impact
measures in the RAE/REF and also allows space for the opposite theory. See further
(cf. McNay 1997:66).
Figure 6: Research interest, panel criteria and academic freedom
Contributions to the academic community
As regards the RAE/REF's effect on the extent to which academics contribute to the
academic community, six interviewees said that editorial board, reviewer functions
and subject associations are more likely to be disadvantaged by the RAE/REF
(01_03; 02_05; 02_09; 03_01; 03_03; 03_06). On the other hand, some interviewees
stated that it encouraged collaborations (01_02; 01_04; 02_08; 03_01).
4.1.5 The relationship between teaching and research
I will explore what kind of effects the RAE/REF has on the dimension of teaching and
research. Overall, it can be said that it strengthens the connection towards research.
Most of the interviewees said that teaching is now more research driven (01_01;
01_02; 01_03; 01_04; 02_02; 02_03; 02_08: 03_03; 03_06) and students are
interested in their research. On the other hand, this implies that the chance to
discuss new topics which are unrelated to that research is more difficult to develop.
When your teaching is closely connected to your research this means that all
restrictions of the RAE/REF are focused more directly on the students' academic
lives. One interviewee claimed that the RAE/REF had an impact on teaching as
he/she had less time to focus on research. So he/she had to work on weekends to
get publications out (09_02). This effect supports hypothesis 1e), which suggests
Research interest
Intrinsic motivation
High impact journals
Trendy
Glamorous
Perception of panel
members criteria

www.researchcghe.org
24
that there is a stronger connection between teaching and research, with research-
based teaching being an exception here. It is possible that certain knowledge will be
transmitted to student generations.
4.1.6 Recruitment policies
In this section I will analyse the RAE/REF's effects on recruitment policies. Overall
this has been perceived as a major effect. An increasing number of strategies are
used to increase and improve RAE/REF scores. This creates an instrumental
strategic mode to look at different rankings to further stimulate research
collaborations at the lower ranked universities. This is a huge shift in the dimension
of research collaboration. Whereas prior to RAE/REF, research ideas prospered due
to research interests, there are now strategic motives to establish research
collaborations with highly ranked researchers. But this development is not only a
result of the RAE/REF. It is accompanied by historical shifts in the way the science
and academia system develops. There is a global trend for rankings and assessment
of science and academia systems.
We can therefore develop a model of research interest which includes a further
dimension, such as instrumental decision-making (Fig. 7). Furthermore, evidence
suggests that top researchers are recruited by means of REF-able criteria to boost
REF scores (01_02; 01_03; 01_04; 01_05; 02_02; 02_03; 02_04; 02_08; 02_09;
03_01; 03_02; 03_03; 03_05; 03_07). Oancea also identifies an effect of REF
inclusion on the possibility of being recruited (cf. Oancea 2014:90). Recruitment
happens just before the REF cycle ends to boost RAE/REF results. This is a serious
concern because it undermines long-term relationships. As a result, the star
researchers might feel instrumentalised and be less integrated in the scientific and
academic community.
It also has an impact on academic quality, because star researchers are inhibited in
their contributions to the academic and scientific community. Furthermore, in
recruitment an increasing number of bibliometric tools are used to judge the quality
of the new researcher. The recruitment process attempts to ascertain in which area
the researcher is working and how well he/she performs with regard to bibliometric
indicators (02_01). One interviewee indicated that there are few jobs available after
the REF, and the main job offers happen before the REF to ensure good scores
through the recruitment of bright people during the assessment period (01_04).

www.researchcghe.org
25
Figure 7: Strategic decisions triggered by the RAE/REF
Employees are increasingly looking for a researcher with a highly REF-able research
profile to boost institutional REF scores. Summarising the effects, one can say that a
second mode of working in science and academia starts to develop. In addition to
the orientation towards knowledge development and the creation of new ideas, and
innovative, creative methods and theories, an increasing amount of time is spent on
instrumental and strategic behaviour, managerial and departmental peer review
processes before and after the RAE/REF to improve research results (01_04; 01_06;
02_03; 02_04; 02_06; 02_09; 03_05; 03_07) and orientation towards metrics in
ranking systems (01_05; 01_06; 02_04; 03_01).
This implies a huge change in academic behaviour. Whereas formerly academics
and scientists were mainly focused on the search for truth, the RAE/REF
academic/scientist is focused on thinking about what criteria will be applied in the
next RAE/REF cycle. Huge efforts are made to predict the expected results. To this
end computer models are more frequently used to ‘catch the cash cow’. The playing
field has changed completely in the wake of the RAE/REF as an instrument to
implement rules, regulate behaviour and improve research quality. This playing field
now demands specific competences both on an individual and an institutional level to
improve the RAE/REF ranking position. This supports hypotheses 2c), 2d) and 2f)
which assume that there is commercialisation and managerial control of academics
and scientists. The relevant performance processes provide information for a global
quality seal.
On an individual and an institutional level the academics and scientists have to meet
the REF-able criteria to gain better scientific and academic scores of excellence
(01_04; 01_05; 02_06; 02_08; 02_09; 03_05; 03_07). One interviewee indicated that
there is concern about inequality of income and that there is much gaming about
salaries (01_05). This subject matter could be further explored to discover new
issues related to salaries. Oancea's empirical results identify a very low impact on
earnings (cf. Oancea 2014:89). These research results support hypothesis 1f)
Research
interest
Instrumental and
strategic behaviour
-Choice of
collaboration
partners at high
ranked
universities
Research collaboration

www.researchcghe.org
26
claiming that more and more selective REF-able criteria are being used in
recruitment processes. Those scientists and academics who are performing well in
the RAE/REF are more likely to get employed.
4.1.7 Research motivation and pressure
In this section I will analyse the effects of the RAE/REF on motivation and pressure.
Some interviewees do not feel affected (01_03; 02_05; 02_08; 03_01; 03_04). For
some it is more like an extra motivation (02_03; 02_04; 02_07). For others it feels
depressing (01_01); it feels pressuring (01_04; 03_07); it leads to burn out (01_05);
it does not help much (02_01); it creates stress (02_02; 03_06); and it demotivates
(03_03; 03_05). Further analysis shows that it creates pressure on academics and
scientists (01_01; 01_02; 01_04; 02_01; 02_02; 02_04; 02_05; 02_07; 02_09;
03_01; 03_03; 03_05; 03_06; 03_07). This has important implications for the quality
of the academic work produced. In this context it is likely for the RAE/REF to
increase pressure, which in turn decreases motivation for quality academic research
(Fig. 8). The fact that the RAE/REF affects the perceptions of pressure has also
been identified by Oancea (cf. Oancea 2014:97–98) and McNay (cf. McNay 1997:65,
67). Oancea discovered an effect of decreasing motivation for non-submitted staff
(cf. Oancea 2014:91). In terms of institutional differentiation, four interviewees from a
research intensive university indicated that pressure has a decreasing effect on their
research motivation (01_01; 01_02; 01_04; 01_05). Chemistry as a discipline seems
to be under particularly high pressure (01_01; 01_02; 02_01; 02_04; 03_01; 03_05)
Figure 8: Motivational effects of the RAE/REF
4.1.8 Power in performance assessment in science and academia
As regards power structures affecting performance assessment in science and
academia many interviewees replied that they have evidence of power relationships
influencing performance assessment in science and academia (01_01; 01_02;
01_06; 02_01; 02_02; 02_03; 02_05; 02_06; 02_08; 02_09; 03_01; 03_02; 03_04;
03_05; 03_06; 03_07). To go into more detail, the respondents said that there is
power of important areas (01_01); competitive politics to lead a brand (01_02);
managerialism and power of the vice chancellor and academic senate (01_06);
people favouring their own agenda (02_01); top-down power (02_03); management
in the departments for better RAE/REF outcomes (02_05); power of who is on the
panel, senior roles, gatekeepers to the REF and personal experience (02_06);
pressure which drives people (02_08); gender bias (02_09; 03_07); more prestigious
universities (03_02); and group and society commitments and influencing
Pressure and Stress
Motivation
Academic Quality

www.researchcghe.org
27
mechanisms in panels (03_04; 03_05). This further supports hypothesis 3) which
assumes that the power-based model of performance assessment in science and
academia is crucial to understanding academic processes related to performance
measurement.
4.2 Connections between the academic, scientific and political
sphere
In this section I carried out interviews with five policy advisers from a funding body,
one policy adviser from a higher education agency, a leading adviser to the
government and two research experts in the field of the effects of the RAE/REF.
4.2.1 Research performance
Policy advisers from the funding body and a higher education agency said that
performance in the UK has increased as a result of the latest REF (04_02; 04_03;
04_05). It exerts pressure on academics to raise quality and it is a bureaucratic and
costly procedure (04_01; 04_02; 04_05). It goes hand in hand with increased
managerialism within universities (04_01; 04_02; 04_03; 04_04). Therefore, the
effects are due to increased management of the RAE/REF rather than being a direct
result of the RAE/REF procedure. It is a selective system of resource distribution. It
leads to a concentration of funding (04_01; 04_02; 04_04). The drive for
bibliometrics comes from the government (04_01; 04_02) and the latter also
stimulated the introduction of impact (04_01). The major problems are the
decreasing level of GDP and the need to maintain the resource level for universities
(04_01; 04_02; 04_03; 04_05).
In regards to the effects on academic performance, the interviewees said that it is an
incentive to produce high quality research by peers (04_01; 04_02; 04_03; 04_04;
04_05) and to broaden interdisciplinary research and collaboration (04_01). An
expert in the field of the RAE/REF stated that there is a dual perception of the effects
on research performance. It is a more legitimate activity. There are knock-on effects
such as more micro-management, more support and more strategic behaviour
(04_06). Furthermore, one of the experts said that there is a drive towards a strategy
of gaming. It encourages a short-term approach. It encourages a drive towards the
norm of mean panel requirements. A reduction and control of quality becomes
visible. Money is a driving force for decisions (04_07).
4.2.2 Inequality
Concerning the RAE/REF's effect on inequality, we establish gender imbalances in
the staff selected. This is completely in line with the arguments I identified in the
interviews with scientists and academics. To balance it out there are codes of
practice and the possibility of reducing the number of publications that need to be
submitted to the RAE/REF (04_01). It is possible to reduce the number of

www.researchcghe.org
28
publications under special circumstances (such as periods of maternity leave, certain
life events, illness and disability) (04_01; 04_03; 04_04). One research expert said
that there are gender effects, and effects on the stages of careers for part-time
workers. There is also implicit and potential pressure on disclosure, in such a way
that researchers fear being perceived as less productive and a ‘half peer’ (04_06).
One expert also said that there is inequality in the attendance of conferences, in
direct funding, and in time and emotional support. There is a drive towards a
‘research control’ culture (04_07). A former leading adviser to the government
explained that this depends on the principles of the RAE/REF. If the RAE/REF
makes it possible to put more money into the universities that did not previously
obtain funding, then this is a good thing (04_08).
4.2.3 Diversity
The diversity issue is handled in a similar way to the inequality issue (04_01). It is
assumed that short-term work is encouraged rather than long-term, ‘risky’ research
(04_03). Interdisciplinary research is discouraged (04_04). One research expert said
that there is an effect on multidisciplinary research and humanities research. One
research expert stated that the boundaries of what is acceptable are set by the
panels. Efforts are increased for people who are producing less than four outputs
because of special circumstances. There is a lack of female staff and professors of
minorities (04_07).
4.2.4 Academic freedom
Choice of research themes
As mentioned above, the results suggest that short-term work is encouraged rather
than long-term, risky research (04_03). Interdisciplinary research is discouraged
(04_04) and impact influences topics (04_01; 04_04). The panel structure is likely to
inhibit interdisciplinary research (04_02). Nevertheless, another interviewee said that
interdisciplinary research scored as highly as other research. It encourages all types
of outputs and areas, but there is conservatism in the sector, which might prevent
some output types and areas (04_03). Citations are not used as criteria in the REF
process (04_05). One research expert claimed that there is not a massive effect, but
that this is rather more symptomatic. Another research expert said, in contrast, that
there is a significant effect. It is a highly managed context, and the devising of a
submission strategy is observable (04_07).
Research freedom
An interviewee from a funding body said that the judgement criteria of the panels are
broad (04_01). It is more the measurement regime which limits research freedom to
invest in mainstream, rather than unusual, research (04_02). But the direction is not
predetermined. It is a process of quality selection by peers from the academic and
scientific community (04_03; 04_04; 04_05). There is an effect on the institutional,
system and central senior level, as one research expert said. There are filters of
freedom (04_07). A former leading adviser to the government said that he hoped

www.researchcghe.org
29
people were doing what they want to do and that he believes the RAE/REF does not
affect research freedom (04_08).
Teaching freedom
Interviewees said that the pressure to publish keeps them from teaching (04_01;
04_02; 04_03; 04_04). But other countries are also driven by research. It is likely
that the RAE/REF accentuates a problem which is rooted in the academic and
scientific system (04_05). One research expert said that there is a positive nexus
with teaching. It is scholarly research informed. But teaching is also affected by the
workload management for junior staff and the individual work balance (04_06).
Another research expert claimed that no effect can be felt on teaching freedom
(04_07).
Learning freedom
The policy adviser from the funding body explained that the possibility of discovering
a new area could be limited due to time reductions because there is a drive towards
the ongoing production of publications. Radical changes in research are less likely
(04_03; 04_04). One research expert said that there is no effect at all (04_06).
Another research expert stated that there might be an effect on in-depth reading due
to reasons of time (04_07). A former leading adviser to the government said that he
suspects there are few effects on learning freedom (04_08).
Contributions to the academic community
The interviewees argued that there are incentives to contribute to the academic
community due to certain indicators such as esteem and environment in the
RAE/REF process to stimulate community behaviour (04_01). Furthermore, the
nomination for the RAE/REF could have a knock-on effect on the people who are not
selected. Their willingness to contribute could be reduced (04_02). Further
interviewees said that there are no limitations to collaboration (04_03; 04_05). One
research expert claimed that there are different types of esteem indicators used to
recognise the contributions of the academic community (04_06). Another research
expert replied that the RAE/REF procedure reduces the willingness to contribute to
the academic community because researchers are more interested in highlighting
their own work: it therefore encourages a less collegial and more selfish environment
(04_07).
4.2.5 The relationship between teaching and research
There is an emphasis on research rather than teaching, but this will change with the
introduction of the Teaching Excellence Framework (TEF) (04_02; 04_05). One
research expert refers to the effects on teaching freedom (04_06). Another research
expert claims that the research intensive universities have a research-led teaching
culture which ignores what students need and what employers expect (04_07).

www.researchcghe.org
30
4.2.6 Recruitment policies
As regards the RAE/REF's effect on recruitment policies, the policy advisers
perceive this as a major effect. It creates a transfer market (04_01; 04_02; 04_03). A
rise in staff movement can be observed by the census date (04_03). Certain
characteristics are required, such as a strong publication record, by recruiters. These
characteristics promise better jobs and better salaries (04_01). There is continued
poaching and enticement of research stars (04_02), and it discourages international
researchers (04_02). Fixed dates and timing of recruitment are relevant (04_04).
Early career researchers are struggling in the assessment cycle (04_04). It enables
greater mobility of researchers, and encourages hiring the best people (04_05). One
research expert said that it creates a transfer market (04_06) while another research
expert claimed that it depends on where you are in the system (04_07). A former
leading adviser to the government said he hoped that it has no effects (04_08).
4.2.7 Research motivation and pressure
Concerning the effect of the RAE/REF on motivation, one interviewee stated that the
RAE/REF does not have an effect on this. Rather, it is the way the management and
performance assessment is implemented in universities (04_01). One interviewee
from a higher education agency said that you have to differentiate between the
people whose work is submitted to the RAE/REF and those whose work is not
selected. For the first group, it means a confirmation of success and underlines the
fact that they are highly employable. Those who are not selected will experience a
loss of morale (04_02). This situation might also change the way academics and
scientists select their research topics. They will have the highest research
motivations for areas which will promise them the best outcome. There is further
motivation for promotion (04_03). The RAE/REF rewards excellence and there is a
strong motivation to produce good quality work. The scientists and academics are
motivated to produce major breakthroughs that are visible to everyone (04_05). One
research expert said that there is a dual structure of effects (04_06). Another
research expert argued there is a more demotivating effect (04_07).
4.2.8 Power
In regards to the power relationships influencing performance assessment in science
and academia, the policy advisers see wide evidence for this (04_01; 04_02; 04_03;
04_04). There is a growing need to justify public investments in the spending review
and to provide evidence for the Treasury (04_01). The peer review is a further
vehicle of power structures. People know each other very well; there are good and
bad relationships between the assessors and the assessed, and institutional
affiliation plays a role as people care about the name of the universities. They use
their power to achieve better outcomes in the RAE/REF (04_02). Furthermore, the
institution is seen as a driving power (04_03), on the level of the individual, the
institution, the panel and researcher (04_04), the government (04_04), lobby groups,
society, and scientific communities and academic unions (04_01). One research

www.researchcghe.org
31
expert said that there are power relationships affecting performance assessment in
science and academia (04_06) while another research expert went into more detail
by describing evidence of management power, cliques of experts, hegemonial
definitions and hierarchies of esteem in the disciplines (04_07).
5. Summary
The academic life theory was extended and dimensions of research performance vs.
research power, equality vs. inequality, diversity vs. uniformity, academic freedom
vs. academic unfreedom, research and teaching vs. research and recruitment
policies were added. These dimensions are vertically stratified by individual,
organisational and macrological actors. Six theories of higher educational alterations
were applied, which include elements for explaining science and academia in
general, and, in the case of performance assessment in science and academia, in
the UK. First, there is a belief in numbers. Second, there is reactivity due to
indicators and judgement criteria. Third, there are market models affecting
performance assessment in science and academia. Fourth, there is business control
of scientific and academic entities. Fifth, there is discipline and control of universities.
Sixth, this is accompanied by quality seals.
In a second step, functional and conflict-theoretical models were developed for the
explanation of science and academia. They were applied for performance
assessment in science and academia to identify the RAE/REF's effects in the UK.
The first model explains the production of knowledge in universities as a stratification
of meritocratic performance entities like performance, quality, talent, achievement on
the basis of equality of opportunity. The second model explains the production of
knowledge as being stratified by power, capital and status on the basis of inequality
of opportunity. These are two theoretical models that were combined to explain
performance assessment in science and academia in the UK. The empirical results
support the application of both theories for the understanding of science and
academia in general, and for considering the effects of performance assessment in
science and academia in the UK.
Empirical results
Scientists and academics
First, I will summarise the research results of the academics and scientists from
three different institutional ranks and three disciplines. The main results in reference
to research performance show that the UK is a leading country worldwide in science
and academia. It is not yet clear whether this is attributable to the RAE/REF.
Scientists and academics believe in the increased performance of UK science and
academia in terms of publication and citation rates. The RAE/REF and its
interpretation at universities by management, departments and staff leads to unequal

www.researchcghe.org
32
treatment in early career developments, progression and promotion, and promotes
gender inequality.
Furthermore, the RAE/REF is likely to inhibit diversity in research topics due to
scientists and academics conforming to the RAE/REF time cycle. Also, the RAE/REF
does not enable scientists and academics to change their research tracks
fundamentally, even though it does not force academics and scientists to take up
research on specific areas only. It is likely to reduce academic freedom in the choice
of research themes due to the introduction of impact and an orientation towards high
impact journal culture.
There are few effects on freedom of teaching and learning. It is likely to inhibit the
possibility to contribute to the academic and scientific community in terms of the
commitment of research stars, non-submitted scientists and academics, and
activities other than the four publications in the context of the RAE/REF. It enables a
teaching culture directed towards research-led teaching and affects recruitment
policies of scientists and academics in such a way that short-term rather than long-
term relationships are favoured. A kind of transfer market is created. It is likely to
reduce long-term, innovative research.
Due to managerialism and quality management procedures in the university it
encourages a strategic shift in academia and science to maintain a position on or
climb up the ranking ladder. Lower ranked institutions are increasing their efforts to
strategically establish research collaborations with research stars at excellenct
universities and invest resources by using computer models and information systems
to increase their institutional ranks. The RAE/REF is likely to reduce motivation for
the production of knowledge and creates pressure for scientists and academics.
Research intensive universities are coming under strong pressure to publish
worldwide leading research. This might go hand in hand with a decrease of
academic quality due to pressure and stress. Furthermore, there is evidence of
power relationships in performance assessment of science and academia.
Connections between scientists, academics and politics
Second, I will provide insights into the perception of the effects of the RAE/REF from
the viewpoint of policy advisers, research experts in the field and a leading adviser to
the government. The performance of UK science and academia has improved. But
the RAE/REF is a bureaucratic mechanism that generates costs and adds pressure
to academics to increase their quality of work. Increasing managerialism that leads
to a concentration of funding in the university system is observable. We witness a
drive for metrics, which emanates from the government. Major problems include the
decreasing level of GDP and the need to maintain the financial level of the university
sector.

www.researchcghe.org
33
There is a dual perception of the RAE/REF's effects on academic life. It is
increasingly regarded as a legitimate activity. But there are knock-on effects, such as
micro-management and more strategic behaviour. There is a tendency to produce
research results to meet the mean of the panel structure. Inequality has an effect on
gender and the staff who are selected. To balance this out codes of practice have
been developed. Special circumstances can allow the amount of publications
required to be reduced. Short-term work instead of long-term work is increased.
Interdisplinary research could be inhibited. But there is also an expectation that it
should score as highly as other research. The conservatism in the sector is the
reason for the prevention of some output types. The judgement criteria are broad
and there is a measurement regime that restricts unusual research ideas.
Teaching and research are in a positive nexus in as much as teaching is research
informed. But teaching is affected by workload management. The RAE/REF means
academics are less likley to switch research directions and fundamentally change
their research track. Different types of esteem indicators favour certain contributions
to the academic community. The Teaching Excellence Framework (TEF) will be
implemented to compensate for the negative effects of the REF. The effect on
recruitment policies is particularly noteworthy. We frequently come across poaching
and enticement of star researchers. The timing of recruitment is relevant. The
RAE/REF has no effect on motivation. The latter is instead affected by the way the
management operates at universities. Motivation generates groundbreaking
research ideas. Power structures affect performance assessment in science and
academia.
Research results in the light of the state of research
This study refers to certain studies related to the state of research. The results can
be connected to theoretical expectations raised by Münch (cf. Münch 2007, 2011,
2014) and Marginson (cf. Marginson 1997, 2007a, 2007b) in as far as academic
freedom, autonomy and academic independence are concerned. Whereas academic
autonomy is likely to persist, there is serious concern about the RAE/REF’s effect on
academic independence, since changing established research tracks, radical breaks
and critical thinking seem less likely as a result of performance assessment in
science and academia (cf. Marginson 1997, 2007a, 2007b). There is evidence that
inequality in the treatment of scientists and academics is rising in the wake of the
RAE/REF mechanism.
These results confirm the results with regard to career promotion and progression
identified by Oancea (cf. Oancea 2010b). Some indications show that the
hypotheses claiming that established institutions are gaining more resources are
partly true (cf. Orr 2003:45–46). However, there are also more and more new
universities gaining more resources. The study confirms the hypothesis of increased
gaming strategies and a better scoring of informed institutions (cf. Harley 2000:555).
It supports the short-term hypothesis that long-term speculative, creative, innovative,
risky and blue sky research is inhibited (cf. McNay 1997:70; Talib 2002; Oancea

www.researchcghe.org
34
2010a). This confirms the short-term hypothesis of McNay, Oancea and Talib (cf.
McNay 1997; Oancea 2010b; Talib 2002).
Research results also confirm the short-term recruitment hypothesis of McNay (cf.
McNay 2016:7) and McNay's transfer market hypothesis (cf. McNay 2016). The
decreasing academic freedom hypothesis in relation to managerial control can be
confirmed and differentiated for middle and lower ranked institutions (cf. Marginson
1997:366; Palfreyman 2006:17–19; Melo et al. 2010:251; Teelken 2012:282).
Evidence shows that the workload hypothesis for teachers and reseachers is true (cf.
Harley 2000:565), but it requires further analysis. At higher ranked institutions there
are a huge number of practices working against a lowering of academic freedom due
to university administration and quality management. There is some evidence that
vital academic community contributions are inhibited (cf. Talib 2002:359). The
hypothesis suggesting a separation of research and teaching (cf. Meier and
Schimank 2009:55) cannot be confirmed. It is more likely that the teaching style is
differentiated in terms of a research-led teaching culture and the new universities
play an ever increasing role in the research battle. But the hypothesis has to be
analysed in more detail. In quantitative terms there are indications that new
universities are gaining more research income in absolute numbers and experience
a reduction in the relative proportion of research income of total income. Additionally
Schmid shows that the ex-polytechnics are systematically dropping out of the
RAE/REF process in the disciplines chemistry and sociology. Furthermore, the ex-
polytechnics are predominantly financed through teaching income instead of
research income according to statistical data of HESA. But the universities founded
before 1992 also have a great amount of teaching income (cf. Schmid 2016).
The interviewees placed strong emphasis on teaching and research across all
university ranks, which could lead to a decrease in teaching quality (cf. Jenkins
1995). Furthermore, it is likely that the emphasis on a research-led teaching strategy
leads to an exclusion of other important aspects of teaching (cf. Broadhead and
Howard 1998a:10).
There are effects on the teaching research nexus that need to be analysed with
regard to different functions of teaching and research (cf. Marginson 2014). The
decreasing status of lecturer hypothesis could not be corroborated: there is a need
for further analysis (cf. Pritchard 2005:447). The findings suggest that the
bureaucracy hypothesis (cf. Pritchard 2005:445) is true, especially for the middle and
lower ranked institutions. The discrimination against new universities hypothesis (cf.
Kreckel 2008:186–87) has to be differentiated. There are more and more new
universities that are playing the RAE/REF game and have thus entered the national
competition for resources. Being a part of the RAE/REF community lowers the
discrimination rate of new universities. But there might still be a significant amount of
discrimination in the RAE/REF system. One has to clearly differentiate in further
analyses between new universities and ex polytechnics. Furthermore it could be
useful to develop questionnaires for departments that dropped out of the RAE/REF
process. This could give insights into processes of inclusion and exclusion in the

www.researchcghe.org
35
academic and scientific sphere. The hypothesis of research potential (cf. McNay
1997:65–67) as a relevant criterion in the recruitment process can be extended to
REF-able criteria, which includes research publications, but also other relevant
indicators and definitions of scientific and academic excellence. The research results
can be connected with further research in the area of the RAE/REF (cf. Broadhead
and Howard 1998b; Williams 1998; Elton 2000; Bence and Oppenheim 2005; Barker
2007; Bridges 2009a, 2009b; Leisyte 2007; Martin 2011; Tourish 2011; Bridges
2011; Shattock 2012; Enders 2014; Enders and Westerheijden 2014; Locke 2014;
Pettigrew, Cornuel, and Hommel 2014; Bleiklie, Enders, and Lepori 2015; Derrick
and Samuel 2015; Enders, Kehm, and Schimank 2015; McKenna 2015; Samuel and
Derrick 2015; Seeber et al. 2015; Shattock 2015; Souto-Otero and Enders 2015;
Wooding et al. 2015; Hamann 2016).
Future research
In my study I considered scientists and academics from three different university
ranks and three different disciplines, four policy advisers from a funding body, one
policy adviser from a higher education agency, two research experts in the field of
the RAE/REF and one leading adviser to the government. With a total of 31 in-depth
interviews concerning the effects of the RAE/REF on academic life I covered a broad
range of the perceptions of the effects exerted by the RAE/REF. Nevertheless,
further research still needs to be done. Interested researchers should broaden their
samples and add quantifiable material in their questions. Furthermore, I will look at
certain differentiations by institutional rank, discipline, age and position of scientists
and academics. Future studies could investigate the impact of the RAE/REF on non-
research active staff who are widely disadvantaged in the performance assessment
process in science and academia. A first introduction of the effects on non-submitted
staff is supplied by the following literature (cf. Oancea 2014). It is also possible to
carry out research on international comparisons (cf. McNay 2015b, 2015a) and
different disciplines (cf. Broadbent 2010).
It should be mentioned in this context that I am myself a researcher and part of the
academic and scientific community. I carried out my research in an as objective and
neutral way as possible. The present text is a working paper where I tried to address
the relevant literature in this field. Due to the complexity of the subject matter, I
would be most grateful for recommendations of relevant literature and empirical
results that I should take into account. Email: len-ole.schaefer@uni-bamberg.de
Thank you very much for your time.

www.researchcghe.org
36
Literature
Ackers, Louise. 2007. Legislating for Equality? Working Hours and Progression in Science
Careers. Rochester, NY: Social Science Research Network. Retrieved 3 May 2016
(http://papers.ssrn.com/abstract=973957).
Adams, Jonathan. 1998. ‘Benchmarking International Research’. Nature 396(6712):615–18.
Adams, Jonathan. 2011. ‘GLOBAL RESEARCH REPORT UNITED KINGDOM. Leeds:
Evidence.’ Retrieved 10 June 2015
(http://researchanalytics.thomsonreuters.com/m/pdfs/globalresearchreport-uk.pdf).
Auranen, Otto and Mika Nieminen. 2010. ‘University Research Funding and Publication
Performance - An International Comparison’. Research Policy 39(6):822–34.
Bagilhole, Barbara. 1993. ‘Survivors in a Male Preserve: A Study of British Women
Academics’ Experiences and Perceptions of Discrimination in a UK University’.
Higher Education 26(4):431–447.
Barbezat, Debra A. and James W. Hughes. 2005. ‘Salary Structure Effects and the Gender
Pay Gap in Academia’. Research in Higher Education 46(6):621–40.
Barker, Katharine. 2007. ‘The UK Research Assessment Exercise: The Evolution of a
National Research Evaluation System’. Research Evaluation 16(1):3–12.
Bence, Valerie and Charles Oppenheim. 2005. ‘The Evolution of the UK’s Research
Assessment Exercise: Publications, Performance and Perceptions’. Journal of
Educational Administration and History 37(2):137–55.
Bentley, Alex. 2016. ‘Devil’s in the REF’. Times Higher Education (THE). Retrieved 28 April
2016 (https://www.timeshighereducation.com/letters/devils-in-the-research-
excellence-framework).
Blake, Margaret and Ivana La Valle. 2000. ‘Who Applies for Research Funding’. Key factors
shaping funding application behaviour among women and men in British higher
education institutions. London: Wellcome Trust. Retrieved 3 May 2016
(http://www.wellcome.ac.uk/stellent/groups/corporatesite/@policy_communications/d
ocuments/web_document/wtd003210.pdf).
Bleiklie, Ivar, Jürgen Enders, and Benedetto Lepori. 2015. ‘Organizations as Penetrated
Hierarchies: Environmental Pressures and Control in Professional Organizations’.
Organization Studies 36(7):873–896.
Bornmann, Lutz and Loet Leydesdorff. 2013. ‘Macro-Indicators of Citation Impacts of Six
Prolific Countries: InCites Data and the Statistical Significance of Trends’. PLoS
ONE 8(2):e56768.
Bourdieu, Pierre. 1979. ‘Les Trois États Du Capital Culturel’. Actes de la recherche en
sciences sociales 30(1):3–6.
Bourdieu, Pierre. 1980. ‘Le Capital Social’. Actes de la recherche en sciences sociales
31(1):2–3.
Bourdieu, Pierre. 1983. ‘Ökonomisches Kapital, Kulturelles Kapital, Soziales Kapital’. Pp.
183–98 in In: Reinhard Kreckel (Hg.): Soziale Ungleichheiten. Soziale Welt.
Sonderband 2.
Bourdieu, Pierre. 1987. Sozialer Sinn. Frankfurt am Main: Suhrkamp.
Bourdieu, Pierre. 1992a. Homo Academicus. Frankfurt am Main: Suhrkamp.
Bourdieu, Pierre. 1992b. Rede Und Antwort. Frankfurt am Main: Suhrkamp.

www.researchcghe.org
37
Bourdieu, Pierre. 1996. Reflexive Anthropologie. Frankfurt am Main: Suhrkamp.
Bourdieu, Pierre. 1998. Praktische Vernunft. Zur Theorie Des Handelns. Frankfurt am Main:
Suhrkamp.
Bourdieu, Pierre. 2006. Die Feinen Unterschiede. Frankfurt am Main: Suhrkamp.
Bridges, David. 2009a. ‘Assessing the Quality of Research in Higher Education: The United
Kingdom Research Assessment Exercise’. In: Besley, Tina (A.C.) (Ed.): Assessing
the quality of educational research in higher education, Educational futures :
rethinking theory and practice. Rotterdam: Sense Publishers.
Bridges, David. 2009b. ‘Research Quality Assessment in Education: Impossible Science,
Possible Art?’ British Educational Research Journal 35(4):497–517.
Bridges, David. 2011. ‘Research Quality Assessment: Intended and Unintended
Consequences’. Power and Education 3(1):31–38.
Broadbent, Jane. 2010. ‘The UK Research Assessment Exercise: Performance
Measurement and Resource Allocation’. Australian Accounting Review 20(1):14–23.
Broadhead, Lee-Anne and Sean Howard. 1998a. ‘The Art of Punishing: The Research
Assessment Exercise and the Ritualisation of Power in Higher Education’. Education
Policy Analysis Archives 6(8).
Broadhead, Lee-Anne and Sean Howard. 1998b. ‘The Research Assessment Exercise’.
education policy analysis archives 6(0):8.
Clark, Burton. 1998. ‘University Transformation for the Twenty-First Century. World
Conference on Higher Education’. Retrieved 30 September 2014
(http://portal.unesco.org/education/fr/files/10371/10384906150Burton.pdf/Burton.pdf)
.
Clark, Burton R. 1989. ‘The Academic Life Small Worlds, Different Worlds’. Educational
Researcher 18(5):4–8.
Clark, Burton R. 1998. Creating Entrepreneurial Universities. Oxford: IAU Press.
Cuthbertson, Keith. 2016. ‘The Gender Pay Gap: It’s Not as Simple as It Looks’. Times
Higher Education (THE). Retrieved 28 April 2016
(https://www.timeshighereducation.com/comment/the-gender-pay-gap-its-not-as-
simple-as-it-looks).
Davis, K. and W. E. Moore. 1945. ‘Some Principles of Stratification’. American sociological
review 10(2):242–49.
Derrick, G. E. and G. Samuel. 2015. ‘The Impact Evaluation Scale: Group Panel Processes
and Outcomes in Societal Impact Evaluation’. Retrieved 12 May 2016
(http://www.sti2015.usi.ch/sites/www.sti2015.usi.ch/files/media/derrick_samuel_2_pa
ge_abstract.pdf).
Doherty, Liz and Simonetta Manfredi. 2006. ‘Women’s Progression to Senior Positions in
English Universities’. Employee Relations 28(6):553–72.
Elton, Lewis. 2000. ‘The UK Research Assessment Exercise: Unintended Consequences’.
Higher Education Quarterly 54(3):274–83.
Enders, Jürgen. 2014. ‘The Academic Arms Race. International Rankings and Global
Competition for World-Class Universities.’ In: Pettigrew, A.W., Cornuel, E. and
Hommel, U. (Eds.): The Institutional Development of Business Schools. Oxford:
Oxford University Press. Retrieved 12 May 2016
(https://books.google.com/books?hl=en&lr=&id=SoMeBQAAQBAJ&oi=fnd&pg=PA15
5&dq=%22that+produce+similar+services+and+products%E2%80%99+(DiMaggio%

www.researchcghe.org
38
22+%22and+their+members+as+passive+recipients+of+institutional%22+%22field+
of+higher+education%E2%80%94research%22+%22signals+to+the+universities+an
d+other+actors+within+the+field,+for%22+&ots=m9MjJmaSLs&sig=omLGIj0Z4_VHv
7LCsVTmeMjGCK8).
Enders, Jürgen, Barbara M. Kehm, and Uwe Schimank. 2015. ‘Turning Universities into
Actors on Quasi-Markets: How New Public Management Reforms Affect Academic
Research’. Pp. 89–103. In: Jansen, Dorothea, Pruisken, Insa (Eds.): The Changing
Governance of Higher Education and Research. Switzerland: Springer International
Publishing. Retrieved 12 May 2016 (http://link.springer.com/chapter/10.1007/978-3-
319-09677-3_5).
Enders, Jürgen and Don F. Westerheijden. 2014. ‘The Dutch Way of New Public
Management’. Policy and Society 33(3):189–98.
Espeland, Wendy Nelson and Michael Sauder. 2007. ‘Rankings and Reactivity: How Public
Measures Recreate Social Worlds’. American Journal of Sociology 113(1):1–40.
Foucault, Michel. 1976. Überwachen Und Strafen. Frankfurt am Main: Suhrkamp.
Foucault, Michel. 2000. ‘Die Gouvernementalität’. In: Ulrich Bröckling, Susanne Krasmann
und Thomas Lemke (Eds.): Gouvernementalität der Gegenwart. Studien zur
Ökonomisierung des Sozialen. Frankfurt am Main: Suhrkamp Verlag.
Frølich, Nicoline. 2008. The Politics of Steering by Numbers: Debating Performance-Based
Funding in Europe. Oslo: NIFU STEP.
Grove, Jack. 2015. ‘UCU Calls for Wage Audits as Gender Pay Gap Endures’. Times
Higher Education (THE). Retrieved 27 April 2016
(https://www.timeshighereducation.com/news/ucu-calls-for-wage-audits-as-gender-
pay-gap-endures/2019452.article).
Hamann, Julian. 2016. ‘The Visible Hand of Research Performance Assessment’. Higher
Education 1–19.
Harley, Sandra. 2000. ‘Accountants Divided: Research Selectivity and Academic
Accounting Labour in UK Universities’. Critical Perspectives on Accounting
11(5):549–82.
Harley, Sandra and Frederic S. Lee. 1997. ‘Research Selectivity, Managerialism, and the
Academic Labor Process: The Future of Nonmainstream Economics in U.K.
Universities’. Human Relations 50(11):1427–60.
Hazelkorn, Ellen. 2008. ‘Learning to Live with League Tables and Ranking: The Experience
of Institutional Leaders’. Higher Education Policy 21(2):193–215.
Hazelkorn, Ellen. 2009. ‘Rankings and the Battle for World-Class Excellence’. Higher
Education Management and Policy 21(1):1–22.
Hazelkorn, Ellen. 2014. ‘Reflections on a Decade of Global Rankings: What We’ve Learned
and Outstanding Issues’. European Journal of Education 49(1):12–28.
Hazelkorn, Ellen and M. Ryan. 2013. ‘The Impact of University Rankings on Higher
Education Policy in Europe: A Challenge to Perceived Wisdom and a Stimulus for
Change’. In: Zgaga, P., Teichler, U. and Brennan, J. (Eds.) The Globalization
Challenge for European Higher Education: Convergence and Diversity, Centres and
Peripheries. Frankfurt: Peter Lang.
Henkel, Mary. 1999. ‘The Modernisation of Research Evaluation: The Case of the UK’.
Higher Education 38(1):105–22.

www.researchcghe.org
39
Hey, Valerie et al. 2011. The Experience of Black and Minority Ethnic Staff in Higher
Education in England. London: Equality Challenge Unit. Retrieved 3 May 2016
(http://www.ecu.ac.uk/wp-content/uploads/external/experience-of-bme-staff-in-he-
final-report.doc).
Jenkins, Alan. 1995. ‘The Research Assessment Exercise, Funding and Teaching Quality’.
Quality Assurance in Education 3(2):4–12.
King, David A. 2004. ‘The Scientific Impact of Nations’. Nature 430(6997):311–16.
Knights, David and Wendy Richards. 2003. ‘Sex Discrimination in UK Academia’. Gender,
Work & Organization 10(2):213–38.
Kreckel, Reinhard. 2008. ‘Zwischen Universeller Inklusion Und Neuer Exklusivität.’ In:
Barbara M. Kehm (Ed.): Hochschule im Wandel. Die Universität als
Forschungsstand. Festschrift für Ulrich Teichler, vol. 26. Frankfurt/New York:
Campus Verlag. Retrieved 26 November 2015
(https://books.google.com/books?hl=en&lr=&id=moDlAgAAQBAJ&oi=fnd&pg=PA181
&dq=%22In+den+Augen+von+Ulrich+Teichler+(2005,+S.+165)+ist+f%C3%BCr+Deu
tschland+das+Erreichen%22+%22an+Boden+(vgl.+Kreckel+2006,+M%C3%BCnch+
2007).+Das+Hauptaugenmerk+gilt+dabei+der%22+&ots=c8V5tTBWLP&sig=GQG0c
eUXXrMjOkQ4r3wT7o9iWtU).
Lee, Frederic S., Xuan Pham, and Gyun Gu. 2013. ‘The UK Research Assessment
Exercise and the Narrowing of UK Economics’. Cambridge Journal of Economics
37(4):693–717.
Leisyte, Liudvika. 2007. University Governance and Academic Research: Case Studies of
Research Units in Dutch and English Universities. University of Twente. Retrieved 12
May 2016 (http://doc.utwente.nl/58088).
Locke, William. 2014. ‘The Intensification of Rankings Logic in an Increasingly Marketised
Higher Education Environment’. European Journal of Education 49(1):77–90.
Locke, William. 2016. ‘University Careers Aren’t What They Used to Be’. Times Higher
Education (THE). Retrieved 28 April 2016
(https://www.timeshighereducation.com/blog/university-careers-arent-what-they-
used-be).
Manfredi, Dottore Simonetta and Lucy Vickers. 2009. ‘The Impact of the Process to
Promote Equality and Diversity in the Research Assessment Exercise 2008
Research Report 2009’. Retrieved 3 May 2016 (http://www.ecu.ac.uk/wp-
content/uploads/external/equality-and-diversity-in-the-research-assessment-
exercise.doc).
Marginson, Simon. 1997. ‘How Free Is Academic Freedom?’ Higher Education Research &
Development 16(3):359–69.
Marginson, Simon. 2006. ‘Dynamics of National and Global Competition in Higher
Education’. Higher Education 52(1):1–39.
Marginson, Simon. 2007a. ‘Are Neo-Liberal Reforms Friendly to Academic Freedom and
Creativity? Some Theoretical and Practical Reflections on the Constituents of
Academic Self-Determination in Research Universities’. Centre for the Study of
Higher Education, The University of Melbourne Seminar ‘Ideas and Issues in Higher
Education’, 28 May 2007. Retrieved 4 May 2016
(http://www.tandfonline.com/doi/abs/10.1080/0305764X.2010.549456).

www.researchcghe.org
40
Marginson, Simon. 2007b. ‘Freedom as Control and the Control of Freedom: F. A. Hayek
and the Academic Imagination’. Pp. 67–104. In: Kayrooz, Carole, Gerlese S.
Åkerlind, Malcolm Tight (Eds.): Autonomy in Social Science Research, vol. 4,
International Perspectives on Higher Education Research. Emerald Group
Publishing Limited. Retrieved 4 May 2016
(http://www.emeraldinsight.com/doi/abs/10.1016/S1479-3628%2806%2904004-4).
Marginson, Simon. 2011. ‘Equity, Status and Freedom: A Note on Higher Education’.
Cambridge Journal of Education 41(1):23–36.
Marginson, Simon. 2014. ‘Teaching and Research in the Contemporary University’. Pp. 11–
18. In: Tong, Vincent C.H. (Ed.): Geoscience Research and Education. Teaching at
Universities., Innovations in Science Education and Technology. Springer
Netherlands. Retrieved 4 May 2016 (http://link.springer.com/chapter/10.1007/978-
94-007-6946-5_2).
Marginson, Simon. 2015. ‘UK Research Excellence: Getting Better All the Time?’
International Higher Education (81):10–12.
Marginson, Simon, Peter Murphy, and Michael A. Peters. 2010. Global Creation: Space,
Mobility, and Synchrony in the Age of the Knowledge Economy. Peter Lang.
Marginson, Simon and Marijk van der Wende. 2007. ‘To Rank or To Be Ranked: The
Impact of Global Rankings in Higher Education’. Journal of Studies in International
Education 11(3–4):306–29.
Martin, Ben R. 2011. ‘The Research Excellence Framework and the “impact Agenda”: Are
We Creating a Frankenstein Monster?’ Research Evaluation 20(3):247–54.
Matthews, David. 2015a. ‘Lord Stern Review: No “foregone Conclusions” about Future of
REF’. Times Higher Education (THE). Retrieved 28 April 2016
(https://www.timeshighereducation.com/news/lord-stern-review-no-foregone-
conclusions-about-future-of-research-excellence-framework).
Matthews, David. 2015b. ‘REF Sceptic to Lead Review into Research Assessment’. Times
Higher Education (THE). Retrieved 28 April 2016
(https://www.timeshighereducation.com/news/ref-sceptic-lead-review-research-
assessment).
Matthews, David. 2016a. ‘REF “leads to Short-Term Research Approach”’. Times Higher
Education (THE). Retrieved 28 April 2016
(https://www.timeshighereducation.com/news/ref-leads-short-term-research-
approach).
Matthews, David. 2016b. ‘REF Should Include All Academics, Says University of
Cambridge’. Times Higher Education (THE). Retrieved 28 April 2016
(https://www.timeshighereducation.com/news/ref-should-include-all-academics-says-
university-cambridge).
Matthews, David. 2016c. ‘Universities Clash over Future of REF’. Times Higher Education
(THE). Retrieved 28 April 2016
(https://www.timeshighereducation.com/news/universities-clash-over-future-of-
research-excellence-framework-ref).
Matthews, David. 2016d. ‘Universities Clash over Future of REF | THE News’. Retrieved 28
April 2016 (https://www.timeshighereducation.com/news/universities-clash-over-
future-of-research-excellence-framework-ref).

www.researchcghe.org
41
McKenna, Hugh P. 2015. ‘Research Assessment: The Impact of Impact’. International
Journal of Nursing Studies 52(1):1–3.
McNay, Ian. 1997. The Impact of the 1992 RAE on Institutional and Individual Behaviour in
English Higher Education. The Evidence from a Research Project. HEFCE M 5/97,
May.
McNay, Ian. 2015a. ‘Debate: Does Research Quality Assessment Increase Output and
Give Value for Money?’ Public Money & Management 35(1):67–68.
McNay, Ian. 2015b. ‘Learning from the UK Research Excellence Framework: Ends and
Means in Research Quality Assessment, and the Reliability of Results in Education’.
Higher Education Review 47(3):24–47.
McNay, Ian. 2016. Imbalancing the Academy: The Impact of Research Quality Assessment.
Meier, Frank and Uwe Schimank. 2009. ‘Matthäus Schlägt Humboldt? New Public
Management Und Die Einheit von Forschung Und Lehre’. Beiträge zur
Hochschulforschung 31(1):42–61.
Melo, Ana Isabel, Cláudia S. Sarrico, and Zoe Radnor. 2010. ‘The Influence of Performance
Management Systems on Key Actors in Universities’. Public Management Review
12(2):233–54.
Merton, Robert K. 1968. ‘The Matthew Effect in Science’. Science 159(3810):56–63.
Merton, Robert K. 1988. ‘The Matthew Effect in Science, II: Cumulative Advantage and the
Symbolism of Intellectual Property’. Isis 79(4):606–23.
Merton, Robert K. 1995. Soziologische Theorie Und Soziale Struktur. Berlin: Walter de
Gruyter.
Münch, Richard. 2007. Die Akademische Elite. Frankfurt am Main: Suhrkamp.
Münch, Richard. 2011. Akademischer Kapitalismus. Berlin: Suhrkamp.
Münch, Richard. 2014. Academic Capitalism: Universities in the Global Struggle for
Excellence. London und New York: Routledge.
Münch, Richard and Len Ole Schäfer. 2014. ‘Rankings, Diversity and the Power of Renewal
in Science. A Comparison between Germany, the UK and the US’. European Journal
of Education 49(1):60–76.
Oancea, Alis. 2010a. ‘Research Assessment in the United Kingdom’. in In: UNESCO (Hg.):
World social science report: Knowledge Divides. Paris: UNESCO Publishing.
Oancea, Alis. 2010b. The BERA / UCET Review of the Impacts of RAE 2008 on Education
Research in UK Higher Education Institutions. London: British Educational Research
Association.
Oancea, Alis. 2014. ‘Research Assessment as Governance Technology in the United
Kingdom: Findings from a Survey of RAE 2008 Impacts’. Zeitschrift für
Erziehungswissenschaft 17(6):83–110.
Oancea, Alis. 2016. ‘Challenging the Grudging Consensus behind the REF’. Times Higher
Education (THE). Retrieved 28 April 2016
(https://www.timeshighereducation.com/blog/challenging-grudging-consensus-
behind-ref).
Orr, Dominic. 2003. Verfahren Der Forschungsbewertung Im Kontext Neuer
Steuerungsverfahren Im Hochschulwesen: Analyse von Vier Verfahren Aus
Niedersachsen, Großbritannien, Den Niederlanden Und Irland. Hannover:
HOCHSCHUL-INFORMATIONS-SYSTEM.

www.researchcghe.org
42
Palfreyman, David. 2006. ‘Is Academic Freedom under Threat in UK and US Higher
Education?’ Retrieved 19 October 2015
(http://oxcheps.new.ox.ac.uk/MainSite%20pages/Resources/OxCHEPS_OP23ii.pdf).
Pettigrew, Andrew M., Eric Cornuel, and Ulrich Hommel, eds. 2014. The Institutional
Development of Business Schools. First edition. Oxford: Oxford University Press.
Porter, Theodore M. 1996. Trust in Numbers: The Pursuit of Objectivity in Science and
Public Life. 2. Auflage. Princeton: Princeton University Press.
Power, Michael. 1999. The Audit Society. Rituals of Verification. New York: Oxford
University Press.
Pritchard, Rosalind. 2005. ‘The Influence of Market Force Culture on British and German
Academics’. Comparative Education 41(4):433–54.
Probert, Belinda. 2005. ‘I Just Couldn’t Fit It In: Gender and Unequal Outcomes in
Academic Careers’. Gender, Work & Organization 12(1):50–72.
Rhoades, Gary and Sheila Slaughter. 1997. ‘Academic Capitalism, Managed Professionals,
and Supply-Side Higher Education’. Social Text 51:9–38.
Samuel, Gabrielle N. and Gemma E. Derrick. 2015. ‘Societal Impact Evaluation: Exploring
Evaluator Perceptions of the Characterization of Impact under the REF2014’.
Research Evaluation rvv007.
Schäfer, Len Ole. 2012. The Construction of Rankings in Science: Chemistry in Germany
and the USA. Diploma thesis: University of Bamberg.
Schmid, Fabian. 2016. Academic capitalism and research funding in the UK - selectivity
and institutional separation of research and teaching. Working paper: University of
Bamberg.
Science and Technology Committee. 2002a. ‘House of Commons - Science and
Technology - Second Report’. Retrieved 15 September 2015
(http://www.publications.parliament.uk/pa/cm200102/cmselect/cmsctech/507/50702.
htm).
Science and Technology Committee. 2002b. ‘THE RESEARCH ASSESSMENT
EXERCISE: GOVERNMENT RESPONSE to the Committee’s Second Report. Fifth
Special Report of Session 2001-02. London: The Stationery Office Limited.’
Retrieved 15 September 2015
(http://www.publications.parliament.uk/pa/cm200102/cmselect/cmsctech/995/995.pdf
).
Science and Technology Committee. 2004a. ‘Research Assessment Exercise: A Re –
Assessment: Government Response to the Committee’s Eleventh Report of Session
2003–04. London: The Stationery Office Limited.’ Retrieved 15 September 2015
(http://www.parliament.the-stationery-
office.co.uk/pa/cm200405/cmselect/cmsctech/34/34.pdf).
Science and Technology Committee. 2004b. ‘Research Assessment Exercise: A Re-
Assessment. Eleventh Report of Session 2003–04. London: The Stationery Office
Limited’. Retrieved 15 September 2015 (http://www.parliament.the-stationery-
office.co.uk/pa/cm200304/cmselect/cmsctech/586/586.pdf).
Seeber, Marco et al. 2015. ‘European Universities as Complete Organizations?
Understanding Identity, Hierarchy and Rationality in Public Organizations’. Public
Management Review 17(10):1444–74.

www.researchcghe.org
43
Shattock, Michael. 2012. Making Policy in British Higher Education 1945-2011. McGraw-Hill
Education (UK).
Shattock, Michael. 2015. ‘The Impact of the UK Research Assessment Exercise’.
International Higher Education (56). Retrieved 12 May 2016
(https://ejournals.bc.edu/ojs/index.php/ihe/article/download/8438/7572).
Shepherd, Jessica. 2011. ‘14,000 British Professors – but Only 50 Are Black’. The
Guardian, May 27. Retrieved 3 May 2016
(http://www.theguardian.com/education/2011/may/27/only-50-black-british-
professors).
Souto-Otero, Manuel and Jürgen Enders. 2015. ‘International Students’ and Employers’
Use of Rankings: A Cross-National Analysis’. Studies in Higher Education 1–28.
Strathern, Marilyn. 1997. ‘“Improving Ratings”: Audit in the British University System’.
European Review 5(3):305–21.
Talib, Ameen Ali. 2002. Performance Measures and Resource Allocation: The Behavioural
Consequences of the University Research Assessment Exercise in England.
University of Warwick, Warwick Business School.
Teelken, Christine. 2012. ‘Compliance or Pragmatism: How Do Academics Deal with
Managerialism in Higher Education? A Comparative Study in Three Countries’.
Studies in Higher Education 37(3):271–90.
Tourish, Dennis. 2011. ‘Leading Questions: Journal Rankings, Academic Freedom and
Performativity: What Is, or Should Be, the Future of Leadership?’ Leadership
7(3):367–81.
Tumin, M. M. 1953. ‘Some Principles of Stratification: A Critical Analysis’. American
Sociological Review 18(4):387–94.
Wieczorek, Oliver and Len Ole Schäfer. 2016. ‘Verwaltungspraktiken: Die Auswirkung von
Leistungsindikatoren Am Beispiel Des Britischen Research Assessment Exercise.’
In: Nina Baur, Cristina Besio, Maria Norkus und Grit Petschick (Eds.): Wissen -
Organisation - Forschungspraxis. Der Makro-Meso-Mikro-Link in der Wissenschaft.
Landsberg: Beltz Juventa.
Williams, Gareth. 1998. ‘Misleading, Unscientific, and Unjust: The United Kingdom’s
Research Assessment Exercise’. BMJ 316(7137):1079.
Wooding, Steven, Thed N.Van Leeuwen, Sarah Parks, Shitij Kapur, and Jonathan Grant.
2015. ‘UK Doubles Its “World-Leading” Research in Life Sciences and Medicine in
Six Years: Testing the Claim?’ PLOS ONE 10(7):e0132990.

!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
Centre for Global Higher Education
UCL Institute of Education
London WC1H 0AL
www.researchcghe.org
@ResearchCGHE