CHI 1994-2013: Mapping Two Decades of Intellectual Progress through Co-word Analysis

Abstract

This study employs hierarchical cluster analysis, strategic diagrams and network analysis to map and visualize the intellectual landscape of the CHI conference on Human Computer Interaction through the use of co-word analysis. The study quantifies and describes the thematic evolution of the field based on a total of 3152 CHI articles and their associated 16035 keywords published between 1994 and 2013. The analysis is conducted for two time periods (1994-2003, 2004-2013) and a comparison between them highlights the underlying trends in our community. More significantly, this study identifies the evolution of major themes in the discipline, and highlights individual topics as popular, core, or backbone research topics within HCI.

Full text

CHI 1994-2013: Mapping Two Decades of Intellectual
Progress through Co-word Analysis
Yong Liu1, Jorge Goncalves1, Denzil Ferreira1, Bei Xiao2, Simo Hosio1, Vassilis Kostakos1
1Department of Computer Science and Engineering, University of Oulu, Finland
2Abo Akademi University, Finland
1firstname.lastname@ee.oulu.fi, 2xiaobei89@gmail.com
ABSTRACT
This study employs hierarchical cluster analysis, strategic
diagrams and network analysis to map and visualize the
intellectual landscape of the CHI conference on Human
Computer Interaction through the use of co-word analysis.
The study quantifies and describes the thematic evolution of
the field based on a total of 3152 CHI articles and their
associated 16035 keywords published between 1994 and
2013. The analysis is conducted for two time periods (1994-
2003, 2004-2013) and a comparison between them
highlights the underlying trends in our community. More
significantly, this study identifies the evolution of major
themes in the discipline, and highlights individual topics as
popular, core, or backbone research topics within HCI.
Author Keywords
Co-word analysis; bibliometric study; conceptual evolution;
HCI; cohesion; coherence
ACM Classification Keywords
K.2. History Of Computing: Theory.
INTRODUCTION
The CHI conference has a long and rich history. In the last
20 years alone its 3152 publications have shaped and
defined the field of human-computer interaction, making
CHI a flagship HCI venue characterized by its strong
multidisciplinarity. In this paper, we are interested in
mapping how the landscape of the HCI field has evolved, as
reflected in the record of CHI publications.
Harrison et al. [16] characterized the field of HCI into three
intertwined and non-exclusive paradigms: human-factors;
classical cognitivism/information processing based; and
phenomenologically-situated. This simplified categorization
makes it challenging to understand the field’s evolution as a
whole. As the authors note, it is difficult to assess
“marginal” contributions that are hard to precisely place.
HCI is indisputably a multidisciplinary field requiring a
more in-depth analysis to reveal the intricacies of its
evolution.
To contribute towards understanding the big picture of HCI
evolution, we analyzed CHI’s publications keywords since
1994, and for our convenience, we divided them into two
10-year periods: 1994-2003 and 2004-2013. Between 1994-
2003, CHI was predominantly focused on fixed (or non-
mobile) HCI. Since 2004, however, the field has grown at a
high pace, due to the introduction of extended abstracts and
electronic proceedings. The popularity of mobile phones,
ambient media and social technologies has shifted HCI
research towards mobile and social interaction, while new
issues involving humans, such as crowdsourcing and
privacy have taken the spotlight. We attempt to study and
analyze HCI research foci transitions and reflect on their
drivers and present status.
Our analysis relies on techniques from hierarchical cluster
and graph theory, through the use of co-word analysis
artifacts such as strategic diagrams and graphs. Co-word
analysis is part of the co-occurrence analysis methods. It is
a widely-applied bibliometric approach to describe the
interactions among concepts, ideas, and problems and to
explore the concept network within a scientific area [7,8]. A
recently published paper of a co-citation analysis of the
CHI proceedings [2] focused on authorship aspects of the
proceedings and citation metrics for papers. Here we focus
on the concepts that reflect our community and their
evolution over time.
Co-word analysis rests on the assumption that a paper’s
keywords constitute an adequate description of its content
as well as the links the paper established between problems:
two keywords co-occurring within the same paper are an
indication of a link between the topics to which they refer
to [9]. The presence of many co-occurrences around the
same word or pair of words points to a locus of strategic
alliance within articles that may embody a research theme
[9,30].
More importantly, by measuring the association strength of
terms produced in a specific scientific discipline, co-word
analysis allows researchers to identify key patterns and
trends within the area [13,18,20]. It is assumed that a
specific keyword with adequate frequency refers to a
particular research topic while a cluster or pattern of
keywords refers to a specific research direction or research
theme. A change of research theme (i.e., declining or
emerging research interest) as well as the change of
research topics within a research theme implies a paradigm
change.
RELATED WORK
The main concepts we use in our analysis are keywords,
networks, and clusters. Keywords appear on research
papers, and two keywords appearing on the same paper are
linked to form a network (or graph) of keywords. Analysis
of this network helps us identify clusters (a set of closely-
related keywords).
Our co-word analysis reduces a large space of descriptors
(i.e., keywords) into a network graph (i.e., multiple related
smaller spaces). Easier to comprehend but still retaining
Permissio n to make dig ital or hard c opies of all or p art of this wo rk for person al or
classroom use is granted without fee pro vided that copies are not made or distributed for
profit or commercial advantage and that copies bear this notice and the full citation on the
first page. Copyrights for components of this work owned by others than the author(s) must
be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on
servers or to redistribute to lists, requires prior specific permission and/or a f ee. Req ue st
permissions from permissions@acm.org.
CHI 2014, April 26–May 1, 2014, Toronto, Ontario, Canada.
Copyright is held by the owner/author (s). Publication rig hts licensed to ACM.
ACM 978-1-4503-2473-1/14/04...$15.00.
http://dx.doi.org/10 .1145/2556288.2556969

crucial information, this approach visualizes the interrelated
concepts [11] and intellectual structure of a discipline into a
map of the conceptual space of this field, and a time-series
of such maps produces a trace of the changes in this
conceptual space [13]. Co-word analysis has been widely
utilized in mapping the conceptual networks of a diversity
of disciplines, like business intelligence [29], consumer
behavior [22], software engineering [11], patent analyses
[10], biology [1,9], education [25], and library and
information science [13,20,30]. As such, it makes sense to
apply this technique to enrich our understanding of CHI.
Given a network of keywords, we can use network analysis
and strategic diagrams to characterize the field. Keywords
and clusters have different properties, depending on how
they are linked with each other. For instance, bridges
between two nodes (i.e., linked nodes) in a network perform
a valuable function in allowing communication and
facilitating the flow between otherwise isolated regions of
the network, also known as structural holes [24]. The
greater the number of bridges associated with a research
topic or theme, the more it serves to connect otherwise
isolated research topics or themes. Keywords with a great
number of structural holes serve as the backbone of the
whole network. If these are removed from the network, the
whole network will collapse into a number of separated and
unconnected research sub-fields, therefore losing its
scientific cohesion and identity.
When computing a network’s core-periphery structure, it
becomes possible to determine which nodes are part of a
densely connected core (i.e., with a higher number of
bridges) and which are part of a sparsely connected
periphery [5,26]. Core nodes are typically well connected to
peripheral nodes. Peripheral nodes are sparingly connected
to a core or to each other. In a keyword network it is
expected that, as the body of knowledge grows, peripheral
nodes become core nodes, thus allowing for the emergence
of new peripheral nodes. Research topics with a high core
value delimit the main body of HCI knowledge, and
represent important knowledge-growing points of the main
body of the field.
In our work we rely on two graph theory concepts to map
the field of HCI: density and centrality, defined as follows:
• Density, or internal cohesion, measures the strength of the
links that tie together the cluster of keywords making up
the research theme. This can be understood as a measure
of the theme’s development [17,22]. Density offers a
good representation of the cluster’s capacity to maintain
itself and to develop over the course of the time in the
field [7,17]. The higher the density, the more coherent the
cluster is and the more likely it is to contain inseparable
expressions;
• Centrality measures the degree of interaction of a theme
with other parts of the network [24]. In other words, it
measures the strength of external ties of a research theme
to other research themes, and can be referred to as a
measure of the importance of a theme in the development
of the entire research field [22]. The greater the number
and the strength of a theme’s connections with other
themes, the more central this theme will be to the whole
network [3].
By combining both concepts we then created a strategic
diagram. Strategic diagrams are two-dimensional plots that
have been widely used in prior co-word analysis studies
[7,11,20,22]. The x-axis shows the strength of interaction
between a specific research theme with others (i.e.,
centrality). The y-axis reflects the density of the research
theme, or the internal cohesion of a specific research theme
(see Figure 1).
Figure 1. Strategic diagram’s degree of density and centrality.
The location of a given research theme within this strategic
diagram characterizes the theme in the context of the whole
discipline:
Quadrant I (Figure 1, top-right): both internally coherent
and central to the research network in question. Known as
the motor-themes of the discipline given that they present
strong centrality and high density;
Quadrant II (Figure 1, top-left): coherent but low centrality
themes. These themes are internally well structured and
indicate that a constituted social group is active in them.
However, they have rather unimportant external ties
resulting in specialized work that is rather peripheral to the
work being carried out in the global research network;
Quadrant III (Figure 1, bottom-left): weakly developed
with marginal interest in the global research network. These
themes have low density and low centrality, mainly
representing either emerging or disappearing themes;
Quadrant IV (Figure 1, bottom-right): weakly structured
themes. These are strongly linked to specific research
interests throughout the network but are only weakly linked
together. In other words, prior works in these themes is
under-developed yet transversal, with potential to be of
considerable significance to the entire research network.
DATA
The ACM digital library provided us data on the papers
published at the CHI conference between 1994 and 2013.
According to Bradford’s law [6], a fundamental theory in
bibliometric analysis, a small core of publications will
account for a sizeable portion of the significant literature in
terms of citations received (i.e., as high as 90%), while
attempts to gather 100 percent of it will add articles to the
core at an exponential rate [14]. Considering the relevance
of the CHI conference to the field of HCI, an analysis on
the CHI articles should enable us to attain a fair overview
of the field’s development: a total of 3152 CHI articles (full
papers and notes) were published between 1994 and 2013,
containing 16035 keywords (mean of 5.09 per article) (see
Figure 2). For a small number of papers we had to
manually extract the keywords from the electronic version
of the manuscript (PDF) using a script.

Figure 2. Number of publications per year at CHI.
The sample was split in two datasets of ten years each, to
investigate the paradigm change in HCI over the past 20
years. The number of the papers published in 1994-2003
(N=702) is smaller than the number of the papers published
in 2004-2013 (N=2450), suggesting that the research in
HCI has grown considerably in the last ten years. We
manually standardized the keywords through synonyms
mergence (e.g., “mobile devices” and “handheld devices”
were merged; “ubicomp” and “ubiquitous computing” and
so on) and filtered broad items (e.g., “HCI” and “human
computer interaction”) [18,20,27,30]. The synonyms
mergence considered the top 2029 keywords that appear at
least twice in a dataset with regard to the merging of
singular and plural forms of nouns, gerunds, nouns,
abbreviations and acronyms.
The frequency of keywords follows a power-law
distribution (see Figure 3) with an alpha of 3.46 (R2=0.51),
indicative that the research structure of HCI in the past 20
years is a scale-free network, a network where a small
number of popular nodes (i.e., keywords) act as hubs
connecting other concepts. These hubs shape the overall
network, which in this case reflects the intellectual structure
of HCI represented by and through keywords. This scale-
free characteristic suggests that a small number of popular
keywords can capture major research directions and major
influences in the field [20,30]. Therefore, in our analysis we
retained only those keywords which appeared more than six
times during 1994-2003, or more than 14 times during
2004-2013. Thus, 94 keywords were selected for the period
of 1994-2003 (total frequency=1154), covering 556
(79.2%) of the 702 papers published during this period.
Similarly, for the period of 2004-2013, 95 keywords were
selected (total frequency=2692), covering 1602 of the 2450
papers published, i.e., 65.3% of the publications. With
fewer but popular keywords we could then reliably
characterize the entire network of keywords.
Figure 3. Power-law distribution of keyword frequency (in
logarithmic scale). Power-law distributions resemble a straight
line when on logarithmic scale.
RESULTS
Identifying the major research themes
First, we conducted hierarchical clustering using Ward’s
method with Squared Euclidean Distance as the distance
measurement [28]. We adopted a supervised clustering
method to reach as many clusters as possible while
maintaining content validity and cluster fitness [18,20,30].
The 94 keywords for the 1994-2003 samples led to 14
clusters (labeled as A1-A14, in Table 1). Each cluster
represents a research theme or subfield [20,30]. Similarly,
the 95 keywords of 2004-2013 samples led to 14 clusters
(labeled as B1-B14, in Table 2). The top-3 most frequent
keywords are shown in bold, and are used to label each
cluster [18,20,30]. In Tables 1 and 2 we show for each
theme:
• Keywords: the set of keywords that constitute this theme;
• Size: the number of keywords in the theme;
• Frequency: how often, on average, a keyword in this
theme appears in our dataset;
• Co-word frequency: how often, on average, two
keywords in this theme appear on the same paper;
• Cohesion coefficient: measures the extent to which when
a keyword of this theme appears on a paper then another
keyword of this theme also appears on a paper. Indicates
the similarity or dissimilarity of keywords in a theme.
Themes with higher cohesion coefficient are more
developed or bridging research themes [20];
• Centrality: the degree of interaction of a theme with other
parts of the network [24]. We calculate a localized
version of this metric using the standard value 2 for the
K-step reach. Thus, our centrality metric evaluates how
the keyword connects all other keywords that can be
reached through 2 connections;
• Density: measures the internal cohesion, or the strength,
of the links that tie together the cluster of keywords
making up the research theme [7,17]. To minimize the
possible bias caused by the different sample sizes of the
two periods, when calculating the overall network
density, we rely on a binary version of the keyword co-
occurrence matrix. This matrix only uses values 1
(“connected”) or 0 (“not connected”) to characterize
every pair of keywords.
We constructed two strategic diagrams to visualize the
cohesion and maturity of the research themes in HCI, using
the centrality and density of each cluster as proposed by
[7,11,18,20,22]. We plotted a strategic diagram for each
period of analysis: 1994-2003 and 2004-2013 (Figure 4a
and Figure 4b, respectively), based on Tables 1 and 2. The
plots’ origins are set to the average centrality and density
across all the clusters for the designated sample, i.e., (0.571,
2.305) for 1994-2003 and (0.635, 3.127) for 2004-2013.
Comparing the intellectual structure of other fields (as
shown in Figure 5), HCI lacks motor themes, and has lots
of under-developed, but transversal research themes (see
Figure 4).
We also calculated the overall network density for each
network, to analyze whether the whole research field
became more cohesive or not. The overall density of the
HCI intellectual map has increased from 0.148 in 1994-
1"
10"
100"
1000"
1" 10" 100" 1000" 10000"
Frequency)
Keyword)rank)

2003 to 0.206 in 2004-2013, meaning that the research field
has become more cohesive over time.
Keyword network maps
For each of the two periods in our datasets we constructed a
granular network of keywords using the following
procedure. Each keyword is represented as a node in a
graph, and we link together keywords that appear together
on a paper. In Figure 6 and 7 we show the result of this
process for each of the two periods of analysis. We note
that in these figures the size of a node is proportional to the
frequency of the keyword, and the thickness of links is
proportional to the co-occurrence correlation for that pair of
keywords. Nodes of the same color belong to the same
cluster, as presented next. To reduce visual clutter we only
show a subset of the complete networks, omitting weaker
ties and isolated nodes. A downside of this simplification is
that, for example, “privacy” in Figure 6 appears to be
disconnected from its own cluster. This is simply because
weaker links are not included. Popular, core and backbone
topics of HCI research
We next focused our analysis on individual keywords rather
than underlying themes. A core-periphery analysis was
conducted to determine the core research topics in the field
from the perspective of the whole network structure.
Twenty keywords (concentration=0.824) and 28 keywords
(concentration=0.841) were identified to be the core
research topics of the whole network in 1994-2003 and
2004-2013 respectively. Keywords or research topics were
categorized as follows:
• Popularity: how frequently a research keyword is used;
• Core: [0-1] how connected is a research keyword with
other topics;
• Structural holes: how connected is a research keyword
with other otherwise distinct topics, thus supporting the
topic structure (i.e., the backbone of the field).
A higher core value indicates a topic that is well connected
to other topics. A higher structural holes count suggests a
keyword that brings together otherwise isolated topics.
Topics with high scores on both of these metrics can be
considered as the driving force for advancements in the
field: without these topics, the field of HCI would be
fragmented. We show these results in Tables 3 and 4.
ID
Keywords
Size
F
CW-F
Cohesion
Centr.
Density
A1
computer supported cooperative work, interaction design, computer-mediated
communication, awareness, media spaces, audio, social interfaces
7
18.42
34.14
0.767
0.981
2.048
A2
world wide web, empirical study, email, Internet
4
15.25
27
0.662
0.532
2.500
A3
ubiquitous computing, augmented reality, tangible user interface, ethnography, mobile
computing, PDA, learning, GOMS, education, mobile/handheld devices, groupware
11
13.90
21.90
0.437
0.888
0.909
A4
visualization, user interface design, cognitive modeling, evaluation, navigation, direct
manipulation, agents, user modeling, animation, graphical user interfaces, design rationale, two-
handed interaction, metaphor, prototypes, trust, haptic, mobile phone, pen computing, design, two-
handed input, intelligent systems, speech recognition, intelligent interfaces
23
10.34
13.78
0.353
1.160
0.174
A5
input devices, virtual reality, information visualization, interaction techniques, 3D user
interfaces, motor control, virtual environments, human performance
8
16.62
30.12
0.734
0.655
2.179
A6
user interface, user studies, usability, methodology, Empirical Evaluation
5
15
24.2
0.468
0.570
1.600
A7
Fitts’ law, information retrieval, hypertext, browsing
4
18
33.25
0.624
0.465
4.000
A8
children, educational applications, participatory design, design techniques
4
13.25
25
0.795
0.287
3.833
A9
multimedia, Interface design, collaboration, video, mouse, gestures, field study, e-commerce,
hypermedia, privacy, social computing
11
8.81
16.09
0.876
1.069
0.473
A10
user-centered design, usability testing, usability engineering, design process, videoconferencing
5
9.6
15.4
0.715
0.368
1.400
A11
eye tracking, eye movements, multimodal interfaces, gaze
4
8
14.25
0.855
0.376
2.000
A12
annotation, digital libraries, documents, dynamic query
4
7.5
10.5
0.617
0.276
1.167
A13
programming by demonstration, end-user programming
2
8.5
12
0.609
0.195
4.000
A14
information foraging, information scent
2
8
16.5
1.001
0.184
6.000
Table 1. Major research themes in HCI during 1994-2003 (size, frequency (F), co-word frequency (CW-F), cohesion, centrality
(Centr.), density)
ID
Keywords
Size
F
CW-F
Cohesion
Centr.
Density
B1
mobile phone, sustainability, ethnography, online communities, HCI4D/ICTD, health, persuasive
technology, motivation, user-centered design, behavior change, community
11
30.09
30.27
0.358
0.899
1.036
B2
ubiquitous computing, privacy, mobile, augmented reality, wearable computing, field study, mobile
computing, context-aware, navigation, haptic, large displays, human-robot interaction, music,
computer vision, GPS, feedback, mobile interaction
17
26.94
28.58
0.416
1.064
0.654
B3
visualization, collaboration, user interface, wikis, social computing, tagging, annotation, personal
information management
8
30.62
35.5
0.516
0.866
1.393
B4
mobile/handheld devices, gestures, Fitts' Law, touch screens, text entry, pointing, touch
7
36
43.71
0.470
0.631
3.619
B5
computer-mediated communication, computer supported cooperative work, eye tracking,
communication, empirical study, trust, videoconferencing
7
30.71
36
0.496
0.722
2.048
B6
user studies, interaction techniques, web search, input devices, personalization
5
26.4
28.2
0.442
0.642
1.500
B7
design, games, usability, user experience, older adults, accessibility, memory
7
30.14
32.14
0.368
0.790
1.476
B8
children, tangible user interface, multi-touch, education, tabletop, learning
6
34
44.16
0.551
0.748
3.333
B9
evaluation, information visualization, interaction design, participatory design, assistive
technology, Methodology, design methods, creativity, prototypes, Security, end-user programming
11
25.63
27
0.419
0.842
0.855
B10
social networks, SNS, social media, twitter, Facebook
5
25.6
34
0.705
0.453
3.700
B11
crowdsourcing, human computation
2
23
25.5
0.533
0.268
7.000
B12
awareness, video, families, coordination
4
19
23.5
0.690
0.449
2.167
B13
multitasking, attention, interruption
3
25.33
31
0.656
0.293
9.000
B14
emotion, affect
2
18
24.5
0.792
0.236
6.000
Table 2. Major research themes in HCI during 2004-2013 (size, frequency (F), co-word frequency (CW-F), cohesion, centrality
(Centr.), density)

#
Popular Topic
(Frequency)
Core Topic
(Coreness value)
Backbone Topic
(Structural holes)
1
CSCW (50)
CSCW (0.375)
CWCW (42)
2
world wide web (35)
two-handed interaction (0.355)
world wide web (31)
3
ubicomp (28)
ubicomp (0.226)
interaction design (31)
4
visualization (27)
world wide web (0.222)
user interface (30)
5
input devices (27)
CMC (0.191)
visualization (29)
6
user interface (26)
information retrieval (0.186)
input devices (27)
7
virtual reality (26)
infoviz (0.171)
interaction techniques (27)
8
Fitts' law (24)
awareness (0.161)
CMC (26)
9
infoviz (23)
tangible user interface (0.160)
ubicomp (25)
10
augmented reality (22)
virtual reality (0.160)
information retrieval (24)
11
interaction design (22)
user interface (0.159)
multimedia (24)
12
interaction techniques (21)
augmented reality (0.149)
infoviz (23)
13
information retrieval (20)
children (0.148)
children (23)
14
tangible user interface (20)
user studies (0.146)
virtual reality (22)
15
CMC(20)
multimedia (0.145)
Fitts' law (22)
16
children (20)
interaction techniques (0.142)
Interface design (22)
17
multimedia (20)
visualization (0.142)
mobile computing (20)
18
user studies (18)
interaction design (0.137)
empirical study (20)
19
user interface design (18)
hypertext (0.133)
augmented reality (19)
20
cognitive modeling (18)
ethnography (0.113)
agents (19)
Table 3. Summary of popular, core and backbone topics of HCI in 1994-2003.
In bold are keywords that appear in every column.
#
Popular Topic
(Frequency)
Core Topic
(Coreness value)
Backbone Topic
(Structural holes)
1
mobile phone (67)
handheld devices (0.229)
ubicomp (44)
2
ubicomp (65)
gestures (0.229)
collaboration (43)
3
visualization (62)
collaboration (0.226)
evaluation (43)
4
handheld devices (60)
mobile phone (0.224)
mobile phone (41)
5
CMC (59)
CMC (0.211)
children (39)
6
gestures (59)
ubicomp (0.210)
visualization (38)
7
user studies (58)
CSCW (0.208)
design (38)
8
collaboration (57)
touch (0.207)
gestures (34)
9
privacy (54)
children (0.203)
user studies (34)
10
CSCW (52)
evaluation (0.195)
CSCW (34)
11
design (49)
privacy (0.161)
CMC (33)
12
children (48)
user studies (0.158)
mobile (32)
13
sustainability (45)
design (0.153)
handheld devices (31)
14
ethnography (45)
education (0.152)
games (29)
15
evaluation (43)
learning (0.149)
ethnography (28)
16
infoviz (43)
games (0.146)
augmented reality (28)
17
mobile (42)
visualization (0.146)
social computing (28)
18
TUI (38)
TUI (0.142)
privacy (27)
19
games (38)
touch screens (0.134)
social networks (26)
20
Fitts' Law (37)
mobile (0.134)
mobile computing (25)
21
online communities (36)
tabletop (0.123)
sustainability (24)
22
HCI4D/ICTD (35)
augmented reality (0.117)
infoviz (24)
23
interaction design (35)
communication (0.116)
education (24)
24
augmented reality (34)
infoviz (0.115)
learning (24)
25
participatory design (33)
social networks (0.113)
communication (24)
26
social networks (33)
awareness (0.112)
TUI (23)
27
usability (33)
SNS (0.109)
awareness (23)
28
crowdsourcing (32)
wikis (0.106)
participatory design (22)
Table 4. Summary of popular, core and backbone topics of HCI in 2004-2013.
In bold are keywords that appear in every column.
Figure 4. Strategic diagram for CHI for the period 1994-2003 (left), and 2004-2013 (right).
Figure 5. Indicative strategic diagrams
from other scientific disciplines.
Psychology [21]
Consumer Behavior [22]
Software Engineering [11]
Stem Cell Research [1]
Quadrant I
Quadrant II
Quadrant III
Quadrant IV
Quadrant IV
Quadrant I
Quadrant II
Quadrant III

Figure 6. Keywords networking map 1994-2003 (the line represents the link between two keywords with
correlation coefficient ≥ 0.14). An interactive version of this graph is available at http://goo.gl/BAjyMt.
Figure 7. Keywords networking map 2004-2013 (the line represents the link between two keywords with
correlation coefficient ≥ 0.09). An interactive version of this graph is available at http://goo.gl/v8j1Nh.
computer supported cooperative work
World Wide Web
ubiquitous co mputing
input devices
user interfa ce
virtual reali ty
information visualiza tion
Fitts ' law
augmented reality
interaction design
interaction techniques
Information Retrieval
tangible user interface
computer-m ediate d communica tion
Children
multimedia
user studie s
User interface design
cognitive modeling
ethnography
Usabili ty
Awareness
evaluation
Hypertext
3D user interfaces
user-centere d design
mobile computing
educational applications
empirical study
browsing
PDA
navigation
participatory design
Usability test ing
Agents
user model ing
eye tracking learning
annotation
graphical user int erfaces
usability enginee ring
two-handed interaction
end-user programming
Method ology
GOMS
collab oration
Programming by Demonstration
eye movements
motor control
virtual environments media spaces
video
audio
information foraging
design techniques
multimodal interfaces
education
digital l ibraries
information scent
mobile/handhel d devices
mobile phone
mouse
design process
email
gestures
pen computing
videoconferencing
documents
Design
e-commerce
dynamic query
gaze
intellige nt systems Intern et
human performa nce priva cy
Empirical Evaluati on
social computing
groupware
social interfaces
mobile phone
ubiquitous computing
mobile/handhel d devices
computer-mediat ed communi cation
gestures
collaboration
privacy
computer supported cooperative work
design
children
sustainability
evaluation
information visualization
mobile
tangible user interface
games
Fitts' Law
online communities
interaction design
augmented reality
participatory design
social networks
Usabili ty
crowdsourcing
SNS
touch screens multi-touch
education
wearable computing
awareness
user experience
multitasking
tabletop
learning
attention
social media
context -aware
wikis
social computing
text entry
communica tion
pointing
older adults
assistive technology
Methodology
accessibility
interaction techniques
navigation
web search
twitter
empirical study
health
design methods
emotion
interruption
video hapti c
persuasive technology
input devices
motivation
touch
creativity
tagging
annotation
facebo ok
trust
affect
music
prototypes
computer vision
behavior change
communit y
GPS
videoconferencing
famil ies
human computa tion
coordination
personalization
mobile interaction

DISCUSSION
Underlying trends within HCI
While previous work [16] has outlined major paradigms
within the field of HCI, our work provides a novel
perspective towards seeing the big picture within our
discipline. Our analysis has identified a number of research
themes that are based on the co-presence of keywords on
published papers – as opposed to a tacit interpretation of the
field and its methods. Thus, our findings reflect the research
that was actually conducted and published, not how a
researcher would subjectively or intuitively map the field.
Orthogonal to this analysis we add the dimension of time,
and focus on analyzing our field in two distinct periods.
This gives us the benefit of hindsight when interpreting our
findings for the first period (1994-2003), since we are able
to validate our claims for that period on the subsequent
period of analysis.
1994-2003
In Figure 4a, quadrant II, we observe clusters A2 (www),
A7 (hypertext), A8 (children), A13 (end-user programming)
and A14 (information foraging) have a high density but low
centrality. This indicates that these research topics are fairly
isolated from other research topics but internally well
connected. In particular, research in clusters A13 and A14
is less popular, and in hindsight we observe that in period
2004-2013 these clusters have disappeared.
In quadrant III, clusters A6, A10, A11 and A12 exhibit low
centrality and density. These are indicative of research
topics that are either emerging or fading, with a higher
likelihood of change. In hindsight we can identify that one
of these clusters was actually fading (A12: digital libraries),
while the other three emerging (A6: methodology, A10:
user-centered design, A11: eye tracking).
In quadrant IV, clusters A1 (CSCW), A3 (ubicomp), A4
(visualization), A5 (UI design) and A9 (multimedia) have
high centrality but low density, sign of an important yet
immature research topic in the field. Their importance is
evidenced by the frequency in which the keywords appear,
often leading to more concrete research subfields. In
hindsight, new conferences were spun-off from these
clusters: Ubicomp in 2001 and Pervasive in 2003 (from
A3), and IUI in 1997 (from A5).
Surprisingly, we found no research topics in quadrant I, i.e.,
with a high centrality and density. Closest to quadrant I we
found cluster A5 (input devices, virtual reality, information
visualization), with a high centrality, reflective of an
important area for CHI as a conference and HCI as a field
in the early days of computer-human interaction.
2004-2013
In Figure 4b, in quadrant I we find B8 (children, tangible
user interfaces, and multi-touch), the maturing theme
relating to children and learning through the use of tangible
and tabletop technologies. Our analysis suggests that these
themes are likely to become motor themes in the future.
In contrast, located in quadrant II are clusters B4, B10, B11,
B13 and B14, clusters with high density but low centrality,
well-focused and developed research topics, yet fairly
isolated from other research topics. Some of these themes
focus on relatively recent technology and trends (e.g., B11:
crowdsourcing, B10: social networks, B14: emotion &
affect) that have not had time to establish strong ties to
other research themes. Yet some of these themes represent
more traditional work that has remained relatively isolated
(B4: text entry & Fitt’s law, B13: multitasking).
In quadrant III we expect clusters that are emerging or
fading. Here we find theme B12 (awareness, video), which
is most likely a fading theme judging by its relatively small
frequency.
Finally, themes in quadrant IV are likely to be core and
transversal for HCI. Here we find multiple clusters (B1, B2,
B3, B5, B6, B7, and B9) of high frequency. Given that
these large clusters have low density, they are evidence of
field expansion during this time period. We note that in the
period 1994-2003 new conferences emerged from themes in
this quadrant that went on to become mainstream, and so
we may expect the same from these themes here.
Trending topics
Next, our analysis focused on specific topics or keywords.
A limitation of our previous analysis was that some of the
research themes contained multiple and diverse keywords,
making it hard to precisely characterize each theme. Here
we overcome this challenge by conducting a core-periphery
analysis of individual keywords to more precisely map their
role and evolution over time.
1994-2003
For the period 1994-2003, 14 of the 20 keywords appear as
popular, core and backbone topics simultaneously (Table 3,
in bold). This indicates a consistency between research
interests, knowledge acquired, as well as effort to maintain
the field. In contrast, the research topics of “tangible user
interface” and “user studies” are popular and core topics,
but have a relatively low number of structural holes. This
indicates that whilst these research topics have the potential
to prosper the field, they are not the ‘backbone’ during the
period.
As yet another example, research topics of “input devices”
and “Fitts’ law” are popular and backbone topics, but are
not core topics, indicating that research on these topics has
not yet effectively extended the knowledge landscape of the
field. Interestingly, research topics of “mobile computing,”
“empirical study” and “agents” were not popular or core
research topics, but they played an important role in
bridging different research efforts to establish an internally
cohesive research field of HCI (i.e., higher structural holes
count).
Lastly, despite the research on “two-handed interaction,”
“awareness,” “hypertext” and “ethnography” effectively
extending the HCI knowledge scope (i.e., high core and
structural holes count), a limited attention was given to
these research topics (i.e., low popularity).
2004-2013
Compared to the period of 1994-2003, we identified a
higher number of keywords (N=28) as core research topics
for the period of 2004-2013, indicating growth of the
knowledge field of HCI (Table 4). Of the top 28 keywords,
18 keywords were simultaneously popular, core and
backbone topics (Table 4, in bold). “Sustainability” and
“ethnography” are both popular and backbone research
topics, however not core topics. “Education,” “learning,”

“communication” and “awareness” are both core and
backbone research topics, but not hot topics. These results
suggest that an increased attention towards these topics is
required in order to develop and maintain the development
of the field.
Many keywords are found to exist only in one group:
“Fitts’ law”, “online communities,” “HCI4D/ICTD,”
“interaction design,” “usability,” “crowdsourcing,”
“touch,” “touch screens,” “tabletop,” “SNS,” “wikis,”
“social computing” and “mobile computing.” These
keywords indicate a paradigm change in the field, as they
disappear or emerge. In addition, despite their popularity, if
the keywords are neither in the core or backbone topics,
they are potentially a mismatch of research efforts.
In summary, comparing the popularity of the keywords
between the two periods, only 42 of 94 keywords (44.7%)
between 1994-2003 are found again as top keywords
between 2004-2013 (italic font in table 2). In other words,
most top research topics of the first ten years were replaced
by new research topics in recent years. The whole field
witnessed a paradigm change during this period.
Fluxionary Research
The field of HCI grew considerably in the last 10 years,
from an average of approximately 70 (1994-2003) to 245
publications per year (2004-2013). We observed an overall
increase in research clusters’ centrality (from 0.571 in
1994-2003 to 0.635 in 2004-2013), and density (from 2.305
in 1994-2003 to 3.127 in 2004-2013) (Figure 4a and Figure
4b, respectively). This means that HCI is becoming
increasingly cohesive. However, the field is lacking a major
driving theme that could potentially accelerate this process,
but instead consists of multiple themes competing for
recognition despite cooperating with each other.
While the underlying dynamics of themes point to gradual
maturity, the field has witnessed a recent explosion in the
number of specific topics or keywords. Overall keyword
centralization has decreased from 31.04% in 1994-2003 to
26.79% in 2004-2013, indicating that the leading research
keywords are constantly becoming less central in the
network. This is inevitable given that more new research
connections have been established between different
research topics in the later ten years.
For example, our analysis reveals that “social networks”
and “crowdsourcing” are completely new research themes
established during 2004-2013, located in quadrant II,
clusters (B10, B11) (Figure 4b). However, this should not
come as a surprise if the reader takes into account the
emergence of several social networking web sites during
this period (e.g., Facebook and Twitter opened to the
general public in 2006, Google+ in 2011). Similarly,
crowdsourcing presents itself as an emerging research
theme in 2004-2013 even though the first publication with
this keyword only appeared at CHI in 2009. However, due
to its rapid growth it has in merely 4 years positioned itself
as an important emerging research paradigm despite its low
centrality and therefore weak connection to other research
paradigms. During the same time period, in clusters B13
and B14, “multi-tasking” and “emotion” are hand-in-hand
with the highest density. This indicates a cluster that
contains “inseparable” expressions that are usually co-
present, much unlike the previous cluster in which, more
often than not, only one of those keywords appear.
In parallel to the emergence of research themes, there are
others that decline or merge. For instance, “End-user
programming,” and “information foraging,” from clusters
A13 and A14, have faded from the landscape of HCI
research as major independent research subfields.
A theme can also merge with others for several reasons,
such as the introduction of novel technology leading to
appropriation, or because a new advance is beneficial to
both fields. For example, in the early days of CHI,
“annotation” from A12 took form in physical documents.
With the availability of collaborative tools, such as “wikis”,
and social “tagging” (from cluster B3), annotation is now in
the context of digital formats. Another example is the
merging of “computer supported cooperative work” from
cluster A1 with “eye tracking” from cluster A11 resulting to
cluster B5 in 2004-2013, as eye-tracking methodologies
began to be used in collaborative settings, such as [23].
Research themes merging can lead also to new research
topics: “ubiquitous computing,” “augmented reality,” and
“ethnography” (from A3) and “visualization,” “user
interface design,” and “cognitive modeling” (from A4)
triggered the creation of three novel subfields: “mobile
phone,” “sustainability” and “ethnography” (B1);
“ubiquitous computing,” “privacy” and “mobile” (B2); and
“visualization,” “collaboration” and “user interface” (B3).
Research on the older topics is now intertwined with these
new topics, contributing to the appearance of several
research directions like sustainability [4], large-scale
ethnography [12] and ubiquitous public displays [15,19].
Where is the accumulated knowledge?
As it stands, the only tradition in HCI is that of having no
tradition in terms of research topics. HCI has a long enough
history for knowledge to accumulate, but to what extent has
this happened? Do prior studies help us when it comes to
new technologies? Judging from our findings the answer is
no, when a new technology comes along it seems that
researchers start from scratch leading to relatively isolated
research themes. There seems to be no single well-defined
way to study a new technology in the context of HCI. As a
result, different approaches or perspectives are adopted
when studying a new technology, leading to a relative
fragmentation within HCI.
Reflecting on our own experience, we believe that the
accumulated knowledge in HCI is almost exclusively
grounded on very specific technological contexts. For
instance when it comes to improving the design of a mouse,
previous studies on ergonomics are helpful. But when the
mouse is replaced by a touch-screen or voice input,
previous findings on mouse performance tend to be
inapplicable. This is not an HCI phenomenon: the transition
from gramophone to music tapes to CDs to iPods had a
similar effect on multiple disciplines. Due to the rapid pace
of technology designed for humans, however, knowledge in
HCI tends to be highly contextual instead of universal like
in the field of biology or physics. So we argue that by
nature HCI research is like nomads chasing water and
grasslands, making it challenging for the community to
accumulate knowledge.

Of course, the Human in HCI does not change as rapidly as
technology, even though practices and habits do. Hence one
potentially solid ground for HCI to develop accumulated
knowledge is on the human aspects of HCI, and this was
acknowledged in the session “celebrating the psychology of
human-computer interaction” in CHI 2008 [2]. However,
our analysis shows that this is far from likely to happen in
the community, with no discernable research theme
emerging on this topic.
Note that a motor theme should be derived from well-
established knowledge (high density), and have
implications to new HCI topics (high centrality). Therefore,
the existence of accumulated knowledge that is applicable
to the context of new technologies is an important condition
for the formation of motor-themes. Based on the above
discussion, we believe that the ‘nomads’ nature of HCI
research largely contributes to the lack of motor theme in
the field.
Should CHI break up into multiple conferences?
The diversity of the CHI conference, and more broadly of
the HCI field, has often prompted discussion. The diversity
of the papers submitted to the CHI conference often
backfires when authors feel that their work is not evaluated
by ‘true’ experts, or indeed by someone of an appropriate
background. Furthermore, researchers complain that some
kind of work is “more valued”, specifically raising the issue
of one-off novelty experiments being preferred over
laborious system development. On the other hand,
researchers feel that the diversity of the field is one of its
key strengths. Thus, the issues of rigor, diversity, and
reviewing process become intertwined in discussion. We
attempt to relate our findings to this discussion and shed
some light on the underlying processes of our discipline and
how we should approach rigor and diversity.
First, our results show that HCI is a diverse field. However,
the field is diverse not in the sense that it consists of
multiple disconnected research themes, like a pot-pouri, but
in the sense that there are a lot of links within and between
diverse themes, rather like a cobweb. In fact, only a handful
of clusters fall in Quadrant II (isolated themes) in Figure 4,
with most large clusters falling in Quadrant IV (transversal
themes) indicating an expansion of the field.
Our results also show that over time, the themes have
become more cohesive, while at the same time there is a
much larger number of topics or keywords in the discipline.
To a large extent, this is stimulated and driven by factors
external to the community, for instance through the
introduction of new technological products and services
(e.g., iPhone, Facebook) that have a direct impact on
humans’ life. As technology advances, and the rate of
innovation remains high, we can expect this trend to
remain: more new topics will constantly be of relevance to
the HCI community.
The key insight we obtain from our results is that any
breakup of the CHI conference today, or the HCI field, is
likely to be pointless in a few years. The community simply
lacks the motor themes along which a potentially
meaningful break up could be achieved (Quadrant 1 in
Figure 4). Our community is slowly maturing in terms of
themes, but is not transversally mature and the recent
expansion of topics is likely to delay this process.
A further insight from our analysis attests to the value of
diversity in our community. We identify many instances
where topics merge or interact with each other in
unpredictable ways, sometimes establishing new themes,
sometimes declining. This strong interaction is indicative of
the adaptability of our community, constantly evaluating
alternative approaches and attempting to conquer new
ground. A break up of the community would only hinder
this process, making it much harder to cope with the
introduction of new topics. This diversity and constant state
of flux is crucial in assimilating and dealing with new
topics.
The polycentric nature of the knowledge map of HCI, as
opposed to a unicentric one, reveals a key property of our
community. Our analysis of the keywords making up the
various clusters suggests that when a new technology is
introduced, our community tackles it and approaches it
from a number of perspectives. For instance, the
introduction of tabletop technology prompted usability and
Fitts’ law studies, studies on security and privacy, studies
on education and learning. Similarly the introduction of
smartphones and social media has been tackled from
multiple perspectives. This pluralism is a characteristic of
our community, for better or worse.
In summary, our analysis suggests that the HCI community:
• is having to deal with an increasing number topics that
are externally driven (e.g., new products, services,
advances in other sciences);
• is responding to this challenge by maintaining a diverse
yet intertwined research profile which remains in flux;
• is gradually maturing in terms of its themes, but it is
simply not transversally mature enough to undergo a
meaningful breakup.
LIMITATIONS
We considered only a single source of publications, the CHI
conference, which despite being the flagship conference of
the discipline has a strong geographical bias with most
papers coming from the US, UK and Canada [2]. The fact
that no journals were included in our analysis means that
work on topics more likely submitted directly to journals is
likely to be underrepresented in our sample.
Furthermore, the CHI conference has an acceptance rate of
about 24%, so most papers that were submitted to CHI were
eventually published somewhere else – and therefore not
included in our sample. Finally, a crucial issue is the extent
to which keywords accurately reflect the contents of a
paper. It is not clear whether all authors follow the same
approach for assigning keywords to their papers, and this is
likely to lead to some inconsistencies. Also, it is possible
that some change of keyword frequency may come from a
change in practices of how authors assign keywords.
However most of the keywords refer to specific
technologies, rather than generic concepts that can be used
interchangeably due to authors’ habit. So we feel that it is
very unlikely that the change of major keywords during the
two periods comes from authors’ habits.
CONCLUSION
In summary, our findings suggest that the field of HCI has
undergone dramatic change in the past 20 years. We can see
a clear paradigm change from the top keywords list, more
than half of which in 1994-2003 have disappeared from the

top list of 2004-2013. No research theme seems to be
immune from the influence of evolution. Rapid technology
change, including the prevalence of mobile devices and
technologies and the availability of new service like SNS
and crowdsourcing appear as a sort of driving force.
From the perspective of the whole network, the study
reported an enhanced cohesion of the field. The overall
network density increased while the whole network became
more internally connected. This implies progress
towards the formation of a concrete research field of HCI as
a whole. However, the results also indicate unmatched
research efforts on hot, core and backbone topics in recent
years, suggesting an ongoing and rapid paradigm shift.
REFERENCES
1. An, X.Y. and Wu, Q.Q. Co-word analysis of the trends
in stem cells field based on subject heading weighting.
Scientometrics 88, 1 (2011), 133–144.
2. Bartneck, C. and Hu, J. Scientometric analysis of the chi
proceedings. Proc. CHI 2009, ACM Press (2009), 699–
708.
3. Bauin, S., Michelet, B., Schweighoffer, M.G., and
Vermeulin, P. Using bibliometrics in strategic analysis:
“understanding chemical reactions” at the CNRS.
Scientometrics 22, 1 (1991), 113–137.
4. Blevis, E. Sustainable interaction design: invention &
disposal, renewal & reuse. Proc. CHI 2007, ACM Press
(2007), 503–512.
5. Borgatti, S.P. and Everett, M.G. Models of
corer/periphery structures. Social Networks 21, (1999),
375–395.
6. Bradford, S.C. Sources of information on specific
subjects. J. Information Science 10, 4 (1985), 173–180.
7. Callon, M., Courtial, J.., and Laville, F. Co-word
analysis as a tool for describing the network of
interactions between basic and technological research:
The case of polymer chemsitry. Scientometrics 22, 1
(1991), 155–205.
8. Callon, M., Courtial, J.P., Turner, W.A., and Bauin, S.
From translations to problematic networks: An
introduction to co-word analysis. Social Science
Information 22, 2 (1983), 191–235.
9. Cambrosio, A., Limoges, C., Courtial, J.P., and Laville,
F. Historical scientometrics? Mapping over 70 years of
biological safety research with co-word analysis.
Scientometrics 27, 2 (1993), 119–143.
10. Chang, P.-L., Wu, C.-C., and Leu, H.-J. Using patent
analyses to monitor the technological trends in an
emerging field of technology: a case of carbon nanotube
field emission display. Scientometrics 82, 1 (2010), 5–
19.
11. Coulter, N., Monarch, I., and Konda, S. Software
engineering as seen through its research literature: A
study in co!word analysis. J. of the American Society for
Information Science 49, 13 (1998), 1206–1223.
12. Dalsgaard, P. and Eriksson, E. Large-scale participation:
a case study of a participatory approach to developing a
new public library. Proc. CHI 2013, ACM Press (2013),
399–408.
13. Ding, Y., Chowdhury, G.G., and Foo, S. Bibliometric
cartography of information retrieval research by using
co-word analysis. Information Processing &
Management 37, 6 (2001), 817–842.
14. Garfield, E. The mystery of the transposed journal
lists—wherein Bradford’s Law of Scattering is
generalized according to Garfield's Law of
Concentration. Current Contents 17, (1971), 5–6.
15. Goncalves, J., Ferreira, D., Hosio, S., Liu, Y.,
Rogstadius, J. Kukka, H. and Kostakos, V.
Crowdsourcing on the spot: altruistic use of public
displays, feasibility, performance, and behaviours. Proc.
UbiComp 2013, ACM Press (2013), 753–762.
16. Harrison, S., Tatar, D., and Sengers, P. The three
paradigms of HCI. Proc. CHI 2007, ACM Press (2007),
1–21.
17. He, Q. Knowledge Discovery through Co-Word
Analysis. Library trends 48, 1 (1999), 133–159.
18. Hu, C.-P., Hu, J.-M., Deng, S.-L., and Liu, Y. A co-
word analysis of library and information science in
China. Scientometrics, (2013).
19. Kukka, H., Oja, H., Kostakos, V., Gonçalves, J., and
Ojala, T. What makes you click: exploring visual signals
to entice interaction on public displays. Proc. CHI 2013,
ACM Press (2013), 1699–1708.
20. Liu, G.-Y., Hu, J.-M., and Wang, H.-L. A co-word
analysis of digital library field in China. Scientometrics
91, 1 (2011), 203–217.
21. Marc, M., Courtial, J.-P., Senkovska, E.D., Petard, J.-P.,
and Py, Y. The dynamics of research in the psychology
of work from 1973 to 1987: From the study of
companies to the study of professions. Scientometrics
21, 1 (1991), 69–86.
22. Muñoz-Leiva, F., Viedma-del-Jesús, M.I., Sánchez-
Fernández, J., and López-Herrera, A.G. An application
of co-word analysis and bibliometric maps for detecting
the most highlighting themes in the consumer behaviour
research from a longitudinal perspective. Quality &
Quantity 46, 4 (2011), 1077–1095.
23. Nguyen, D. and Canny, J. MultiView: spatially faithful
group video conferencing. Proc. CHI 2005, ACM Press
(2005), 799–808.
24. Nielsen, A. and Thomsen, C. Sustainable development:
the role of network communication. Sustainable
development: the role of network communication 18, 1
(2011), 1–10.
25. Ritzhaupt, A., Stewart, M., Smith, P., and Barron, A. An
investigation of distance education in North American
research literature using co-word analysis. Int. Review of
Research in Open and Distance Learning 11, 1 (2010),
37–60.
26. Rombach, M.P., Porter, M., Fowler, J., and Mucha, P.
Core-Periphery Structure in Networks. (2013), 1–27.
27. Wang, Z.-Y., Li, G., Li, C.-Y., and Li, A. Research on
the semantic-based co-word analysis. Scientometrics 90,
3 (2011), 855–875.
28. Ward, J.H. Hierarchical Grouping to Optimize an
Objective Function. J. of the American Statistical
Association 58, 301 (1963), 236–244.
29. Vaughan, L., Yang, R., and Tang, J. Web co-word
analysis for business intelligence in the Chinese
environment. Aslib Proceedings 64, 6 (2012), 653–667.
30. Zong, Q.-J., Shen, H.-Z., Yuan, Q.-J., Hu, X.-W., Hou,
Z.-P., and Deng, S.-G. Doctoral dissertations of Library
and Information Science in China: A co-word analysis.
Scientometrics 94, 2 (2013), 781–799.