![Difference in accuracy with respect to a contextless approach in all works analyzed, per dataset. The results for [1] have been removed due to their unusually high accuracy (Table 4).](https://www.researchgate.net/profile/J-Fernando-Sanchez-Rada/publication/333067922/figure/fig4/AS:761734499422210@1558623065976/Difference-in-accuracy-with-respect-to-a-contextless-approach-in-all-works-analyzed-per_Q320.jpg)
Content uploaded by J. Fernando Sánchez-Rada
Author content
All content in this area was uploaded by J. Fernando Sánchez-Rada on May 23, 2019
Content may be subject to copyright.
Social Context in Sentiment Analysis: Formal Definition, Overview of Current Trends and Framework for Comparison
Accepted Manuscript
Social Context in Sentiment Analysis: Formal Definition, Overview of
Current Trends and Framework for Comparison
J. Fernando S´
anchez-Rada, Carlos A. Iglesias
PII: S1566-2535(18)30870-4
DOI: https://doi.org/10.1016/j.inffus.2019.05.003
Reference: INFFUS 1097
To appear in: Information Fusion
Received date: 11 December 2018
Revised date: 8 May 2019
Accepted date: 13 May 2019
Please cite this article as: J. Fernando S ´
anchez-Rada, Carlos A. Iglesias, Social Context in Sentiment
Analysis: Formal Definition, Overview of Current Trends and Framework for Comparison, Information
Fusion (2019), doi: https://doi.org/10.1016/j.inffus.2019.05.003
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service
to our customers we are providing this early version of the manuscript. The manuscript will undergo
copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please
note that during the production process errors may be discovered which could affect the content, and
all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Highlights
•We propose a definition of social context for sentiment analysis
•We provide a framework for sentiment analysis approaches that use social
context
•We conduct a structured review of sentiment analysis with social context
•Sentiment analysis benefits from the inclusion of social context
•We discuss insights about different techniques and their performance
1
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Social Context in Sentiment Analysis: Formal
Definition, Overview of Current Trends and Framework
for Comparison
J. Fernando S´anchez-Rada and Carlos A. Iglesias
Intelligent Systems Group,
Universidad Polit´ecnica de Madrid.
{jf.sanchez,carlosangel.iglesias}@upm.es
Abstract
Sentiment analysis in social media is harder than in other types of text due
to limitations such as abbreviations, jargon, and references to existing content
or concepts. Nevertheless, social media provides more information beyond text,
such as linked media, user reactions, and relations between users. We refer to
this information as social context. Recent works have successfully leveraged
the fusion of text with social context for sentiment analysis tasks. However,
these works are usually limited to specific aspects of social context, and there
have not been any attempts to analyze and apply social context systematically.
This work aims to bridge this gap by providing three main contributions: 1) a
formal definition of social context; 2) a framework for classifying and comparing
approaches that use social context; 3) a review of existing works based on the
defined framework.
Keywords: sentiment analysis, social context, social network analysis, online
social networks
1. Introduction
Recent years have witnessed the rise of social media. Platforms such as
Twitter or Facebook have become the de facto way to share thoughts and opin-
ions with a wide audience [41]. Studies of Twitter usage show that about 19%
of tweets contain a reference to a brand or product, 20% of which also show
some expression of brand sentiment [39]. As a consequence, companies and
researchers have grown interested in social media as a way to monitor public
opinion. The sheer amount of social media content makes it impractical or im-
possible to manually process it. Hence, automatic sentiment analysis has grown
very popular.
Sentiment analysis has been applied for many years in other types of opin-
ionated content, such as online reviews or news articles. However, social media
Preprint submitted to Information Fusion May 13, 2019
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
content poses several unique challenges to natural language processing in gen-
eral, and to sentiment analysis in particular [64]. Some of these challenges are
imposed by the very nature of social media platforms, such as limited length and
relying on associated media. Other difficulties are caused by the characteristics
of human interaction in these types of media. e.g., short attention span, need
for immediacy, and use of specialized language. The result is a type of text that
is short, full of jargon or abbreviations, ephemeral, and rife with references to
contextual information.
There are different approaches to sentiment analysis in social media [3,71,
14]. Most techniques are content-centric. They exploit specific linguistic char-
acteristics of social media, just like previous research has done for other media
(e.g., news articles) and domains (e.g., movie reviews). Some works try to over-
come abbreviations and short texts in social media by finding external sources
to link text to, such as news articles [32] or Wikipedia pages [29]. Other works
leverage the specific language in these media by finding cues for sentiment (e.g.,
smileys and hashtags) [21]. When the textual content is also accompanied by
multimedia, such as images or videos, the sentiment information in these media
obtained with multimodal analysis [69] may also be exploited.
Nevertheless, these approaches fail to use the fact that information shared
on social networks is not isolated. The meaning of a particular piece of content
(e.g., a Tweet, a Facebook status or a blog post) may only be understood when
its context is taken into consideration. This context includes visible information
such as previous content that belongs to the same conversation, previous inter-
actions between users, or people that interacted with the content (e.g., by liking
it). It also includes seemingly unrelated social features. For instance, some
demographic factors such as age and gender have been shown to correlate with
sentiment and vocabulary [89], and they have been used to improve sentiment
classification [37].
New sentiment analysis techniques are starting to incorporate the fusion of
information from text and social context. Social context has also been intro-
duced in other fields related to sentiment analysis, such as spam detection, where
clues to identify spammers are usually hidden in multiple aspects of context,
such as previous content, behavior, relationship, and interaction [15]. Unfortu-
nately, the definition of social features, the methods employed to extract them,
and how they are applied to sentiment analysis tasks vary greatly from work
to work. These differences in notation and approaches are taxing, which makes
comparing different works harder.
Thus, further research is needed to delve more deeply into the notion of
social context and the fusion of social context with traditional textual sentiment
analysis. This work seeks to answer the following questions:
•Q1. What is social context?
•Q2. Can social context improve sentiment analysis?
•Q3. What elements of social context are more relevant for sentiment
analysis purposes?
3
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
As a result, the contributions herein are threefold. First, this work proposes
a formal and general definition of social context. Secondly, a framework to
compare existing works in the field is proposed. In this framework, each work is
described using a multi-level taxonomy that classifies each approach in terms of
the proposed definition of social context, and other factors such as the machine
learning techniques applied. Thirdly, the state of the art in sentiment analysis
using social context is organized and compared using the defined framework.
Moreover, the results reported by each work in the analysis have been aggregated
and analyzed, to simplify the comparison of approaches.
The remaining of this paper is structured as follows. Section 2presents an
overview of the state of the art in sentiment analysis prior to social context,
and an introduction to social network analysis; Section 3introduces a formal
definition of social context; Section 4presents the framework for comparison
of approaches to sentiment analysis using social context; Section 5provides an
overview of the state of the art, using the framework presented in the previous
section; Lastly, Section 6discusses the main conclusions drawn from this work
and future lines of research.
2. Related Work
This section is overview of relevant work in the fields of sentiment analy-
sis and social network analysis. Each field is discussed in a separate section.
The former discusses different approaches in sentiment analysis, including deep
learning and ensemble techniques. The latter introduces Social Network Anal-
ysis (SNA), and it focuses on community detection due to its importance in
several of the works reviewed.
2.1. Sentiment Analysis
Although sentiment analysis has been an active research topic for decades, it
has grown in popularity with the advent of online opinion-rich resources [64]. In
turn, these resources have also added their own set of limitations and challenges.
Over the last two decades, numerous works have explored sentiment analy-
sis in different applications and using different approaches. These approaches
can be grouped into machine learning, lexicon based, and hybrid [71]. Of the
three, machine learning techniques and hybrid approaches seem to be domi-
nant [3,65,90], and lexicon techniques are typically incorporated into machine
learning approaches to improve their results. Machine learning approaches ap-
ply a predictor (a classifier, or an estimator) on a set of features that represent
the input. The set of predictors is not very different from those used in other
areas. Instead, the complexity in these approaches lies in extracting complex
features from the text, filtering only relevant features, and selecting a good
predictor [78].
One of the most straightforward features is the Bag Of Words (BOW) model.
In BOW, each document is represented by the multiset (bag) of its constituent
words. Word order is disrupted, and syntactic structures are broken. As a
4
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
result, a great deal of information from natural language is lost [94]. Therefore,
various types of features have been exploited, such as higher order n-grams [63].
A more sophisticated feature is Part of Speech (POS) tagging [30]. In it, a
syntactic analysis process is run, and each word is labeled (tagged) with its
syntactic function (e.g., noun). Additionally, syntactic trees can be calculated.
Using these trees, the words in the input can be rearranged to a more convenient
position while still conveying the same meaning. Note how these two types of
features only rely on lexical and syntactical information. For this reason, they
are sometimes referred to as surface forms.
Surface forms can also be combined with other prior information, such as
word sentiment polarity [28,11,44,54,57]. This prior knowledge usually takes
the form of sentiment lexicons, i.e., dictionaries that associate words in a domain
or language with a sentiment. Some lexicons also include non-words such as
emoticons [40,36] and emoji [60]. These alternative forms of writing have been
shown very useful, as they can dominate textual cues and form a good proxy
for text polarity [36].
The use of lexicon-based techniques has many advantages [82], most of which
stem from their combination with other methods. For instance, it is possible
to generate lexicons that are domain dependent or that incorporate language-
dependent characteristics. Lexicons and syntactic information can also be com-
bined with linguistic context to shift valence [68]. On the other hand, there are
several disadvantages to lexicon approaches. First, creating lexicons is an ardu-
ous task, as it needs to be consistent and reliable [82]. It also needs to account
for valence variability across domains, contexts, and languages. These depen-
dencies make it hard to maintain domain-independent lexicons. An alternative
to retain independence while encoding domain, language, and context variabil-
ity is through semantic representation of the lexical resources in the form of
ontologies. An ontology can encode both lexical [52] and affective [81] nuances,
both in the lexicons and in the automatic annotations [9]. This is especially
useful for aspect-based sentiment analysis, as the differences between aspects
can be incorporated into the ontology [91].
In recent years, new approaches based on deep learning have shown ex-
cellent performance in Sentiment Analysis [19,5]. In contrast with traditional
techniques, deep learning techniques learn complex features from data with min-
imum human interaction. These algorithms do not need to be passed manually
crafted features: they automatically learn new complex features. The downside
is that the quality of the features heavily depends on the size of the training
data set. Hence, they often require large amounts of data, which is not al-
ways available. They also raise other concerns such as interpretability [51,49]
or its inability to adapt to deal with edge cases [51]. In the realm of Natural
Language Processing (NLP), most of the focus is on learning fixed-length word
vector representations using neural language models [42]. These representations,
also known as word embeddings, can then be fed into a deep learning classifier,
or used with more traditional methods. One of the most popular approaches in
this area is word2vec [55]. The downside of these methods is that they require
enormous amounts of training data. Luckily, several researchers have already
5
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
applied these methods to large corpora such as Wikipedia and released the
resulting embeddings.
Lastly, it is also possible to combine independent predictors to achieve a
more accurate and reliable model than any of the predictors on their own. This
approach is known as ensemble learning. Many ensemble methods have been
previously used for sentiment analysis. Ensemble methods can be classified ac-
cording to two main dimensions Rokach [73]: how predictions are combined
(rule-based and meta-learning), and how the learning process is done (concur-
rent and sequential). A new application of ensemble methods is the combi-
nation of traditional classifiers based on feature selection and deep learning
approaches [3].
2.2. Social Network Analysis and Community Detection
Social Network Analysis (SNA) is the investigation of social structures [62].
It provides techniques to characterize and study the connections between people,
and their interactions. SNA is not limited to Online Social Network (OSN), but
to any kind of social structure. Other examples of social network would be a
network of citations in publications or a network of relatives. Through SNA
techniques, it is possible to extract information from a social network that may
be useful for sentiment analysis, such as chains of influence between users, groups
of like-minded users, or metrics of user importance.
There are several ways in which SNA techniques can be exploited in senti-
ment analysis, but most of them fall under one of two categories: those that
transform the network into metrics or features that can be used to inform a
classifier; and those that limit the analysis to certain groups or partitions of the
network.
A simple example of metrics provided by SNA could be user’s follower in-
degree (number of users that follow the user) and out-degree (number of users
followed by the user), which could be used as features for each user [79]. How-
ever, these metrics are not very rich, as they only cover users directly connected
to a user, and it does so in a very naive way: all connections are treated equally.
Other more sophisticated metrics could be used instead of in/out-degree, such
as centrality, a measure of the importance of a node within a network topology,
or PageRank, an iterative algorithm that weights connections by the importance
of the originating user. Several works have introduced alternative metrics for
user and content influence in a network [33,59].
The second category of approaches is what is known either as network parti-
tion or as community detection, depending on whether the groupings may over-
lap. Intuitively, community detection aims to find subgroups within a larger
group. This grouping can be used to inform a classifier, or to limit the analysis
to relevant groups only. More precisely, community detection identifies groups
of vertices that are more densely connected to each other than to the rest of the
network [66]. The motivation is to reduce the network into smaller parts that
still retain some of the features of the bigger network. These communities may
be formed due to different factors, depending on the type of link used to connect
users, and the technique used to detect the communities. Each definition has
6
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
its own set of characteristics and shortcomings. For instance, if users are con-
nected after messaging each other, community detection may reveal groups of
users that communicate with each other often [22]. By using friendship relations,
community detection may also provide the groups of contacts of a user [25].
The reader is referred to other publications [66,61] for further details of the
different definitions of community and algorithms to detect them.
3. A Definition of Social Context
This section introduces a novel definition of social context and its compo-
nents. The definition is focused on OSN aspects, and it is based on previous
definitions and on the observed usage of social context features in the state of
the art.
Since the inception of Twitter and its API in late 2006, several works have
used social features to complement text [6]. This section aims to introduce a
general definition of social context that both encompasses existing definitions
and formalizes the loose or implicit definitions used in most works.
To the best of our knowledge, the first formal definition of social context was
introduced by Lu et al. [50]. They defined the social context of a set of Reviews
Ras the triple C(R) = hU, A, Si, of the set of reviewers U, the authorship
function A, and the social network relation S. Although their work is focused
on reviews, it identifies the three main entities of this social context: the content
(review), the content producer (the author) and the user-relations (the social
network relations). Later works have also referred to social context in different
terms [93,58], but a formal definition is seldom provided. For instance, Ren and
Wu [72] define both Social Context and Topical Context, based on the graph
of relations and their adjacency matrix. Namely, Social Context is defined as
GS={u, S}, where uis the set of users and Sis the adjacency matrix between
users, and Topical Context is defined as Gt={t, T }, where tis the set of topics,
and Tis the adjacency matrix of topics.
Based on these definitions, and our analysis of the state of the art, we have
identified four types of elements that make up Social Context (Fig. 1): content
(C), users (U), relations (R) and interactions (I). These elements are related
as follows.
Users are connected through relations and interactions. Relations are sta-
ble connections between two or more users (Ru). There are multiple types of
relations, such as friendship, or belonging to the same group. Some types of
relations are undirected or mutual, like kinship, whereas others are directed or
asymmetrical, such as liking and following relations. Interactions appear when
a user communicates with others (Iu). The types of interactions include di-
rect messages, replies, and user mentions. Most of these types also involve the
creation of content. When a user creates or posts new content, an authorship
relation between the user and the content is formed (Ruc). New content may
also be related to existing content (e.g., as a reply or a mention, Rc), or to other
users (e.g., the user is mentioned in the content, Ruc). Users may then interact
with the newly created content (Iuc), by replying to it, liking it, saving it, etc.
7
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
IuIuc
Rc
Ruc
Ru
User Content
Figure 1: Model of Social Context, including: content (C), users (U), relations (Rc,Ruand
Ruc), and interactions (Iuand Iuc).
All elements are rich entities with different attributes. The specific attributes
that can be used depend on the type of element and the OSN. Content attributes
(e.g., text, creation date) and user attributes (e.g., name, age, gender) are com-
monly used. Although interaction and relation attributes are not as widespread,
they are also important. They provide information such as when the interaction
happened, or the weight of the relation. These attributes make it possible to
filter specific connections, and to apply algorithms that rely on weighted graphs.
An additional concept to take into account is temporal dependence. New
content is continuously created, and existing content is changed or removed.
Relations are similar, as they are forged and dissolved naturally; and users can
join, delete their accounts or become inactive. The relevance of social context
variation over time is illustrated in Section 4.3 with the introduction of dynamic
approaches.
These ideas about the elements of Social Context and their dynamic nature
are condensed in the following definitions. First, Definition 1covers Social
Context as a whole and establishes its constituent elements.
Definition 1. Social Context is the collection of users, content, relations, and
interactions which describe the environment in which social activity takes place.
Namely:
SocialC ontext(τ) = hC, U, R, I i(τ) = hC(τ), U(τ), R(τ), I (τ)i
At any point in time τ:C(τ)is the set of content (Definition 2) generated
by these users; U(τ)is the set of users (Definition 3); I(τ)is the set of inter-
actions (Definition 5) between users, and of users with content; R(τ)is the set
of relations (Definition 4) between users, between pieces of content, and between
users and content.
This is a very general definition which only sets up the main elements, and it
relies on the definition of each element to fully characterize context. To simplify
8
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
the notation in the remaining definitions, time dependence will be implicit from
here on: SocialContext =hC, U, R, I i. This can be done without loss of gen-
erality. Whenever time dependence is relevant, we will refer to time-dependent
social context as dynamic social context and to time-independent social context
as static social context.
To illustrate the definitions, we will model an example of social context for
a sentiment analysis task on Facebook content. For this analysis, we only need
access to status updates by some users, and photos uploaded to a set of Facebook
pages (groups).
The first element in social context is content:
Definition 2. The collection of content is defined as:
C={ct,i |t∈Tc}(1)
Where Tcare all the types of content available, and each ct,i is a piece of
content of a certain type t. Each piece of content should be unambiguously
identified by its type and an identifier (i).
Our example context only includes two types of contents: status updates
and photos. Each type of content may be given some attributes. Some of these
attributes are common, such as the creation date. Others are specific for that
type, such as the keywords for status updates, and the link to the image file for
photos. Additionally, each photo and each status has to be given an identifier,
which may also be the one given by the Facebook API. So far, the context
defined is not very useful, as it would only allow us to analyze the sentiment of
the status updates and the photos (using other modalities).
The next element in Social Context is the collection of users in the network.
Definition 3. Let the set of users be:
U={u1, u2, . . . , un}(2)
Where each uiis a specific user that is unambiguously identified by its user
identifier i. Each user may have one or more roles. The set of roles for a user
is:
ρ(ui) = {t|ρt(ui)=1, ui∈U, t ∈Tρ}(3)
Where Tρare all possible roles in a context, and ρt(ui)is a function that
determines whether user uihas been assigned role t.
Roles define the function of users within the network. They usually restrict
the type of interactions and relations a user may have, and with what content
and users. e.g., online fora have the role of topic moderators, in addition to
regular users. The aim of moderators is to decide what content should be
allowed, to edit it, and to manage users that misbehave. Hence, new relations
9
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
(e.g., edited-by) and interactions (e.g., ban) are available to this specific role. If
the user is a moderator of more than one topic, several roles will apply.
Our example context will include the profiles of the users in our study and
their attributes. Since we are only interested in age and location, users will just
have those attributes. Our users may also have roles. In our case, we will be
interested in page administrators. At this point, the lack of connection between
users and content hampers other types of analysis.
The categorization of connections in Social Context is based on the concept
of social ties in the social sciences, i.e., dyadic relations [8]. Social ties are
grouped into one of four categories: similarities, such as co-location or being
the same gender; social relations, such as kinship (e.g., family ties), role (e.g.,
friendship), or affection (e.g., liking); interactions, such as having talked to each
other, or harming one another; and flows, such as sharing information, beliefs,
or resources. For the sake of simplicity, and based on the use of context in
the state of the art, only two types of connections are modeled as part of Social
Context: relations (Definition 4) and interactions (Definition 5). The remaining
social ties (similarities and flows) can be modeled as an equivalent relation or
interaction, depending on the case. Similarities are not typically considered as
ties in themselves but rather as conditions or states that increase the probability
of forming other kinds of ties. Flows are typically inferred from interactional
and relational data [8] so, for the sake of simplicity, they can be thought of as
another type of relation or interaction.
Hence, relations are connections such as friendship, kinship, group member-
ship or liking each other, whereas interactions are connections such as getting in
touch, re-sharing each other’s content, etc. There are two main differences be-
tween relations and interactions that motivate their distinction. First, relations
are few and slow-changing, whereas interactions are plentiful and short-lived.
Secondly, content can be related to other content (e.g., a reply and the original
content), while interactions are always performed by a user agent.
Formally, relations and interactions are defined as follows:
Definition 4. Given a set of content C, and a set of users U. Relations are the
connections between users (Ru), between users and content (Ruc) and between
different content (Rc). Formally:
R≡ {rt|t∈Tr}=Ru∪Ruc ∪Rc(4)
Ru
t={ru
t,ui,uj|ui, uj∈U, ui6=uj, t ∈Tr,u}(5)
Ruc
t={ruc
t,ui,cj|ui∈U, cj∈C, t ∈Tr,uc}(6)
Rc
t={rc
t,ci,cj|ci, cj∈C, ci6=cj, t ∈Tr,c}(7)
Where Tr,c are the types of relations between two pieces of content, Tr,uc
are the types of relations between users and content, and Tr,u are the types of
relations between users.
10
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Definition 5. Given a set of content C, and a set of users U. Interactions are
the activities carried on by a user that involve either another user (Iu), or a
piece of content (Iuc). Formally:
I≡ {it|t∈T i}=Iu∪Iuc (8)
Iu
t={iu
t,ui,uj,i |ui, uj∈U, t ∈Ti,u}(9)
Iuc
t={iuc
t,ui,uj,i |ui∈U, cj∈C, t ∈Ti,uc}(10)
Where Ti,uc are the types of interactions between user and content, Ti,u are
the types of interactions between users, and iis an identifier for the interactions,
as multiple interactions of the same type are possible.
With all elements defined, we can go back to the previous example of Social
Context on Facebook. From the possible types of relations between users (Ru),
we may add two: user friendship and kinship. These two relations would allow
us to group users that are closely related. To link users with content, we will
choose two types of user-content relations (Ruc): authorship, and mentions (i.e.,
the link between the content and the users it mentions). As for relations between
content (Rc), we may choose replies (i.e., the link between two pieces of content
when one mentions the other). Lastly, we will only have access to interactions
between users and content (Iuc) in the form of likes, reactions, and replies. Due
to technical limitations, we will not have access to user interactions, such as
direct messages.
The resulting example context would allow for richer analyses that exploit
information such as inferred groups of people based on how often they interact
with each other or appear in photos together. Sentiment analysis may exploit
prior knowledge about the sentiment of the user (via the authorship relation),
or even knowledge about the sentiment of friends and acquaintances (through
either relations or interactions between users). It may even be possible to find
people within the group that have changed the opinion of the people with whom
they interact.
Table 1shows other types of user, content, relations and interactions found
in popular OSN. It includes common elements in the OSN analyzed in the
state of the art: Twitter, Weibo, Reddit, Facebook, blogging platforms and
Wikipedia.
The tabular format does not capture how different types of relations or
interactions are unique to certain types of content and/or user roles. We will
exemplify this fact using Facebook since it has different types of content and
users roles. In Facebook, we may consider four main types of content. There
are statuses, which are posts by users which are shown on their own profile
(i.e., user feed). Statuses are very rich, they may mention other users, include
location information, link to other content, or even express the mood of the
author. The visibility of the status is governed by the user’s privacy settings,
and the relationship of the user to others. For instance, privacy-minded users
11
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
may make their statuses only available to their close friends, while other users
may make theirs public. Similarly, users can create pages, which are public
profiles created around a specific topic, such as a business, a brand, or a cause.
Pages are similar to user profiles, but they can be administered by one or more
users. Another type of content is photos, which may be linked to a user profile or
to a page. Photos can include information about the users that appear in them,
which creates a relation between the photo and the users. Events are a different
type of content that is used to organize gatherings and to give information about
them. Users may indicate whether they will attend, comment on the event, and
invite other users to join.
Users may interact with content to which they have access in different ways:
by liking it; by commenting to it, which creates new content that other users
may interact with; or by expressing their reaction or emotion to it, such as
surprise. These types of interaction are common for all types of content. Some
types of content provide other means of interaction, such as re-sharing of posts,
which allows users to share a post by other user in their own profiles.
The primary means for interaction between users is through content, either
by interacting with the content, e.g., users may reply to each other’s content, by
including other users in their content, e.g., by adding a mention in a comment
or a tag in a photo. Lastly, they may interact through special actions such as
poking each other, or through private instant messages. Since these interactions
are private, they have not been included in the table.
OSN Content
(Tc)
User
roles
(Tρ)
Relations (Tr) Interactions (Ti)
User-
User
(Tr,u)
User-
Content
(Tr,uc)
Content-
Content
(tr,c)
User-
User
(ti,u)
User-
Content
(ti,uc)
Twitter Tweet User Follow
Friend
Author
Mentioned
Favorite
Reply
Retweet
Mention
Reply
Reply
Retweet
Mention
Weibo Weibo User Follow
Friend
Author
Mentioned
Favorite
Reply
Reshare
Mention
Reply
Reply
Reshare
Reddit Post
Comment
User
Admin Follow Author
Mentioned
Link
Reply
Mention
Reply
Vote
Gild
Reply
Mention
Facebook
Status
Page
Comment
Photo
Event
User
Page
admin
Friend
Relative
Author
Admin
Fan
Own
Tagged
Attend
Like
React
Link
Reply
Contain
Mention
Reply
Tag
Comment
Re-share
Blog Post
Comment
Author
Reader Follow Author
Like
Link
Reply
Mention
Reply
Reshare
Comment
Wiki Page
Comment
Editor
Reviewer -
Author
Edit
Review
Link
Parent
Reply
- Edit
Table 1: Types of Social Context elements in different OSN.
Some researchers are concerned that the typical follower-friend relation might
12
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
not be enough to capture the richness of relations in online media [20]. They
also propose researching into new multifaceted approaches which take into con-
sideration more aspects of the network simultaneously. Social context has been
intentionally defined with those approaches in mind. The definition of Social
Context can be interpreted in the form of sets, or in its equivalent graph form,
where users and content are vertices, and both relations and interactions are
edges. The graph form can be combined with different types of links (Tc,Tu,
Tr,Ti) to generate multiplex networks [27] (i.e. a multilayered network of users
and content), which can be exploited in multifaceted approaches.
To conclude, the usage of the social network [43] and the effect of the social
network on user behaviour [18] depend on other aspects such as cultural dif-
ferences, factual information and events. This type of information falls outside
the scope of social context, and will need to be encoded through other means
such as a knowledge graph, or a description of events. However, social context
will capture information such as language of a user or creation time of content,
which can be used to link the user or content to that external information. This
concept will be further explained in Sect. 4.2.
4. Framework for Research on Social Context in Sentiment Analysis
This section defines a novel framework to compare sentiment analysis ap-
proaches that exploit social context. The framework is centered around a multi-
levelled taxonomy for structuring research in the field. The first level refers to
the dataset used. The second level covers the scope of Social Context built from
the dataset. The third level covers machine learning methods applied. The
fourth level covers the type of social context used (static and dynamic). Each
level is further explained in a separate section.
4.1. Dataset
The datasets used for analyzing social context can be identified by several
characteristics. The first of them is the online social network from which the
data was gathered. Twitter predominates in this area, due to its relatively open
API and abundance of content. The second characteristic is the type of anno-
tation on content. Likewise, the third characteristic is the type of annotation
on users. In this work, we focus on sentiment (polarity), but other annotations
such as stance, emotion, and quality of the content are often used. In the case
of polarity, the classes used may also differ. i.e. positive (+), negative (−) and
neutral (0). The fourth, fifth, and sixth characteristics are the type of link be-
tween users, between pieces of content, and between users and content. These
links can stem either from a relation or from an interaction, as mentioned in
the definition of social context.
4.2. Context Scope
Researchers have to choose what information from their datasets to select for
the social context in their work. They may also complement the original data
13
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
with information from external sources. As a consequence, every work employs
a different context. Nonetheless, a closer inspection reveals some patterns: some
elements are commonly used together (e.g., users and friendships), and some el-
ements are harder to obtain or rarer than others (e.g., follower-followee relations
are more common than retweets or favorites). As contexts get more and more
complex, they start including more unusual elements in addition to the more
basic ones.
Hence, we propose a classification of works based on the complexity or scope
of their context. Our proposal is inspired by the micro, meso and macro levels of
analysis typically used in social sciences [7]. The two differences are: 1) a level
of analysis is added to account for analysis without social context, and 2) the
meso level is further divided into three sub-levels (mesor,mesoi, and mesoe),
to better capture the nuances at the meso level. The result is shown in Fig. 2,
and the levels are:
Social Context Analysis
Micro Meso MacroContextless
MesorMesoiMesoe
Figure 2: Taxonomy of approaches, and the elements of Social Context involved.
•Contextless: The approaches in this category do not use social context,
and they rely solely on textual features.
•Micro: These approaches exploit the relation of content to its author(s),
and may include other content by the same author. For instance, they may
use the sentiment of previous posts [1] or other personal information such
as gender and age to use a language model that better fits the user [88].
•Meso-relations (Mesor): In this category, the elements from the micro
category are used together with relations between users. This new infor-
mation can be used to create a network of users. The slow-changing nature
of relations makes the network very stable. The network can be used in
two ways. First, to calculate user and content metrics, which can later be
used as features in a classifier. e.g., a useful metric could be the ratio of
positive neighboring users [1]. Second, the network can be actively used
in the classification, with approaches such as label propagation [80].
•Meso-interactions (Mesoi): This category also models and utilizes inter-
actions. Interactions can be used in conjunction with relations to create
14
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
a single network or be treated individually to obtain several independent
networks. The resulting network is much richer than the previous cate-
gory, but also subject to change. In contrast to relations, interactions are
more varied and numerous. To prevent interactions from becoming noisy,
they are typically filtered. For instance, two users may only be connected
only when there have been a certain number of interactions between them.
•Meso-enriched (Mesoe): A natural step further from M esoi, this category
uses additional information inferred from the social network. A common
technique in this area is community detection. Community partitions may
inform a classifier, influence the features used for each instance [87], or be
used to process groups of users differently [22]. Other examples would
be metrics such as modularity and betweenness, which can be thought
of as proxies for importance or influence. Some works have successfully
explored the relationship between these metrics and user behavior, in order
to model users. However, these results are seldom used in classification
tasks.
•Macro: At this level, information from other sources outside the social
network is incorporated. For instance, Li et al. [48] use public opposi-
tion of political candidates in combination with social theories to improve
sentiment classification. Another example of external information is facts
such as the population of a country, or current government, which can
be combined with geo-location information in social media content. A
more complex example would be events in the real world or in other types
of media, such as television, which can be analyzed in combination with
social media activity [34].
The six levels of approaches are listed in increasing order of detail, measured
as the number of elements social context may include. The specific elements that
are available at each level are represented in Fig. 3. The essential elements have
already been covered in the definition of social context: content (C), users (U),
relations (Rc,Ruand Ruc), and interactions (Iuand Iuc). Social Context can
also be enriched through SNA with techniques such as community detection
(CD). Additionally, external sources of information can be used at a macro
level, such as facts or hyperlinks to external media, which are not part of the
definition of Social Context.
4.3. Dynamic approaches
Social context can be represented and analyzed as static or dynamic, as
mentioned in the definition. Static approaches present a quasi-static view of
social context and do not take its evolution into account. Note that this does
not prevent context from being updated at a later point. For instance, a user
label may be changed, or more content may be added. However, these changes
are not integrated into the model. In most of the works analyzed, context
is modeled as static. Conversely, dynamic approaches both use and need a
15
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Contextless Micro Mesor Mesoi Mesoe Macro
Media
Other
OSN
Facts
Ru
U
Ruc
Rc
CIu
Iuc
SNA
Figure 3: List of Social Context features available at each level of analysis
dynamic social context, as they exploit the changing nature of social networks.
These changes are an intrinsic part of the analysis and need to be part of the
model.
Although none of the surveyed works use dynamic social contexts for sen-
timent classification, several works use dynamic social context in tasks related
to sentiment analysis. Based on those and related works, we suggest dynamic
approaches for sentiment analysis may adhere to the following taxonomy, de-
pending on the parts of social context that are dynamic.
At the Micro-dynamic level, content is dynamic, and the changes in its
activity are taken into consideration. These changes could be the increase in
some metrics such as retweets and likes. For instance, the evolution in content
activity (number of retweets and mentions) can be used to classify content [96].
At the Meso-dynamic level, inter-personal communication starts to be appar-
ent and available. Several elements of the context can be studied in a dynamic
fashion. Two types of approaches could be considered, to subdivide this level.
First, approaches that focus on virality, and are content-centric. They use
the evolution of interactions, and the links between users in the network, to
measure and predict future activity, or to classify content according to the ac-
tivity related to it. This classification may be useful for sentiment analysis.
For instance, previous works have shown different types of content are linked to
different temporal patterns [96]. And by using certain features of content and
its activity, it is also possible to predict further spreading in the network (i.e., a
cascade) [17]. These content cascades are also linked to specific sentiments [2].
Garas et al. [26] could be relevant in this area, as it studies emotion persistence
in online communications (IRC).
Second, contagion-based approaches, which are user-centric. They focus on
user sentiment and emotion, instead of content. They apply social theories
and experimental results regarding sentiment and emotion contagion [35]. For
instance, a massive experiment on Facebook showed that emotional states can
be transferred to others via emotional contagion, leading people to experience
the same emotions without their awareness [45]. Hence, it may be possible
16
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
to improve the prediction of a user’s sentiment (and their content’s) by using
the sentiment of the content to which she is being exposed. On the other hand,
studies of social media activity regarding grassroots movements have shown that
social integration, as measured through social network metrics, increases with
their level of engagement and of expression of negativity [2]. This suggests a
connection between the groups to which a user belongs, and the sentiment the
user expresses. The connection could be exploited for user classification and, in
turn, for classification of the content created by them.
4.4. Analysis methods and Social Theories
Lastly, works differ in the type of classification performed. The options here
range from using traditional classification algorithms (e.g., random forest, SVM)
or neural networks, to network-based approaches such as label propagation.
However, two types of algorithms stand out from those of contextless analysis:
models that directly benefit from the networked nature of context, and deep
learning approaches. Several works also use a hybrid approach, where traditional
techniques are combined with network techniques, either via multiple processing
steps or by combining the techniques into one.
There are several ways in which algorithms could leverage the networks in
social context. Firstly, some algorithms are already network-oriented. Label
propagation, in particular, has shown promising results [80], and it can be made
to treat lexical resources and the subject of the analysis equally. Secondly, the
structure of the network can be directly incorporated into the learning process
through modified cost functions [38,92]. Thirdly, the output of a classifier
can be later complemented with a network-based algorithm. For example, Li
et al. [48] apply standard classification, then tweets or users are clustered, and
within each cluster, every piece of content or every user are given the same label
according to different criteria (i.e., most confident result, majority label, and
weighted majority). Fourthly, a multi-step or ensemble classification strategy
can be used, where the structure of the network and social theories are used to
combine the results of different classifiers.
On the deep learning front, recent works are incorporating different types of
neural networks that have been used for contextless analysis and subjectivity
analysis [14], such as convolutional neural networks (CNN). At the same time,
concepts such as word embeddings have inspired network embedding as an al-
ternative way of including features from social context in the analysis [97]. The
range of features that can be captured through network embeddings is vast,
including several types of relations [13]. Moreover, new research is complement-
ing and extending node embedding (i.e., nodes are represented as vectors) with
other methods such as edge and community embedding [10]. In particular, com-
munity embedding has shown promising results in community prediction and
node classification [12].
In general, network approaches usually follow well-known social theories.
Social theories usually model how users with different views or status arrange
themselves in the network. In other words, they are rules of attachment. They
may also model how users behave.
17
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Some examples of social theories or attributes include homophily, consis-
tency, social balance, and status theory. Homophily [53] is one of the commonly
used theories in the works we have examined and in the social sciences. In simple
terms, homophily means a connection between two people is more likely when
they are similar in some aspects (i.e., birds of a feather flock together). Under
the hypothesis of homophily, when two users are connected, certain features can
be propagated. Consistency [50] usually means that users tend to maintain their
views over time. So, two pieces of content shared by the same user in a short
period are likely to express a similar sentiment or opinion if they are about the
same topic. The social status theory [47] models the balance of power in social
networks. It states that, if three nodes A,Band Cform a clique, and the status
relation between Aand Bis the same as between Band C, it must also be true
of Aand C. In other words, the superior of your superior is your superior, and
the inferior of your inferior is your inferior. Social balance models the balance of
opinions in cliques. The rules in social balance translate to: a friend of a friend
is a friend, and an enemy of my enemy is my friend. Tang et al. [84] presents
a more detailed explanation of social theories that can be used to mine social
media.
5. Review of Social Context and Sentiment Analysis works
This section is the result of reviewing the state of the art in using social con-
text for sentiment analysis. The review is composed of five subsections. The first
one presents and compares the different works that have been reviewed. The
second subsection describes and compares the datasets that have been used in
these works. The third subsection covers common social context features that
are useful for sentiment analysis. The fourth one presents a performance com-
parison of the works on different datasets. The last subsection discusses ways
in which sentiment analysis has been used to improve social network analysis.
5.1. Works
This section introduces recent works in the area of sentiment analysis that
use social context. The aim is to compare how social context is defined and
exploited in each of them. The main features of each of the works are sum-
marized in Table 2. The table shows the gradual introduction of interactions
to complement interactions, as works evolve from mesorto mesoiand mesoe
approaches. It also highlights the most commonly used types of elements and
social theories used.
To the best of our knowledge, the first work to make explicit mention of
social context in the context of sentiment analysis is Lu et al. [50]. Their goal
was to predict the quality of reviews, rather than their sentiment, but the work
is worth mentioning for three reasons. First of all, they provide the first formal
mention of social context in the sense covered in this work. Secondly, their
novelty is that they merge traditional features (text) with what they call Social
Network Features. They provide a categorization of features, including author
18
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Table 2: Comparison of works using sentiment analysis and social context. The number of polarity labels is shown in parentheses.
Reference OSN Level lulciu.iuc rcru,c ruSocial Theories
Pennacchiotti and
Popescu [67]Twitter mesoi
political
orientation,
ethnicity
polarity (3) replies,
retweets retweet authorship friends
Speriosu et al. [80] Twitter mesorpolarity (2) polarity (2) authorship follower
Tan et al. [83] Twitter mesoipolarity (2) - (mutual)
mention authorship follower consistency,
homophily
Li et al. [48]Twitter,
Fora
mesor,
Macro
stance
(targets) polarity (2) stance (targets) balance,
consistency
Aisopos et al. [1] Twitter micro,
mesoipolarity (2) mention authorship follower
Hu et al. [38] Twitter mesorpolarity (3) polarity (3) authorship follower consistency and
contagion
Pozzi et al. [70] Twitter mesoipolarity (2) retweet retweet authorship mutual
follower
Ren and Wu [72] Twitter mesorpolarity (2) homophily
Deng et al. [23] Fora mesorpolarity (3) reply
friends,
inferred
friends
homophily,
consistency
West et al. [92] Wiki mesoipolarity (3) polarity (3) votes, mentions authorship social status,
social balance
Yang and Eisenstein
[97]Twitter mesoipolarity (2) retweet,
mention retweet follow language
homophily
Cheng et al. [16] Reddit mesoipolarity (2) reply
Sixto et al. [79] Twitter mesoipolarity (5) retweet favorite follow
Xiaomei et al. [95] Twitter mesoepolarity (2) authorship follow emotion
contagion
19
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
and social network features, which are calculated with social network analysis.
Lastly, the network is used to extract constraints based on several hypotheses
of consistency (of authors, links, citations, and trust).
On a related note, Pennacchiotti and Popescu [67] leverage replies, retweets
and friendship relations to infer user attributes, such as ethnicity and political
orientation. Their definition of political orientation can be considered stance
detection. Although their work is implicitly motivated by a hypothesis of ho-
mophily, they do not make any mention of specific social theories, and no
constraints or rules based on them are constructed. Instead, classification is
achieved via Gradient Boosted Decision Trees.
Speriosu et al. [80] introduce an alternative approach to infer polarity that
exploits the networked nature of social context. They compare three different
approaches: a lexicon-based classifier (baseline), a maximum entropy classifier
and Label Propagation (LPROP). The best results were achieved with LPROP,
which is also appealing because it yields annotations for resources (e.g., lexicon),
content and users indistinctly.
Similarly, Tan et al. [83] use a network approach based on SampleRank
with a Markovian model. The model assumes that the sentiment of a given
user is only influenced by the sentiment label of tweets generated by that user
(consistency), and the sentiment of neighboring users (homophily).
Li et al. [48] compare an approach based on linguistic features with a com-
bination of linguistic features and social features (referred to as global social
evidence). The goal is sentiment analysis about political figures (targets) on
Twitter and fora. In their hybrid approach, users, targets and issues (topics
targets are vocal about) form a network. Three different hypotheses are then
exploited on the data: 1) global consistency on indicative target-issue pairs, 2)
global consistency on indicative target-target pairs, and 3) social balance. The
results are slightly better than the baseline in the case of Twitter and widely
better for forum data. A similar comparison of linguistic and social features is
made by Aisopos et al. [1]. In their work, several classification algorithms are
compared using different feature models, some of which include social context
features.
Hu et al. [38] are the first in our review to include a classification algorithm
specially tuned to incorporate social context. Their work is also interesting
because they overcome the fact that most existing datasets only contain texts,
which makes them unsuitable for social context analysis. They do so by com-
bining text datasets with the friendship graph extracted from Kwak et al. [46].
Other works focus on user classification, such as Pozzi et al. [70]. They
leverage connections in the network to infer user polarity, with highly positive
results. User connections can also be exploited for content polarity classification.
Ren and Wu [72] use both friendship and user-topic relations (calculated from
user tweets) to calculate user-topic polarity. In addition to friendship, Deng
et al. [23] use reply-to relations in online fora, as well as inferred friendship.
West et al. [92] showed that the assumption of homophily in networks can
improve polarity detection from short texts. They use social ties to infer the
stance of users in Wikipedia. In particular, they exploit the social balance and
20
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
social status theories. They also point out the effect that the selection strategy
of training and testing nodes has on accuracy. Tang et al. [84] use similar social
theories to improve sentiment analysis on Twitter.
Lately, some works have introduced novel approaches such as Convolutional
Networks [97]. In doing so, they add new types of features such as network
embeddings, i.e., a vector representation of the network of a user, which can
be fed into a classifier. The motivation behind these embeddings is to leverage
language homophily in the analysis. Cheng et al. [16] follow in these steps, with
a similar premise using content from a different social network (Reddit). In this
case, the analysis also exploits the fact that comments are nested at different
levels.
5.2. Datasets
The usual drawback with sentiment analysis datasets is that they rarely
incorporate social context. This is either because social context was not taken
into consideration when the dataset was collected or because of data protection
policies and terms of use of the original OSN. The latter is usually easier
to circumvent, as these datasets usually have IDs or pointers to the original
resources, so that the necessary data can be recovered with the appropriate
credentials and access to the OSN. This process is known as hydration, and it
can be used to recover more data than was initially considered. i.e., it enables
the expansion of the social context. The limitation is the fact that resources can
be removed or made private before hydration. Table 3shows basic statistics of
the datasets used in the works reviewed.
RT Mind [70] contains a set of 62 users and 159 tweets, with positive or
negative annotations. To collect this dataset, Pozzi et al. [70] crawled 2500
Twitter users who tweeted about Obama during two days in May 2013. For
each user, their recent tweets (up to 3200, the limit of the API) were collected.
At that point, only users that tweeted at least 50 times about Obama were
considered. The tweets from those users that relate to Obama were kept and
manually labeled by 3 annotators. The dataset contains ID of the tweet, ID of
the author, text of the tweet, creation time, and sentiment (positive or negative).
The OMD dataset (Obama-McCain debate) [77] contains tweets about the
televised debate between Senator John McCain, and then-Senator Barack Obama.
The tweets were detected by following three hashtags: #current,#tweetdebate,
and #debate08. The dataset contains tweets captured during the 97-minute
debate, and 53 after it, to a total of 2.5 hours. There were 3238 tweets from
1160 people. There were 1824 tweets from 647 people during the actual debate
and 1414 tweets from 738 people after it. Of those, only 1261 tweets, from 679
users, have sentiment annotations. The dataset includes tweet IDs, publication
date, text, author name and nickname, and individual annotations of up to 7
annotators.
The Health Care Reform (HCR) [80] dataset contains tweets about the run-
up to the signing of the health care bill in the USA on March 23, 2010. It was
collected using the #hcr hashtag, from early 2010. A subset of the collected
tweets were annotated with polarity (positive, negative, neutral and irrelevant)
21
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Table 3: Datasets used in the experiments
Source Users Entries
RT Mind [70] Twitter 62 159
OMD [77] Twitter 679 1261
HCR-DEV [80] Twitter 806 1434
HCR-TEST [80] Twitter 806 1434
STS [31] Twitter 498 490
PF1901 [23] Forum 412 1901
MF1560 [23] Forum 320 1560
SemEval 2013 [56] Twitter 3813 3813
SemEval 2014 [76] Twitter 5749 5749
SemEval 2015 [75] Twitter 2379 2379
Ciao [85] Ciao 257682 10569
TASS [74] Twitter 158 68017
YANG2011 [96] Twitter 20M 476M
Li-Twitter [48] Twitter ? 4646
Li-Forum [48] Forum ? 762
AskMen [16] Reddit ? 1057K
AskWomen [16] Reddit ? 814K
Politics [16] Reddit ? 2180K
and polarity targets (health care reform, Obama, Democrats, Republicans, Tea
Party, conservatives, liberals, and Stupak) by Speriosu et al. [80]. The tweets
were separated into training, dev (HCR-DEV) and test (HCR-TEST) sets. The
dataset contains tweet ID, user ID and username, text of the tweet, sentiment,
target of the sentiment, annotator and annotator ID.
The Stanford Twitter Sentiment (STS) [31] contains manually annotated
tweets that mention a wide range of topics such as consumer products (40d, 50d,
kindle2), companies (aig, at&t), and people (Bobby Flay, Warren Buffet). The
version of the dataset used by Speriosu et al. [80] contains only 216 annotated
tweets, 108 of which tweets are positive, and 75 are negative. However, the
original paper [31] mentions 359 tweets with positive or negative sentiment.
These figures are aligned with the content of the dataset at the authors’ website1,
which also includes neutral tweets, to a total of 498 tweets by 490 authors. The
discrepancy should be noted, both because Speriosu et al. [80] use the reduced
dataset, and because they have released a collection of three datasets together
with the source code they used to process it2. The collection is well documented,
which might make it easier for other researchers to reuse their reduced dataset.
In their work, Deng et al. [23] include two datasets. The first dataset
(PF1901) is crawled from the “Election & Campaigns” board of a political
1http://cs.stanford.edu/people/alecmgo/trainingandtestdata.zip
2https://bitbucket.org/speriosu/updown/
22
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
forum3, There are 1901 labeled posts in total written by 232 unique users from
March 2011 to April 2012. Out of those, 419 positive and 553 negative posts
are also labeled with associated candidates. The rest are considered neutral
or unsure. The second dataset (MF1560) is crawled from a military forum4,
containing 43 483 threads and 1 343 427 posts. In total, there are 1560 labeled
posts written by 320 unique users, out of which 437 positive and 618 negative
posts also had their topic labeled. The rest are considered neutral or unsure.
The collection of SemEval datasets originate from the competition set up
for the different editions of the International Workshop on Semantic Evaluation
(SemEval). SemEval includes several individual tasks, which focus on different
types of classification, on different types of data. For this paper, we focus on
the Tweet sentiment classification tasks. There is a dataset for each edition:
SemEval 2013 [56], SemEval 2014 [76], SemEval 2015 [75]. For each tweet,
the dataset contains the ID of the tweet, the ID of the author, and the sen-
timent label (positive, negative or neutral). To use the dataset, participants
are encouraged to hydrate it, using the tools provided by the organizers of the
competition.
The General Corpus TASS dataset is one of the three datasets created for
the Taller de an´alisis de sentimientos (workshop on sentiment analysis) [74].
The other two datasets are the SocialTV dataset and the STOMPOL dataset,
and they are focused on aspect based analysis. The dataset contains tweets in
Spanish, authored by 150 well-known personalities and celebrities of the world
of politics, economy, communication, mass media and culture. The original
corpus is released in XML format, and it includes date, author and ID of each
tweet.
The AskMen, AskWomen and Politics datasets Cheng et al. [16]5contain
posts from popular subreddits (subcategories within the Reddit OSN6with dif-
ferent topics and styles: AskWomen (814K comments), AskMen (1057K com-
ments), and Politics (2180K comments).
Yang and Leskovec [96] collected a dataset of nearly 476 million Twitter
posts from 20 million users covering eight months, from June 2009 to February
2010. Aisopos et al. [1] filter the dataset in their work down to 6.12 million
negative and 14.12 million positive tweets using emoticons. From those tweets,
they finally used a sample of 1 million tweets with each polarity.
Li et al. [48] collected datasets from two OSN: an online forum and Twitter.
The forum dataset was collected from the most recent posts at the “Elections &
Campaigns” forum (similarly to Deng et al. [23]), from March 2011 to December
2011. 97.3% of those posts subjective, i.e., they contain positive or negative
sentiments. The tweet data set was automatically collected by retrieving positive
instances with #Obama2012 or #GOP2012 hashtags, and negative instances
3http://www.politicalforum.com/elections-campaigns/
4http://forums.military.com/
5https://github.com/hao-cheng/factored_neural/
6https://reddit.com
23
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
with #Obamafail or #GOPfail hashtags. All tweets where the hashtags of
interest were not located at the very end of the message were filtered.
Lastly, the Ciao dataset [85] includes opinions on the Ciao website7in May
2011. The authors started the collection of the dataset with a set of most active
users and then did a breadth-first search until no new users could be found. The
sentiment in the dataset is expressed with a 5-star rating system.
5.3. Features
This section briefly covers some of the features that can be extracted from
social context at different levels.
5.3.1. Micro features
At the micro level, features may be related to the content author, or to the
content itself. From the user, the main set of features is:
•Number of followees. In OSN such as Twitter, users (followers) are ex-
posed only to the content of their followees. This is typically an asym-
metrical relation. Following another user does not require the followee to
accept, except for private accounts and blocked users. For this reason,
it is typical for users to follow hundreds or even thousands of users [46].
Hence, this feature is rather noisy. Some works refer to followees as friends,
whereas other works reserve the term friend for mutual followers.
•Number of followers. In contrast with the previous feature, only a fraction
of users tend to accumulate most of the followers [46]. As a result, the
number of followers is more informative.
•Number of friends. In some instances, the number of followers that the
user follows back is known. Otherwise, it has to be calculated from the
meso network.
•Ratio of positive / negative / neutral content (per topic). This may in-
dicate the typical sentiment polarity for a user. Some theories such as
author coherence indicate that the sentiment we show about a topic tends
to be stable over short periods. Moreover, studies show that different types
of users exhibit characteristic sentiment patterns in their posts. Namely,
popular users are more likely to post positive content.
•Age, gender and nationality. All these features influence the way we com-
municate, from the language we use to the sentiment we are more likely
to express, and they have been shown to help in sentiment analysis [88].
Content may also be linked to features such as:
7http://www.ciao.co.uk
24
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
•Number of favorites, retweets, and replies. These values gradually increase
as more users interact with the content. For this reason, it may take some
time for them to stabilize or become meaningful, and it is not available in
online analysis unless some delay is added. By using specific time windows,
it is also possible to snapshot the value of the metric at different times, to
create derived metrics. e.g., number of replies during the first hour, and
number of replies during the first day. This type of analysis also borders
dynamic social context, which we have discussed earlier.
•Topic(s). The topic could either be extracted from content and metadata
such as hashtags or automatically inferred with topic detection.
•Sentiment of the original message. It is only available for replies. It may
be beneficial to know the original creator and the views of the creators,
as that enables the use of social theories (e.g., Li et al. [48]).
•Sentiment ratio of replies. This information is not typically used because
it requires a posteriori knowledge. However, for some types of offline
classification, this information is known at the time of prediction.
Additionally, it is also possible to generate user and topic-specific models or
to embed the context of the topical context of the content [23,16]. Network-
based algorithms such as label propagation and algorithms that take arbitrary
input sizes, such as recurrent neural networks, are not constrained by a fixed
input space. As a result, they can incorporate features of the context without
aggregation, such as averaging.
5.3.2. Mesorfeatures
At this level, a network of users and content also starts to form. Connections
in this network may be directed or undirected. Some examples of relations that
can originate a network are:
•Follower relation (directed). This is the relation that, when aggregated,
gives rise to the number of followees and number of followers in the pre-
vious section. It is the most common type of relation, and it typically
requires further filtering, given both the tendency of users to follow hun-
dreds of users and the lack of confirmation from the other side.
•Mutual follower relation (undirected). A simple follower relation often
yields poor results. The cause could be that this type of relation is too
weak [20], and is non-reciprocal. Most works use mutual relations instead,
where users are only connected if they follow each other.
•Ratio of Common Followers/followees relation (undirected). This is a mea-
sure of how many followers/followees two users have in common. Under
the hypothesis of homophily, it may be a proxy for user similarity. More
elaborate versions may take into account the number of followees/followers
of the followers/followees, via a weighted sum.
25
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
•Ratio of Common Topics/Keywords relation (undirected). Similar to the
ratio of Common Followers/followees, it is related to the similarity of two
users, based on the content they share.
5.3.3. Mesoifeatures
Interactions can also be used to create a network. For instance:
•Reply interaction (directed). The act of replying forms one relation be-
tween the original content, and the content to which it replies. However,
two interaction links can be formed as well: one between both users, and
another one between the user and the original content. Since replies are
less likely to occur than retweets, they tend to be more informative.
•Mention interaction (directed). When a user mentions another user in
their content, two links are formed: a mention interaction between the two
users, and a relation between the content and the user that was mentioned.
•Like/favorite interaction (directed). In most OSN, users can mark content
they like. As opposed to a reply, liking is usually achieved with a single
click. Hence, this is amongst the most common types of interactions.
•Retweet/reshare interaction (directed). Retweeting is the act of sharing
content from a different user verbatim.
•Shared a conversation (undirected). When two users engage in a conver-
sation (a series of replies), it can be encoded as a new interaction between
the users.
The ability to relate an author to other users enables the propagation of
micro features over the meso network, which yields a new set of features, such
as:
•Sentiment ratio of neighbors. The ratio of positive/negative/neutral neigh-
bors. Neighbors could be adjacent users (those sharing an edge), or users
that belong to the same group (e.g., the same community). These neigh-
bors could be filtered, e.g., to only take new neighbors into account, or
neighbors that have had recent activity. The sentiment for each neigh-
bor could also be calculated in time windows or weighted so that recent
content is more important.
•Sentiment ratio of content by neighbors. Similar to the previous one,
without aggregating on the user level.
Lastly, some techniques allow embedding large information networks (be it
content, user or mixed networks) into low-dimensional vector spaces. These
types of techniques are increasingly popular in contextless analysis due to their
excellent performance [3]. The components of the embedding can then be used
as features, either on their own or combined with other features. One example
26
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
of network embedding is the LINE method [86], which is used in one of the
works reviewed [16]. However, LINE does not take different types of nodes or
relationships into account. The heterogeneous network embedding model [13]
is an alternative. Although it was conceived to embed networks of text and
images, it could be adapted to encode mixed networks of content and users.
5.3.4. Mesoefeatures and Enrichment through Social Network Analysis
Social Network Analysis provides several methods to process, examine and
describe a social network. These methods use the network topology and its
attributes and infer information that could be useful for sentiment analysis tasks.
For instance, there are several ways to measure user popularity and influence in
a social network, according to different criteria. As a result, the impact of each
user in the sentiment prediction can be weighted. Similarly, the importance
of user connections (relations and interactions) can be measured. Thus, the
granularity can be set at the connection level, where sentiment prediction is not
only influenced by neighboring users, but also on the strength of the connection
to those neighbors. Another example is community detection, which could help
segment the user base into smaller groups that exhibit similar behavior.
5.3.5. Macro features
Macro features include any type of information that is outside of the realm
of the OSN. Hence, the possibilities for features in this category are unlimited.
Of all the works we have reviewed, only one [48] uses macro features. In par-
ticular, it uses known enmity or opposition between politicians, together with
social theories about user and target consistency. Other possibilities include the
analysis of links to external sources or attachments.
5.4. Performance
Having described these works, it is also important to compare their per-
formance. Few works use the same dataset in the same conditions. Instead of
providing that comparison, Table 4summarizes the best results for content-level
classification in every work surveyed, at every level of analysis identified in the
taxonomy in Section 4. The table shows both results for F1-score and accuracy,
when available. As expected, the results show that social context improves the
performance over the contextless baseline.
For completeness, Figure 4and Figure 5show all the results reported in these
works, grouped by the level of analysis. The performance is shown relative to
the contextless baseline in every dataset.
5.5. Other Approaches
Although this paper focuses on using social context to improve sentiment
analysis, there are other ways in which sentiment information can be fused with
other sources or types of information [4]. For instance, sentiment information
can be included into existing social network analysis. This can be done to char-
acterize or explain a given phenomenon. When adding sentiment information,
27
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Table 4: Maximum Accuracy score reported in each work, per level of analysis and dataset.
Level Metric Baseline micro mesormesoimesoemacro
Work Dataset
[1] YANG2011 Acc. 97.42 60.40 - 80.08 - -
[23] MF1560 Acc. 46.64 - 55.60 - - -
PF1901 Acc. 61.24 - 72.75 - - -
[48] Li-Forum Acc. 59.61 67.24 62.89 - - 71.97
Li-Twitter Acc. 83.97 - 85.35 - - -
[79] TASS Acc. 79.30 - - 89.80 - -
[80] HCR-DEV Acc. 58.60 65.70 65.20 - - -
HCR-TEST Acc. 62.90 71.20 71.00 - - -
OMD Acc. 61.30 66.70 66.50 - - -
STS Acc. 83.10 84.70 84.70 - - -
[95] HCR Acc. 69.00 - - - 77.5 -
OMD Acc. 76.00 - - - 76.0 -
[16] AskMen F1 51.70 - - 52.70 - -
AskWomen F1 55.20 - - 56.30 - -
Politics F1 53.00 - - 54.80 - -
[79] TASS F1 69.20 - - 90.20 - -
[97] Ciao F1 - - - 80.19 - -
SE 2013 F1 69.31 - 71.49 71.91 - -
SE 2014 F1 72.73 - 74.17 75.07 - -
SE 2015 F1 63.24 - 66.00 66.75 - -
some patterns and trends emerge, which would otherwise be lost in the global
aggregate. For instance, sentiment information can be used to analyze different
Twitter communities separately instead of aggregating their results [22].
Sentiment and social network analysis can also be combined to find poten-
tially radicalized users [6], or to highlight emotionally charged content [24]. Ad-
ditionally, sentiment information alone has proved to yield very high precision
and a low recall in some user classification tasks [67]. This suggests that senti-
ment information could be crucial in positively identifying members of specific
groups.
6. Conclusions and future work
The question that motivated this work was whether there is valuable infor-
mation in social networks that has the potential to improve sentiment analysis
in specific scenarios. We refer to this information as social context. To answer
this question, three related questions need to be answered: “what is social con-
text?”(Q1), “can social context improve sentiment analysis?”(Q2) and “what
elements of social context are more relevant for sentiment analysis?”(Q3).
To answer the first question (Q1), we analyzed the use and definitions of
28
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Figure 4: Difference in accuracy with respect to a contextless approach in all works analyzed,
per dataset. The results for [1] have been removed due to their unusually high accuracy
(Table 4).
social context in the state of the art. Our analysis revealed that there are com-
monalities between these works, despite differences in notation. We formalized
these commonalities in a formal definition of social context. This definition
enables a richer and more precise description of social media information.
We used this definition in a new framework for comparison of approaches to
sentiment analysis using social context. Part of this framework is a taxonomy of
approaches, which shows the different levels of social context that are possible.
Using this taxonomy, we compared works in the literature. The results of this
comparison, which are included in this work, support the notion that using
social context may improve performance in sentiment analysis (Q2), both in
content classification and user classification tasks.
Once these levels of analysis have been identified, the natural question is
what performance gains can be achieved by using more complex features. Di-
rectly comparing their results is not straightforward, but the taxonomy can be
used to group approaches and to compare these groups. Higher results corre-
spond to more detailed definitions of Social Context, as shown by mesoiap-
proaches outperforming mesorones in most works (Q3). The trend seems to
support these results, as recent works are starting to incorporate mesoiap-
29
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Figure 5: Difference in F1 score with respect to a contextless approach in all works analyzed,
per dataset.
proaches. Unfortunately, the number of works in the field is not enough to
provide an accurate evaluation of the specific elements of content (e.g., whether
retweet interactions are more informative than community detection).
On the other hand, the trend suggests that there is room for improvement
in the processing of social context and its use with different classifiers. For
instance, techniques such as network embeddings could be used to condense
several aspects of social context.
We expect that the formal definition of context and the framework in this
work foster the use of social context in sentiment analysis in two ways. Firstly,
by providing a common language to express social context. Secondly, by allowing
future works to perform a more systematic comparison with existing approaches.
As more works start leveraging social context, the taxonomy of approaches
will likely grow and add novel ideas. Similarly, more elements may need to
be included in the definition of social context to account for more complex
scenarios.
Acknowledgments
This work is supported by the Spanish Ministry of Economy and Competi-
tiveness under the R&D project SEMOLA (TEC2015-68284-R) and the Euro-
30
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
pean Union under the project Trivalent (H2020 Action Grant No. 740934, SEC-
06-FCT-2016). The authors also want to mention earlier work that contributed
to the results in this paper. More specifically, the MixedEmotions (European
Union‘s Horizon 2020 Programme research and innovation programme under
grant agreements No.644632) and SoMeDi (ITEA3 16011) projects.
References
[1] Aisopos, F., Papadakis, G., Tserpes, K., Varvarigou, T., 2012. Content vs.
context for sentiment analysis: a comparative analysis over microblogs. In:
Proceedings of the 23rd ACM conference on Hypertext and social media.
ACM, pp. 187–196.
[2] Alvarez, R., Garcia, D., Moreno, Y., Schweitzer, F., Dec. 2015. Sentiment
cascades in the 15m movement. EPJ Data Science 4 (1).
[3] Araque, O., Corcuera-Platas, I., S´anchez-Rada, J. F., Iglesias, C. A., Jun.
2017. Enhancing Deep Learning Sentiment Analysis with Ensemble Tech-
niques in Social Applications. Expert Systems with Applications.
[4] Balazs, J. A., Vel´asquez, J. D., Jan. 2016. Opinion Mining and Information
Fusion: A survey. Information Fusion 27, 95–110.
[5] Bengio, Y., Nov. 2009. Learning Deep Architectures for AI. Foundations
and Trends R
in Machine Learning 2 (1), 1–127.
[6] Bermingham, A., Conway, M., McInerney, L., O’Hare, N., Smeaton, A. F.,
2009. Combining social network analysis and sentiment analysis to explore
the potential for online radicalisation. In: Social Network Analysis and Min-
ing, 2009. ASONAM’09. International Conference on Advances in. IEEE,
pp. 231–236.
[7] Bol´ıbar, M., Sep. 2016. Macro, meso, micro: broadening the ‘social’ of so-
cial network analysis with a mixed methods approach. Quality & Quantity
50 (5), 2217–2236.
[8] Borgatti, S. P., Mehra, A., Brass, D. J., Labianca, G., Feb. 2009. Network
Analysis in the Social Sciences. Science 323 (5916), 892–895.
[9] Buitelaar, P., Arcan, M., Iglesias, C., Sanchez-Rada, F., Strapparava, C.,
2013. Linguistic linked data for sentiment analysis. In: Proceedings of the
2nd Workshop on Linked Data in Linguistics (LDL-2013): Representing
and linking lexicons, terminologies and other language data. pp. 1–8, 00015.
[10] Cai, H., Zheng, V. W., Chang, K. C., Sep. 2018. A Comprehensive Survey of
Graph Embedding: Problems, Techniques, and Applications. IEEE Trans-
actions on Knowledge and Data Engineering 30 (9), 1616–1637, 00123.
[11] Cambria, E., 2016. Affective computing and sentiment analysis. IEEE In-
telligent Systems 31 (2), 102–107.
31
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
[12] Cavallari, S., Zheng, V. W., Cai, H., Chang, K. C.-C., Cambria, E., 2017.
Learning community embedding with community detection and node em-
bedding on graphs. In: Proceedings of the 2017 ACM on Conference on
Information and Knowledge Management. ACM, pp. 377–386.
[13] Chang, S., Han, W., Tang, J., Qi, G.-J., Aggarwal, C. C., Huang, T. S.,
2015. Heterogeneous network embedding via deep architectures. In: Pro-
ceedings of the 21th ACM SIGKDD International Conference on Knowledge
Discovery and Data Mining. ACM, pp. 119–128.
[14] Chaturvedi, I., Cambria, E., Welsch, R. E., Herrera, F., Nov. 2018. Dis-
tinguishing between facts and opinions for sentiment analysis: Survey and
challenges. Information Fusion 44, 65–77.
[15] Chen, H., Liu, J., Lv, Y., Li, M. H., Liu, M., Zheng, Q., Nov. 2018. Semi-
supervised clue fusion for spammer detection in Sina Weibo. Information
Fusion 44, 22–32.
[16] Cheng, H., Fang, H., Ostendorf, M., 2017. A Factored Neural Network
Model for Characterizing Online Discussions in Vector Space. In: Proceed-
ings of the 2017 Conference on Empirical Methods in Natural Language
Processing. pp. 2296–2306.
[17] Cheng, J., Adamic, L., Dow, P. A., Kleinberg, J. M., Leskovec, J., 2014.
Can Cascades Be Predicted? In: Proceedings of the 23rd International
Conference on World Wide Web. WWW ’14. ACM, New York, NY, USA,
pp. 925–936.
[18] Cho, H., Lee, J.-S., Aug. 2008. Collaborative Information Seeking in In-
tercultural Computer-Mediated Communication Groups: Testing the In-
fluence of Social Context Using Social Network Analysis. Communication
Research 35 (4), 548–573, 00090.
[19] Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa,
P., 2011. Natural language processing (almost) from scratch. The Journal
of Machine Learning Research 12, 2493–2537.
[20] Darmon, D., Omodei, E., Garland, J., Aug. 2015. Followers Are Not
Enough: A Multifaceted Approach to Community Detection in Online So-
cial Networks. PLOS ONE 10 (8).
[21] Davidov, D., Tsur, O., Rappoport, A., 2010. Enhanced sentiment learn-
ing using twitter hashtags and smileys. In: Proceedings of the 23rd inter-
national conference on computational linguistics: posters. Association for
Computational Linguistics, pp. 241–249.
[22] Deitrick, W., Hu, W., 2013. Mutually Enhancing Community Detection
and Sentiment Analysis on Twitter Networks. Journal of Data Analysis
and Information Processing 01 (03), 19–29.
32
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
[23] Deng, H., Han, J., Li, H., Ji, H., Wang, H., Lu, Y., 2014. Exploring and
inferring user–user pseudo-friendship for sentiment analysis with hetero-
geneous networks. Statistical Analysis and Data Mining: The ASA Data
Science Journal 7 (4), 308–321.
[24] Gamon, M., Basu, S., Belenko, D., Fisher, D., Hurst, M., K¨onig, A. C.,
2008. BLEWS: Using blogs to provide context for news articles. In:
ICWSM. pp. 60–67.
[25] Gao, B., Berendt, B., Clarke, D., De Wolf, R., Peetz, T., Pierson, J.,
Sayaf, R., 2012. Interactive grouping of friends in OSN: Towards online
context management. In: Data Mining Workshops (ICDMW), 2012 IEEE
12th International Conference on. IEEE, pp. 555–562.
[26] Garas, A., Garcia, D., Skowron, M., Schweitzer, F., May 2012. Emotional
persistence in online chatting communities. Scientific Reports 2.
[27] Garcia, D., Abisheva, A., Schweighofer, S., Serd¨ult, U., Schweitzer, F.,
Mar. 2015. Ideological and Temporal Components of Network Polarization
in Online Political Participatory Media: Ideological and Temporal Compo-
nents of Network. Policy & Internet 7 (1), 46–79.
[28] Garc´ıa-Pablos, A., Cuadros Oller, M., Rigau Claramunt, G., 2016. A com-
parison of domain-based word polarity estimation using different word em-
beddings. In: Proceedings of the Tenth International Conference on Lan-
guage Resources and Evaluation. Portoroz, Slovenia.
[29] Genc, Y., Sakamoto, Y., Nickerson, J., 2011. Discovering context: classify-
ing tweets through a semantic transform based on wikipedia. Foundations
of augmented cognition. Directing the future of adaptive systems, 484–492.
[30] Gimpel, K., Schneider, N., O’Connor, B., Das, D., Mills, D., Eisenstein,
J., Heilman, M., Yogatama, D., Flanigan, J., Smith, N. A., 2011. Part-of-
speech Tagging for Twitter: Annotation, Features, and Experiments. In:
Proceedings of the 49th Annual Meeting of the Association for Computa-
tional Linguistics: Human Language Technologies: Short Papers - Volume
2. HLT ’11. Association for Computational Linguistics, Stroudsburg, PA,
USA, pp. 42–47.
[31] Go, A., Bhayani, R., Huang, L., 2009. Twitter sentiment classification using
distant supervision. CS224N Project Report, Stanford 1 (12).
[32] Guo, W., Li, H., Ji, H., Diab, M. T., 2013. Linking Tweets to News: A
Framework to Enrich Short Text Data in Social Media. In: ACL (1). pp.
239–249.
[33] Hajian, B., White, T., Oct. 2011. Modelling Influence in a Social Network:
Metrics and Evaluation. In: 2011 IEEE Third International Conference
on Privacy, Security, Risk and Trust and 2011 IEEE Third International
Conference on Social Computing. pp. 497–500.
33
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
[34] Heo, Y.-C., Park, J.-Y., Kim, J.-Y., Park, H.-W., May 2016. The emerging
viewertariat in South Korea: The Seoul mayoral TV debate on Twitter,
Facebook, and blogs. Telematics and Informatics 33 (2), 570–583, 00014.
[35] Hill, A. L., Rand, D. G., Nowak, M. A., Christakis, N. A., Jul. 2010.
Emotions as infectious diseases in a large social network: the SISa
model. Proceedings of the Royal Society of London B: Biological Sciences,
rspb20101217.
[36] Hogenboom, A., Bal, D., Frasincar, F., Bal, M., De Jong, F., Kaymak, U.,
2015. Exploiting Emoticons in Polarity Classification of Text. J. Web Eng.
14 (1&2), 22–40, 00043.
[37] Hovy, D., 2015. Demographic Factors Improve Classification Performance.
In: ACL (1). pp. 752–762.
[38] Hu, X., Tang, L., Tang, J., Liu, H., 2013. Exploiting Social Relations for
Sentiment Analysis in Microblogging. In: Proceedings of the Sixth ACM
International Conference on Web Search and Data Mining. WSDM ’13.
ACM, New York, NY, USA, pp. 537–546.
[39] Jansen, B. J., Zhang, M., Sobel, K., Chowdury, A., 2009. Twitter Power :
Tweets as Electronic Word of Mouth. Journal of the American Society for
Information Science 60 (11), 2169–2188.
[40] Jiang, F., Liu, Y.-Q., Luan, H.-B., Sun, J.-S., Zhu, X., Zhang, M., Ma, S.-
P., 2015. Microblog sentiment analysis with emoticon space model. Journal
of Computer Science and Technology 30 (5), 1120–1129, 00026.
[41] Kaplan, A. M., Haenlein, M., Jan. 2010. Users of the world, unite! The
challenges and opportunities of Social Media. Business Horizons 53 (1),
59–68.
[42] Kim, Y., Oct. 2014. Convolutional neural networks for sentence classifi-
cation. In: Proceedings of the 2014 Conference on Empirical Methods in
Natural Language Processing (EMNLP). Association for Computational
Linguistics, Doha, Qatar, pp. 1746–1751.
[43] Kim, Y., Sohn, D., Choi, S. M., Jan. 2011. Cultural difference in motiva-
tions for using social network sites: A comparative study of American and
Korean college students. Computers in Human Behavior 27 (1), 365–372,
00708.
[44] Kiritchenko, S., Zhu, X., Mohammad, S. M., Aug. 2014. Sentiment Analysis
of Short Informal Texts. Journal of Artificial Intelligence Research 50, 723–
762.
[45] Kramer, A. D., Guillory, J. E., Hancock, J. T., 2014. Experimental evidence
of massive-scale emotional contagion through social networks. Proceedings
of the National Academy of Sciences.
34
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
[46] Kwak, H., Lee, C., Park, H., Moon, S., 2010. What is Twitter, a Social
Network or a News Media? In: Proceedings of the 19th International
Conference on World Wide Web. WWW ’10. ACM, New York, NY, USA,
pp. 591–600.
[47] Leskovec, J., Huttenlocher, D., Kleinberg, J., 2010. Signed networks in
social media. In: Proceedings of the SIGCHI conference on human factors
in computing systems. ACM, pp. 1361–1370.
[48] Li, H., Chen, Y., Ji, H., Muresan, S., Zheng, D., 2012. Combining So-
cial Cognitive Theories with Linguistic Features for Multi-genre Sentiment
Analysis. In: PACLIC. pp. 127–136.
[49] Lipton, Z. C., 2016. The mythos of model interpretability. arXiv preprint
arXiv:1606.03490.
[50] Lu, Y., Tsaparas, P., Ntoulas, A., Polanyi, L., 2010. Exploiting social con-
text for review quality prediction. In: Proceedings of the 19th international
conference on World wide web. ACM, pp. 691–700.
[51] Marcus, G., 2018. Deep learning: A critical appraisal. arXiv preprint
arXiv:1801.00631.
[52] McCrae, J., Spohr, D., Cimiano, P., 2011. Linking lexical resources and
ontologies on the semantic web with lemon. In: Extended Semantic Web
Conference. Springer, pp. 245–259, 00210.
[53] McPherson, M., Smith-Lovin, L., Cook, J. M., 2001. Birds of a feather:
Homophily in social networks. Annual review of sociology 27 (1), 415–444.
[54] Melville, P., Gryc, W., Lawrence, R. D., 2009. Sentiment Analysis of Blogs
by Combining Lexical Knowledge with Text Classification. In: Proceed-
ings of the 15th ACM SIGKDD International Conference on Knowledge
Discovery and Data Mining. KDD ’09. ACM, New York, NY, USA, pp.
1275–1284.
[55] Mikolov, T., Chen, K., Corrado, G., Dean, J., 2013. Efficient estimation of
word representations in vector space. arXiv preprint arXiv:1301.3781.
[56] Nakov, P., Rosenthal, S., Kozareva, Z., Stoyanov, V., Ritter, A., Wilson, T.,
2013. SemEval-2013 Task 2: Sentiment analysis in Twitter. In: Proceedings
of the 7th International Workshop on Semantic Evaluation. SemEval ’13.
Vol. 7. Atlanta, Georgia, USA, pp. 312–320.
[57] Nasukawa, T., Yi, J., 2003. Sentiment Analysis: Capturing Favorability
Using Natural Language Processing. In: Proceedings of the 2Nd Interna-
tional Conference on Knowledge Capture. K-CAP ’03. ACM, New York,
NY, USA, pp. 70–77.
35
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
[58] Nguyen, M.-T., Tran, D.-V., Nguyen, L.-M., Dec. 2017. Social con-
text summarization using user-generated content and third-party sources.
Knowledge-Based Systems.
[59] Noro, T., Tokuda, T., Jul. 2016. Searching for Relevant Tweets Based on
Topic-related User Activities. J. Web Eng. 15 (3-4), 249–276.
[60] Novak, P. K., Smailovi´c, J., Sluban, B., Mozetiˇc, I., 2015. Sentiment of
emojis. PloS one 10 (12), e0144296, 00226.
[61] Orman, G. K., Labatut, V., Cherifi, H., 2011. Qualitative comparison of
community detection algorithms. In: International conference on digital
information and communication technology and its applications. Springer,
pp. 265–279.
[62] Otte, E., Rousseau, R., Dec. 2002. Social network analysis: a powerful
strategy, also for the information sciences. Journal of Information Science
28 (6), 441–453.
[63] Pak, A., Paroubek, P., 2010. Twitter as a corpus for sentiment analysis and
opinion mining. In: LREc. Vol. 10. pp. 1320–1326.
[64] Pang, B., Lee, L., 2008. Opinion mining and sentiment analysis. Founda-
tions and Trends R
in Information Retrieval 2 (1–2), 1–135.
[65] Pang, B., Lee, L., Vaithyanathan, S., 2002. Thumbs Up?: Sentiment Clas-
sification Using Machine Learning Techniques. In: Proceedings of the ACL-
02 Conference on Empirical Methods in Natural Language Processing - Vol-
ume 10. EMNLP ’02. Association for Computational Linguistics, Strouds-
burg, PA, USA, pp. 79–86.
[66] Papadopoulos, S., Kompatsiaris, Y., Vakali, A., Spyridonos, P., 2012. Com-
munity detection in social media. Data Mining and Knowledge Discovery
24 (3), 515–554.
[67] Pennacchiotti, M., Popescu, A.-M., 2011. A Machine Learning Approach
to Twitter User Classification. Icwsm 11 (1), 281–288.
[68] Polanyi, L., Zaenen, A., 2006. Contextual valence shifters. In: Computing
attitude and affect in text: Theory and applications. Springer, pp. 1–10.
[69] Poria, S., Cambria, E., Bajpai, R., Hussain, A., Sep. 2017. A review of
affective computing: From unimodal analysis to multimodal fusion. Infor-
mation Fusion 37, 98–125.
[70] Pozzi, F. A., Maccagnola, D., Fersini, E., Messina, E., 2013. Enhance user-
level sentiment analysis on microblogs with approval relations. In: Congress
of the Italian Association for Artificial Intelligence. Springer, pp. 133–144.
36
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
[71] Ravi, K., Ravi, V., Nov. 2015. A survey on opinion mining and sentiment
analysis: Tasks, approaches and applications. Knowledge-Based Systems
89 (Supplement C), 14–46.
[72] Ren, F., Wu, Y., Oct. 2013. Predicting User-Topic Opinions in Twitter with
Social and Topical Context. IEEE Transactions on Affective Computing
4 (4), 412–424.
[73] Rokach, L., Feb. 2010. Ensemble-based classifiers. Artificial Intelligence
Review 33 (1-2), 1–39.
[74] Rom´an, J. V., C´amara, E. M., Morera, J. G., Zafra, S. M. J., 2015. TASS
2014-the challenge of aspect-based sentiment analysis. Procesamiento del
Lenguaje Natural 54, 61–68.
[75] Rosenthal, S., Nakov, P., Kiritchenko, S., Mohammad, S., Ritter, A., Stoy-
anov, V., 2015. Semeval-2015 task 10: Sentiment analysis in twitter. In:
Proceedings of the 9th international workshop on semantic evaluation (Se-
mEval 2015). pp. 451–463.
[76] Rosenthal, S., Ritter, A., Nakov, P., Stoyanov, V., 2014. SemEval-2014
Task 9: Sentiment Analysis in Twitter. In: Proceedings of the 8th Interna-
tional Workshop on Semantic Evaluation (SemEval 2014). Dublin, Ireland,
pp. 73–80.
[77] Shamma, D. A., Kennedy, L., Churchill, E. F., 2009. Tweet the Debates:
Understanding Community Annotation of Uncollected Sources. In: Pro-
ceedings of the First SIGMM Workshop on Social Media. WSM ’09. ACM,
New York, NY, USA, pp. 3–10.
[78] Sharma, A., Dey, S., 2012. A comparative study of feature selection and
machine learning techniques for sentiment analysis. In: Proceedings of the
2012 ACM research in applied computation symposium. ACM, pp. 1–7.
[79] Sixto, J., Almeida, A., L´opez-de Ipi˜na, D., 2018. Analysis of the Struc-
tured Information for Subjectivity Detection in Twitter. Transactions on
Computational Collective Intelligence XXIX, 163–181.
[80] Speriosu, M., Sudan, N., Upadhyay, S., Baldridge, J., 2011. Twitter Polar-
ity Classification with Label Propagation over Lexical Links and the Fol-
lower Graph. In: Proceedings of the Conference on Empirical Methods in
Natural Language Processing. Association for Computational Linguistics,
pp. 53–56.
[81] S´anchez-Rada, J. F., Iglesias, C. A., 2016. Onyx: A linked data approach
to emotion representation. Information Processing & Management 52 (1),
99–114, 00026.
37
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
[82] Taboada, M., Brooke, J., Tofiloski, M., Voll, K., Stede, M., Apr. 2011.
Lexicon-Based Methods for Sentiment Analysis. Computational Linguistics
37 (2), 267–307.
[83] Tan, C., Lee, L., Tang, J., Jiang, L., Zhou, M., Li, P., 2011. User-level
Sentiment Analysis Incorporating Social Networks. In: Proceedings of the
17th ACM SIGKDD International Conference on Knowledge Discovery and
Data Mining. KDD ’11. ACM, New York, NY, USA, pp. 1397–1405.
[84] Tang, J., Chang, Y., Liu, H., Jun. 2014. Mining Social Media with Social
Theories: A Survey. SIGKDD Explor. Newsl 15 (Iid), 20–29.
[85] Tang, J., Gao, H., Liu, H., 2012. mTrust: discerning multi-faceted trust in
a connected world. In: Proceedings of the fifth ACM international confer-
ence on Web search and data mining - WSDM ’12. ACM Press, Seattle,
Washington, USA, p. 93.
[86] Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., Mei, Q., 2015. Line:
Large-scale information network embedding. In: Proceedings of the 24th
International Conference on World Wide Web. International World Wide
Web Conferences Steering Committee, pp. 1067–1077.
[87] Tommasel, A., Godoy, D., Mar. 2018. A Social-aware online short-text
feature selection technique for social media. Information Fusion 40, 1–17.
[88] Volkova, S., Wilson, T., Yarowsky, D., 2013. Exploring demographic lan-
guage variations to improve multilingual sentiment analysis in social media.
In: Proceedings of the 2013 Conference on Empirical Methods in Natural
Language Processing. pp. 1815–1827.
[89] Volkova, Svitlana, 2015. Predicting Demographics and Affect in Social Net-
works. Ph.D. thesis, Johns Hopkins University, Baltimore, Maryland.
[90] Wang, S., Manning, C. D., 2012. Baselines and Bigrams: Simple, Good
Sentiment and Topic Classification. In: Proceedings of the 50th Annual
Meeting of the Association for Computational Linguistics: Short Papers -
Volume 2. ACL ’12. Association for Computational Linguistics, Strouds-
burg, PA, USA, pp. 90–94.
[91] Wei, W., Gulla, J. A., 2010. Sentiment learning on product reviews via
sentiment ontology tree. In: Proceedings of the 48th Annual Meeting of the
Association for Computational Linguistics. Association for Computational
Linguistics, pp. 404–413, 00134.
[92] West, R., Paskov, H. S., Leskovec, J., Potts, C., 2014. Exploiting So-
cial Network Structure for Person-to-Person Sentiment Analysis. CoRR
abs/1409.2450.
38
ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
[93] Wu, F., Shu, J., Huang, Y., Yuan, Z., Aug. 2016. Co-detecting social spam-
mers and spam messages in microblogging via exploiting social contexts.
Neurocomputing 201, 51–65.
[94] Xia, R., Zong, C., 2010. Exploring the Use of Word Relation Features for
Sentiment Classification. In: Proceedings of the 23rd International Confer-
ence on Computational Linguistics: Posters. COLING ’10. Association for
Computational Linguistics, Stroudsburg, PA, USA, pp. 1336–1344.
[95] Xiaomei, Z., Jing, Y., Jianpei, Z., Hongyu, H., Feb. 2018. Microblog sen-
timent analysis with weak dependency connections. Knowledge-Based Sys-
tems 142, 170–180.
[96] Yang, J., Leskovec, J., 2011. Patterns of temporal variation in online media.
In: Proceedings of the fourth ACM international conference on Web search
and data mining. ACM, pp. 177–186.
[97] Yang, Y., Eisenstein, J., Aug. 2017. Overcoming Language Variation in
Sentiment Analysis with Social Attention. Transactions of the Association
for Computational Linguistics 5, 295–307.
39
