Content uploaded by John Kalung Leung
Author content
All content in this area was uploaded by John Kalung Leung on Sep 29, 2020
Content may be subject to copyright.
arXiv:2007.00636v1 [cs.IR] 1 Jul 2020
Making Use of Affective Features from Media Content Metadata for
Better Movie Recommendation Making
John Kalung Leung1, Igor Griva2and William G. Kennedy3a
1Computational and Data Sciences Department, Computational Sciences and Informatics, College of Science, George
Mason University, 4400 University Drive, Fairfax, Virginia 22030, USA
2Department of Mathematical Sciences, MS3F2, Exploratory Hall 4114, George Mason University,4400 University Drive,
Fairfax, Virginia 22030, USA
3Center for Social Complexity, Computational and Data Sciences Department, College of Science, George Mason
University, 4400 University Drive, Fairfax, Virginia 22030, USA
{jleung2, igriva, wkennedy}@gmu.edu
Keywords: context-aware, emotion mining, affective computing, recommender systems, movie recommendations, deep
learning
Abstract: Our goal in this paper aims to investigate the causality in the decision making of movie recommendations
from a Recommender perspective through the behavior of users’ affective moods. We illustrate a method of
assigning emotional tags to a movie by auto-detection of the affective attributes in the movie overview. We
apply a text-based Emotion Detection and Recognition model, which trained by the short text of tweets, and
then transfer the model learning to detect the implicit affective features of a movie from the movie overview.
We vectorize the affective movie tags through embedding to represent the mood of the movie. Whereas we
vectorize the user’s emotional features by averaging all the watched movie’s vectors, and when incorporated
the average ratings from the user rated for all watched movies, we obtain the weighted vector. We apply
the distance metrics of these vectors to enhance the movie recommendation making of a Recommender. We
demonstrate our work through an SVD based Collaborative Filtering (SVD-CF) Recommender. We found
an improved 60% support accuracy in the enhanced top-5 recommendation computed by the active test user
distance metrics versus 40% support accuracy in the top-5 recommendation list generated by the SVD-CF
Recommender.
1 Introduction
Movie recommendations come from different
sources. A more traditional way to make a movie
recommendation is by word of mouth through
moviegoers who have watched the movie, or relying
on elite movie critics who wrote about their opinions
of the film, or through news media, publications, and
advertisements. Since the dawn of the Internet era
in the last century, we rely on machine automation
to make movie recommendations using various Rec-
ommender methodologies (Bobadilla et al., 2013),
(Zhang et al., 2011), (Scheel et al., 2012), and
(Kompan and Bielikova, 2014). More recently, we
have applied the more advanced Deep Learning (DL)
techniques in Recommenders (Zhang et al., 2017).
After a century, the field in recommendation making
is still in active research (Jannach et al., 2010).
ahttps://orcid.org/0000-0001-9238-1215
Regardless of the efforts, we have invested in Rec-
ommenders research, and they always seem to be
more ways to make improvements even in the field
(Beel et al., 2015). In this paper, we shall include
primary human emotions as an aspect of making
movie recommendations through a Recommender
(Canales and Mart´ınez-Barco, 2014).
Emotion affects human experience and influences our
daily activities on all levels of the decision-making
process. When a user ponders over a list of recom-
mended items such as songs, books, movies, prod-
ucts, or services, his affective state of preferences in-
fluences his decision making on which recommended
item he chooses to consume. Emotion plays a role in
our decision-making process in preference selection
(Naqvi et al., 2006). However, up to now, informa-
tion retrieval (IF) and Recommender Systems (RS)
give little attention to include human emotion as a
source of user context (Ho and Tagmouti, 2006). Our
goal is to make affective awareness a component of
movie recommendations for users. The challenge is
that no film database or movie dataset in the public
domain contains any explicit textual oriented human
emotional tag in the metadata. Though, the film meta-
data fields such as plot, overview, storyline, script,
watcher reviews, and critics reviews contain excellent
subjective data that describes the general mood of a
movie. We can apply Machine Learning (ML) tech-
niques to identify and extract affective features im-
plicitly from the film metadata and leverage the film’s
emotional characteristics when making movie recom-
mendations to users.
1.1 Motivation
We envision that the affective elements of a cinema
represented by a low dimension continuous emotion
embedding vector denoted as the movie’s emotional
vector (mvec). The fact that a film is unique and
mutually exclusive from its peers, thus, in theory,
a mvec. Some film databases, such as the Movie-
Lens, track users’ movie-watching history, and feed-
back (Harper and Konstan, 2016). Using the movie-
watching records of a user, we can formulate a low
dimension continuous emotion embedding vector for
the user denoted as the user emotional vector (uvec).
We obtain a user’s uvec embedding value by taking
the average over all the movie’s mvec and the av-
erage rating the user has given to all the movies he
has watched. Note that, uvec may not be unique. It
merely represents the current measure of a user’s af-
fective preference for films that he has enjoyed. The
difference between mvec and uvec is that mvec of a
movie is static with enduring value throughout its life-
time. Whereas, uvec is dynamic with value change as
the user watched and rated movies. The advantage
of using the dynamic nature of uvec in making a rec-
ommendation is that we are taking the most updated
user’s affective preference into consideration of the
user decision-making process every time. As the user
emotional preference change, so will the movie Rec-
ommender to adjust the recommendation making pro-
cess accordingly. We may be the first party making
use of the novelty in leveraging the dynamic nature
of uvec over mvec to enhance the process of making
movie recommendations of a Recommender.
Also, we can leverage mvec to analyze the emotional
features of a movie. The continuous embedding na-
ture of the mvec represents the range and strength of
moods in a film. In this study, we track six primary
human affective features in emotion: “joy”, “sad-
ness”, “hate”, “anger”, “disgust”, and “surprise ”. We
added “neutral” as the seventh affective feature for
convenience in our computing tasks. We normalized
the affective features when we compute mvec for a
film. Thus, all affective features in mvec will add up
to one (1). We can interpret each affective feature in
a film as a percentage in decimal. For example, Inter-
net Movie Database (IMDb) is a popular online movie
information database that has rated “The Godfather
(1972)” as the top movie of all time (IMDb, 2020).
Our emotion detector classified the movie with the
affective class “hate” and the mvec for the movie de-
picts in table 1.
Table 1: Affect values of movie: “The Godfather (1972)”.
neutral
0.0840931
joy sadness hate
0.059261046 0.08991193 0.23262443
anger disgust surprise
0.20177138 0.19720455 0.13513364
Glancing through table 1, we perform the affec-
tive analysis for the movie in the following way.
The movie mvec affective attribute “hate” stands as
the dominant feature followed by “anger”, “disgust”,
“surprise”, ”sadness”, “neutral”, and closed out by
“joy”. With the mvec affective features distribution
value, we can describe “The Godfather (1972)” movie
as a film full of hate. It is an angry movie. The con-
tent is disgusting due to the violent nature of the film.
However, the story of the movie is full of surprises,
but a sad movie. None can say the movie celebrates
happiness. By reciting the mvec attributes, we explain
the movie. On the same token, we can use the uvec of
a specific user who watched “The Godfather (1972)”
to explain how well the user has enjoyed the movie.
Surprisingly, we can leverage mvec and uvec as an ex-
planation tool in the Recommender recommendation
making process.
2 Related Work
Detecting primary human emotion expression in
text is a relatively new area of research in Nat-
ural Language Processing (NLP). A common ap-
proach in identifying the general thought, feel-
ing, or sense in writing is to classify the con-
textual polarity orientation (positive, neutral, and
negative) of opinionated text through the polarity
Sentimental Analysis (SA) (Wilson et al., 2005), and
(Maas et al., 2011). When applying fine-grained Sen-
timent Analysis (Fink et al., 2011), researchers can
identify the intensity level of the polarity as a multi-
class single-label classification problem (e.g., very
positive, positive, neutral, negative, and very nega-
tive) (Bhowmick et al., 2009). However, to determine
the mental, emotional state or composure (i.e., mood)
in subjective text, Emotional Analysis (EA) can bet-
ter suit to handle the task (Tripathi et al., 2016). Here
the researcher wants to know the feeling of the
writing under examination is in one of the follow-
ing primary human emotions or moods. The study
of primary human emotional expressions started
in the era of Aristotle in around 4th century BC
(Konstan and Konstan, 2006). However, not until
Charles Darwin (1872 - 1998) revisited the inves-
tigation of human emotional expression in the 19th
century, which propelled the field to its present stage
of modern psychology research (Ekman, 2006). Paul
Ekman et alia in the 1970s developed a Facial Ac-
tion Coding System (FACS) to carry out a series
of research on facial expressions that have identi-
fied the following six primary universal human emo-
tions: happiness, sadness, disgust, fear, surprise,
and anger (Ekman, 1999). Ekman later added con-
tempt as the seventh primary human emotion to his
list (Ekman et al., 2013). Robert Plutchik invented
the Wheel of Emotions avocated eight primary emo-
tions: anger, anticipation, joy, trust, fear, surprise,
sadness, and disgust. Adding to the primary eight
emotions are secondary and complementary emo-
tions for a total of 32 emotions depicted on the ini-
tial Wheel of Emotions (Plutchik, 2001). More re-
cent research by Glasgow University in 2014 amened
that couple pairs of emotions such as fear and sur-
prise elicited similar facial muscles response, so are
disgust and anger. The study broke the raw hu-
man emotions down to four fundamental emotions:
happiness, sadness, fear/surprise, and disgust/anger
(Tayib and Jamaludin, 2016).
In our study, we shall focus our emotion detec-
tion work on Ekman’s six primary human emotions.
We follow other emotion detection researchers’ foot-
steps on the same decision, for many have based
their work on Ekman’s six primary human emotions
(Canales and Mart´ınez-Barco, 2014). Also, we can
make use of the WordNet-Affect, a linguistic re-
source for a lexical representation of affective knowl-
edge in affective computing on human interaction
such as attention, emotions, motivation, pleasure,
and entertainment (Valitutti et al., 2004). It is wor-
thy to note that emotional expression research usu-
ally aims at detecting and recognizing emotion types
from human facial expression and vocal intonation
(De Silva et al., 1997). However, our EDR study fo-
cuses on the mood of text expression instead. The
question remains how much of an emotion we can
convey through writing.
3 Methodology
Emotion Detection and Recognition on text is a text
classification problem, one of the favorite Natural
Language Processing and Supervised Machine Learn-
ing tasks (Danisman and Alpkocak, 2008). It is a Su-
pervised Machine Learning task because text clas-
sification requires a labeled dataset containing both
text documents and associated labels for training the
classifier (Medhat et al., 2014). In the absence of an
explicit emotion labeled movie metadata dataset, we
build an affective text aware model in two steps, first,
through a steadily available domain using tweets data
from the Twitter database. Next, we feed the movie
text metadata, such as storyline and overviews, to
the Emotion Detection and Recognition (EDR) model
built from tweets’ affective tags to classify the affec-
tive labels for the movie.
3.1 Affective Computing and Machine
Learning Modeling
Researchers apply Machine Learning (ML) method-
ologies to solve two classes of problems: regression
and classification (Stone and Veloso, 2000). For ex-
ample, to predict tomorrow, we use the stock market
index is moving up or down and by how much. It is
a regression problem, whereas, to state the polarity or
fine-grained sentiment of stock market traders regard-
ing the market performance to come tomorrow, it is a
classification problem. In the ML classification, the
polarity SA is a multi-class single-label classification
problem (Nakov et al., 2019). However, human emo-
tion usually expresses in a combination of affective
moods, such a classification task, a multi-class multi-
label classification problem (Li and Ren, 2012). To
simplify our study, we treat the movie metadata text-
based emotion detection as a multi-class single-label
classification problem instead. Our text-based EDR
model’s final step output seven nodes each represents
the probability distribution of an affective feature. We
then take the argmax (Gould et al., 2016) from the
probability distribution of the candidate nodes as the
prediction mood value, as depicted in equation 1.
argmax
x∈Df(x) = {x|f(x)≥f(y),∀y∈D}(1)
The argmax function expresses xas a set of data
points for which f(x)obtains the most significant val-
ues, if present, of the function.
3.2 Data Preperation
The challenge for our study in emotion detection
from movie text-based metadata is to obtain a large
enough movie metadata set with mood labels. No
such dataset is readily available. We, therefore,
need to build the required dataset by deriving it
from four different sources. For the movie rat-
ing datasets, we obtained the datasets from the
MovieLens datasets stored in the GroupLens repos-
itory (Harper and Konstan, 2016). We scraped The
Movie Database (TMDb) (TMDb, 2018) for movie
overviews and other metadata. We derived our emo-
tional word sense set as contextual emotional words
synonymous from WordNet (Miller, 1995). Finally,
we scraped the Twitter database for tweets with key-
word tags that matched our contextual emotion word
synonymous (Marres and Weltevrede, 2013). Movie-
Lens contains a “links” file that provides with cross-
reference links between MovieLens’ movie id and
TMDb’s tmdb id. We connect MovieLens and TMDb
datasets through the “links” file.
3.2.1 Extract emotion synonymous from
WordNetAffect EmotionLists
WordNet developed an affective knowledge linguis-
tic resource known as WordNet-Affect for lexical
representation (Strapparava et al., 2004). The se-
lection and tagging of a subset of synsets convey
the emotional meaning of a word in WordNet-
Affect. WordNet-Affect emotion lists contain
lists of concepts extracted from WordNet-Affect,
synsets with six emotions of interest: anger, disgust,
hate, joy, sadness, and surprise stored in a com-
pressed file: “WordNetAffectEmotionLists.tar.gz”
(Poria et al., 2012). We downloaded the “.gz” file
and uncompressed it into six emotion text files.
The alternative is to download from GitHub the
already uncompressed of the six emotion files from
https://github.com/robert-jm/twit-ranker/tree/master/dictionaries/WordNetAffectEmotionLists.
Each emotion file contains two columns of infor-
mation: synsets and the synonymous. Here, the
synonymous set of the synset corresponds to an
emotion class and store in the corresponding emo-
tion text file. We extract the synonymous column
from each emotion text file. We removed duplicate
synonymous, sorted the cleansed synonymous, and
stored the result in comma-separated values (CSV)
format in the corresponding emotion synonymous
file. Below in table 2 contains the statistic of the six
emotion synonymous files after performed the data
cleansing task.
Table 2: Synonymous statistic of six emotion.
mood count synonymous list
anger 255 ”abhor”,...,”wrothful”
disgust 53 ”abhorrent”,...,”yucky”
hate 147 ”affright”,...,”unsure”
joy 400 ”admirable”,...,”zestfulness”
sadness 202 ”aggrieve”,...,”wretched”
surprise 71 ”admiration”,...,”wondrously”
3.2.2 Extracting tweets from Twitter database
There are many types of tweets on Twitter, a popu-
lar social network, and the microblogging platform.
In this study, we only work with the regular tweet,
140 characters, or less short message, which posts on
Twitter. Almost every user’s tweets are extractable
and available to the public. Each tweet is search-
able by keyword. We wrote a simple Python script to
extract tweets through Twitter’s API (Makice, 2009).
We treat each synonymous in an emotion correspond-
ing file as a keyword of a tweet. By looping through
all the synonymous in Twitter’s search-by-keyword
API, we extract all the tweets with such keyword and
store them in a CSV file. The alternative is to extract
tweets and store them in a JSON file, as illustrated
in (Makice, 2009). For example, if the emotion syn-
onymous is belonging to the anger concept, we will
store the retrieved tweet in anger raw.csv file. As
depicted in table 2, the anger emotion corresponding
file, anger syn.txt, has 255 synonymous, we will store
all tweets retrieved from the corresponding keywords
in anger raw.csv.
We performed text pre-processing cleansing steps,
which involved removing any punctuation, HTML
tags, limit the set of special characters such as “,
′′, “.
′′,
or “#′′, stopwords removal, duplicate removal, stem-
ming the word phrases, and stored the cleansed. Our
effort yield the following gathered affective feature
records illustrated in the table 3.
Table 3: Mood datasets gathered from Twitter.
mood class record size
neutral 19108
joy 138019
sadness 60381
hate 38651
anger 17830
disgust 19887
surprise 15002
unbalance 7 mood classes 308878
each balanced mood 15000
each balanced mood train 12000
each balanced mood test 3000
Our mood dataset extracted from Twitter shows an
unbalance dataset with “joy” emotion class occupied
slightly over a third of the dataset. “sadness” emotion
class is half the amount of “joy” while “hate” is half
that of “sadness”. These three emotion classes, when
combined, represent 77% of the emotion dataset. The
distribution of the other four emotions ranges from
15,002 to 19,887, more or less evenly distributed.
If we apply the dataset directly to Machine Learn-
ing modeling without adjustment for the imbalance
classes, we will skew our result toward the dominant
mood types. We decide to balance the mood dataset
by subsampling each affective attribute dataset size to
15,000. We further split each affective dataset into
a training dataset with 80% of the samples (12,000)
and 20% of the samples for the test dataset (3,000).
Using a brute force method, we scrape the TMDb
database for movie metadata, particularly for movie
overview or storyline, which contains the subjective
writing movie description that we can classify the
mood of the text. Knowing we can query the TMDb
database by tmdb id, a unique movie identifier as-
signed to a movie. The tmdb id starts from 1 and
up. However, in the sequence of tmdb id, there may
be a gap between consecutive numbers. Our effort
yields 452,102 records after cleansing the raw data
we scraped from TMDb.
3.3 Emotion Modeling
Inspired by the recent advance in Natural Language
Processing (NLP) for text classification described by
(Sosa, 2017), we develop our text-based EDR model
by combining Long Short-Term Memory (LSTM),
a variant of Recurrent Neural Network (RNN), and
Conv-1D of Convolutional Neural Network (CNN).
We build our model similar to the method used in
(Liu, 2020). We define our model architecture con-
sists of two half. The first half is RNN LSTM-CNN
Conv-1D architecture, as described in (Sosa, 2017)
that text input process by an LSTM architecture be-
fore follow up data processing by a CNN Conv-1D ar-
chitecture. In contrast, the second half of the model is
to reverse the processing order of architecture, CNN
Conv-1D-RNN LSTM. Input first process by a CNN
Conv-1D architecture before feeding it to an RNN
LSTM architecture. The two half of the architecture
then combine to feed data into a max-pooling layer
of a CNN for a pooling operation to select the domi-
nant feature in the filter’s regional feature map. Next,
data passes into a flattening layer of a CNN to convert
data into a one-dimensional array before feed data to
a fully connected dense layer of a CNN. The dense
layer’s output will feed to a set of nodes that are equal
to the number of classes the architecture aims to clas-
sify. Each of the output nodes holds the output dis-
tribution value of its class. In the final act, a softmax
activation function examine and activate the appropri-
ate class node accordingly.
We use bi-directional RNN LSTM and CNN Conv1D
architectures to build our model. In the first half of the
model, the RNN LSTM-CNN Conv-1D phase, we use
two bi-directional LSTM for the RNN LSTM archi-
tecture and seven Conv1D of the CNN architecture. In
the second half of the model, the CNN-LSTM phase,
we apply seven pairs of Conv1D of the CNN architec-
ture and two bi-directional LSTM for the LSTM ar-
chitecture. Follow the idea illustrated in (Kim, 2014);
we prepared two identical input layers of embedding
matrix constructed from a pre-trained GloVe embed-
ding matric similar to (Pennington et al., 2014). We
build the two input layers of the embedding matrix
with one of the input embedding layers set to “train-
able”, while the other is not, i.e. “frozen”. Dur-
ing the processing of the first half of the model, the
RNN LSTM-CNN Conv-1D phase, the “trainable” in-
put layer occupies one of the bi-directional LSTM ar-
chitecture. In contrast, the “frozen” input layer fills
the other.
Similarly, when processing the second half of the
model, the CNN Conv-1D - RNN LSTM phase, a
“trainable” input layer, and a “frozen” input layer will
occupy each pair of the Conv1D units. We obtain
55.6% accuracy in classifying the emotion class of
the tweets’ balanced dataset, as depicted in table 4
and the confusion matrix in figure 1 depicts the per-
formance of the seven emotion classifier. The classi-
fication result is acceptable to serve our purpose since
our goal is not to build the best emotion text classifier,
but a usable one to classify the emotion class of movie
overviews.
Table 4: EDR performance on balanced mood tweets
dataset.
precision recall f1-score support
neutral 0.47 0.77 0.59 2992
joy 0.63 0.53 0.58 3030
sadness 0.64 0.44 0.52 3034
hate 0.64 0.51 0.57 2933
anger 0.62 0.68 0.65 2984
disgust 0.44 0.45 0.44 2987
surprise 0.55 0.51 0.53 3040
accuracy 0.56 21000
macro avg 0.57 0.56 0.55 21000
weighted
avg 0.57 0.56 0.55 21000
Figure 1: Confusion matrix of 7 emotion balanced dataset.
For comparison purposes, we include the ERD perfor-
mance result on unbalanced mood tweets data listed
in table 5. We also depicted the confusion matrix of
the seven emotion unbalanced tweets dataset in table
2.
Figure 2: Confusion Matrix of 7 emotion unbalanced
dataset.
Although the result showed much better performance
metrics when compared with its counterpart, we de-
cide not to use the ERD model trained by the un-
balanced dataset because the dataset is heavily fa-
vor “joy”, “sadness”, and “hate” with F1-score 78%,
61%, and 58% respectively. The F1-score for the
other four moods, except for “anger”, stand at 56%,
“disgust”, “surprise”, and “neutral” are 30%, 36%,
and 41%, respectively. Besides, when it comes to
evaluating the performance result involving unbal-
anced data, the macro-average F1-score is a more
suitable metric to gauge. Here, the macro-average
F1-score for balanced and unbalanced datasets is 55%
versus 0.51%, respectively.
Table 5: EDR performance on unbalanced mood tweets
dataset.
precision recall f1-score support
neutral 0.34 0.50 0.41 3758
joy 0.73 0.83 0.78 27719
sadness 0.57 0.66 0.61 12126
hate 0.68 0.51 0.58 7710
anger 0.74 0.46 0.56 3518
disgust 0.49 0.22 0.30 3972
surprise 0.70 0.24 0.36 2972
accuracy 0.65 61775
macro avg 0.61 0.49 0.51 61775
weighted
avg 0.65 0.65 0.64 61775
Let us revisit the example we gave in the mood anal-
ysis of the most top movie of all time rated by IMDb,
“The Godfather (1972)”. We built another version
of our emotion detection classifier model using the
unbalanced mood dataset. The classifier classified
“hate” is the most dominant mood feature for the
movie, and the mvec for the film with the model built
from the unbalanced dataset depicts in 6. The mvec
depicts the mood attributes descending order of the
movie is “hate”, “anger”, “joy”, “sadness”, “disgust,
“neutral”, and “surprise”. Comparing to table 1 which
shows the mood attributes descending order as “hate”,
“anger”, “disgust”, “surprise”, “sadness”, “neutral”,
and “joy”. Any person who has watched “The God-
father (1972)” movie would probably favor the mood
analysis result done by the emotion detection classi-
fier model using the balanced mood dataset.
Table 6: Affect values of movie: “The Godfather (1972)”
derived from unbalanced mood dataset.
neutral
0.04276474
joy sadness hate
0.16501102 0.076094896 0.4305178
anger disgust surprise
0.1993026 0.053966276 0.03234269
3.4 Emotion Preduction
We build a seven text-based emotion predictor for
movie overviews from the seven emotion tweet clas-
sifier model. We run the predictor through all the
452,102 overviews scraped from the TMDb database
to generate a TMDb movie overview with an emo-
tion label dataset. As mentioned before, Movie-
Lens datasets come in different sizes. We will
work with the following MovieLens datasets: ml-1m,
which contains about one million rating information
of movies; ml-20m dataset, 20 million rating infor-
mation; ml-latest-small dataset, about ten thousand
rating information of 610 users; ml-latest-full dataset,
holds 27 million rating information; and the recently
leased ml-25m dataset, with 25 million rating infor-
mation. (Note the name of the MovieLens dataset
coveys the number of ratings, movies, users, and tags
contained in the dataset.) Table 7 depicts the num-
ber of ratings, users, and movies; each of the Movie-
Lens datasets contain. In each of the depicted Movie-
Lens dataset, it provides a links file to cross-reference
between MovieLens and two other movie databases,
TMDb and IMDb, through movie id, tmdb id, and
imdb id. MovieLens maintains a small number of
data fields, but users can link to TMDb and IMDb
databases via the links file to access other metadata
that MovieLens is lacking.
Table 7: MovieLens datasets.
dataset ratings users movies
ml-1m 1M 6000 4000
ml-20m 20M 138000 27000
ml-25 25M 162000 62000
ml-latest-small 100K 600 9000
ml-latest-full 27M 280K 58000
The ml-latest-full datasets maintain the most signifi-
cant number of movies in MovieLens dataset collec-
tion. However, the ml-latest dataset will change over
time and is not a proper use for reporting research re-
sults. We use the ml-latest-small, and ml-latest-full
datasets in proof of concept and prototyping, not re-
search reporting work. The other MovieLens 1M,
20M, and 25M datasets are stable benchmark datasets
which we will use for research reporting work.
Although we have scraped 452,102 movie overviews
from TMDb when merging with MovieLens, we only
make use of one-eighth of the number of overviews
we have collected. Showing in table 8 is the number
of movie overviews the MovieLens datasets can ex-
tract from TMDb after performing the data cleaning
task.
Table 8: Number of overview in MovieLens extracted from
TMDb.
dataset number of overviews
ml-1m 1M
ml-20m 26603
ml-25m 25M
ml-latest-small 9625
ml-latest-full 56314
4 Implementation
4.1 Uvec and Mvec
For our study, we need a movie Recommender Sys-
tem to evaluate the performance of uvec and mvec.
We envision uvec and mvec play a role in the tail
end of making movie Recommendations, i.e., during
the stage of the top-N movie recommendation making
process. Any movie Recommenders can fit to sup-
port the evaluation of uvec and mvec. In our case, we
develop a Collaborative Filtering-based movie Rec-
ommender System (CFRS) based on the SVD al-
gorithm. We then added functions to support uvec
and mvec operations in the enhancement of making
movie recommendations. We start by adding func-
tions in the SVD-CFRS to support the loading, stor-
ing, extraction, and manipulation operations of uvec
and mvec. The added uvec and mvec support func-
tions do not interfere with any normal RS operation,
including the ordinary making movie recommenda-
tions process. We let our SVD-CFRS run in its usual
way serving routine recommendation requests: no
uvec and mvec involvement in the movie recommen-
dation making just yet. Before we start to evaluate
uvec and mvec, we prepare each user has the uvec
by computing the average of all mvec of the movies
the user has watched. We deployed the MovieLens
ml-latest-small dataset to be the test set and ran-
domly pick a user, user id 400, as the active test user.
Our active test user’s uvec depicted in table 9 repre-
senting the overall average of 43 movies’ mvec the
user id 400 has watched. We ask the Recommender
to make movie recommendations for the user id 400
in a business-as-usual way. The Recommender fur-
nishes a top-N, where N is 20, movie recommenda-
tion list for the user id 400 as depicted in table 11.
Using the uvec of the user id 400, we compute the
pairwise similarity between the user id 400 and each
recommended movie’s mvec on the top-N list. We
sorted the pairwise distance metrics top-N list com-
puted using uvec in the descending order to match up
the presentation ordering of the Recommender’s top-
N list. We applied five different distance metrics for
computing the uvec’s pairwise distance metrics top-
N list. We depicted the comparative top-N results in
table 12. The five distance metrics we employed in
the comparison were Euclidean distance, Manhattan
distance, Minkowski distance, Cosine similarity, and
Pearson correlation with their formula illustrated in
equation 2 through 7.
Euclideandistance (x,y) = sn
∑
i=1
(xi−yi)2(2)
Manhattandistance (x,y) = ||x−y||1=
n
∑
i=1
|xi−yi|
(3)
Minkowskid istance (x,y) = n
∑
i=1
|xi−yi|p!1
p
(4)
Inner (x,y) = ∑
i
xiyi=hx,yi(5)
CosSim (x,y) = ∑ixiyi
q∑ix2
iqy2
i
=hx,yi
||x||||y|| (6)
PearCorr (x,y) = ∑i(x−¯x)(y−¯y)
q∑i(x−¯x)2q∑i(y−¯y)2
=hx−¯x,y−¯yi
||x−¯x||||y−¯y||
=CosSim (x−¯x,y−¯y)
(7)
where (x,y)are vectors x= (x1,x2,··· ,xn)and
y= (y1,y2,··· ,yn).
5 Evaluation
5.1 Findings
The following table 9 depicts the average mood value
uvec of the active test user id 400, while table 10
shows the weighted average mood value uvec of the
active test user.
Table 9: The average mood value of user id 400 user.
neutral
0.16352993
joy sadness hate
0.08873525 0.12708998 0.20331840
anger disgust surprise
0.11933819 0.15881287 0.13917538
Table 10: The weighted average mood value of user id 400
user.
neutral
0.14755723
joy sadness hate
0.08006808 0.11467653 0.18345939
anger disgust surprise
0.10768190 0.1433009 0.125581509
The following table 11 depicts the top-N movie rec-
ommendation list generated by our SVD-CF Recom-
mender for the active test user id 400.
Table 11: TopN recommendation list generated by SVD CF
recommender for user id 400 user.
no. mid title
0 527 Schindler’s List (1993)
1 5952 Lord of the Rings:
The Two Towers,
The (2002)
2 2858 American Beauty (1999)
3 2329 American History X (1998)
4 2028 Saving Private Ryan (1998)
5 1089 Reservoir Dogs (1992)
6 110 Braveheart (1995)
7 1291 Indiana Jones and
the Last Crusade
(1989)
8 4226 Memento (2000)
9 91529 Dark Knight Rises,
The (2012)
10 68157 Inglourious Basterds
(2009)
11 6016 City of God
(Cidade de Deus)
(2002)
12 589 Terminator 2:
Judgment Day
(1991)
13 1704 Good Will Hunting
(1997)
14 1200 Aliens (1986)
15 1214 Alien (1979)
16 1 Toy Story (1995)
17 99114 Django Unchained
(2012)
18 7361 Eternal Sunshine of
The (2012)
19 1136 Monty Python and
the Holy Grail
(1975)
The following table 12 depicts the comparison of the
top-N recommendation list generated by the five dis-
tance metrics for user id 400 user.
The following table 13 depicts the comparison of the
top-N recommendation list generated by the five dis-
tance metrics for the active test user id 400 using the
weighted mood values for the uvec. To get the aver-
age weighted uvec for the active test user id 400, we
multiply the averaged user id 400 uvec with the av-
erage normalized rating value of the watched movie.
We depicted the average weighted uvec of user id 400
in table 10. Using the weighted uvec of user id 400,
we compute the five weighted distance metrics de-
Table 12: Comparison of the top-N recommendation list
generated by the five distance metrics for user id 400 user.
no. mid Euc Man Min Cos Pear
0 527 1291 1291 1291 91529 110
1 5952 4226 5952 4226 68157 2329
2 2858 5952 4226 7361 527 91529
3 2329 7361 1089 5952 2329 527
4 2028 1089 7361 1089 2858 2028
5 1089 1200 2028 1200 6016 68157
6 110 2028 589 1704 110 1214
7 1291 1704 1704 2028 1 1
8 4226 589 1200 589 99114 1200
9 91529 1214 1214 1214 1136 1089
10 68157 99114 1 1136 1214 6016
11 6016 110 1136 99114 2028 2858
12 589 1136 110 110 589 99114
13 1704 1 2329 1 1200 1136
14 1200 6016 527 2858 1704 1704
15 1214 2329 99114 6016 1089 589
16 1 527 6016 527 7361 5952
17 99114 2858 91529 2329 5952 7361
18 7361 68157 2858 68157 4226 1291
19 1136 91529 68157 91529 1291 4226
picted in table 13. Interestingly enough, the top-N
recommendation list is identical whether the results
generated through Cosine similarity or Pearson Cor-
relation using weighted or non-weighted uvec. How-
ever, the other three metrics slightly differed in the
ranking of top-N recommendation using the weighted
or non-weighted uvec. Cosine similarity seems to act
like an invert ranking of its top-N list when compared
to Euclidean, Manhattan, Minkowski distance met-
rics.
Table 13: Comparison of the top-N recommendation list
generated by the five distance metrics for user id 400 user
using weighted uvec.
no. mid wEuc wMan wMin wCos wPear
0 527 1291 1291 1291 91529 110
1 5952 4226 5952 4226 68157 2329
2 2858 5952 7361 1200 527 91529
3 2329 7361 4226 1089 2329 527
4 2028 1089 1089 5952 2858 2028
5 1089 2028 589 7361 6016 68157
6 110 1200 2028 2028 110 1214
7 1291 1704 1704 1704 1 1
8 4226 589 1214 589 99114 1200
9 91529 1214 1200 1214 1136 1089
10 68157 110 1136 110 1214 6016
11 6016 99114 1 99114 2028 2858
12 589 1 110 1 589 99114
13 1704 1136 99114 1136 1200 1136
14 1200 2329 2329 527 1704 1704
15 1214 527 527 2329 1089 589
16 1 6016 6016 6016 7361 5952
17 99114 2858 91529 68157 5952 7361
18 7361 91529 2858 2858 4226 1291
19 1136 68157 68157 91529 1291 4226
5.2 Comparison of Top-N made by
Distance Metrics
We combined the following rating datasets, except the
ml-latest-small from the MovieLens into one dataset
by merging ml-1m, ml-20m, ml-25m, and ml-latest-
full datasets. We extracted all data points of user id
400 as the active test user under examination from the
combined rating dataset. We removed duplicated data
points based on the movie id and timestamp from the
combined rating dataset that matched the data points
in user id 400. The combined rating dataset em-
ploys here as movies to-be-watched by the active test
user id 400 in some future timeline. We employed
ml-latest-small as the input dataset for our SVD-CF
Recommender to get the top-N recommendation list
for our active test user id 400. The SVD-CF Recom-
mender is never aware of the combined user id 400
data points except those contained in the ml-latest-
small dataset. Once we excluded all the duplicated
data points found in ml-latest-small from the com-
bined rating datasets, we have 209 user id 400 data
points for validation work. We extracted data points
from the combined dataset, which matched the top-N
recommendation generated by the Recommender, as
depicted in Table 14. We put the extracted data points
into two groups: data points found in the top-N and
not found in the top-N. There are 8 data points in the
combined dataset sorted by timestamp in ascending
order to reflect the order of the movie the active user
has watched. That is a good sign because it shows the
SVD-CF Recommender works at a 40% supporting
rate, eight out of twenty recommendations coincided
with the preference of the active test user. If we draw a
cutoff point at top-5 across the comparison of the top-
N recommendation list, the following four movie id
data points in the combined dataset matched: 1291,
4226, 7361, and 2028. Cosine takes fifth place; it
only hit the 4th spot in the top-5 list. Pearson takes
the fourth place and hit the top second and fifth spots
on the top-5 list. Manhattan takes third place, hitting
the top 1, 3, and 5. Euclidean takes second place, hit-
ting the top 1, 2, and 4. Minkowski wins the round by
hitting the top 1, 2, and 3 spots.
6 Future Work
We plan to elaborate our affective computing study by
building an Emotion Aware Recommender using the
emotion labeled tags obtained from movie overviews
through the Tweets Affective Classifier. We also plan
Table 14: Common data points found in TopN recommen-
dation list and in active user user id 400.
no. movie id movie id
in topN not in topN
1 1 110
2 2329 527
3 1291 1089
4 589 1136
5 1704 1200
6 2028 1214
7 4226 2858
8 7361 5952
9 6016
10 68157
11 91529
12 99114
to make use of affective features in users’ emotion
profiles to enhance Group Recommender in group
formation, group dynamic, and group decision mak-
ing. In our study, we would like to find better metrics
to measure the performance of affective computing.
7 Conclusion
In this paper, we illustrate a strategy to generate af-
fective features for movies by transfer learning tech-
niques utilizing a different domain Emotion Detec-
tion and Recognition (EDR) model classifier. We
developed the EDR model to detect and recognize
seven emotional features in tweets through affective
tags stored in the Twitter database. We then transfer
the learning of the EDR model from classifying the
emotional features of tweets to predict the moods of
a movie through the movie description in the movie
overview. We scraped the TMDb database for movie
overviews and metadata. Through the EDR program,
we generate emotional features, mvec, for each col-
lected movie from TMDb. We gather movie datasets
that contain rating information from the MovieLens
repository. We use the rating dataset of MovieLens to
build an SVD-CF Recommender. We add functions
to support uvec and mvec in enhancing the Recom-
mender in generating the top-N movie recommenda-
tions. We randomly pick an active user, user id 400,
as the test candidate. We generate a top-N with N set
to 20 to generate movie recommendations through the
SVD-CF Recommender. We compute the uvec for the
user id 400 by average all the mvec of movies that the
testing candidate has watched. We calculate the Eu-
clidean, Manhattan, Minkowski, Cosine, and Pearson
distance metrics and compare the five distance met-
rics’ rankings against the Recommender top-N. We
found Minkowski distance metrics performed the best
at 60% support accuracy versus the 40% accuracy
made by the Recommender. A 20% improvement in
top-N movie recommendations. We also make the
following observations:
•Text-based NLP EDR modeling technique works
and can apply to solve a real-world problem where
the abundance of subjective writing is available.
•Text-based EDR model is transferable from one
domain to another, and all it requires is that the
target text is in subjective writing form.
REFERENCES
Beel, J., Gipp, B., Langer, S., and Breitinger, C. (2015).
Research-paper recommender systems: a literature
survey. International Journal on Digital Libraries,
pages 1–34.
Bhowmick, P. K., Basu, A., Mitra, P., and Prasad, A. (2009).
Multi-label text classification approach for sentence
level news emotion analysis. In International Con-
ference on Pattern Recognition and Machine Intelli-
gence, pages 261–266. Springer.
Bobadilla, J., Ortega, F., Hernando, A., and Guti´errez, A.
(2013). Recommender systems survey. Knowledge-
Based Systems, 46:109–132.
Canales, L. and Mart´ınez-Barco, P. (2014). Emotion de-
tection from text: A survey. In Proceedings of the
Workshop on Natural Language Processing in the 5th
Information Systems Research Working Days (JISIC),
pages 37–43.
Danisman, T. and Alpkocak, A. (2008). Feeler: Emotion
classification of text using vector space model. In
AISB 2008 Convention Communication, Interaction
and Social Intelligence, volume 1, page 53.
De Silva, L. C., Miyasato, T., and Nakatsu, R. (1997). Fa-
cial emotion recognition using multi-modal informa-
tion. In Proceedings of ICICS, 1997 International
Conference on Information, Communications and Sig-
nal Processing. Theme: Trends in Information Sys-
tems Engineering and Wireless Multimedia Commu-
nications (Cat., volume 1, pages 397–401. IEEE.
Ekman, P. (1999). Basic emotions. Handbook of cognition
and emotion, 98(45-60):16.
Ekman, P. (2006). Darwin and facial expression: A century
of research in review. Ishk.
Ekman, P., Friesen, W. V., and Ellsworth, P. (2013). Emo-
tion in the human face: Guidelines for research and
an integration of findings, volume 11. Elsevier.
Fink, C. R., Chou, D. S., Kopecky, J. J., and Llorens, A. J.
(2011). Coarse- and fine-grained sentiment analysis of
social media text. Johns hopkins apl technical digest,
30(1):22–30.
Gould, S., Fernando, B., Cherian, A., Anderson, P., Cruz,
R. S., and Guo, E. (2016). On differentiating pa-
rameterized argmin and argmax problems with ap-
plication to bi-level optimization. arXiv preprint
arXiv:1607.05447.
Harper, F. M. and Konstan, J. A. (2016). The movielens
datasets: History and context. ACM Transactions on
Interactive Intelligent Systems (TiiS), 5(4):19.
Ho, Ai Thanh, M. I. L. and Tagmouti, Y. (2006). E-
mrs: Emotion-based movie recommender system. In
Proceedings of IADIS e-Commerce Conference. USA:
University of Washington Both-ell, pages 1–8.
IMDb (2020). Top 100 greatest movies of all
time (the ultimate list). Published online at
http://www.imdb.com/list/ls055592025/. Accessed:
2020-05-17 16:40:27.
Jannach, D., Zanker, M., Felfernig, A., and Friedrich,
G. (2010). Recommender systems: an introduction.
Cambridge University Press.
Kim, Y. (2014). Convolutional neural networks for sentence
classification. arXiv preprint arXiv:1408.5882.
Kompan, M. and Bielikova, M. (2014). Group recommen-
dations: survey and perspectives. Computing and In-
formatics, 33(2):446–476.
Konstan, D. and Konstan, D. (2006). The emotions of the
ancient Greeks: Studies in Aristotle and classical lit-
erature, volume 5. University of Toronto Press.
Li, J. and Ren, F. (2012). Emotion recognition of we-
blog sentences based on an ensemble algorithm of
multi-label classification and word emotions. IEEJ
Transactions on Electronics, Information and Sys-
tems, 132(8):1362–1375.
Liu, T. (2020). Multi-class emotion classification for short
texts. Published online at https://tlkh.github.io. Ac-
cessed: 2020-05-18 16:11:30.
Maas, A. L., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y.,
and Potts, C. (2011). Learning word vectors for sen-
timent analysis. In Proceedings of the 49th Annual
Meeting of the Association for Computational Lin-
guistics: Human Language Technologies, pages 142–
150, Portland, Oregon, USA. Association for Compu-
tational Linguistics.
Makice, K. (2009). Twitter API: Up and running: Learn
how to build applications with the Twitter API. ”
O’Reilly Media, Inc.”.
Marres, N. and Weltevrede, E. (2013). Scraping the social?
issues in live social research. Journal of cultural econ-
omy, 6(3):313–335.
Medhat, W., Hassan, A., and Korashy, H. (2014). Sentiment
analysis algorithms and applications: A survey. Ain
Shams engineering journal, 5(4):1093–1113.
Miller, G. A. (1995). Wordnet: a lexical database for en-
glish. Communications of the ACM, 38(11):39–41.
Nakov, P., Ritter, A., Rosenthal, S., Sebastiani, F., and Stoy-
anov, V. (2019). Semeval-2016 task 4: Sentiment
analysis in twitter. arXiv preprint arXiv:1912.01973.
Naqvi, N., Shiv, B., and Bechara, A. (2006). The role
of emotion in decision making: A cognitive neuro-
science perspective. Current directions in psychologi-
cal science, 15(5):260–264.
Pennington, J., Socher, R., and Manning, C. D. (2014).
Glove: Global vectors for word representation. In
Proceedings of the 2014 conference on empirical
methods in natural language processing (EMNLP),
pages 1532–1543.
Plutchik, R. (2001). The nature of emotions: Human emo-
tions have deep evolutionary roots, a fact that may ex-
plain their complexity and provide tools for clinical
practice. American scientist, 89(4):344–350.
Poria, S., Gelbukh, A., Cambria, E., Yang, P., Hussain,
A., and Durrani, T. (2012). Merging senticnet and
wordnet-affect emotion lists for sentiment analysis. In
2012 IEEE 11th International Conference on Signal
Processing, volume 2, pages 1251–1255. IEEE.
Scheel, C., Castellanos, A., Lee, T., and De Luca, E. W.
(2012). The reason why: A survey of explanations
for recommender systems. In International Work-
shop on Adaptive Multimedia Retrieval, pages 67–84.
Springer.
Sosa, P. M. (2017). Twitter sentiment analysis using com-
bined lstm-cnn models. Eprint Arxiv.
Stone, P. and Veloso, M. (2000). Multiagent systems: A sur-
vey from a machine learning perspective. Autonomous
Robots, 8(3):345–383.
Strapparava, C., Valitutti, A., et al. (2004). Wordnet affect:
an affective extension of wordnet. In Lrec, volume 4,
page 40. Citeseer.
Tayib, S. and Jamaludin, Z. (2016). An algorithm to de-
fine emotions based on facial gestures as automated
input in survey instrument. Advanced Science Letters,
22(10):2889–2893.
TMDb (2018). Tmdb about. Published online
at www.themoviedb.org. Accessed: 2018-05-11
17:29:37.
Tripathi, V., Joshi, A., and Bhattacharyya, P. (2016). Emo-
tion analysis from text: A survey. Center for Indian
Language Technology Surveys.
Valitutti, A., Strapparava, C., and Stock, O. (2004). Devel-
oping affective lexical resources. PsychNology Jour-
nal, 2(1):61–83.
Wilson, T., Wiebe, J., and Hoffmann, P. (2005). Recogniz-
ing contextual polarity in phrase-level sentiment anal-
ysis. In Proceedings of human language technology
conference and conference on empirical methods in
natural language processing, pages 347–354.
Zhang, S., Yao, L., and Sun, A. (2017). Deep learning based
recommender system: A survey and new perspectives.
arXiv preprint arXiv:1707.07435.
Zhang, Z.-K., Zhou, T., and Zhang, Y.-C. (2011). Tag-aware
recommender systems: a state-of-the-art survey. Jour-
nal of computer science and technology, 26(5):767.
