Content uploaded by Ioannis Livieris
Author content
All content in this area was uploaded by Ioannis Livieris on Apr 02, 2020
Content may be subject to copyright.
TECHNICAL
RE P O R T
University of Patras
Department of Mathematics
GR-265 00 Patras, Greece.
http://www.math.upatras.gr/
Fundamental Research Questions and
Proposals on Predicting Cryptocurrency
Prices using DNNs
E. Pintelas1, I.E. Livieris1†, S. Stavroyiannis2,
T. Kotsilieris3, P. Pintelas1
NO. TR20-01
1University of Patras, Department of Mathematics, GR-265 00, Patras, Greece.
2Department of Accounting & Finance, University of the Peloponesse, GR 241 00,
Antikalamos, Greece.
3Department of Business Administration, University of the Peloponnese, GR 241 00,
Antikalamos, Greece.
†corresponding author
Part of this work has been submitted in Artificial Intelligence Application and Innova-
tions (AIAI) 2020.
TECHNICAL REPORT
NO. TR20-01
Fundamental Research Questions and Proposals on
Predicting Cryptocurrency Prices using DNNs
E. Pintelas, I.E. Livieris, S. Stavroyiannis, T. Kotsilieris and P. Pintelas
February 2020
Abstract. In last decade, cryptocurrency has emerged in financial area as a key factor in
businesses and financial market opportunities. Accurate predictions can assist cryptocur-
rency investors towards right investing decisions and lead to potential increased profits.
Additionally, they can also support policy makers and financial researchers in studying
cryptocurrency markets behavior. Nevertheless, cryptocurrency price prediction is consid-
ered a very challenging task, due to its chaotic and very complex nature. In this study we
investigate three major research questions: i) Can deep learning efficiently predict cryp-
tocurrency prices? ii) Are cryptocurrency prices a random walk process? iii) Is there a
proper validation method of cryptocurrency price prediction models? To this end, we eval-
uate some of the most successful and widely used in bibliography deep learning algorithms
forecasting cryptocurrency prices. The results obtained, provide significant evidence that
deep learning models are not able to solve this problem efficiently and effectively. Follow-
ing detailed experimentation and results analysis, we conclude that it is essential to invent
and incorporate new techniques, strategies and alternative approaches such as: more sophis-
ticated prediction algorithms, advanced ensemble methods, feature engineering techniques
and other validation metrics.
keywords. Deep learning; CNN; LSTM; BiLSTM; Cryptocurrency price prediction; time
series.
PAGE 2 NO. TR20-01
Contents
1 Introduction 3
2 Brief description of advanced deep learning models 4
3 Experimental methodology 5
3.1 Description of the dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.2 Validation metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.3 Experimental results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
4 Discussion 13
4.1 Can deep learning efficiently predict cryptocurrency prices? ............ 13
4.2 Are cryptocurrency prices a random walk process? . . . . . . . . . . . . . . . . . 14
4.3 What is a proper validation method of cryptocurrency price prediction models? . . 15
5 Revisiting the problem 16
6 Conclusions 17
NO. TR20-01 PAGE 3
1 Introduction
Cryptocurrency is a new type of digital currency which utilizes blockchain technology and crypto-
graphic functions to gain transparency, decentralization and immutability [27,28]. Bitcoin (BTC)
is considered the first and the most popular cryptocurrency, which was invented by an anonymous
group or person in 2009. Since then, 4000 alternative cryptocurrencies like Etherium (ETH) and
Ripple (XRP) were created proving that the cryptocurrency market has emerged in financial area.
BTC, ETH and XRP are the three most popular cryptocurrencies, since they almost hold the 79.5%
of the global cryptocurrency market capitalization.
Cryptocurrency price prediction can provide a lending hand to cryptocurrency investors for
making proper investment decisions in order to acquire higher profits while it can also support
policy decision-making and financial researchers for studying cryptocurrency markets behavior.
Cryptocurrency price prediction can be considered as a common type of time series problems, like
the stock price prediction. Traditional time series methods such as the well-known AutoRegressive
Integrated Moving Average (ARIMA) model, have been applied for cryptocurrencies price and
movement prediction [5,8,29,36,38]. However, these models are not able to capture non-linear
patterns of very complicated prediction problems in contrast to Deep Learning algorithms which
achieve greater performance on forecasting time series problems [17,34].
Deep Learning algorithms (DL) are powerful machine learning algorithms which specialize in
solving nonlinear and complex problems exploiting most of the times big amounts of data in order
to become efficient predictor models. The accurate cryptocurrency price prediction is by nature
a significantly challenging and complex problem since its values have very big fluctuations over
time following an almost chaotic and unpredictable behavior. Therefore, deep learning techniques
may constitute the proper methodology to solve this problem.
Recent research efforts have adopted deep learning techniques for predicting cryptocurrency
price. Ji et al. [16] conducted a comparison of state-of-the-art deep neural networks such as Long
Short-Term Memory (LSTM), Deep Neural Networks (DNNs), deep residual network and their
combinations for predicting Bitcoin price. Their results demonstrated slightly better accuracy of
LSTM compared to other models for regression problem while DNNs outperformed all models
on price movement prediction. Shintate and Pichl [33] developed a trend prediction classifica-
tion framework for predicting non-stationary cryptocurrency time series utilizing deep learning.
Their results revealed that their proposed model outperformed LSTM baseline model while the
profitability analysis showed that simple buy-and-hold strategy was superior to their model and
thus it cannot yet be used for algorithmic trading. Lahmiri and Bekiros [21] utilized LSTM and
generalized regression neural networks for cryptocurrency price prediction. Their results showed
that LSTM was superior to the generalized regression neural architecture concluding that deep
learning is a very efficient method in predicting the inherent chaotic dynamics of cryptocurrency
prices. Amjad and Shah [4] used live streaming Bitcoin data for predicting price changes (in-
crease, decrease or no-change), building a model based on the most confident predictions, in order
to perform profitable trades. The classification algorithms which they used were Random Forest,
Logistic Regression and Linear Discriminant Analysis. Their results seem to be very impressive
PAGE 4 NO. TR20-01
since they achieved a high prediction accuracy (>60−70%) and about 5.33xaverage return on
investments on a test set.
In this work, we evaluatethe performance of advanced deep learning algorithms for predicting
the price and movement of the three most popular cryptocurrencies (BTC, ETH and XRP). The
main contribution of this research lies in investigating three major questions:
i) Can deep learning efficiently predict cryptocurrency prices?
ii) Are cryptocurrency prices a random walk process?
iii) Is there a proper validation method of cryptocurrency price prediction models?
Furthermore, it also lies in the recommendation for new algorithms and alternative approaches for
the cryptocurrency prediction problem.
The remainder of this research is organized as follows: Section 2performs a brief introduc-
tion to the advanced deep learning models utilized in our experiments. Section 3presents our
research methodology and experimental results. Section 4discusses and answers the three re-
search questions, while Section 5presents our suggestions on possible alternative solutions for the
cryptocurrency prediction problem. Finally, Section 6presents our concluding remarks.
2 Brief description of advanced deep learning models
Deep learning algorithms constitute one of the most powerful machine learning algorithms cat-
egories which have been successfully applied on a multitude of commercial applications. Long
Short Term Memory and Convolutional Neural Networks are probably the most popular, success-
ful and widely used deep learning techniques [7,11,22].
Long Short-Term Memory (LSTM) [12,13] constitute a special type of deep neural networks,
which are able to learn long-term dependencies by utilizing feedback connections in order to “re-
member” past network cell states. These networks have become very popular since they have been
successfully applied on a wide range of applications and have shown remarkable performance on
time series forecasting [10,18,19,25,31]. More specifically, LSTM networks are composed by a
memory cell, an input, output and forget gate. The input gate controls the new stored information
into the memory cell, while the forget gate controls the information which must be vanished. Fi-
nally, the output gate controls the final output information value which is given after a delay into
the forget, input gate utilizing a feedback connection loop. In this way, LSTM is able to create
a controlled information flow filtering unnecessary information and thus achieving to learn long
term dependencies.
Bidirectional Long Short-Term Memory (BiLSTM) [12,32] are a special type of recurrent
neural networks which connect two LSTM layers of converse directions to the same output, in
order to remember past and future network cell states. The principle idea is that each training
sequence is presented forwards and backwards into two separate LSTM layers aiming in accessing
both past and future contexts for a given time. More specifically, the first hidden layer possesses
NO. TR20-01 PAGE 5
recurrent connections from the past time steps; while in the second one, the recurrent connections
are reversed, transferring activation backwards along the sequence.
Convolutional Neural Networks (CNN) [1,3,20] constitute another type of deep neural net-
works which utilize convolution and pooling layers in order to filter the raw input data and extract
valuable features, which will feed a fully connected layer in order to produce the final output. More
specifically, they apply convolution operations in the input data and in order to produce new more
useful features. The convolutional layers are usually followed by a pooling layer which extracts
values from the convolved features producing a lower dimension instance. In fact, a pooling layer
produces new features which can be considered as summarized versions of the convolved features
produced by the convolutional layer. This implies that pooling operations can significantly assist
the network to be more robust since small changes of the inputs, which are usually detected by the
convolutional layers, will become approximately invariant.
3 Experimental methodology
In this work, we evaluate the performance of advanced DL models for predicting the price of BTC,
ETH and XRP. The evaluated DL models are constituted by CNN, LSTM, BiLSTM and dense
layers. Table 1depicts our DL models for the best identified topologies. We have to mention that
exhaustive and thorough experiments were performed in order to identify the DL topologies which
incur the best performance results.
We recall that the basic idea of utilizing LSTM and BiLSTM on cryptocurrency price prediction
problems, is that they might be able to capture useful long or short sequence pattern dependencies,
due to their special architecture design, assisting on prediction performance, while the convolu-
tional layers of a CNN model might filter out the noise of the raw input data and extract valuable
features producing a less complicated dataset which would be more useful for the final prediction
model [23]. Therefore, we expect that a noticeable performance increase will be achieved by the
incorporation of these advanced models comparing to classic machine learning algorithms.
Additionally, the performance of the DL models was compared against that of traditional state-
of-the-art ML models: Support Vector Regressor (SVR) [9], 3-Nearest Neighbors (3NN) [2] and
Decision Tree Regressor (DTR) [24]. The implementation code was written in Python 3.4 while for
all deep learning models we utilized Keras library [14] and Theano as back-end while Scikit-learn
library [30] was used for the machine learning models.
3.1 Description of the dataset
For the purpose of this research, we utilized data from Jan-2018 to Aug-2019, concerning the
hourly prices in USD and were divided into training set consisting of data from Jan-2018 to Feb-
2019 (10176 values) and testing set from Mar-2019 to Aug-2019 (4416 values).
The data were obtained from www.kraken.com website, which is a trading platform for cryp-
tocurrency exchanges. Also, we utilized four different values for the forecasting horizon (number
PAGE 6 NO. TR20-01
of past prices taken into consideration), i.e., 4, 9, 12 and 16 hours. From a forecasting aspect, the
value of the forecasting horizon Fis considered essential crucial for the prediction accuracy of an
intelligent model. The forecasting horizon stands for the number of hourly prices which are taken
into consideration by a forecasting model for predicting the next price. More specifically, in case
the forecasting horizon is equal to 12 means that the model takes into account prices collected from
12 past hours and for predicting the price on the next hour.
Model Description
LSTM1LSTM layer with 50 units.
Output layer of 1 neuron.
LSTM2Two LSTM layers with 30 and 15 units, respectively.
Output layer of 1 neuron.
BiLSTM1BiLSTM layer with 60 units.
Output layer of 1 neuron.
BiLSTM2TwoBiLSTM layers with 40 and 20 units, respectively.
Output layer of 1 neuron.
CNN-LSTM1Convolutional layer with 64 of filters of size (2, ).
Convolutional layer with 128 of filters of size (2, ).
Max pooling layer with size (2, ).
LSTM layer with 100 units.
Output layer of 1 neuron.
CNN-LSTM2Convolutional layer with 64 of filters of size (2, ).
Convolutional layer with 128 of filters of size (2, ).
Max pooling layer with size (2, ).
LSTM layer with 70 units.
Dense layer with 16 neurons.
Output layer of 1 neuron.
CNN-BiLSTM1Convolutional layer with 64 of filters of size (2, ).
Convolutional layer with 128 of filters of size (2, ).
Max pooling layer with size (2, ).
LSTM layer with 100 units.
CNN-BiLSTM2Convolutional layer with 64 of filters of size (2, ).
Convolutional layer with 128 of filters of size (2, ).
Max pooling layer with size (2, ).
BiLSTM layer with 70 units.
Dense layer with 16 neurons.
Table 1: Best identified topologies for our deep learning models
3.2 Validation metrics
The most common validation metrics for measuring the performance of regression problems are
Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) [6]. Since the cryptocurrency
price prediction problem can be considered a regression problem, in our experiments we utilized
these two evaluation metrics. Moreover, by comparing the predicted prices of our models, with
the real ones, we managed to compute the classification accuracy of price movement direction pre-
NO. TR20-01 PAGE 7
diction (if the price will increase or decrease). Therefore, we utilized two additional performance
metrics: Accuracy (Acc) and F1-score (F1).
Finally, in order to check the reliability of our prediction models, we have utilized a test of
autocorrelation in the residuals [26,37]. This test examines the presence of autocorrelation be-
tween the residuals (differences between predicted and real values). In case autocorrelation exists,
then the prediction model may be inefficient since it did not manage to capture all the possible
information which lies into the data [26].
3.3 Experimental results
Tables 2,3,4and 5present the experimental results of our DL models ML models for forecasting
horizon 4, 9, 12 and 16, respectively. CNN-LSTM and CNN-BiLSTM models exhibited the best
overall performance among all prediction models. In particular, the CNN-LSTM exhibited the
highest RMSE performance for all datasets for every forecasting horizon comparing to other DL
models, while the CNN-BiLSTM exhibited the best Acc and F1score in most cases. Nevertheless,
the performance variations for all DL models seem to be minimal. The 3NN model reported
the highest RMSE performance for forecasting horizon 4 among all the ML models on BTC and
ETH datasets, while the DTR exhibited the highest RMSE on XRP. For forecasting horizon 9 the
DTR outperformed all ML models for all dataset, regarding to RMSE score. Furthermore, the
3NN model managed to achieve the best overall performance in Acc score almost in all cases. In
summary, advanced DL models seem to slightly outperform ML models while they did not manage
to achieve a noticeable performance increase comparing to our ML models.
BTC ETH XRP
Model MAE RMSE Acc F1MAE RMSE Acc F1MAE RMSE Acc F1
SVR 0.0178 0.0209 50.80% 0.484 0.0131 0.0163 49.14% 0.482 0.0323 0.0370 47.67% 0.644
3NN 0.0102 0.0161 49.51% 0.469 0.0106 0.0149 51.56% 0.481 0.0073 0.0115 52.33% 0.492
DTR 0.0119 0.0170 49.22% 0.465 0.0127 0.0199 50.11% 0.467 0.0099 0.0157 50.14% 0.504
LSTM10.0089 0.0138 52.86% 0.524 0.0086 0.0130 53.59% 0.531 0.0065 0.0106 53.40% 0.509
LSTM20.0092 0.0144 53.63% 0.522 0.0117 0.0165 52.68% 0.501 0.0072 0.0116 53.13% 0.514
BiLSTM10.0096 0.0142 52.05% 0.508 0.0084 0.0130 53.89% 0.538 0.0069 0.0114 53.21% 0.519
BiLSTM20.0090 0.0132 52.56% 0.519 0.0090 0.0134 53.30% 0.502 0.0074 0.0118 52.75% 0.518
CNN-LSTM10.0072 0.0109 53.21% 0.530 0.0066 0.0106 53.77% 0.533 0.0059 0.0097 52.42% 0.472
CNN-LSTM20.0061 0.0099 52.50% 0.502 0.0067 0.0106 54.20% 0.524 0.0061 0.0097 53.03% 0.436
CNN-BiLSTM10.0064 0.0107 54.51% 0.544 0.0078 0.0121 53.91% 0.524 0.0076 0.0119 53.61% 0.534
CNN-BiLSTM20.0060 0.0101 55.43% 0.548 0.0072 0.0115 54.18% 0.541 0.0076 0.0117 53.46% 0.506
Table 2: Performance of DL and ML forecasting models for F=4
PAGE 8 NO. TR20-01
BTC ETH XRP
Model MAE RMSE Acc F1MAE RMSE Acc F1MAE RMSE Acc F1
SVR 0.0152 0.0192 52.57% 0.546 0.0101 0.0146 49.30% 0.501 0.0214 0.0292 47.92% 0.457
3NN 0.0139 0.0197 51.57% 0.484 0.0144 0.0195 51.50% 0.504 0.0088 0.0132 54.35% 0.485
DTR 0.0134 0.0179 49.54% 0.459 0.0148 0.0228 49.64% 0.465 0.0100 0.0161 49.89% 0.504
LSTM10.0108 0.0159 51.99% 0.479 0.0113 0.0158 53.06% 0.500 0.0083 0.0117 51.34% 0.413
LSTM20.0141 0.0207 52.45% 0.499 0.0153 0.0208 52.75% 0.527 0.0077 0.0115 51.66% 0.456
BiLSTM10.0117 0.0170 52.00% 0.489 0.0111 0.0165 53.31% 0.512 0.0078 0.0126 52.73% 0.517
BiLSTM20.0098 0.0168 52.97% 0.530 0.0109 0.0166 53.80% 0.527 0.0074 0.0121 55.22% 0.534
CNN-LSTM10.0074 0.0119 53.92% 0.536 0.0086 0.0130 53.92% 0.530 0.0058 0.0096 51.06% 0.453
CNN-LSTM20.0065 0.0107 54.20% 0.532 0.0081 0.0124 54.45% 0.537 0.0065 0.0100 51.54% 0.493
CNN-BiLSTM10.0092 0.0149 53.44% 0.533 0.0108 0.0158 53.10% 0.522 0.0097 0.0148 54.01% 0.540
CNN-BiLSTM20.0077 0.0125 54.89% 0.541 0.0101 0.0152 53.95% 0.533 0.0104 0.0157 53.95% 0.532
Table 3: Performance of DL and ML forecasting models for F=9
BTC ETH XRP
Model MAE RMSE Acc F1MAE RMSE Acc F1MAE RMSE Acc F1
SVR 0.0133 0.0179 51.97% 0.512 0.0177 0.0218 51.99% 0.511 0.0201 0.0285 49.86% 0.518
3NN 0.0168 0.0238 50.77% 0.512 0.0198 0.0277 51.04% 0.509 0.0102 0.0153 53.91% 0.535
DTR 0.0173 0.0263 49.95% 0.503 0.0157 0.0227 49.04% 0.490 0.0100 0.0159 51.39% 0.514
LSTM10.0119 0.0178 52.75% 0.511 0.0085 0.0135 53.76% 0.528 0.0071 0.0109 52.71% 0.489
LSTM20.0155 0.0217 51.81% 0.489 0.0151 0.0213 52.76% 0.521 0.0086 0.0123 52.11% 0.449
BiLSTM10.0119 0.0179 52.46% 0.516 0.0135 0.0192 52.70% 0.499 0.0087 0.0127 51.60% 0.460
BiLSTM20.0135 0.0200 52.77% 0.523 0.0141 0.0201 53.31% 0.521 0.0081 0.0126 52.49% 0.487
CNN-LSTM10.0095 0.0145 53.38% 0.525 0.0108 0.0160 53.62% 0.528 0.0068 0.0103 50.51% 0.426
CNN-LSTM20.0075 0.0121 53.26% 0.528 0.0098 0.0148 53.81% 0.525 0.0073 0.0106 50.95% 0.441
CNN-BiLSTM10.0104 0.0161 53.37% 0.519 0.0110 0.0166 53.80% 0.533 0.0068 0.0103 50.06% 0.459
CNN-BiLSTM20.0110 0.0167 53.49% 0.516 0.0112 0.0168 53.68% 0.525 0.0080 0.0112 49.94% 0.499
Table 4: Performance of DL and ML forecasting models for F=12
BTC ETH XRP
Model MAE RMSE Acc F1MAE RMSE Acc F1MAE RMSE Acc F1
SVR 0.0139 0.0185 50.70% 0.512 0.0208 0.0267 51.97% 0.512 0.0188 0.0279 49.45% 0.514
3NN 0.0194 0.0280 50.40% 0.509 0.0211 0.0291 50.99% 0.508 0.0120 0.0176 53.42% 0.530
DTR 0.0172 0.0262 49.05% 0.494 0.0161 0.0226 48.30% 0.482 0.0102 0.0159 51.44% 0.513
LSTM10.0143 0.0198 51.29% 0.465 0.0145 0.0196 51.74% 0.464 0.0090 0.0122 49.90% 0.407
LSTM20.0145 0.0199 51.80% 0.473 0.0213 0.0294 52.49% 0.507 0.0114 0.0148 52.22% 0.430
BiLSTM10.0154 0.0221 52.20% 0.509 0.0142 0.0202 53.38% 0.508 0.0088 0.0127 54.36% 0.489
BiLSTM20.0154 0.0222 52.27% 0.514 0.0148 0.0213 53.27% 0.523 0.0087 0.0130 55.90% 0.506
CNN-LSTM10.0092 0.0147 53.31% 0.531 0.0119 0.0172 53.61% 0.517 0.0074 0.0108 52.00% 0.419
CNN-LSTM20.0094 0.0140 51.30% 0.501 0.0113 0.0166 53.20% 0.527 0.0066 0.0102 53.17% 0.444
CNN-BiLSTM10.0129 0.0195 51.99% 0.506 0.0134 0.0199 53.22% 0.529 0.0066 0.0102 51.71% 0.445
CNN-BiLSTM20.0133 0.0200 51.51% 0.501 0.0138 0.0203 53.38% 0.524 0.0066 0.0101 50.90% 0.466
Table 5: Performance of DL and ML forecasting models for F=16
NO. TR20-01 PAGE 9
In the sequel, in order to identify the reliability of models’ predictions, we applied a residual
autocorrelation test for the models which achieved the best overall scores regarding all performance
metrics and dataset, namely CNN-LSTM and CNN-BiLSTM for forecasting horizons 4 and 9.
Figures 1-12 present the Auto-Correlation Function (ACF) plot of both selected convolutional-
based models for BTC, ETH and XRP. Notice that the confident limits (blue dashed line) are con-
structed assuming that the residuals follow a Gaussian probability distribution. Clearly, all present
ACF plots reveal that some correlation coefficients were not within the confidence limits (dashed
lines), violating the assumption of no auto-correlation in the errors. Therefore, the presence of cor-
relation indicates that the advanced DL models are unreliable for cryptocurrency price predictors
since there exists some significant information left over which should be taken into account for
obtaining better predictions.
-0.2
0
0.2
0.4
0.6
0.8
1
Auto-Correlation
0 1 2 3 4 5 6 7 8 9 10
Lag
Figure 1: ACF plots on the residuals for BTC using CNN-LSTM for F=4
-0.2
0
0.2
0.4
0.6
0.8
1
Auto-Correlation
0 1 2 3 4 5 6 7 8 9 10
Lag
Figure 2: ACF plots on the residuals for BTC using CNN-LSTM for F=9
PAGE 10 NO. TR20-01
-0.2
0
0.2
0.4
0.6
0.8
1
Auto-Correlation
0 1 2 3 4 5 6 7 8 9 10
Lag
Figure 3: ACF plots on the residuals for BTC using CNN-BiLSTM for F=4
-0.2
0
0.2
0.4
0.6
0.8
1
Auto-Correlation
0 1 2 3 4 5 6 7 8 9 10
Lag
Figure 4: ACF plots on the residuals for BTC using CNN-BiLSTM for F=9
-0.2
0
0.2
0.4
0.6
0.8
1
Auto-Correlation
0 1 2 3 4 5 6 7 8 9 10
Lag
Figure 5: ACF plots on the residuals for ETH using CNN-LSTM for F=4
NO. TR20-01 PAGE 11
-0.2
0
0.2
0.4
0.6
0.8
1
Auto-Correlation
0 1 2 3 4 5 6 7 8 9 10
Lag
Figure 6: ACF plots on the residuals for ETH using CNN-LSTM for F=9
-0.2
0
0.2
0.4
0.6
0.8
1
Auto-Correlation
0 1 2 3 4 5 6 7 8 9 10
Lag
Figure 7: ACF plots on the residuals for ETH using CNN-BiLSTM for F=4
-0.2
0
0.2
0.4
0.6
0.8
1
Auto-Correlation
0 1 2 3 4 5 6 7 8 9 10
Lag
Figure 8: ACF plots on the residuals for ETH using CNN-BiLSTM for F=9
PAGE 12 NO. TR20-01
-0.2
0
0.2
0.4
0.6
0.8
1
Auto-Correlation
0 1 2 3 4 5 6 7 8 9 10
Lag
Figure 9: ACF plots on the residuals for XRP using CNN-LSTM for F=4
-0.2
0
0.2
0.4
0.6
0.8
1
Auto-Correlation
0 1 2 3 4 5 6 7 8 9 10
Lag
Figure 10: ACF plots on the residuals for XRP using CNN-LSTM for F=9
-0.2
0
0.2
0.4
0.6
0.8
1
Auto-Correlation
0 1 2 3 4 5 6 7 8 9 10
Lag
Figure 11: ACF plots on the residuals for XRP using CNN-BiLSTM for F=4
NO. TR20-01 PAGE 13
-0.2
0
0.2
0.4
0.6
0.8
1
Auto-Correlation
0 1 2 3 4 5 6 7 8 9 10
Lag
Figure 12: ACF plots on the residuals for XRP using CNN-BiLSTM for F=9
Regarding BTC, the ACF plots reveal that some correlation coefficients were not within the
confidence limits, violating the assumption of no auto-correlation in the errors. More specifically,
the ACF plots of CNN-LSTM for F=4 and F=9 as well as the ACF plot of CNN-BiLSTM for
F=9 presented that there are significant spikes at lags 1 −5 while the ACF plot of CNN-BiLSTM
presented that there are significant spikes at lags 1 and 2. Regarding ETH, both ACF plots of
CNN-LSTM and CNN-BiLSTM for F=4 revealed that there are significant spikes at lags 1 and 2
while for F=9 revealed that that there exist significant spikes at lags 1-5. Finally, regarding XRP,
both ACF plots of CNN-LSTM and the ACF-plot of CNN-BiLSTM for F=4 presented that there
exist some small spikes at lag 1 and 2 while the ACF-plot of CNN-BiLSTM for F=9 presented a
negligible spike at lags 1 and 2.
Summarizing, the interpretation of Figures 1-12 illustrated the presence of auto-correlation in
the residuals of all advanced DL models. Thus, we concluded that these models are unreliable
for cryptocurrency price predictors since there exists some significant information left over which
should be taken into account for obtaining better predictions.
4 Discussion
Following our experiments, this section is dedicated in providing a thorough and sufficiently de-
tailed discussion of our findings with regard to the predefined three research questions: Can deep
learning algorithms efficiently predict cryptocurrency prices? Are cryptocurrency prices a random
walk process? Which is a proper validation method of cryptocurrency price prediction models?
4.1 Can deep learning efficiently predict cryptocurrency prices?
Deep learning algorithms are considered to be the most powerful and the most effective methods in
approximating extremely complex and non-linear classification and regression problems, therefore
it was expected that a noticeable performance increase will be achieved by the incorporation of
PAGE 14 NO. TR20-01
these models comparing to classic machine learning algorithms. Surprisingly, our results demon-
strated that the utilized DL algorithms, slightly outperformed the other ML algorithms utilized
in our experiments, whereas instead a noticeable performance increase was anticipated. So, it is
paramount importance to investigate the reason why that happened. To this end, we summarize
two possible reasons: The problem we are trying to solve is a random walk process or very close
to it, thus any attempt for prediction might be of poor quality or the problem is just too compli-
cated that even advanced deep learning methods cannot find any pattern that would lead to any
reliable prediction. Thus, more sophisticated methodologies, techniques and innovative strategies
are needed to be investigated.
When a time series prediction problem follows a random walk process or it is so complicated
that most models face it as a random process, then the more efficient method to face it, is the
employment of present values as the prediction values for the next state [26]. That is exactly what
a persistence model does and maybe what most prediction models really do and possibly that’s
the reason why ML models used in our experiments achieve almost the same performance score
compared to the deep learning models used in our experiments. In contrast, the deep learning
models may attempt forecasting based on patterns that were traced and as a result are unable to
achieve high performance because either those patterns are false or because there exist no such
patterns at all, in the case that the cryptocurrency price prediction problem is a random walk
process.
Nevertheless, as mentioned before, the DL models did not manage to achieve a noticeable
performance score in our experiments, since their score was almost the same with the ML models.
Thus, we conclude that these advanced DL models cannot efficiently predict cryptocurrency prices
because the utilized datasets with the specific form which we “fed” them to our prediction models,
probably follow almost a random walk process and thus not sufficient information lies on them in
order to perform accurate and reliable future predictions.
4.2 Are cryptocurrency prices a random walk process?
Towards the construction of a model which performs reliable and accurate predictions, firstly, we
have to identify if the cryptocurrency price prediction problem is a random walk process. In a
recent study, Stavroyiannis et al. [35] proved that Bitcoin prices follow a random walk process
since their experiments revealed the presence of unit roots, for several time intervals from 1-min to
180-min, and thus reliable profitable trading opportunities may not be possible in Bitcoin markets.
However, since this problem is highly affected by time evolution and external changes, these results
maybe temporary and reverse in future.
However, there are numerous technical strategies that the majority of the professional traders
utilize in order to make trading decisions in stock market and cryptocurrency investments. Most
of them seem to be heuristic and empirical strategies which are based on various technical indica-
tors and patterns such as the “Engulfing Pattern” and the “Evening Star”. A recent study utilized
those technical indicators and trading patterns strategies in order to predict stock market and cryp-
tocurrency prices [15]. Their results provide evidence that technical analysis strategies have strong
NO. TR20-01 PAGE 15
predictive power and thus can be useful in cryptocurrencies markets like Bitcoin.
Therefore, we conclude that the cryptocurrency prices in general are not totally a random walk
process but they may be close to it, which means that probably exist some actual patterns on historic
data that could assist on forecasting attempts. In other words, we liken this problem as a “huge
sea of random walk points where small hidden islands (patterns) may exist in”. As a result, more
research is required for the discovery of alternative, innovative and more sophisticated methods
such as the incorporation of new feature engineering strategies and the creation of new algorithmic
and ensemble methods.
4.3 What is a proper validation method of cryptocurrency price prediction
models?
As mentioned above, the most common validation metrics for measuring the performance of most
regression algorithms are MAE and RMSE. However, finding a proper validation metric for cryp-
tocurrency price prediction models can be a very complicated and tricky task and cannot be con-
sidered an easy and straightforward process. The MAE and RMSE may constitute an incomplete
way for validating cryptocurrency price prediction problems since a prediction model may have
excellent MAE and RMSE performance but cannot properly predict the cryptocurrency price di-
rection move (classification problem). A cryptocurrency trader or investor may be more interested
in the future price direction movement rather than knowing the exact future cryptocurrency price.
Profitability analysis for algorithmic trading strategies reveal that classification prediction models
were more effective than regression models [16].
Even if we utilize a third evaluation metric which will measure the performance accuracy of
cryptocurrency price direction movement, that may still constitute an incomplete method for val-
idating cryptocurrency prediction algorithms. Consider the following example: Suppose we wish
to validate 2 cryptocurrency prediction models utilizing a test set of 100 questions, e.g. what is the
future price direction movement on the next 100 time steps? The first model answers (predicts) all
questions while it answers correctly 52 questions achieving an accuracy score of 52%. The second
model answers only 5 from 100 questions but it cannot answer the other 95 questions, while these
5 answers are correct. So, the second model achieves a score of 5%. Thus, an important question
is raised, “which is the best model”? A cryptocurrency trader or investor will probably choose the
second model since it acts in a more reliable way and it would be more valuable for him to possess
a model which performs accurate predictions on random times (specified by the model), rather
than possessing a model which performs unreliable predictions on every moment (specified by the
user).
Therefore, we conclude that finding a proper validation metric for cryptocurrency price pre-
diction models is a very challenging task and thus alternative and new methods for evaluating
cryptocurrency prediction models are essential.
PAGE 16 NO. TR20-01
5 Revisiting the problem
One of the most significant steps in order to solve any problem, especially the really hard and chal-
lenging ones, lies in finding a proper strategy approach and securing the complete understanding of
the problem we try to solve. A proper strategy approach should answer questions such as: should
we have to predict prices, price movement direction, price trends, price spikes and so on. Next,
should we apply data preprocessing and feature engineering strategies (e.g. which features should
we use in order to efficiently train a prediction model ?) Also, what is the best prediction model to
apply (e.g. DNNs, other sophisticated prediction models, ensemble models and so on) and finally,
which is a proper method to validate this model? All these issues, considered as discrete steps in
the process, should be taken into serious consideration since each one of them can significantly
contribute to any prediction attempt in order to efficiently approximate the problem.
These steps are not a straightforward process, since we should always have to consider its
chaotic and extremely complicated nature with respect to its practical contribution after a possible
solution. For example, it may be an easier task to solve and possibly more beneficial for the invest-
ment and trading world to predict if the price will just increase or decrease (classification problem
for price direction movement prediction) rather than predicting the exact value of cryptocurrency
price. Some strategy approaches examples are presented in brief below.
Instead of adopting a specific time interval, one could utilize various time intervals of higher
and lower frequency historic datasets for predicting the prices on a specific future interval in order
to utilize and exploit in a more efficient way all possible information that a historic dataset may
contain. Another approach could be instead of predicting the price or the movement direction on
one discrete future time value, to predict the average and movement direction price or peak price
inside a future time window frame (this approach would be more similar to a trend prediction
problem).
Pattern identification and recognition could be another approach. This approach would be more
similar to a pattern detection framework in which the model would detect specific pattern areas in
order to perform a prediction. More specifically, if we are able to identify the feature characteristics
of possible useful patterns that a prediction model found, then we could filter out useless sequence
inputs which have no predictive information and then utilize only the useful sequence inputs which
will possibly assist on reliable and accurate predictions. In this case the prediction model will
perform prediction operations only when the input sequence falls into the same category with the
chosen patterns. This framework would be more similar to the way that a professional trader often
acts, who performs investment decisions based on his/her personal chosen patterns and indicators
recipes on technical analysis of historic price charts.
Finally, another approach could be the investigation of heuristic patterns and other financial
indicators which professional traders and bankers utilize in their trading and financial technical
analysis. It is essential to identify how these methods actually assist predictions and investment de-
cisions in a more mathematic way (if they actually work) and maybe incorporate these techniques
in a machine learning framework for developing co-operative prediction models. That could be an
effective cryptocurrency prediction framework.
NO. TR20-01 PAGE 17
6 Conclusions
In this work, we evaluated advanced DL models for predicting cryptocurrency prices and also
investigated three research question concerning this problem in a review and discussion approach.
Our results revealed that the presented models are inefficient and unreliable cryptocurrency price
predictors, probably due to the fact that this problem is a very complicated one, that even advanced
deep learning techniques such as LSTM and CNNs are not able to solve efficiently. Also, based on
our experimental results and investigation regarding to our research questions about cryptocurrency
price problem, we conclude that cryptocurrency prices follow almost a random walk process while
few hidden patterns may probably exist in, where an intelligent framework has to identify them in
order a prediction model to make accurate and reliable forecasts. Therefore, new sophisticated
algorithmic methods, alternative approaches, new validation methods and the incorporation of
ensemble methods should be explored.
References
[1] Hamed Habibi Aghdam and Elnaz Jahani Heravi. Guide to convolutional neural networks.
New York, NY: Springer. doi, 10:978–3, 2017.
[2] David W Aha. Lazy learning. Springer Science & Business Media, 2013.
[3] Neena Aloysius and M Geetha. A review on deep convolutional neural networks. In 2017
International Conference on Communication and Signal Processing, pages 588–592. IEEE,
2017.
[4] MuhammadAmjad and Devavrat Shah. Trading bitcoin and online time series prediction. In
NIPS 2016 Time Series Workshop, pages 1–15, 2017.
[5] Nashirah Abu Bakar and Sofian Rosbi. Autoregressive integrated moving average (ARIMA)
model for forecasting cryptocurrency exchange rate in high volatility environment: A new
insight of bitcoin transaction. International Journal of Advanced Engineering Research and
Science, 4(11), 2017.
[6] Alexei Botchkarev. Performance metrics (error measures) in machine learning regression,
forecasting and prognostics: Properties and typology. arXiv preprint arXiv:1809.03006,
2018.
[7] Eugene Charniak. Introduction to deep learning. The MIT Press, 2019.
[8] Shi Chen, Cathy Yi-Hsuan Chen, Wolfgang Karl H¨ardle, TM Lee, and Bobby Ong.
Econometric analysis of a cryptocurrency index for portfolio investment. In Handbook of
Blockchain, Digital Finance, and Inclusion, Volume 1, pages 175–206. Elsevier, 2018.
PAGE 18 NO. TR20-01
[9] N. Deng, Y. Tian, and C. Zhang. Support vector machines: optimization based theory, algo-
rithms, and extensions. Chapman and Hall/CRC, 2012.
[10] Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, and Pierre-
Alain Muller. Deep learning for time series classification: a review. Data Mining and Knowl-
edge Discovery, 33(4):917–963, 2019.
[11] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning book. MIT Press,
521(7553):800, 2016.
[12] Alex Graves. Long short-term memory. In Supervised sequence labelling with recurrent
neural networks, pages 37–45. Springer, 2012.
[13] Klaus Greff, Rupesh K Srivastava, Jan Koutn´ık, Bas R Steunebrink, and J¨urgen Schmidhu-
ber. LSTM: A search space odyssey. IEEE Transactions on Neural Networks and Learning
Systems, 28(10):2222–2232, 2016.
[14] A. Gulli and S. Pal. Deep Learning with Keras. Packt Publishing Ltd, 2017.
[15] Jing-Zhi Huang, William Huang, and Jun Ni. Predicting Bitcoin returns using high-
dimensional technical indicators. The Journal of Finance and Data Science, 5(3):140–155,
2019.
[16] Suhwan Ji, Jongmin Kim, and Hyeonseung Im. A comparative study of Bitcoin price predic-
tion using deep learning. Mathematics, 7(10):898, 2019.
[17] ES Karakoyun and AO Cibikdiken. Comparison of ARIMA time series model and LSTM
deep learning algorithm for bitcoin price forecasting. In The 13th MultidisciplinaryAcademic
Conference in Prague, pages 171–180, 2018.
[18] Fazle Karim, Somshubra Majumdar, Houshang Darabi, and Shun Chen. Lstm fully convolu-
tional networks for time series classification. IEEE Access, 6:1662–1669, 2017.
[19] Fazle Karim, Somshubra Majumdar, Houshang Darabi, and Samuel Harford. Multivariate
LSTM-FCNS for time series classification. Neural Networks, 116:237–245, 2019.
[20] Salman Khan, Hossein Rahmani, Syed Afaq Ali Shah, and Mohammed Bennamoun. A
guide to convolutional neural networks for computer vision. Synthesis Lectures on Computer
Vision, 8(1):1–207, 2018.
[21] Salim Lahmiri and Stelios Bekiros. Cryptocurrency forecasting with deep learning chaotic
neural networks. Chaos, Solitons & Fractals, 118:35–40, 2019.
[22] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521(7553):436–
444, 2015.
NO. TR20-01 PAGE 19
[23] Ioannis E Livieris, Emmanuel Pintelas, and Panagiotis Pintelas. A CNN-LSTM model for
gold price time series forecasting. Neural Computing and Applications, 2020 (under revi-
sion).
[24] Wei-Yin Loh. Classification and regression tree methods. Wiley StatsRef: Statistics Reference
Online, 2014.
[25] Pankaj Malhotra, Lovekesh Vig, Gautam Shroff, and Puneet Agarwal. Long short term mem-
ory networks for anomaly detection in time series. In Proceedings, volume 89. Presses uni-
versitaires de Louvain, 2015.
[26] Douglas C Montgomery, Cheryl L Jennings, and Murat Kulahci. Introduction to time series
analysis and forecasting. John Wiley & Sons, 2015.
[27] Ujan Mukhopadhyay, Anthony Skjellum, Oluwakemi Hambolu, Jon Oakley, Lu Yu, and
Richard Brooks. A brief survey of cryptocurrency systems. In 14th Annual Conference
on Privacy, Security and Trust, pages 745–752. IEEE, 2016.
[28] Arvind Narayanan, Joseph Bonneau, Edward Felten, Andrew Miller, and Steven Goldfeder.
Bitcoin and cryptocurrency technologies: a comprehensive introduction. Princeton Univer-
sity Press, 2016.
[29] D Olvera-Juarez and E Huerta-Manzanilla. Forecasting bitcoin pricing with hybrid models:
A review of the literature. International Journal of Advanced Engineering Research and
Science, 6(9), 2019.
[30] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel,
P. Prettenhofer, R. Weiss, and V. Dubourg. Scikit-learn: Machine learning in Python. Journal
of Machine Learning Research, 12:2825–2830, 2011.
[31] Alaa Sagheer and Mostafa Kotb. Time series forecasting of petroleum production using deep
LSTM recurrent networks. Neurocomputing, 323:203–213, 2019.
[32] Mike Schuster and Kuldip K Paliwal. Bidirectional recurrent neural networks. IEEE trans-
actions on Signal Processing, 45(11):2673–2681, 1997.
[33] Takuya Shintate and Luk´aˇs Pichl. Trend prediction classification for high frequency bitcoin
time series with deep learning. Journal of Risk and Financial Management, 12(1):17, 2019.
[34] Sima Siami-Namini, Neda Tavakoli, and Akbar Siami Namin. A comparison of ARIMA and
LSTM in forecasting time series. In 2018 17th IEEE International Conference on Machine
Learning and Applications, pages 1394–1401. IEEE, 2018.
[35] Stavros Stavroyiannis, Vassilios Babalos, Stelios Bekiros, Salim Lahmiri, and Gazi Salah Ud-
din. The high frequency multifractal properties of Bitcoin. Physica A: Statistical Mechanics
and its Applications, 520:62–71, 2019.
PAGE 20 NO. TR20-01
[36] Gergana Taneva. An analysis and a forecast of the cryptomarket based on the ARIMA model.
Economic Thought journal, (4):66–84, 2019.
[37] Ruey S Tsay. Analysis of financial time series, volume 543. John Wiley & Sons, 2005.
[38] Is¸il Yenido˘gan, Aykut C¸ ayir, Ozan Kozan, Tu˘gc¸e Da˘g, and C¸ i˘gdem Arslan. Bitcoin forecast-
ing using ARIMA and prophet. In 3rd International Conference on Computer Science and
Engineering, pages 621–624. IEEE, 2018.
