Table 1 - uploaded by Dmitry Vengertsev
Content may be subject to copyright.
Source publication
This paper studies the problem of applying machine learning with deep architecture to time series forecasting.
While these techniques have shown promise for modeling static data, applying them to sequential data is gaining
increasing attention. This paper overviews the particular challenges present in applying Conditional Restricted
Boltzmann Machi...
Contexts in source publication
Context 1
... tuning three parameters of ARIMA -number of au- toregressive terms, number of nonseasonal differences needed for stationarity, and number of lagged forecast errors in the prediction equation, high prediction accuracy was obtained, Table 1. ...
Context 2
... Multi-Layer Perceptron, sliding window of size N = 3 gave the model with the lowest combination of variance and bias for given number of hidden units, which is consistent with other papers where smaller sliding window shows smaller bias, Fig. 2. Table 1 summarizes performance of MLP and SV regression. ...
Context 3
... neural network are able to accurately predict time series. It is clear from Table 1 that Conditional Restricted Boltzmann Machines are comparable or better than our competing meth- ods. As mentioned above, the prediction accuracy can be improved by tuning parameters of CRBM (computationally intensive). ...
Similar publications
Recently, deep learning has gained great popularity in the area of recommender systems. Various combinations of deep learning, collaborative recommendation and content-based recommendation have occurred. However, as one of the three most significant recommendation techniques, hybrid recom-mendation has little cooperation with deep learning. Besides...
Deep Learning provided powerful tools for forecasting financial time series data. However, despite the success of these approaches on many challenging financial forecasting tasks, it is not always straightforward to employ DL-based approaches for highly volatile and non-stationary time financial series. To this end, in this paper, an adaptive input...
Citations
... Time series forecasting has attracted substantial attention in the academic community and has a vast area of applications in energy, communication, business, financial, health and sports domains [1][2][3][4][5][6][7][8]. Time series feature extraction and feature learning is a key step used in unsupervised, semi-supervised and supervised applications [9,10]. ...
Sensor-based human activity classification requires time and frequency domain feature extraction techniques. The set of choice in time and frequency domain features may have a significant impact on the overall classification accuracy. Another problem is to train deep learning models with sufficient dataset. The use of generative models eliminates the requirement of choosing certain features of the signal. As a generative model, restricted Boltzmann machine (RBM) is an energy-based probabilistic graphical model which factorises the probability distribution of a random variable over a binary probability distribution. Conditional restricted Boltzmann machines (CRBMs) is an extension to RBM, which can capture temporal information in time-series signals and can be deployed as a generative model in classification. In this study, the authors show how CRBMs can be trained to learn signal features. They present four generative model training results, RBM, CRBM, generative adversarial network, Wasserstein generative adversarial network – gradient penalty and compare the models' performances with a performance criterion. They show that the CRBM model can generate signals closest to true signals with a significantly higher success rate as compared to other presented generative models. They present a statistical analysis of the findings and show that the findings significantly hold.
... However, FFNNs also find application in sequential learning approaches. Within the scope of [7], D. Vengertsev investigates in his work the suitability of deep FFNNs in comparison to previous models for time series prediction. The author shows that MLPs are also capable of predicting time series. ...
... With the success of machine learning (ML) algorithms in different domains, ML techniques for time series forecasting is also popular [35], [36]. However, among these methods, only a few (when compared to the non-DL methods) have considered DL methods for time series forecasting [37]- [39]. ...
Multivariate time series (MTS) regression tasks are common in many real-world data mining applications including finance, cybersecurity, energy, healthcare, prognostics, and many others. Due to the tremendous success of deep learning (DL) algorithms in various domains including image recognition and computer vision, researchers started adopting these techniques for solving MTS data mining problems, many of which are targeted for safety-critical and cost-critical applications. Unfortunately, DL algorithms are known for their susceptibility to adversarial examples which also makes the DL regression models for MTS forecasting also vulnerable to those attacks. To the best of our knowledge, no previous work has explored the vulnerability of DL MTS regression models to adversarial time series examples, which is an important step, specifically when the forecasting from such models is used in safety-critical and cost-critical applications. In this work, we leverage existing adversarial attack generation techniques from the image classification domain and craft adversarial multivariate time series examples for three state-of-the-art deep learning regression models, specifically Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU). We evaluate our study using Google stock and household power consumption dataset. The obtained results show that all the evaluated DL regression models are vulnerable to adversarial attacks, transferable, and thus can lead to catastrophic consequences in safety-critical and cost-critical domains, such as energy and finance.
... ARIMA model can also be applied on non-stationary time series with differencing method. It is the most widely used linear model and has been used as a comparison reference [18,19]. ...
... SVM models generated for multiple inputs can produce nonlinear solutions. In many studies that forecast time series, use SVM [9,18]. ...
Time series forecasting is currently used in various areas. Energy management is also one of the most prevalent application areas. As a matter of fact, energy suppliers and managers have to face with the energy mix problem. Electricity can be produced from fossil fuels, from nuclear energy, from bio-fuels or from renewable energy resources. Concerning electricity generation system based on solar irradiation, it is very important to know precisely the amount of electricity available for the different sources and at different horizons: minutes, hours, and days. Depending on the horizon, two main classes of methods can be used to forecast the solar irradiation: statistical time series forecasting methods for short to midterm horizons and numerical weather prediction methods for medium to long-term horizons. On this paper, we focus only on statistical time series forecasting methods. The aim of this study is to assess if deep learning can be suitable and competitive on the solar irradiation data time series forecasting. In this context, studies using deep learning and other machine learning methods for time series forecasting were investigated. A special Recurrent Neural Network variations Long Short-Term Memories and Gated Recurrent Unit models are introduced.
... Research shows that deep neural network architectures yield better results than the commonly used shallow ones [15]. Starting with ARIMA, Support Vector Regressions, and Multi-layer Perceptron, researchers have used deep architectures such as Restricted Boltzmann Machine (RBM) and conditional RBM for performing times -series forecasting [16]. However, utilization of deep architectures in predicting expenditures in health care field is still missing in the literature. ...
Managing expenditure on medications is a serious
challenge faced by patients, in particular for those who cannot
afford costly health care. Predicting patient’s spending on
medications becomes crucial for efficient planning, budgeting, and
decision-making. However, little attention has been given to
predicting patient expenditure using deep time -series forecasting
methods. The primary objective of this paper is the time-series
forecasting of patient expenditures on medications using both
traditional and deep time-series forecasting methods. A traditional
Auto-Regressive Integrated Moving Average (ARIMA) model; and,
two deep models, a standard Long Short-Term Memory (LSTM)
model and a stacked LSTM model were calibrated to predict the
monthly expenditure on medication for 50,000+ patients in the US
between 2011 and 2015. The first 48 months were used for training
the models and the remaining 12 months were used for testing the
models. Results revealed that the stacked LSTM model performed
better than both the standard LSTM and ARIMA models during
test conditions. Overall, both the deep time-series models
performed better than the traditional time-series ARIMA model.
We highlight the implications of our results for forecasting time -
series data involving patient journeys.
... The classical ARMA based models are still in use and do effective forecasting for majority of the highly traded stocks. Among the most appealing of the recent contributions in this front are the works on deep neural nets for forecast prediction [3], the variational mode decomposition on carbon prices and subsequent forecasting using Spike Neural Nets [4] and the convex optimization techniques for time series models [5]. While we believe these methods will eventually catch up and replace traditional Time Series approaches, we have refrained from using such cutting edge models for the sheer sake of simplicity of computations and parameter estimations. ...
Financial forecasting is a widely applied area, making use of statistical prediction using ARMA, ARIMA, ARCH and GARCH models on stock prices. Such data have unpredictable trends and non-stationary property which makes even the best long term predictions grossly inaccurate. The problem is countered by keeping the prediction shorter. These methods are based on time series models like auto regressions and moving averages, which require computationally costly recurring parameter estimations. When the data size becomes considerable, we need Big Data tools and techniques, which do not work well with time series computations. In this paper we discuss such a finance domain problem on the Indian National Stock Exchange (NSE) data for a period of one year. Our main objective is to device a light weight prediction for the bulk of companies with fair accuracy, useful enough for algorithmic trading. We present a minimal discussion on these classical models followed by our Spark RDD based implementation of the proposed fast forecast model and some results we have obtained.
Air pollution is one of the most significant threats to our environment; therefore, it is critical to investigate and monitor air pollution in current context. Moreover, air pollution remains as one of the major causes for environmental change, as well as human illness, and it should not be ignored; instead, it should be redefined by carefully examining its properties and recent modifications. The advanced prediction algorithms like machine learning and deep learning can be used to quantify the changes. Selective LSTM models are used to assess the PM2.5 concentrations present in the atmosphere in order to become connected with numerous estimation methods and discover the one that best accommodates policymakers and directors.
KeywordsAir pollutionPM2.5LSTMDeep learningCNN LSTM
We problems propose the creation of a large-scale deep learning model to analyse the momentum of the price of stocks and predict the direction of its next movement. We have used multiple momentum indicators in combination to try and get an accurate representation of the volatility in the markets. The model is comprised of transpose convolutional and convolutional filters, echo-state networks and gated recurrent units. The proposed model outperforms the state-of-the-art machine learning and deep learning algorithms. The applications of this paper could very well be beyond stock trading and could impact the wider field of time series classification and the analyses of noisy and biased data.KeywordsTranspose convolutional networksTime series classificationInceptionMomentum indicatorsVolatile and biased dataGated recurrent units
The power sector has been widely invested-in for many years. There is a need in finding lasting solutions that can ameliorate the ever-dynamic challenges attached to it which makes the researcher looking for techniques in artificial intelligence solving the complication in the power sector. Since when Artificial intelligence came to existence a lot of problems have been solved through the use of its application such as an artificial neural network (ANN), Neural Network (NN), Deep Neural Network (DNN), Machine learning (ML) and deep learning (DL). Deep learning has become a very good solving tool which makes research focus more on it to tackle a lot of problems such as forecasting tasks, modeling the non-linearity in data of many fields, computer vision, natural language processing, speech recognition, and signal processing. This updated review paper focuses on the application of deep learning (DL) that applied to solar load forecasting; the common algorithm used was shown in the literature reviews. The main reason for this review is to show the latest updated techniques using DL for forecasting that will help the researcher to select the best methods in DL for forecasting accurately. After the review it shows that deep learning performs better for forecasting showing good accuracy, finding the hidden layer.
