No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Abstract
Fuzzy cognitive maps (FCMs) have been successfully applied to time series forecasting. However, it still remains challenging to handle multivariate long nonstationary time series, such as EEG data, which may change rapidly and have patterns of trend. To overcome this limitation, in this article, we propose a fast prediction model to deal with multivariate long nonstationary time series based on the combination of elastic net and high order fuzzy cognitive map (HFCM), which is termed as ElasticNet
<sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">HFCM</sub>
. The designed FCM models each variable by one node and the high-order FCM helps to capture the patterns of trend. A case study on predicting human actions through the Electroencephalogram (EEG) data in the form of multichannel long nonstationary time series is investigated based on the proposed prediction model. Specifically, we first predict EEG signals based on the historical data, then a 1D-convolutionary neural network (1D-CNN) is developed to classify the predicted time series. The experimental results on the Grasp-and-Lift dataset show that the proposal can predict the EEG data with lower prediction error compared with the other regression methods. The area under the curve scores obtained on the Grasp-and-Lift dataset by 1D-CNN are higher than those obtained by state-of-the-art classification methods for EEG data in most cases. These results illustrate that the proposal can predict and classify multivariate long nonstationary time series with high accuracy and efficiency.
... Then, the FCM is employed to capture the temporal dependencies and the causal relationships among these features, i.e., latent feature series or fuzzy time series. Many FCM learning methods [24], such as regression-based methods [9]- [12], [14], [25], [26] and evolutionary-based methods [3], [27]- [32], are used to learn the weights of the FCM. Finally, the output of FCM or the combination of the output of FCM and the extracted feature is employed to handle the prediction task. ...
... However, the above methods remain challenging when coping with long nonstationary time series like Electroencephalogram data. Shen et al. [14] proposed a model with elastic net and HFCM to represent the original time series to overcome this limitation. However, these methods may fall short in handling the multivariate time series prediction task. ...
... In STFCM, the weights matrices among concept vertices, e.g., W 2 and W x in (14), in HFCMs are randomly initialized; and then the objective function (19) is optimized with their partial derivatives showing in (20), (21), and (22) in the condition of fixed values of W 1 , b 1, and W 3 . However, STFCM may not perform well if these weights are not suitable. ...
Fuzzy cognitive map (FCM) is a simple but effective tool for modeling and predicting time series. This paper focuses on the problem of multivariate time series prediction, which is essential and challenging in data mining. Although several FCM-based approaches have been designed to solve this problem, their feature extraction module designed for single mode falls short in capturing the nonlinear spatiotemporal dependencies among variates, thereby resulting in low prediction accuracy in forecasting multivariate time series, which shows the single mode learning is not enough. Therefore, in this paper, we propose a joint spatiotemporal feature learning framework for multivariate time series prediction, where a mix-resolution spatial module consisting of multiple sparse autoencoders (SAEs) is designed to extract the feature series with different spatial resolutions and a mix-order spatiotemporal module concluding multiple high-order FCMs (HFCMs) is designed to model the spatiotemporal dynamics of these feature series. Finally, the outputs of the two modules are concatenated to predict the future values. We refer to this framework as the spatiotemporal fuzzy cognitive map (STFCM). Especially, an efficient learning algorithm is designed to update the integral weights of STFCM based on the batch gradient descent algorithm when it deems necessary. We validate the performance of STFCM on four real-world datasets. Compared with the existing state-of-the-art methods, the experimental results not only show the advantages of the two designed modules in STFCM but also show the excellent performance of STFCM.
... Time-series prediction is one of the most sought-after yet, arguably, the most challenging tasks [11]. It has played an important role in a wide range of fields, including the industrial [14], financial [15], health [16], traffic [17,18], and environmental [19] fields for several decades. For multivariate time-series (MTSs), existing methods inherently assume interdependencies among variables. ...
... In other words, each variable not only depends on its historical values but also on other variables. To efficiently and effectively exploit latent interdependencies among variables, many techniques such as deep learning-based ones [14,[19][20][21][22], the matrix or tensor decomposition-based ones [23,24], the k-nearest neighbor (kNN)-based ones [15,17,18,21], and others [16,[25][26][27] have been proposed. However, obtaining richer semantics with similar or better performances is meaningful but rare. ...
... Various techniques have been proposed for predicting time-series. These methods can be categorized into the deep learning-based ones [14,[19][20][21][22], matrix or tensor decomposition-based ones [23,24], k-nearest neighbor (kNN)-based ones [15,17,18,21], etc. [16,[25][26][27]. ...
Recently, predicting multivariate time-series (MTS) has attracted much attention to obtain richer semantics with similar or better performances. In this paper, we propose a tri-partition alphabet-based state (tri-state) prediction method for symbolic MTSs. First, for each variable, the set of all symbols, i.e., alphabets, is divided into strong, medium, and weak using two user-specified thresholds. With the tri-partitioned alphabet, the tri-state takes the form of a matrix. One order contains the whole variables. The other is a feature vector that includes the most likely occurring strong, medium, and weak symbols. Second, a tri-partition strategy based on the deviation degree is proposed. We introduce the piecewise and symbolic aggregate approximation techniques to polymerize and discretize the original MTS. This way, the symbol is stronger and has a bigger deviation. Moreover, most popular numerical or symbolic similarity or distance metrics can be combined. Third, we propose an along–across similarity model to obtain the k-nearest matrix neighbors. This model considers the associations among the time stamps and variables simultaneously. Fourth, we design two post-filling strategies to obtain a completed tri-state. The experimental results from the four-domain datasets show that (1) the tri-state has greater recall but lower precision; (2) the two post-filling strategies can slightly improve the recall; and (3) the along–across similarity model composed by the Triangle and Jaccard metrics are first recommended for new datasets.
... 3) Multivariate Time Series (MTS) Prediction: MTS is used to observe multiple time-dependent variables concurrently, a method particularly useful in complex system analysis [62], [63], [64], [65]. In this study, we applied five single learning models for MTS (as detailed in Section III-B5), selecting the most effective one as the meta-learner for SEL to enhance prediction accuracy. ...
... e) One-Dimensional Convolutional Neural Network (1D CNN): The 1D CNN, optimized for one-dimensional data like time series, uses convolutional layers with 1D filters for local pattern detection, pooling layers for dimensionality reduction, and densely connected layers for abstraction and prediction. It is effective in signal processing, speech recognition, and time series analysis [56], [61], [65], offering computational efficiency. The Adam algorithm optimizes its weights. ...
This paper introduces an innovative deep learning design framework for front-end radio frequency energy harvesting (RFEH), employing stacking ensemble learning (SEL). The framework integrates multi-output regression (MOR), multilabel classification (MLC), and multivariate time series (MTS) in sequence to achieve accurate prediction (R
<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup>
> 0.90) and classification (> 0.95) at each stage. By transforming frequency to datetime and consolidating output parameters into consecutive time intervals, our approach intelligently combines receiving antennas, impedance matching networks (IMNs), and voltage multipliers (VMs) through MOR and MLC, while MTS guides the determination of geometrical parameters and values. To validate our method, we developed an efficient food sensor tag embedded with an RFEH circuit operating at 915 MHz. The tag, featuring antennas designed by MTS (including an inset-fed rectangular microstrip patch antenna [RMPA], a folded collinear antenna [FCA], and a quasi-Yagi antenna [QYA]), achieves a power conversion efficiency (PCE) of 55.17% with an input power of 6.25 dBm, demonstrating stable sensing capabilities. This integrated deep learning framework holds promise for tailored front-end RFEH design in food quality monitoring application, with significant implications across industries.
... Finally, the testing data is used to evaluate the performance accuracy of the obtained models. Shen et al. (2021) implemented a fast prediction hybrid model combining elastic net and HFCM to deal with multivariate long non-stationary time series to predict human actions through the electroencephalogram (EEG) data. In this technique, each node in an FCM represents a variable in the multivariate time series, thus an N-node FCM should be constructed as the prediction model. ...
... On the other side, population-based [or evolutionary algorithm-based (EA-based)] methods are the most popular, because of improving the forecasting accuracy, robustness, generalization abilities as well as simplicity of application in comparison to the Hebbian-based methods' accuracy. In spite of these achievements, as explained in Shen et al. (2021), the application of EAbased algorithms is limited to model short stationary time series and they are unable to dealing with long non-stationary time series. In addition, EA-based models are more time-consuming and this issue becomes more apparent with the increase of the size of time series. ...
Among various soft computing approaches for time series forecasting, fuzzy cognitive maps (FCMs) have shown remarkable results as a tool to model and analyze the dynamics of complex systems. FCMs have similarities to recurrent neural networks and can be classified as a neuro-fuzzy method. In other words, FCMs are a mixture of fuzzy logic, neural network, and expert system aspects, which act as a powerful tool for simulating and studying the dynamic behavior of complex systems. The most interesting features are knowledge interpretability, dynamic characteristics and learning capability. The goal of this survey paper is mainly to present an overview on the most relevant and recent FCM-based time series forecasting models proposed in the literature. In addition, this article considers an introduction on the fundamentals of FCM model and learning methodologies. Also, this survey provides some ideas for future research to enhance the capabilities of FCM in order to cover some challenges in the real-world experiments such as handling non-stationary data and scalability issues. Moreover, equipping FCMs with fast learning algorithms is one of the major concerns in this area.
... A number of methods for predicting time sequences with multiple variables using fuzzy high-order cognitive maps and elastic networks are set out in [7]. In Fig. 4 is shown an example of a fuzzy five-node recognizing map. ...
... Every ECG signal (over time line) is a node in these elastic networks, and the Fig. 4. Example of a fuzzy five-node recognizing map consisting of five nodes. Left side-graphic structure, right side-representation with a weight matrix [7] different cross-correlation values represents the weights between these nodes. This way developed prediction framework and classification of non-stationary time sequences based on one-dimensional convolutional NN and fuzzy recognizing maps allows prediction of six actions with higher accuracy than existing models. ...
... We then applied various classification techniques on this 3-way classification problem and discovered that the combination of Laplacian Eigenmaps (simple-minded, k = 8) with the k-NN classifier resulted in the highest classification accuracy (F1 score 88.2 ±3.5%). As a result, we used automatic feature-extraction directly from the pre-processed EMG time-domain signal [20,[47][48][49][50][51][52]. Importantly, our approach to extract features from EMG signal resulted in relatively high decoding accuracy. ...
... Other studies that relied on the same dataset that we used mostly focused on EEG [47][48][49][50]. However, Cisotto et al. used both EEG and EMG to classify the same dataset [51]; though they attempted classification of only 2 of the 3 available classes (the most extreme weights: 165 and 660 gr). ...
Electromyography (EMG) is a simple, non-invasive, and cost-effective technology for measuring muscle activity. However, multi-muscle EMG is also a noisy, complex, and high-dimensional signal. It has nevertheless been widely used in a host of human-machine-interface applications (electrical wheelchairs, virtual computer mice, prosthesis, robotic fingers, etc.) and, in particular, to measure the reach-and-grasp motions of the human hand. Here, we developed an automated pipeline to predict object weight in a reach-grasp-lift task from an open dataset, relying only on EMG data. In doing so, we shifted the focus from manual feature-engineering to automated feature-extraction by using pre-processed EMG signals and thus letting the algorithms select the features. We further compared intrinsic EMG features, derived from several dimensionality-reduction methods, and then ran several classification algorithms on these low-dimensional representations. We found that the Laplacian Eigenmap algorithm generally outperformed other dimensionality-reduction methods. What is more, optimal classification accuracy was achieved using a combination of Laplacian Eigenmaps (simple-minded) and k-Nearest Neighbors (88% F1 score for 3-way classification). Our results, using EMG alone, are comparable to other researchers’, who used EMG and EEG together, in the literature. A running-window analysis further suggests that our method captures information in the EMG signal quickly and remains stable throughout the time that subjects grasp and move the object.
... F. Shen et al. proposed a prediction model to overcome the limitation in time-series forecasting. The combination of EN model and high order Fuzzy cognitive maps (FCMs) shown predictable of a time-series dataset with less error compared with other regression models [20]. EN model is a combination of lasso regression and ridge regression, it has proven efficiency of predict time series compare with lasso and Ridge. ...
... The results associated with this classification step is given in Table 6. Hidden layer size (10,5), (14,7), (20,10) The performance measures for regression models in Table 5 are close to each other. However, it is clearly seen that MLP performs the prediction with lowest error and three methods (SVM, EN and MLP) has the highest 2 score. ...
With the rapid spread of urbanization, competent authorities become increasingly anxious from air pollution risks and effect on citizens especially those with respiratory diseases. In this work, performances of six machine learning methods were analyzed for prediction of maximum ozone (O_3) concentration for the next-day. The models make the prediction using concentrations of six atmospheric components (PM2.5, PM10, Ozone (O3), Sulfur Dioxide (SO2), Nitrogen Dioxide (NO2), and Carbon Monoxide (CO)). The utilized machine learning methods are multilayer perception (MLP), Support Vector Regression (SVM), k-Nearest Neighbor (K-NN), Random Forests (RF), Gradient Boosting (GB), and Elastic Net (EN). After the predictions made by these models, the predicted values were further processed to be classified into one of the six air quality levels defined by United States Environmental Protection Agency. The prediction performances of the models as well as their corresponding classification results were analyzed. It was shown that MLP model gives the lowest RMSE of 2246 for prediction step while SVR achieved the highest accuracy score of 0.790.
... We then applied various classification techniques on this 3-way classification problem and discovered that the combination of Laplacian Eigenmaps (simple-minded, k=8) with the k-NN classifier resulted in the highest classification accuracy (F1 score 88.2±3.5%). To the best of our knowledge, ours is the first successful attempt to use automatic feature-extraction directly from the (filtered) raw EMG time-domain signal [20,[47][48][49][50][51][52]. What is more, our approach resulted in relatively high decoding accuracy. ...
... Other studies that used the same dataset that we used mostly focused on EEG [47][48][49][50]. However, Cisotto et al. used both EEG and EMG to classify the same dataset [51]. ...
Electromyography (EMG) is a simple, non-invasive, and cost-effective technology for sensing muscle activity. However, EMG is also noisy, complex, and high-dimensional. It has nevertheless been widely used in a host of human-machine-interface applications (electrical wheelchairs, virtual computer mice, prosthesis, robotic fingers, etc.) and in particular to measure reaching and grasping motions of the human hand. Here, we developd a more automated pipeline to predict object weight in a reach-and-grasp task from an open dataset relying only on EMG data. In that we shifted the focus from manual feature-engineering to automated feature-extraction by using raw (filtered) EMG signals and thus letting the algorithms select the features. We further compared intrinsic EMG features, derived from several dimensionality-reduction methods, and then ran some classification algorithms on these low-dimensional representations. We found that the Laplacian Eigenmap algorithm generally outperformed other dimensionality-reduction methods. What is more, optimal classification accuracy was achieved using a combination of Laplacian Eigenmaps (simple-minded) and k-Nearest Neighbors (88% for 3-way classification). Our results, using EMG alone, are comparable to others in the literature that used EMG and EEG together. They also demonstrate the usefulness of dimensionality reduction when classifying movement based on EMG signals and more generally the usefulness of EMG for movement classification.
... In recent years, with the great success of deep learning in signal processing and time series-related tasks [2]- [5] such as wind speed forecasting [6], [7], deep learning-based tourism demand forecasting methods have gradually become a research hotspot [8], [9]. Encouraged by the success of multivariate time series forecasting models [10]- [12], some methods attempt to introduce additional tourism-related variables, such as economic factors [13] and weather factors [14]. As tourism related-variables, the search intensity indices (SII) can reflect tourists' preferences and arouse extensive research interest [15]- [17]. ...
An accurate tourism demand forecasting model is crucial for tourism decision-makers. In recent years, several deep learning-based models have been developed to predict tourist arrivals via search intensity indices. However, few methods consider the lag effect in the long-term time range and the interaction between different search intensity indices factors. To alleviate the above limitations, we propose a graph-guided tourism demand forecasting network, which can model the lag effect of historical variables on future variables. Specifically, each variable is individually encoded via a convolutional neural network in the time dimension. Then, lag effects are modeled dynamically in a bipartite graph, and mined via graph aggregation. Finally, a fully-connected network is designed for regression prediction. Experimental results on two public datasets demonstrate the superiority of the proposed method in both one-step and multi-step prediction compared with existing methods.
... Time series forecasting (TSF) is the process of analyzing time series data using statistics and modeling to make predictions and inform straegic decision-making. TSF plays a vital role in various domains, especially in the fields of financial management 1,2 , social network [3][4][5] , medical science [6][7][8] , and industrial engineering 9,10 . Therefore, there is a growing consensus that it is of great importance to enhance the accuracy and interpretability of TSF due to its widespread application. ...
Long short-term memory (LSTM) based time series forecasting methods suffer from multiple limitations, such as accumulated error, diminishing temporal correlation, and lacking interpretability, which compromises the prediction performance. To overcome these shortcomings, a fuzzy inference-based LSTM with the embedding of a fuzzy system is proposed to enhance the accuracy and interpretability of LSTM for long-term time series prediction. Firstly, a fast and complete fuzzy rule construction method based on Wang–Mendel (WM) is proposed, which can enhance the computational efficiency and completeness of the WM model by fuzzy rules simplification and complement strategies. Then, the fuzzy prediction model is constructed to capture the fuzzy logic in data. Finally, the fuzzy inference-based LSTM is proposed by integrating the fuzzy prediction fusion, the strengthening memory layer, and the parameter segmentation sharing strategy into the LSTM network. Fuzzy prediction fusion increases the network reasoning capability and interpretability, the strengthening memory layer strengthens the long-term memory and alleviates the gradient dispersion problem, and the parameter segmentation sharing strategy balances processing efficiency and architecture discrimination. Experiments on publicly available time series demonstrate that the proposed method can achieve better performance than existing models for long-term time series prediction.
... The Elastic Net is a combination of LR and RR [20], which groups and shrinks the parameters associated with correlated This article has been accepted for publication in IEEE Transactions on Dielectrics and Electrical Insulation. This is the author's version which has not been fully edited and content may change prior to final publication. ...
The prediction of the remaining useful life (RUL) of transformer oil helps in condition monitoring and health monitoring of oil-filled power transformers. However, the prediction of RUL depends on the ageing condition of the insulation system. In this paper, a novel hybrid machine learning (ML)-based regression model is developed for predicting the RUL of the insulating oil in years. A total of 26 features have been taken from different chemical and physical properties and indices of mineral oil. Later, features are selected using the Pearson correlation coefficient and conditional mutual information-based feature selection (CMIFS) techniques. Finally, a hybrid algorithm consisting of support vector regression (SVR), k-nearest neighbor (k-NN), multiple layer perceptron (MLP), ridge regression (RR), ElasticNet, Adaptive Boosting (AdaBoost), and extreme gradient boost (XGBoost) are used to predict the RUL of the oil. The performance of the hybrid model is analyzed by root mean square error (RMSE), root mean square logarithmic error (RMSLE), mean absolute error (MAE), and correlation coefficient (R
<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup>
). The comparison with the individual base regression algorithm showed that the hybrid model performed better. The present study adds to the arguments that data-driven intelligent monitoring systems are essential for the safe and efficient health monitoring of transformers.
... Time series data is widely collected in many fields that describes thing's status evolution process, such as traffic flow [1,2], energy [3,4], financial markets [5,6] and many others fields. It is meaningful to make effective decisions through accurate prediction of the future. ...
Multivariate time series forecasting is an important issue in industries, agriculture, finance, and other applications. There are many challenging problems in it such as non-linear and complicated relationships among the series. The entanglement of latent multiple different sequence patterns maybe one of the most reasons for the time series complex behavior, and decomposition can help reveal the hidden evolution law. Graph is a good modelling tool for multiple entities and graph neural network has showed better learning ability for spatial dependence, but its high memory consumption requires valid solutions. Inspired by the above two points, we propose a Temporal Decomposition enhanced Graph neural network for Multivariate time Series Forecasting, namely TDG4MSF, which mainly consists of four components: temporal decomposition enhanced representation learning, graph structure learning, gated GNNML-based representation learning and MLP-based forecasting. A progressive quadratic decomposition architecture is designed in the temporal decomposition enhanced representation learning that extracts the different periodic patterns of time series. Graph structure learning is used to construct adjacency matrix to represent the spatial topological structure. The temporal decomposition enhanced representation and adjacency matrix are fed into gated GNNML to integrating temporal and spatial information between variables. A MLP-based forecasting is utilized to make multivariate time series prediction. Experimental results show the effectiveness of our model in short- and medium-term prediction scenarios are superior to the state-of-the-art methods, which help make exact decisions timely. Code is available at: https://github.com/TYUT-Theta/MHZN.git.
... This maximizes the likelihood of the provided forecasts. Therefore, efforts that might be useful in terms of the stated needs include: the geo-visualization of forecasts based on spatial clustering to reflect the characteristics of adjacent terrains [32][33][34][35][36]; forecast geo-visualization for sparse data [37][38][39][40]; the geo-visualization of the forecasting of criminal activities using ma-chine learning and deep learning techniques [34][35][36][39][40][41][42][43]; event forecasting using classical, improved classical, machine learning, and deep learning techniques for multivariate time series [44][45][46][47][48][49][50][51][52][53][54][55][56][57][58][59][60]; and, finally, multivariate time series forecasting with sparse data [61][62][63][64][65]. ...
This article presents the development of a geo-visualization tool, which provides police officers or any other type of law enforcement officer with the ability to conduct the spatiotemporal predictive geo-visualization of criminal activities in short and continuous time horizons, according to the real events that are happening: that is, for those geographical areas, time slots, and dates that are of interest to users, with the ability to consider individual events or groups of events. This work used real data collected by the Colombian National Police (PONAL); it constitutes a tool that is especially effective when applied to Real-Time Systems for crime deterrence, prevention, and control. For its creation, the spatial and temporal correlation of the events is carried out and the following deep learning techniques are employed: CNN-1D (Convolutional Neural Network-1D), MLP (multilayer perceptron), LSTM (long short-term memory), and the classical technique of VAR (vector autoregression), due to its appropriate performance in the multi-step and multi-parallel forecasting of multivariate time series with sparse data. This tool was developed with Open-Source Software (OSS) as it is implemented in the Python programming language with the corresponding machine learning libraries. It can be implemented with any geographic information system (GIS) and used in relation to other types of activities, such as natural disasters or terrorist activities.
... Shen et al. describe creating a model that forecasts short-term subway ridership by employing a gradient-boosting decision tree (GBDT) to train and assess the model's performance (Shen, et al., 2020). The accuracy is between 0.98 and 0.99, indicating that the model provides promising results. ...
Since its establishment in 1999, the Metro Rail Transit Line 3 (MRT3) has served as a transportation option for numerous passengers in Metro Manila, Philippines. The Philippine government's transportation department records more than a thousand people using the MRT3 daily and forecasting the daily passenger count may be rather challenging. The MRT3's daily ridership fluctuates owing to variables such as holidays, working days, and other unexpected issues. Commuters do not know how many other commuters are on their route on a given day, which may hinder their ability to plan an efficient itinerary. Currently, the DOTr depends on spreadsheets containing historical data, which might be challenging to examine. This study presents a time series prediction of daily traffic to anticipate future attendance at a particular station on specific days.
... Additionally, the kernel ridge regression approach was effectively used for the prediction of monthly mean precipitation (Ali et al., 2020). Shen et al. (2021) took human action prediction by electroencephalogram (EEG) signals as an example to study multivariate time series prediction based on elastic net, and high-order fuzzy cognitive map normalization of lasso regression is achieved by applying L1 regularization to the loss function. Wang et al. (2018) used lasso regression to accurately predict stock market fluctuations. ...
Flooding is one of the most disruptive natural disasters, causing substantial loss of life and property damage. Coastal cities in Asia face floods almost every year due to monsoon influences. Early notification of flooding events enables governments to implement focused preventive
actions. Specifically, short-term forecasts can buy time for evacuation and emergency rescue, giving flood victims timely relief. This paper proposes a novel multi-strategy-mode waterlogging-prediction (MSMWP) framework for forecasting waterlogging depth based on time series prediction and a machine learning regression method. The framework integrates historical rainfall and waterlogging depth to predict near-future waterlogging in time under future meteorological circumstances. An expanded rainfall model is proposed to consider the positive correlation of future rainfall with waterlogging. By selecting a suitable prediction strategy, adjusting the optimal model parameters, and then comparing the different algorithms, the optimal configuration of prediction is selected. In the actual-value testing, the selected model has high computational efficiency, and the accuracy of predicting the waterlogging depth after 30 min can reach 86.1 %, which is superior to many data-driven prediction models for waterlogging depth. The framework is useful for accurately predicting the depth of a target point promptly. The prompt dissemination of early warning information is crucial to preventing casualties and property damage.
... In electricity, traffic management, financial industry, air pollution control, etc., multivariate time series (MTS) can be seen everywhere, and the MTSF algorithm has a wide range of applications. As an example, forecasting traffic data can help travelers plan their travel routes more effectively and managers direct traffic more efficiently [2]. Researchers has widely used the MTSF method to obtain predicted values for future moments, which is fundamental and important for aiding decision making, optimizing resource allocation, and taking stopping measures in advance [3]. ...
Multivariate time series forecasting has many practical applications in a variety of domains such as commerce, weather, environment, and transportation. There exist so many methods developed for multivariate time series forecasting. However, most forecasting methods do not focus on the intrinsic connections that exist between the various variables in a multivariate time series. These connections can negatively affect the accuracy of multivariate time series forecasting. In this paper, a new deep long- and short-term memory neural network forecasting model named TDLSTM-LS is proposed to solve this problem. In this model, tensor processing with TOS optimization module, SLSQP optimization-based LSTM-LS module, and deep LSTM module constructed based on linking gates are applied to retain, strengthen and convey the intrinsic connections among multiple variables. Experiments on datasets from five different domains show that the evaluation metrics of the proposed model (TDLSTM-LS) outperform all other state-of-the-art baseline methods.
... The popular models have been selected for computed the temporal data and predict future information. The Gaussian NB, SVM-SVC [43], SVM-SVR [44] [45], Linear Regression [46], 47, Logistic Regression, Bayesian Ridge [48], Lasso and Ridge Regression [49], Elastic Regression [50], Decision Tree [51], Random Forest [52], K-neighbours [53], MLP [54], RNN, GRU [55], LSTM [56], and BiLSTM [57] models have been used in this system for prediction purpose. ...
The prediction system is used for decision-making by retrieving the information from the real-world data depending on temporal attributes. Various prediction systems for predicting temporal information are available in the literature, which uses human interaction and take a lot of execution time and effort. This paper proposed an automated temporal prediction system. We used the abbreviation AutoTPS for it. AutoTPS is used to retrieve temporal information without human interaction and reduce the processing time. The Methodology of the proposed automated prediction system is based on the integration of automatic data cleaning, feature selection, prediction type selection, and best model selection for the prediction concerning the different sizes of temporal datasets. The proposed AutoTPS has been illustrated on various temporal datasets, which shows the compatibility of different quality parameters (accuracy, usefulness, timeliness, efficiency, completeness) through comparative analysis.
... In this article, the PM2.5 pollutant prediction is multivariate time series forecasting (Lin et al. 2015;Shen et al. 2021). Fuzzy time series can forecast the PM2.5 concentration more accurately. ...
With the rapid increase of urbanization and industrialization, particulate matter (PM2.5) concentration has increased significantly. PM2.5 profile forecasting has become one of the critical research areas in environmental control and protection. The early detection of PM2.5 as a pollutant is vital because PM2.5 has a significant impact on human health than other pollutants. This paper proposes a deep neuro-fuzzy prediction system (DNFPS) by amalgamating the deep learning and the fuzzy time series algorithm to forecast the PM2.5 concentration. The proposed predictive model consists of three phases; a data preprocessing algorithm to generate a high-quality dataset, a denoising autoencoder using fully convolutional neural networks (FCNNs) to extract the features from the pollutant time series profile as well as reduce the dimension of the time series dataset, and the type-2 fuzzy time series forecasting (FTSF) method to forecast PM2.5 concentration. The butterfly optimization algorithm (BOA) is integrated with the type-2 FTSF method to improve the prediction accuracy of the proposed method. FTSF-BOA is implemented to fine-tune the length of type-2 fuzzy intervals. Experiments employing Sydney data sets to analyze the performance of DNFPS. DNFPS shows that the proposed model achieves an excellent performance than other standard baseline models. It has lower computational time (training time) than the other traditional baseline deep learning models.
Fuzzy Cognitive Maps (FCMs) have demonstrated considerable success in time series forecasting and are adept at handling uncertainties and capturing the dynamics of complex systems. Nevertheless, challenges still remain in the handling of multivariate high-dimensional time series using a time-effective learning algorithm. This paper introduces MRHFCM, a new methodology for predicting high-dimensional time series in multiple-input multiple-output (MIMO) systems. MRHFCM represents a hybrid method that combines data embedding transformation, randomized high-order FCM (R-HFCM), and an echo state network (ESN). The core of MRHFCM involves a cascade of R-HFCMs termed the CR-HFCM model. Each CR-HFCM comprises three layers: the input layer, reservoir (internal layer), and output layer. Notably, only the output layer is trainable, employing the least squares minimization algorithm. The weights within each sub-reservoir are randomly chosen and remain unchanged throughout the training procedure. Three real-world high-dimensional datasets are utilized to assess the performance of the proposed MRHFCM method. The results obtained reveal that our approach outperforms some existing baseline and state-of-the-art machine learning and deep learning forecasting techniques in terms of both accuracy and parsimony.
Fuzzy cognitive maps (FCMs) learning is a hot topic in recent years. However, as the number of concepts increases in FCMs, it is difficult to learn the sparse and robust FCMs from a small amount of data, especially from noise data. In this paper, a new large-scale FCMs learning method based on the sparse regression of adaptive loss function is presented, marked as AQP-FCM. Adaptive loss function and L1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$L_1$$\end{document}-norm are introduced in the model to deal with noise data. We solve the model by ADMM method and quadratic programming method to learn the FCMs better. Moreover, the convergence of model is proved. We did a series of experiments under the synthetic data of time series and noise synthesis data. AQP-FCM is also applied to reconstruct gene regulatory network (GRNs). The results of the experiments show that the proposed AQP-FCM method has good performance.
The chapter presents an FCM-based model for pattern classification termed Long-Term Cognitive Network (LTCN). This model uses the class-per-output architecture discussed in the previous chapter and the quasi-nonlinear reasoning rule to avoid the unique fixed-point attractor. To improve its prediction capabilities, the LTCN-based classifier suppresses the constraint that weights must be in the [–1, 1] interval while using all temporal states produced by the network in the classification process. As for the tuning aspect, this classifier is equipped with two versions of the Moore-Penrose learning algorithm. Besides presenting the mathematical formalism of this model and its ensuing learning algorithm, we will develop an example that shows the steps required to solve classification problems using an existing Python implementation. The chapter also elaborates on a measure that estimates the role of each concept in the classification process and presents simulation results using real-world datasets. After reading this chapter, the reader will have acquired a solid understanding of the fundamentals of these algorithms and will be able to apply them to real-world pattern classification datasets.
Recently network-based method for forecasting time series has become a hot research topic. Although some methods have been recognized for their prediction performance, how to mine more useful information of time series and make accuracy predictions is still an open question. To address this issue, we first propose a novel similarity measure called multi-subgraph similarity (Mss) for nodes in visibility graph. Then, a novel well-performed forecasting method for time series is proposed based on Mss. First, a time series is converted into a visibility graph. Afterward, the similarity distribution is obtained by Mss. Eventually, the prediction of time series is made using the similarity distribution. To demonstrate the proposed method is of better prediction performance, we compare the results of forecasting Construction Cost Index (CCI) and UCR data sets. The experiment results indicate that the proposed method could provide more accuracy predictions than compared methods. Moreover, the robustness test shows that the proposed method is of good robustness.
Ozone prediction, a key role for ozone pollution control, is facing the following challenges, i.e., the complex evolution trend of ozone, the cross-interference phenomena between ozone and other pollutants, and the low-quality monitoring data. To overcome the above challenges, we propose a multi-source and multivariate ozone prediction model based on fuzzy cognitive maps (FCMs) and evidential reasoning theory from the perspective of spatio-temporal fusion, termed as ERC-FCM. In this framework, an FCM-based prediction model is introduced to solve the ozone forecasting problem. Inspired by the multivariate time series forecasting, a multivariate ozone prediction problem is modeled as an FCM learned by the real-coded genetic algorithm, in which each node denotes a variable (pollutant). Thus, both the complex evolution trend of ozone and the cross-interference phenomena can be reflected by the FCM. Further, we propose an ensemble theoretical framework based on evidence reasoning theory and the matrix 2 norm. This theoretical framework relieves the negative factors from the low-quality monitoring data and improves the prediction accuracy when facing multi-source and multivariate time series. The performance of ERC-FCM is validated on two real-world datasets. The experimental results demonstrate that our method yields the best prediction performance by comparison with the other classical FCM-based methods on mean absolute error (MAE), mean square error (MSE), and root mean square error (RMSE). In addition, the Friedman test and Nemenyi test show that ERC-FCM gets relatively better prediction accuracy than other models.
In reality, some kinds of time series have the characteristics of high volatility, where fluctuation patterns of sequential data contain rich semantic knowledge and represent the spatial features from two-dimensional perspective. Especially, some non-trivial fluctuations may provide key information for the forecasting of time series. However, how to appropriately represent the fluctuation patterns and achieve the causal reasoning among them remains open. In this work, by learning the causal knowledge of fluctuation patterns, a novel spatial attention fuzzy cognitive map with high-order structure is proposed for the interpretable prediction of time series with high volatility. Firstly, a kind of extended polar fuzzy information granules is utilized to convert time series into granule sequences with interpretable fluctuation features, based on which fuzzy cognitive maps can be constructed by using full data-driven way. Secondly, in order to capture the key fluctuation patterns, the attention mechanism is first introduced into fuzzy cognitive map, where the spatial features of the focused patterns can be taken fully utilized. Thirdly, the high-order structure is involved into the proposed model for the learning of the temporal knowledge existing in the pattern sequences. Finally, real-world financial time series with strong noises and high volatility are empirically utilized to verify the promising performance of the proposed method.
Load forecasting is an important machine learning problem in the field of power system, which is of great significance to power system source load balance, power supply planning and system maintenance. With the development of power technology, fine-grained load data become more easily available, which puts forward new requirements for load forecasting accuracy. In this paper, we use a deep learning framework, namely conditional autoencoder, to forecast day-ahead load. And the conditional autoencoder extracts the characteristics of load time series through deep neural network and finds the sequence pattern of historical data. In addition, we use the load data of different time scales for training, so as to improve the accuracy of the model. In this paper, the real data of regenerative electric boiler in Jilin Province are used for load forecasting, and the effectiveness of the model is verified.
Human Activity Recognition (HAR) aims to identify the actions performed by humans using signals collected from various sensors embedded in mobile devices. In recent years, deep learning techniques have further improved HAR performance on several benchmark datasets. In this paper, we propose one-dimensional Convolutional Neural Network (1D CNN) for HAR that employs a divide and conquer-based classifier learning coupled with test data sharpening. Our approach leverages a two-stage learning of multiple 1D CNN models; we first build a binary classifier for recognizing abstract activities, and then build two multi-class 1D CNN models for recognizing individual activities. We then introduce test data sharpening during prediction phase to further improve the activity recognition accuracy. While there have been numerous researches exploring the benefits of activity signal denoising for HAR, few researches have examined the effect of test data sharpening for HAR. We evaluate the effectiveness of our approach on two popular HAR benchmark datasets, and show that our approach outperforms both the two-stage 1D CNN-only method and other state of the art approaches.
In recent years, Fuzzy Cognitive Maps (FCMs) have become a convenient knowledge-based tool for economic modeling. Perhaps, the most attractive feature of these cognitive networks relies on their transparency when performing the reasoning process. For example, in the context of time series forecasting, an FCM-based model allows predicting the next outcomes while expressing the underlying behavior behind the investigated system. In this paper, we investigate the forecasting of social security revenues in Jordan using these neural networks. More specifically, we build an FCM forecasting model to predict the social security revenues in Jordan based on historical records comprising the last 120 months. It should be remarked that we include expert knowledge related to the sign of each weights, whereas the intensity in computed by a supervised learning procedure. This allows empirically exploring a sensitive issue in such models: the trade-off between interpretability and accuracy.
Air pollution is related to the concentration of harmful substances in the lower layers of the atmosphere and it is one of the most serious problems threatening the modern way of life. Determination of the conditions that cause maximization of the problem and assessment of the catalytic effect of relative humidity and temperature are important research subjects in the evaluation of environmental risk. This research effort describes an innovative model towards the forecasting of both primary and secondary air pollutants in the center of Athens, by employing Soft Computing Techniques. More specifically, Fuzzy Cognitive Maps are used to analyze the conditions and to correlate the factors contributing to air pollution. According to the climate change scenarios till 2100, there is going to be a serious fluctuation of the average temperature and rainfall in a global scale. This modeling effort aims in forecasting the evolution of the air pollutants concentrations in Athens as a consequence of the upcoming climate change.
Coconut is one of the major perennial food crops that has a long development phase of 44 months. The climatic and seasonal variations affect all stages of coconut’s long development cycle. Besides, the soil composition also plays a vital role in deciding the coconut yield behavior. The present study is focused on categorizing the coconut production level for the given set of agro-climatic conditions using the methodology of fuzzy cognitive map (FCM) enhanced by its learning capabilities. Additionally, an attempt is made to study the impact of climatic variations and weather parameters on the coconut yield behavior using the reasoning capabilities of FCM. Real coconut field data of different seasons for the period from 2009 to 2013 of Kerala state’s Malabar region were used for training and evaluation of the FCM. The present work demonstrates the classification and prediction capabilities of FCM for the described precision agriculture application, with the two most known and efficient FCM learning approaches, viz., nonlinear Hebbian (NHL) and data-driven nonlinear Hebbian (DDNHL). The DDNHL-FCM offers an overall classification accuracy of 96 %. The various case studies furnished in the paper demonstrate the power of NHL-FCM in effectively reasoning new knowledge pertaining to the presented precision agriculture application.
WAY-EEG-GAL is a dataset designed to allow critical tests of techniques to decode sensation, intention, and action from scalp EEG recordings in humans who perform a grasp-and-lift task. Twelve participants performed lifting series in which the object's weight (165, 330, or 660 g), surface friction (sandpaper, suede, or silk surface), or both, were changed unpredictably between trials, thus enforcing changes in fingertip force coordination. In each of a total of 3,936 trials, the participant was cued to reach for the object, grasp it with the thumb and index finger, lift it and hold it for a couple of seconds, put it back on the support surface, release it, and, lastly, to return the hand to a designated rest position. We recorded EEG (32 channels), EMG (five arm and hand muscles), the 3D position of both the hand and object, and force/torque at both contact plates. For each trial we provide 16 event times (e.g., 'object lift-off') and 18 measures that characterize the behaviour (e.g., 'peak grip force').
For various IT systems security is considered a key quality factor. In particular, it might be crucial for video surveillance systems, as their goal is to provide continuous protection of critical infrastructure and other facilities. Risk assessment is an important activity in security management; it aims at identifying assets, threats and vulnerabilities, analysis of implemented countermeasures and their effectiveness in mitigating risks. This paper discusses an application of a new risk assessment method, in which risk calculation is based on Fuzzy Cognitive Maps (FCMs) to a complex automated video surveillance system. FCMs are used to capture dependencies between assets and FCM based reasoning is applied to aggregate risks assigned to lower-level assets (e.g. cameras, hardware, software modules, communications, people) to such high level assets as services, maintained data and processes. Lessons learned indicate, that the proposed method is an efficient and low-cost approach, giving instantaneous feedback and enabling reasoning on effectiveness of security system.
In this paper, we propose an automated computer platform for the purpose of classifying Electroencephalography (EEG) signals associated with left and right hand movements using a hybrid system that uses advanced feature extraction techniques and machine learning algorithms. It is known that EEG represents the brain activity by the electrical voltage fluctuations along the scalp, and Brain-Computer Interface (BCI) is a device that enables the use of the brain’s neural activity to communicate with others or to control machines, artificial limbs, or robots without direct physical movements. In our research work, we aspired to find the best feature extraction method that enables the differentiation between left and right executed fist movements through various classification algorithms. The EEG dataset used in this research was created and contributed to PhysioNet by the developers of the BCI2000 instrumentation system. Data was preprocessed using the EEGLAB MATLAB toolbox and artifacts removal was done using AAR. Data was epoched on the basis of Event-Related (De) Synchronization (ERD/ERS) and movement-related cortical potentials (MRCP) features. Mu/beta rhythms were isolated for the ERD/ERS analysis and delta rhythms were isolated for the MRCP analysis. The Independent Component Analysis (ICA) spatial filter was applied on related channels for noise reduction and isolation of both artifactually and neutrally generated EEG sources. The final feature vector included the ERD, ERS, and MRCP features in addition to the mean, power and energy of the activations of the resulting Independent Components (ICs) of the epoched feature datasets. The datasets were inputted into two machine-learning algorithms: Neural Networks (NNs) and Support Vector Machines (SVMs). Intensive experiments were carried out and optimum classification performances of 89.8 and 97.1 were obtained using NN and SVM, respectively. This research shows that this method of feature extraction holds some promise for the classification of various pairs of motor movements, which can be used in a BCI context to mentally control a computer or machine.
Scikit-learn is an increasingly popular machine learning li- brary. Written
in Python, it is designed to be simple and efficient, accessible to
non-experts, and reusable in various contexts. In this paper, we present and
discuss our design choices for the application programming interface (API) of
the project. In particular, we describe the simple and elegant interface shared
by all learning and processing units in the library and then discuss its
advantages in terms of composition and reusability. The paper also comments on
implementation details specific to the Python ecosystem and analyzes obstacles
faced by users and developers of the library.
In this paper we present different results and studies around the world about the Fuzzy Cognitive Maps. This technique is the fusion of the advances of the fuzzy logic and cognitive maps theories. At this moment, there are several applications in different domains (control, multiagent systems, etc.) and new works (dynamical characteristics, learning procedures, etc.) to improve the performance of these systems. In this paper, we present a survey of different work about this topic. Copyright c 2004-2005 Yang's Scientific Research Institute, LLC. All rights reserved.
Receiver operating characteristics (ROC) graphs are useful for organizing classifiers and visualizing their performance. ROC graphs are commonly used in medical decision making, and in recent years have been used increasingly in machine learning and data mining research. Although ROC graphs are apparently simple, there are some common misconceptions and pitfalls when using them in practice. The purpose of this article is to serve as an introduction to ROC graphs and as a guide for using them in research.
The fuzzy cognitive map (FCM) has gradually emerged as a powerful paradigm for knowledge representation and a simulation mechanism that is applicable to numerous research and application fields. However, since efficient methods to determine the states of the investigated system and to quantify causalities that are the very foundations of FCM theory are lacking, constructing FCMs for complex causal systems greatly depends on expert knowledge. The manually developed models have a substantial shortcoming due to the model subjectivity and difficulties with assessing its reliability. In this paper, we proposed a fuzzy neural network to enhance the learning ability of FCMs. Our approach incorporates the inference mechanism of conventional FCMs with the determination of membership functions, as well as the quantification of causalities. In this manner, FCM models of the investigated systems can automatically be constructed from data and, therefore, operate with less human intervention. In the employed fuzzy neural network, the concept of mutual subsethood is used to describe the causalities, which provides more transparent interpretation for causalities in FCMs. The effectiveness of the proposed approach in handling the prediction of time series is demonstrated through many numerical simulations.
In this paper, a new evolutionary-based fuzzy cognitive map (FCM) methodology is proposed to cope with the forecasting of the patient states in the case of pulmonary infections. The goal of the research was to improve the efficiency of the prediction. This was succeeded with a new data fuzzification procedure for observables and optimization of gain of transformation function using the evolutionary learning for the construction of FCM model. The approach proposed in this paper was validated using real patient data from internal care unit. The results emerged had less prediction errors for the examined data records than those produced by the conventional genetic-based algorithmic approaches.
We introduce a new algorithm for fuzzy cognitive maps learning. The proposed approach is based on the particle swarm optimization method and it is used for the detection of proper weight matrices that lead the fuzzy cognitive map to desired steady states. For this purpose a properly defined objective function that incorporates experts' knowledge is constructed and minimized. The application of the proposed methodology to an industrial control problem supports the claim that the proposed technique is efficient and robust.
Fuzzy cognitive maps (FCMs) are recognized as a flexible and
powerful modeling and simulating technique. However, it is a relatively
new methodology, which exhibits weaknesses mainly in the algorithmic
background. Such weaknesses become evident during heuristic evaluations
of the cause-effect relationships describing FCM-based systems. External
intervention (typically from experts) for the determination and
fine-tuning of the FCM parameters cannot be regarded as an accurate and
efficient way to design and manage FCMs, especially in the case of
highly complicated structures, where even experts meet difficulties in
their attempts at a holistic interpretation. The introduction and
implementation of a training procedure based on a robust and flexible
optimization tool constitutes a promising alternative. This study
focuses on evolutionary computation, since this domain encompasses
optimization techniques possessing the needed features for this type of
problems. Evolution strategies (ESs) appear to be the most appropriate
methodology and, as such, they are tested in this work for a potential
implementation in FCM-based systems. The proposed approach combines FCM
and ES concepts and sets the basis for the establishment and deployment
of structural evolution, which broadens the applicability of FCMs
We propose a new method for estimation in linear models. The ‘lasso’ minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant. Because of the nature of this constraint it tends to produce some coefficients that are exactly 0 and hence gives interpretable models. Our simulation studies suggest that the lasso enjoys some of the favourable properties of both subset selection and ridge regression. It produces interpretable models like subset selection and exhibits the stability of ridge regression. There is also an interesting relationship with recent work in adaptive function estimation by Donoho and Johnstone. The lasso idea is quite general and can be applied in a variety of statistical models: extensions to generalized regression models and tree‐based models are briefly described.
This paper portrays the application of Fuzzy Cognitive Maps (FCMs) to elicit expert views on current condition and future scenario of coastal susceptibility to erosion in Bangladesh. The geomorphological characteristic of the coastal area is highly dynamic where land erosion and accretion with different rates are constant phenomena. This research focuses on three coastal zones: western, central and eastern that comprise the entire coastal area of the country. Using ‘Mental Modeler’ software this study quantified experts' judgements on the issue and developed FCMs by way of arranging workshops. At the basis, this study identified 33 factors of susceptibility to erosion for current baseline conditions. Considering future projections of hydro-climatic phenomena, this study accentuated potential factors of susceptibility to erosion for future scenario under three time-slices: near-future (2020), future (2050) and far-future (2080). The results generated from FCMs show that some factors such as sedimentation, soft and unconsolidated soils, shelf bathymetry, funnel shape of the Bay of Bengal, wave action, river discharge, monsoon wind, cyclone and storm surges, excessive monsoon rain, high tidal energy, variations of tidal range and sea level rise are highly influential that yielded higher centrality scores for both current and future susceptibility of the area to erosion. The experts' interpretations demonstrate that the future susceptibility to erosion might be higher in the central zone compared to the western and eastern zones of the coastal area. This is the first time that FCM based approach was applied to evaluate expert views on coastal susceptibility to erosion for the country. This study suggests coastal managers, planners and policymakers to consider the current and future factors of susceptibility of coastal lands for taking specific measures options. This study is also significant from socio-economic and demographic contexts of any densely populated coastal area like Bangladesh.
Fuzzy cognitive maps (FCMs) have been successfully used to model and predict stationary time series. However, it still remains challenging to deal with large-scale non-stationary time series which have trend and vary rapidly with time. In this paper, we propose a time series prediction model based on the hybrid combination of high-order FCMs with the redundant wavelet transform to handle large-scale non-stationary time series. The model is termed as Wavelet-HFCM. The redundant Haar wavelet transform is applied to decompose original non-stationary time series into multivariate time series, then the high-order FCM is used to model and predict multivariate time series. In learning high-order FCM to represent large-scale multivariate time series, a fast high-order high-order FCM learning method is designed on the basis of ridge regression to reduce the learning time. Finally, summing multivariate time series up yields the predicted time series at each time step. Compared with existing classical methods, the experimental results on eight benchmark datasets show the effectiveness of our proposal, indicating that our prediction model can be applied to various prediction tasks.
The planning and management of river ecosystems affects a variety of social groups (i.e., managers, stakeholders, professionals and users) who have different interests about water uses. To avoid conflicts and reach an environmentally sustainable management, various methods have been devised to enable the participation of these actors. Mathematical modelling of river systems is highly recommended to forecast, but we do not always have enough information to do it. In these cases, the soft and meta-models can be valid alternatives to simulate these complex systems.
Fuzzy
cognitive
map (FCM) is
a
universal tool for modeling dynamic decision support systems. It can be constructed by the experts or learned based on historical data. FCM models learned from data are denser than those created by humans. We developed an evolutionary learning approach for fuzzy cognitive maps based on density and system performance indicators. It allows to select only the most significant connections between concepts and receive the structure more similar to the FCMs initialized by experts. This paper is devoted to the application of the developed approach to model decision support systems with the use of real-life and historical data.
Understanding situations occurring within the physical world by analyzing streams of sensor data is a complex task for both human and software agents. In the area of situation awareness, the observer is typically overwhelmed by information overload and by intrinsic difficulties of making sense of spatially distributed and temporal-ordered sensor observations. Thus, it is desirable to design effective decision-support systems and develop efficient methods to handle sensor data streams. The proposed work is for the comprehension of the situations evolving along the timeline and the projection of recognized situations in the near future. The system analyzes semantic sensor streams, it extracts temporal pattern describing events flow and provides useful insights with respect to the operators’ goals. We implement a hybrid solution for situation comprehension and projection that combines data-driven approach, by using temporal extension of Fuzzy Formal Concept Analysis, and goal-driven approach, by using Fuzzy Cognitive Maps. The cloud-based architecture integrates a distributed algorithm to perform Fuzzy Formal Concept Analysis enabling to deal with deluge of sensor data stream acquired through a sensor-cloud architecture. We discuss the results in terms of prediction accuracy by simulating sensor data stream to early recognize daily life activities inside an apartment.
Interpreting brain waves can be so important and useful in many ways. Having more control on your devices, helping disabled people, or just getting personalized systems that depend on your mood are only some examples of what it can be used for. An important issue in designing a brain-computer interface (BCI) is interpreting the signals. There are many different mental tasks to be considered. In this paper we focus on interpreting left, right, foot and tongue imagery tasks. We use Empirical Mode Decomposition (EMD) for feature extraction and Support Vector Machine (SVM) with Radial Basis Function (RBF) kernel for classification. We evaluate our system on the dataset 2a from BCI competition IV, and very promising classification accuracy that reached 100% is obtained.
This paper describes the construction of intelligent hybrid architectures and the optimization of the fuzzy integrators for time series prediction; interval type-2 fuzzy neural networks (IT2FNN). IT2FNN used hybrid learning algorithm techniques (gradient descent backpropagation and gradient descent with adaptive learning rate backpropagation). The IT2FNN is represented by Takagi–Sugeno–Kang reasoning. Therefore this TSK IT2FNN is represented as an adaptive neural network with hybrid learning in order to automatically generate an interval type-2 fuzzy logic system (TSK IT2FLS). We use interval type-2 and type-1 fuzzy systems to integrate the output (forecast) of each Ensemble of ANFIS models. Particle Swarm Optimization (PSO) was used for the optimization of membership functions (MFs) parameters of the fuzzy integrators. The Mackey-Glass time series is used to test of performance of the proposed architecture. Simulation results show the effectiveness of the proposed approach.
Fuzzy cognitive maps (FCMs) have been used to describe and model the behavior of complex systems. Learning large-scale FCMs from a small amount of data without any a priori knowledge remains an outstanding problem. In particular, a significant challenge arises when limited amounts of data are accompanied by noise. Here, we develop a framework based on the least absolute shrinkage and selection operator (lasso), a convex optimization method, to robustly learn FCMs from noisy data, which is termed LASSOFCM. In LASSOFCM, the task of learning FCMs is decomposed into sparse signal reconstruction problems owing to the sparseness of FCMs. In the experiments, LASSOFCM is applied to learn synthetic data with varying sizes and densities. The results show that LASSOFCM obtains good performance in learning FCMs from time series with or without noise and outperforms the existing methods. Moreover, we apply LASSOFCM to reconstruct gene regulatory networks (GRNs) using the benchmark dataset DREAM3 and DREAM4, and LASSOFCM achieves good performance. LASSOFCM establishes a paradigm for learning large-scale FCMs with high accuracy and has potential applications in a wide range of fields.
People who suffer from neuromuscular disorders and amputated limbs require prosthetic devices that are maneuvered through brain computer interfaces. Electroencephalography is a method to record the activity of the brain that is used for inputs for a brain computer interface. In this paper we propose a method for predicting hand motion phases in grasp-and-lift task from electroencephalography recordings using recurrent networks. Various architectures of recurrent neural networks are compared in terms of performance. For consistent prediction, moving average is applied.
In this paper we propose a new approach to learning fuzzy cognitive maps (FCMs) as a predictive model for time series forecasting. The first contribution of this paper is the dynamic optimization of the FCM structure, i.e., we propose to select concepts involved in the FCM model before every prediction is made. In addition, the FCM transformation function together with the corresponding parameters are proposed to be optimized dynamically. Finally, the FCM weights are learned. In this way, the entire FCM model is learned in a completely new manner, i.e., it is continuously adapted to the current local characteristics of the forecasted time series. To optimize all of the aforementioned elements, we apply and compare 5 different population-based algorithms: genetic, particle swarm optimization, simulated annealing, artificial bee colony and differential evolution. For the evaluation of the proposed approach we use 11 publicly available data sets. The results of comparative experiments provide evidence that our approach offers a competitive forecasting method that outperforms many state-of-the-art forecasting models. We recommend to use our FCM-based approach for the forecasting of time series that are linear and tend to be trend stationary.
The performance evaluation of health, safety and environment management system (HSE-MS) is considered to be an effective way to eliminate out dated measures and help managers adopt proper rectification measures. The objective of this paper is to design a weight distribution model for HSE-MS performance evaluation, the importance of which stems from the current lack of integrated approaches for interpreting and ranking HSE-MS performance evaluation elements. Initially, Fuzzy Cognitive Maps (FCM) is adopted to illustrate the direct and indirect effects of HSE-MS elements on system performance indicators, and the results of FCM are used to develop leading factors helpful for decision making in an intensive management system. Then, the weight distribution from FCM is amended by Relative Degree Analysis (RDA), the aim of which is to combine the advantages of quantitative and qualitative knowledge-driven methods. Finally, the level of HSE-MS performance is obtained and analyzed. The whole performance evaluation framework highlights the potential correlations of evaluation elements as well as expert opinions, which will improve the reasonability of the HSE-MS performance evaluation.
This paper describes an optimization of interval type-2 and type-1 fuzzy integrators in ensembles of ANFIS models with genetic algorithms (GAs), this with emphasis on its application to the prediction of chaotic time series, where the goal is to minimize the prediction error. The Mackey-Glass time series that was considered is to validate the proposed ensemble approach. The methods used for the integration of the ensembles of ANFIS are: type-1 and interval type-2 Mamdani fuzzy inference systems (FIS). Genetic Algorithms are used for optimization of the membership function parameters of the FIS in each integrator. In the experiments we changed the type of the membership functions for each type-1 and interval type-2 FIS, thereby increasing the complexity of the training, The output (Forecast) generated by each integrator is calculated with the RMSE (root mean square error) to minimize the prediction error, therefore we compared the performance obtained by each Fuzzy Integrators
Brain-computer interfaces can be used for motor substitution and recovery; therefore, detection and classification of movement intention is crucial for optimal control. In this study, palmar, lateral and pinch grasps were differentiated from the idle state and classified from single-trial EEG using only information prior the movement onset. Fourteen healthy subjects performed the three grasps 100 times while EEG was recorded from 25 electrodes. Temporal and spectral features were extracted from each electrode, and feature reduction was performed using sequential forward selection (SFS) and principal component analysis (PCA). The detection problem was investigated as the ability to discriminate between movement preparation and the idle state. Furthermore, all task pairs and the three movements together were classified. The best detection performance across movements (79±8%) was obtained by combining temporal and spectral features. The best movement-movement discrimination was obtained using spectral features; 76±9% (2-class) and 63±10% (3-class). For movement detection and discrimination, the performance was similar across grasp types and task pairs; SFS outperformed PCA. The results show it is feasible to detect different grasps and classify the distinct movements using only information prior to the movement onset; which may enable brain-computer interface-based neurorehabilitation of upper limb function through Hebbian learning mechanisms.
This study elaborates on a comprehensive design methodology of Fuzzy Cognitive Maps (FCMs). Here the maps are regarded as a modeling vehicle of time series. It is apparent that whereas time series are predominantly numeric, FCMs are abstract constructs operating at the level of abstract entities referred to as concepts and represented by the individual nodes of the map. We introduce a mechanism to represent a numeric time series in terms of information granules constructed in the space of amplitude and change of amplitude of the time series, which, in turn, gives rise to a collection of concepts forming the corresponding nodes of the FCMs. Each information granule is mapped onto a node (concept) of the map. We identify two fundamental design phases of FCMs, namely (a) formation of information granules mapping numeric data (time series) into activation levels of information granules (viz. the nodes of the map), and (b) optimization of information granules at the parametric level, viz. learning (estimating) the weights between the nodes of the map. The learning is typically realized in a supervised mode on a basis of some experimental data. A construction of information granules is realized with the aid of fuzzy clustering, namely Fuzzy C-Means. The optimization is realized with the use of Particle Swarm Optimization (PSO). The proposed approach is illustrated in detail by a series of experiments using a collection of publicly available data.
Fuzzy cognitive maps (FCMs) are cognition fuzzy influence graphs, which are based on fuzzy logic and neural networks. Many automated learning algorithms have been proposed to reconstruct FCMs from data, but most learned maps using such methods are much denser than those constructed by human experts. To this end, we first model the FCM learning problem as a multiobjective optimization problem, and then propose a multiobjective evolutionary algorithm, labeled as MOEA-FCM, to learn FCM models. MOEA-FCM is able to learn FCMs with varying densities at the same time from input historical data, which can provide candidate solutions with different properties for decision makers. In the experiments, the performance of MOEA-FCM is validated on both synthetic and real data with varying sizes and densities. The experimental results demonstrate the efficiency of MOEA-FCM and show that MOEA-FCM can not only reconstruct FCMs with high accuracy without expert knowledge, but also create a diverse Pareto optimal front which consists of FCMs with varying densities. The significance of this study is that decision makers can choose different FCM models provided by MOEA-FCM based on their practical requirements.
The time series prediction models based on fuzzy set theory have been widely applied to diverse fields such as enrollments, stocks, weather and etc., as they can handle prediction problem under uncertain circumstances in which data are incomplete or vague. Researchers have presented diverse approaches to support the development of fuzzy time series prediction models. While the existing approaches exhibit two evident shortcomings: one is that they have low efficiency of development, which is hardly applicable in the prediction problem involving large-scale time series, and the other is that fuzzy logical relationships mined in an ad hoc way cannot uncover the global characteristics of time series, which reduces accuracy of the resulting model. In this paper, a novel modeling and prediction approach of time series based on synergy of high-order fuzzy cognitive map (HFCM) and fuzzy c-means clustering is proposed, in which fuzzy c-means clustering algorithm is used to construct information granules, transform original time series into granular time series and generate a structure of HFCM prediction model in an automatic fashion. Subsequently depending on historical data of time series, the HFCM prediction model of time series is completely formed by exploiting PSO algorithm to learn all parameters of one. Finally, the developed HFCM prediction model can realize numeric prediction by performing inference in the granular space. Four benchmark time series data sets with different statistical characteristics coming from different areas are applied to validate the feasibility and effectiveness of the proposed modeling approach. The obtained results clearly show the effectiveness of the approach. The developed HFCM prediction models depend on historical data of time series and is emerged in the form of map, which is simpler, legible and have high-level interpretability. Additionally, the proposed approach also exhibits a clear ability to handle the prediction problem of large-scale time series.
Time series are built as a result of real-valued observations ordered in time; however, in some cases, the values of the observed variables change significantly, and those changes do not produce useful information. Therefore, within defined periods of time, only those bounds in which the variables change are considered. The temporal sequence of vectors with the interval-valued elements is called a 'multivariate interval-valued time series.' In this paper, the problem of forecasting such data is addressed. It is proposed to use fuzzy grey cognitive maps (FGCMs) as a non-linear predictive model. Using interval arithmetic, an evolutionary algorithm for learning FGCMs is developed, and it is shown how the new algorithm can be applied to learn FGCMs on the basis of historical time series data. Experiments with real meteorological data provided evidence that, for properly-adjusted learning and prediction horizons, the proposed approach can be used effectively to the forecasting of multivariate, interval-valued time series. The domain-specific interpretability of the FGCM-based model that was obtained also is confirmed.
The area under the receiver operating characteristic (ROC) curve, known as the AUC, is currently considered to be the standard method to assess the accuracy of predictive distribution models. It avoids the supposed subjectivity in the threshold selection process, when continuous probability derived scores are converted to a binary presence-absence variable, by summarizing overall model performance over all possible thresholds. In this manuscript we review some of the features of this measure and bring into question its reliability as a comparative measure of accuracy between model results. We do not recommend using AUC for five reasons: (1) it ignores the predicted probability values and the goodness-of-fit of the model; (2) it summarises the test performance over regions of the ROC space in which one would rarely operate; (3) it weights omission and commission errors equally; (4) it does not give information about the spatial distribution of model errors; and, most importantly, (5) the total extent to which models are carried out highly influences the rate of well-predicted absences and the AUC scores.
In this paper, we offer a new design methodology of type-2 fuzzy models whose intent is to effectively exploit the uncertainty of non-numeric membership functions. A new performance index, which guides the development of the fuzzy model, is used to navigate the construction of the fuzzy model. The underlying idea is that an optimal granularity allocation throughout the membership functions used in the fuzzy model leads to the best design. In contrast to the commonly utilized criterion where one strives for the highest accuracy of the model, the proposed index is formed in such a way so that the type-2 fuzzy model produced intervals, which “cover” the experimental data and at the same time are made as narrow (viz. specific) as possible. Genetic algorithm is proposed to automate the design process and further improve the results by carefully exploiting the search space. Experimental results show the efficiency of the proposed design methodology.
We present a novel per-dimension learning rate method for gradient descent
called ADADELTA. The method dynamically adapts over time using only first order
information and has minimal computational overhead beyond vanilla stochastic
gradient descent. The method requires no manual tuning of a learning rate and
appears robust to noisy gradient information, different model architecture
choices, various data modalities and selection of hyperparameters. We show
promising results compared to other methods on the MNIST digit classification
task using a single machine and on a large scale voice dataset in a distributed
cluster environment.
Fuzzy cognitive maps (FCMs) are fuzzy-graph structures for representing causal reasoning. Their fuzziness allows hazy degrees of causality between hazy causal objects (concepts). Their graph structure allows systematic causal propagation, in particular forward and backward chaining, and it allows knowledge bases to be grown by connecting different FCMs. FCMs are especially applicable to soft knowledge domains and several example FCMs are given. Causality is represented as a fuzzy relation on causal concepts. A fuzzy causal algebra for governing causal propagation on FCMs is developed. FCM matrix representation and matrix operations are presented in the Appendix.
We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together. The elastic net is particularly useful when the number of predictors (p) is much bigger than the number of observations (n). By contrast, the lasso is not a very satisfactory variable selection method in the p≫n case. An algorithm called LARS-EN is proposed for computing elastic net regularization paths efficiently, much like algorithm LARS does for the lasso.
In this paper, we compare the inference capabilities of three different types of fuzzy cognitive maps (FCMs). A fuzzy cognitive map is a recurrent artificial neural network that creates models as collections of concepts/neurons and the various causal relations that exist between these concepts/neurons. In the paper, a variety of industry/engineering FCM applications is presented. The three different types of FCMs that we study and compare are the binary, the trivalent and the sigmoid FCM, each of them using the corresponding transfer function for their neurons/concepts. Predictions are made by viewing dynamically the consequences of the various imposed scenarios. The prediction making capabilities are examined and presented. Conclusions are drawn concerning the use of the three types of FCMs for making predictions. Guidance is given, in order FCM users to choose the most suitable type of FCM, according to (a) the nature of the problem, (b) the required representation capabilities of the problem and (c) the level of inference required by the case.
In order to simulate biological processes, we use multi-agents system. However, modelling cell behavior in systems biology is complex and may be based on intracellular biochemical pathway. So, we have developed in this study a Fuzzy Influence Graph to model MAPK pathway. A Fuzzy Influence Graph is also called Fuzzy Cognitive Map. This model can be integrated in agents representing cells. Results indicate that despite individual variations, the average behavior of MAPK pathway in a cells group is close to results obtained by ordinary differential equations. So, we have also modelled multiple myeloma cells signalling by using this approach.
Fuzzy cognitive maps (FCMs) are a very convenient, simple, and powerful tool for simulation and analysis of dynamic systems. They were originally developed in 1980 by Kosko, and since then successfully applied to numerous domains, such as engineering, medicine, control, and political affairs. Their popularity stems from simplicity and transparency of the underlying model. At the same time FCMs are hindered by necessity of involving domain experts to develop the model. Since human experts are subjective and can handle only relatively simple networks (maps), there is an urgent need to develop methods for automated generation of FCM models. This study proposes a novel learning method that is able to generate FCM models from input historical data, and without human intervention. The proposed method is based on genetic algorithms, and requires only a single state vector sequence as an input. The paper proposes and experimentally compares several different design alternatives of genetic optimization and thoroughly tests and discusses the best design. Extensive benchmarking tests, which involve 200 FCMs with varying size and density of connections, performed on both synthetic and real-life data quantifies the performance of the development method and emphasizes its suitability.
This paper describes an architecture for ensembles of ANFIS (adaptive network based fuzzy inference system), with emphasis on its application to the prediction of chaotic time series, where the goal is to minimize the prediction error. The time series that we are considered are: the Mackey-Glass, Dow Jones and Mexican stock exchange. The methods used for the integration of the ensembles of ANFIS are: integrator by average and the integrator by weighted average. The performance obtained with this architecture overcomes several standard statistical approaches and neural network models reported in the literature by various researchers. In the experiments we changed the type of membership functions and the desired goal error, thereby increasing the complexity of the training.
As a powerful paradigm for knowledge representation and a simulation mechanism applicable to numerous research and application fields, Fuzzy Cognitive Maps (FCMs) have attracted a great deal of attention from various research communities. However, the traditional FCMs do not provide efficient methods to determine the states of the investigated system and to quantify causalities which are the very foundation of the FCM theory. Therefore in many cases, constructing FCMs for complex causal systems greatly depends on expert knowledge. The manually developed models have a substantial shortcoming due to model subjectivity and difficulties with accessing its reliability. In this paper, we propose a fuzzy neural network to enhance the learning ability of FCMs so that the automatic determination of membership functions and quantification of causalities can be incorporated with the inference mechanism of conventional FCMs. In this manner, FCM models of the investigated systems can be automatically constructed from data, and therefore are independent of the experts. Furthermore, we employ mutual subsethood to define and describe the causalities in FCMs. It provides more explicit interpretation for causalities in FCMs and makes the inference process easier to understand. To validate the performance, the proposed approach is tested in predicting chaotic time series. The simulation studies show the effectiveness of the proposed approach.
FCMs are aimed at modeling and simulation of dynamic systems. They exhibit numerous advantages, such as model transparency, simplicity, and adaptability to a given domain, to name a few. FCMs have been applied to numerous industrial and research areas. In some cases generic FCMs suffer from a certain drawback that originates from their definition and concerns a limited, first-order dynamics of processing realized at the nodes of the maps. In this study, we introduce a concept of higher-order memory based FCMs. The proposed extension modifies the simulation model of a generic FCM while it does not negatively impact transparency and simplicity of the model itself. We discuss several architectural alternatives along with the ensuing computing and optimization aspects. Preliminary experimental results included in this paper show superiority of the extended higher-order memory based FCMs over a generic FCM in terms of the modeling accuracy
In this paper, we introduce a novel approach to time-series prediction realized both at the linguistic and numerical level. It exploits fuzzy cognitive maps (FCMs) along with a recently proposed learning method that takes advantage of real-coded genetic algorithms. FCMs are used for modeling and qualitative analysis of dynamic systems. Within the framework of FCMs, the systems are described by means of concepts and their mutual relationships. The proposed prediction method combines FCMs with granular, fuzzy-set-based model of inputs. One of their main advantages is an ability to carry out modeling and prediction at both numerical and linguistic levels. A comprehensive set of experiments has been carried out with two major goals in mind. One is to assess quality of the proposed architecture, the other to examine the influence of its parameters of the prediction technique on the quality of prediction. The obtained results, which are compared with other prediction techniques using fuzzy sets, demonstrate that the proposed architecture offers substantial accuracy expressed at both linguistic and numerical levels.
The area under the ROC (receiver operating characteristics) curve, or simply AUC, has been traditionally used in medical diagnosis since the 1970s. It has recently been proposed as an alternative single-number measure for evaluating the predictive ability of learning algorithms. However, no formal arguments were given as to why AUC should be preferred over accuracy. We establish formal criteria for comparing two different measures for learning algorithms and we show theoretically and empirically that AUC is a better measure (defined precisely) than accuracy. We then reevaluate well-established claims in machine learning based on accuracy using AUC and obtain interesting and surprising new results. For example, it has been well-established and accepted that Naive Bayes and decision trees are very similar in predictive accuracy. We show, however, that Naive Bayes is significantly better than decision trees in AUC. The conclusions drawn in this paper may make a significant impact on machine learning and data mining applications.