No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Design of wavelet neural networks based on symmetry fuzzy C-means for function approximation
Abstract
Specifying the number and locations of the translation vectors for wavelet neural networks (WNNs) is of paramount significance as the quality of approximation may be drastically reduced if initialization of WNNs parameters was not done judiciously. In this paper, an enhanced fuzzy C-means algorithm, specifically the modified point symmetry–based fuzzy C-means algorithm (MPSDFCM), was proposed, in order to determine the optimal initial locations for the translation vectors. The proposed neural network models were then employed in approximating five different nonlinear continuous functions. Assessment analysis showed that integration of the MPSDFCM in the learning phase of WNNs would lead to a significant improvement in WNNs prediction accuracy. Performance comparison with the approaches reported in the literature in approximating the same benchmark piecewise function verified the superiority of the proposed strategy.
... Despite the fact that WNNs have proven its reliability and generality in diverse fields [3][4][5][6][7][8], its sensitivity toward the number and initial prior distribution of the translation parameters still remains an open issue. Randomness in initial translation parameters selection might neglect the contributable attributes residing in the input space that could be constructive in generating an optimal interpolation between the input-output spaces. ...
... The formulated explicit expression may suit a particular data, but may fail miserably with a subtle change in parameter, and the most challenging is, the nature of the data is usually unknown prior to analysis. Other optimization techniques that draw their inspiration from the unsupervised learning, like K-means (KM) algorithm [10], fuzzy C-means algorithm [7], hierarchical clustering [11] and connectionist clustering [12], are also employed for the WNNs initialization. However, these methods are sensitive to the chosen initial cluster centers. ...
The effectiveness of swarm intelligence has been proven to be at the heart of various optimization problems. In this study, a recently developed nature-inspired algorithm, specifically the firefly algorithm (FA), is integrated in the learning strategy of wavelet neural networks (WNNs). The FA, which systematically optimizes the initial location of the translation parameters for WNNs, has reduced the number of hidden nodes while simultaneously improved the generalization capability of WNNs significantly. The applicability of the proposed model was demonstrated through empirical simulations for function approximation study, with both synthetic and real-world data. Performance assessment demonstrated its enhancement over the K-means clustering and random initialization approaches, as well as to the other neural network models reported in the literature, whereby a noteworthy decrease in the approximation error was observed.
... In this study, as the first step in building prediction models, FCM clustering was used to segment the measured PPV datasets of the Tan Dong Hiep quarry. Additional details on FCM can be found in numerous published materials (Bezdek et al. 1984;Cannon et al. 1986;Hung and Yang 2001;Liu et al. 2008;Zainuddin and Ong 2013). ...
Predicting and reducing blast-induced ground vibrations is a common concern among engineers and mining enterprises. Dealing with these vibrations is a challenging issue as they may result in the instability of the surrounding structures, highways, water pipes, railways, and residential areas. In this study, the effects of blasting in a quarry mine in Vietnam were examined. A total of 25 blasting events were investigated with the help of an unmanned aerial vehicle, micromate instruments, and blast patterns, and 83 observations were recorded. Subsequently, the fuzzy C-means clustering (FCM) algorithm was applied to classify the 83 observations based on the blast parameters. Finally, based on the classification of the blasts, quantile regression neural network (QRNN) models were developed. The combination of FCM and QRNN models resulted in a novel, hybrid model (FCM-QRNN) for predicting blast-induced ground vibration. The US Bureau of Mines (USBM), random forest (RF), QRNN (without clustering), and artificial neural network (ANN) models were also considered and compared with the FCM-QRNN model to obtain a comprehensive assessment of the proposed model. The results indicate that the proposed FCM-QRNN model has a higher accuracy than the other models: USBM, QRNN, RF, and ANN. The proposed model can be used to control the undesirable effects of blast-induced ground vibration. Although this study and the proposed FCM-QRNN model are original works with positive results; however, the performance of this model in other locations still needs to be considered as a case study for further scientific information.
... The obtained prediction errors of 9.5235e−04, 7.8581e−04 and 8.5370e −04 for IC-1, IC-2, and IC-3, respectively, are the lowest among all considered forecasting models. This may be attributed to the compact topology of WNN and the constitution of fast-decaying localized wavelet activation functions in the hidden layer, contributing to superior generalization capability in this regard [40]. The low percentage deviations of the WNN prediction values from the actual flank wear shown in Table 7-less than 10% for IC-1, less than 8% for IC-2, and less than 10% for IC-3-demonstrate that the WNN is able to model the tool wear progression satisfactorily. ...
The monitoring of tool condition in machining processes has significant importance to control the quality of machined parts and to reduce equipment downtime. This study investigates the application of a special variant of artificial neural networks (ANNs), in particular, wavelet neural network (WNN) for tool wear monitoring in CNC end milling process of high-speed steel. A mixed level design of experiments with machining parameters of cutting speed, feed rate, cutting depth, and machining time is developed, from which 126 experiments are conducted. For each experiment, tool wear and surface roughness of machined workpiece are measured. The tool wear images are processed, and the descriptor of wear zone is extracted. The WNN is then applied to predict the flank wear of the cutting tool and compared with commonly used types of ANNs and the statistical model. Different input combinations with the inclusion of wear zone descriptor and surface roughness of machined parts are used to evaluate the performance of all models. Results show that the WNN with the input parameters of cutting speed, feed rate, depth of cut, machining time, and descriptor of wear zone predicts the degree of tool wear most accurately.
... In order to solve these problems, people have adopted a variety of optimization measures to improve it, especially, in the aspect of reducing the dependence on the initial value to some extent [6]. However, those improvements which are mainly more complicated to implement have a mass of computational cost and more iterations [7]. ...
Fuzzy C-means (FCM) is a classical algorithm of cluster analysis which has been applied to many fields including artificial intelligence, pattern recognition, data aggregation and their applications in software engineering, image processing, IoT, etc. However, it is sensitive to the initial value selection and prone to get local extremum. The classification effect is also unsatisfactory which limits its applications severely. Therefore, this paper introduces the artificial-fish-swarm algorithm (AFSA) which has strong global search ability and adds an adaptive mechanism to make it adaptively adjust the scope of visual value, improves its local and global optimization ability, and reduces the number of algorithm iterations. Then it is applied to the improved FCM which is based on the Mahalanobis distance, named as adaptive AFSA-inspired FCM(AAFSA-FCM). The optimal solution obtained by adaptive AFSA (AAFSA) is used for FCM cluster analysis to solve the problems mentioned above and improve clustering performance. Experiments show that the proposed algorithm has better clustering effect and classification performance with lower computing cost which can be better to apply to every relevant area, such as IoT, network analysis, and abnormal detection.
... BCO algoritması, bulanık bölünmeli kümeleme tekniklerinden en iyi bilinen ve en yaygın kullanılanıdır. Günümüzde, BCO algoritması, örüntü tanıma [7], görüntü işleme [8] [9], fonksiyon tahmini [10], sınıflandırıcı tasarımı [11] [12] ve belge madenciligi [13] çalışmalarında yaygın olarak kullanılmaktadır. Çalışmanın amaçlarından birisi de, BCO algoritması ile bulanık kümeleme yönteminin uzaktan egitim sistemindeki ögrenci etkinliklerini ne ölçüde modelledigini göstermektir. ...
Bu çalışma, öğrencilerin uzaktan eğitim sisteminde yılsonu akademik performanslarının belirlenmesinde kullanılacak olan ders etkinliklerine katılım derecesini Bulanık C Ortalamalar algoritması kullanarak kümeleyen bir yöntem sunmaktadır. Çalışmada örneklem olarak, Muğla Sıtkı Koçman Üniversitesi’nde uzaktan eğitim yoluyla verilen Bilgi ve İletişim Teknolojileri dersini son üç yıl içerisinde alan 6659 öğrencinin ders etkinlik verileri seçilmiştir. Öğrencilerin Moodle öğrenme yönetim sistemine giriş yaptıktan sonra ders sayfasını ne sıklıkla takip ettikleri, ders materyallerini hangi sıklıkla inceledikleri,
ders videolarını izleme süreleri ve çevrimiçi derslere katılım süreleri bilgileri veri kümesini oluşturmuştur. Gürültülü verilerden arındırılan veri kümesi KNIME veri madenciliği platformunda analiz edilmiştir. Analiz sonucunda ham veri bulanık kümelere otomatik olarak dönüştürülmektedir. Çalışma sonucunda elde edilen bulanık kümeler, öğretim elemanlarına öğrencilerin yılsonu alacakları performans puanının belirlenmesine yönelik bilgiler sağlamaktadır.
... From a finite data set, the basic task of a function approximation method is to find the suitable relationship between variables and their corresponding responses [4]. There are different approaches of the function approximation including analytical methods such as least squares linear approximation, polynomial approximation, and shape-preserving approximation in addition to many intelligent methods such as approximation with Fuzzy [5][6], Neural Networks (NNs) [7][8] or combined between neural and fuzzy [3,[9][10]. Both NNs and fuzzy logic can be recommended as universal function approximators, provided that sufficient hidden neurons in NN or rules in fuzzy logic [3] can give good performance for nonlinear function approximation. ...
This work deals with an approximation of functions which finds the underlying relationship from an available finite input-output data of the function. It is the fundamental problem in a majority of real world applications, such as signal processing, prediction, data mining and control system. In this paper five different methods are used to verify their efficiency of approximation: MLPNN, RBFNN, GRNN, FIS and ANFIS networks. The performance is compared by using the RMSE measurement as an indicator of the fitness of these models in function approximation problem. The experimental results show that the performance of all networks used in this work at the training process is more different at the checking process when the networks have been tested with unknown data points. This depends on many factors such as type of networks used to approximate the function, available training data, noise in the data and values of the required parameters for training each network (No. of layers, No. of neurons, No. of training epochs, etc.).
... The commonly used wavelet activation functions are Mexican Hat, Morlet, or Gaussian wavelet. Further details of the parameter initialization and the learning process are omitted here for brevity and are explained elsewhere [17,18]. ...
Properly determining the discriminative features which characterize the inherent behaviors of electroencephalography (EEG) signals remains a great challenge for epileptic seizure detection. In this present study, a novel feature selection scheme based on the discrete wavelet packet decomposition and cuckoo search algorithm (CSA) was proposed. The normal as well as epileptic EEG recordings were first decomposed into various frequency bands by means of wavelet packet decomposition, and subsequently, statistical features at all developed nodes in the wavelet packet decomposition tree were derived. Instead of using the complete set of the extracted features to construct a wavelet neural networks-based classifier, an optimal feature subset that maximizes the predictive competence of the classifier was selected by using the CSA. Experimental results on the publicly available benchmarks demonstrated that the proposed feature subset selection scheme achieved promising recognition accuracies of 98.43–100%, and the results were statistically significant using z-test with p value <0.0001.
... Cui R et al in [22] use an adaptive NN to control autonomous underwater vehicles (AUVs). As a special kind of NNs, wavelet neural network (WNN) has been widely used [26][27][28][29][30][31][32][33][34][35][36]. Wavelets can not only become a set of standard orthogonal basis in L 2 ðRÞ space, but also to provide unconditional basis for many other spaces, such as Sobolev spaces, Hölder space, L p ðRÞ space. ...
This paper investigates the distributed cooperative learning (DCL) problems over networks, where each node only has access to its own data generated by the unknown pattern (map or function) uniformly, and all nodes cooperatively learn the pattern by exchanging local information with their neighboring nodes. These problems cannot be solved by using traditional centralized algorithms. To solve these problems, two novel DCL algorithms using wavelet neural networks are proposed, including continuous-time DCL (CT-DCL) algorithm and discrete-time DCL (DT-DCL) algorithm. Combining the characteristics of neural networks with the properties of the wavelet approximation, the wavelet series are used to approximate the unknown pattern. The DCL algorithms are used to train the optimal weight coefficient matrix of wavelet series. Moreover, the convergence of the proposed algorithms is guaranteed by using the Lyapunov method. Compared with existing distributed optimization strategies such as distributed average consensus (DAC) and alternating direction method of multipliers (ADMM), our DT-DCL algorithm requires less information communications and training time than ADMM strategy. In addition, it achieves higher accuracy than DAC strategy when the network consists of large amounts of nodes. Moreover, the proposed CT-DCL algorithm using a proper step size is more accurate than the DT-DCL algorithm if the training time is not considered. Several illustrative examples are presented to show the efficiencies and advantages of the proposed algorithms.
... From a finite data set, the basic task of a function approximation method is to find the suitable relationship between variables and their corresponding responses [4]. There are different approaches of the function approximation including analytical methods such as least squares linear approximation, polynomial approximation, and shape-preserving approximation in addition to many intelligent methods such as approximation with Fuzzy [5][6], Neural Networks (NNs) [7][8] or combined between neural and fuzzy [3,[9][10]. Both NNs and fuzzy logic can be recommended as universal function approximators, provided that sufficient hidden neurons in NN or rules in fuzzy logic [3] can give good performance for nonlinear function approximation. ...
A new multi-model predictive control (MMPC) algorithm was proposed and applied in an intelligent vehicle lateral tracking control system in this paper, which is better to adapt the intelligent vehicle lateral tracking control under complex multi-conditions. First, the Gustafson–Kessel algorithm was used for the cluster analysis based on the vehicle test data to obtain the clustering center and train the sample data of each typical steering condition. Then, a multi-model structure was constructed by least squares support vector machines, and the sub-models of each category were taken as the prediction model for the application of MPC. Hence, the objective function of multi-objective optimization can be established and the multi-objective optimization problem was solved by the non-dominated sorted genetic algorithm-II algorithm to obtain the optimal control quantity. Finally, the MMPC-based intelligent vehicle lateral tracking control system was used to control the vehicle lateral tracking under three steering conditions, including straight line, normal right turn, and U-turn, through a simulation study in the MATLAB/Simulink environment. By comparing the vehicle trajectory, steering wheel angle, lateral deviation, and lateral angle, the performance of the proposed control method was verified. The experimental analysis results show that the proposed method can track the steering wheel angle of the vehicle reference trajectory under various working conditions. The vehicle lateral deviation value can be controlled in the range of (−1.0 m, 0.5 m). The high-precision lateral tracking control ensures that the yaw rate of the vehicle can track the yaw velocity under the reference driving track and guarantees the driving stability of intelligent vehicles.
Determining the optimal number of hidden nodes and their proper initial locations are essentially crucial before the wavelet neural networks (WNNs)start their learning process. In this paper, a novel strategy known as the modified cuckoo search algorithm (MCSA), is proposed for WNNs initialization in order to improve its generalization performance. The MCSA begins with an initial population of cuckoo eggs, which represent the translation vectors of the wavelet hidden nodes, and subsequently refines their locations by imitating the breeding mechanism of cuckoos. The resulting solutions from the MCSA are then used as the initial translation vectors for the WNNs. The feasibility of the proposed method is evaluated by forecasting a benchmark chaotic time series, and its superior prediction accuracy compared with that of conventional WNNs demonstrates its potential benefit.
Driver's intention classification and identification is identified as the key technology for intelligent vehicles and is widely used in a variety of advanced driver assistant systems (ADAS). To study driver's steering intention under different typical operating conditions, five driving school coaches of different ages and genders are selected as the test drivers for a real vehicle test. Four kinds of typical car steering condition test data with four different vehicles are collected. Test data are filtered by the Butterworth filter and are used for extracting the driver steering characteristic parameters. Based on Principal Component Analysis (PCA), the three kinds of clustering analysis methods, including the Fuzzy C-Means algorithm (FCM), the Gustafson-Kessel algorithm (GK) and the Gath-Geva algorithm (GG), considered are proposed to classify and identify driver's intention under different typical operating conditions. Results show that the three approaches can successfully classify and identify drivers' intention respectively despite some accuracy error by FCM. Meanwhile, compared with FCM and GK, GG was the best performing in classification and identification of the driver's intention. In order to verify the validity of the identification method designed by this article, five different drivers were selected. Five tests were carried out on the driving simulator. The results show that the results of each identification are exactly the same as the actual driver's intention.
Due to microarray experiment imperfection, spots with various artifacts are often found in microarrayimage. A more rigorous spot recognition approach in ensuring successful image analysis is crucial. In this paper, a novel hybrid algorithm was proposed. A wavelet approach was applied, along with anintensity-based shape detection simultaneously to locate the contour of the microarray spots. The proposedalgorithm segmented all the imperfect spots accurately. Performance assessment with the classicalmethods, i.e., the fixed circle, adaptive circle, adaptive shape and histogram segmentation showed thatthe proposed hybrid approach outperformed these methods.
Recently sparse representation (SR) based clustering has attracted a growing interests in the field of image processing and pattern recognition. Since the SR technology has favorable category distinguishing ability, we introduce it into the fuzzy clustering in this paper, and propose a new clustering algorithm, called sparse learning based fuzzy c-means (SL_FCM). Firstly, to reduce the computation complexity of the SR based FCM method, most energy of discriminant feature obtained by solving a SR model is reserved and the remainder is discarded. By this way, some redundant information (i.e. the correlation among samples of different classes) in the discriminant feature can be also removed, which can improve the clustering quality. Furthermore, to further enhance the clustering performance, the position information of valid values in discriminant feature is also used to re-define the distance between sample and clustering center in SL_FCM. The weighted distance in SL_FCM can enhance the similarity of the samples from the same class and the difference of the samples of different classes, thus to improve the clustering result. In addition, as the dimension of stored discriminant feature of each sample is different, we use set operations to formulate the distance and cluster center in SL_FCM. The comparisons on several datasets and images demonstrate that SL_FCM performs better than other state-of-art methods with higher accuracy, while keeps low spatial and computational complexity, especially for the large scale dataset and image.
Automatic seizure detection is of great importance for speeding up the inspection process and relieving the workload of medical staff in the analysis of EEG recordings. In this study, a method based on an improved wavelet neural network (WNN) is proposed for automatic seizure detection in long-term intracranial EEG. WNN combines the traditional back propagation neural network (BPNN) with wavelet transform. Compared with classic WNN architectures, a modified point symmetry-based fuzzy c-means (MSFCM) algorithm is applied to the initialization of wavelet transform's translations, which has been successful in multiclass cancer classification. In addition, Fast-decaying Morlet wavelet is chosen as the activation function to make the WNN learn faster. Relative amplitude and relative fluctuation index are extracted as a feature vector to describe the variation of EEG signals, and the feature vector is then fed into WNN for classification. At last, post-processing including smoothing, channel fusion and collar technique is adopted to achieve more accurate and stable results. This system performs efficiently with the average sensitivity of 96.72%, specificity of 98.91% and false-detection rate of 0.27 h−1. The proposed approach achieves high sensitivity and low false detection rate, which demonstrates its potential for clinical usage.
This paper presents a new analysis of four time series of paleoclimate data, using wavelet methods to determine the timings and characteristics of the ice-age terminations. We use an antisymmetric wavelet which detects locations in the time series where there are large changes. The results are consistent with previous work, but we have, for the first time, been able to explore the features of the individual terminations in an objective way. We find that at the terminations the interval between atmospheric carbon dioxide changes and sea level is about 7 kyr, compared with 15 kyr obtained using Fourier methods, which average the phase over the whole 100 kyr signal. The sea level change is the fastest but occurs after the other three components.
In this paper, a modified version of the FCM algorithm is presented to deal with clusters with totally different geometrical properties. The proposed algorithm adopts a novel non-metric distance measure based on the idea of "point symmetry". Experimental results on several data sets are presented to illustrate its effectiveness.
A wavelet network is an important tool for analyzing time series especially when it is nonlinear and non-stationary. It takes advantage of high resolution of wavelets and learning and feed forward nature of Neural Networks. Wavelets are a class of functions such that multiple resolution nature of wavelets provides a natural frame work for the anal-ysis of time series. The power of this network to approximate functions from given input-output data is proved and it has utilized the local-ization property of a wavelet to focus on local properties. Guaranteed upper bounds on the accuracy of approximation is established. Here we are analyzing the time series of number of terrorist attacks in the world measured on monthly basis during the period February 1968 to January 2007 for establishing the superiority of this method over other existing methods. The simulation results show that the model is capable of producing a reasonable accuracy within several steps.
In this paper, we propose a Wavelet Neural Network with Hybrid Learning Ap- proach (WNN-HLA). A novel hybrid learning approach, which combines the on-line partition method (OLPM) and the gradient descent method, is proposed to identify a par- simonious internal structure and adjust the parameters of WNN-HLA model. First, the proposed OLPM is an online method and is a distance-based connectionist clustering method. Unlike the traditional cluster techniques that only consider the total variation to update the one mean and deviation. Second, a back-propagation learning method is used to adjust the parameters for the desired outputs. Several simulation examples have been given to illustrate the performance and effectiveness of the proposed model. The com- puter simulations demonstrate that the proposed WNN-HLA model performs better than some existing models.
In this paper, we propose simple but effective two different fuzzy wavelet networks (FWNs) for system identification. The FWNs combine the traditional Takagi-Sugeno-Kang (TSK) fuzzy model and discrete wavelet transforms (DWT). The proposed FWNs consist of a set of if-then rules and, then parts are series expansion in terms of wavelets functions. In the first system, while the only one scale parameter is changing with it corresponding rule number, translation parameter sets are fixed in each rule. As for the second system, DWT is used completely by using wavelet frames. The performance of proposed fuzzy models is illustrated by examples and compared with previously published examples. Simulation results indicate the remarkable capabilities of the proposed methods. It is worth noting that the second FWN achieves high function approximation accuracy and fast convergence.
A wavelet network concept, which is based on wavelet transform theory, is proposed as an alternative to feedforward neural networks for approximating arbitrary nonlinear functions. The basic idea is to replace the neurons by ;wavelons', i.e., computing units obtained by cascading an affine transform and a multidimensional wavelet. Then these affine transforms and the synaptic weights must be identified from possibly noise corrupted input/output data. An algorithm of backpropagation type is proposed for wavelet network training, and experimental results are reported.
We propose a modified version of the K-means algorithm to cluster
data. The proposed algorithm adopts a novel nonmetric distance measure
based on the idea of “point symmetry”. This kind of
“point symmetry distance” can be applied in data clustering
and human face detection. Several data sets are used to illustrate its
effectiveness
Designing a wavelet neural network (WNN) needs to be done judiciously in attaining the optimal generalization performance. Its prediction competence relies highly on the initial value of translation vectors. However, there is no established solution in determining the appropriate initial value for the translation vectors at this moment. In this paper, we propose a novel enhanced fuzzy c-means clustering algorithm - specifically, the modified point symmetry-based fuzzy c-means (MPSDFCM) algorithm - in initializing the translation vectors of the WNNs. The effectiveness of embedding different activation functions in WNNs will be investigated as well. The categorization effectiveness of the proposed WNNs model was then evaluated in classifying the type 2 diabetics, and was compared with the multilayer perceptrons (MLPs) and radial basis function neural networks (RBFNNs) models. Performance assessment shows that our proposed model outperforms the rest, since a 100% superior classification rate was achieved.
It is very difficult to generalize the relationship between MFL signal and the defect geometric parameters of the pipeline because the relationship is nonlinear. Many applications of wavelet neural network on this field show that the defect geometric parameters can be obtained with this method to some extent. However, the initial centers have great influence on performance of the network. Hierarchical clustering algorithm is proposed in this paper and applied to classification of defect samples, centers selection and calculation of basis function width. With this algorithm, clusters similarity is computed to create tree structure and the perfect clustering is obtained. The sample set created from finite element defect simulation are used to train and validate the efficiency and reliability of the network based on hierarchical clustering algorithm. The experiment shows that the training speed and the prediction precision of the network can be improved simulataneously.
The crystal structure of tetracarbonyl-1κ 2 C,2κ 2 C-μ-{2-[α-ethoxy-4-(trifluoromethyl)benzylidene-1κC α ]-1(1,2-η):2(3,4,5,6-η)-1,3,5-cycloheptatriene}-μ-ethoxymethylidyne-1:2κ 2 C-diiron has been determined by X-ray diffraction analysis. The Fe atoms coordinated by carbonyl and cycloheptatriene groups are unsymmetrically bridged by a carbyne ligand. The extensively delocalized bonding may be found over the entire metal-ligand framework
The WaveARX network, a new neural network architecture, is introduced. Its development was motivated by the opportunity to capitalize on recent research results that allow some shortcomings of the traditional artificial neural net (ANN) to be addressed. ANN has been shown to be a valuable tool for system identification but suffers from slow convergence and long training time due to the globalized activation function. The structure of ANN is derived from trial and error procedures, and the trained network parameters often are strongly dependent on the random selection of the initial values. There are not even guidelines on the number of neurons needed. Also, few identification techniques are available for distinguishing linear from nonlinear contributions to a system's behavior. The WaveARX integrates the multiresolution analysis concepts of the wavelet transform and the traditional AutoRegressive eXternal input model (ARX) into a three-layer feedforward network. Additional network design problems are solved as the WaveARX formalisms provide a systematic design synthesis for the network architecture, training procedure, and excellent initial values of the network parameters. The new structure also isolates and quantifies the linear and nonlinear components of the training data sets. The wavelet function is extended to multidimensional input space using the concept of a norm. The capabilities of the network are demonstrated through several examples in comparison with some widely used linear and nonlinear identification techniques. Separately, the wavelet network of the WaveARX model is shown for the example investigated to have a better performance than two other existing wavelet-based neural networks.
A simple model based on the combination of neural network and wavelet techniques named wavelet neural network (WNN) is proposed.
Thanks to the time-frequency analysis feature of wavelet, a selection method that takes into account the domain of input space
where the wavelets are not zero is used to initialize the translation and dilation parameters. A proper choice of initialize
parameters is found to be crucial in achieving adequate training. Training algorithms for feedback WNN is discussed too. Results
obtained for a nonlinear processes is presented to test the effectiveness of the proposed method. The simulation result shows
that the model is capable of producing a reasonable accuracy within several steps.
This paper presents the development of fuzzy wavelet neural network system for time series prediction that combines the advantages
of fuzzy systems and wavelet neural network. The structure of fuzzy wavelet neural network (FWNN) is proposed, and its learning
algorithm is derived. The proposed network is constructed on the base of a set of TSK fuzzy rules that includes a wavelet
function in the consequent part of each rule. A fuzzy c-means clustering algorithm is implemented to generate the rules, that is the structure of FWNN prediction model, automatically,
and the gradient-learning algorithm is used for parameter identification. The use of fuzzy c-means clustering algorithm with the gradient algorithm allows to improve convergence of learning algorithm. FWNN is used
for modeling and prediction of complex time series and prediction of foreign-exchange rates. Exchange rates are dynamic process
that changes every day and have high-order nonlinearity. The statistical data for the last 2years are used for the development
of FWNN prediction model. Effectiveness of the proposed system is evaluated with the results obtained from the simulation
of FWNN-based systems and with the comparative simulation results of previous related models.
KeywordsTime series prediction–Fuzzy wavelet neural network–Prediction of exchange rates–Neuro-fuzzy modeling
A new structure of wavelet neural networks (WNN) with extreme learning machine (ELM) is introduced in this paper. In the proposed wavelet neural networks, composite functions are applied at the hidden nodes and the learning is done using ELM. The input information is first processed by wavelet functions and then passed through a type of bounded nonconstant piecewise continuous activation functions g:R→R. A selection method that takes into account the domain of input space where the wavelets are not zero is used to initialize the translation and dilation parameters. The formed wavelet neural network is then trained with the computationally efficient ELM algorithm. Experimental results on the regression of some nonlinear functions and real-world data, the prediction of a chaotic signal and classifications on serval benchmark real-world data sets show that the proposed neural networks can achieve better performances in most cases than some relevant neural networks and learn much faster than neural networks training with the traditional back-propagation (BP) algorithm.
We propose a novel method for the identification of non-linear system by utilizing some of the important properties of wavelets like denoising, compression, multiresolution along with the concepts of fuzzy logic. Two new type-2 fuzzy wavelet networks (T2FWNs) are proposed here. These T2FWNs can handle rule uncertainties in a better way because of using the type-2 fuzzy sets in modeling and fuzzy differential (FD) and Lyapunov stability during learning. Lot of work has been done in the identification of non-linear system by using the models based on type-1 fuzzy logic system (FLS). But in practice they are unable to handle uncertainties in the rules. The robustness of the system is assured by Lyapunov stability (LS). Also we have explored the properties of wavelets and FLS to handle the uncertainties efficiently. As the stability of the model is highly dependent on the learning of the system we use Lyapunov stability in combination with fuzzy differential. FD gives the range of variation of parameters having lower and upper bound in which the system is stable. The performance of T2FWN is compared with type-1 FLS, FWN [D.W.C. Ho, P.-A. Zhang, J. Xu, Fuzzy wavelet networks for function learning, IEEE Trans. Fuzzy Syst. 9 (February (1)) 2000] and FWNN [S. Srivastava, M. Singh, M. Hanmandlu, A.N. Jha, New fuzzy wavelet neural networks for system identification and control, Intl. J. Appl. Soft Comput. 6 (November (I)) 2005, 1–17]. It is shown that noise and disturbance in the reference signal are reduced with wavelets. A comparison of three learning algorithms: (i) gradient descent (GD) (ii) a combination of Lyapunov stability and fuzzy differential (LSFD) and, (iii) a combination of (i) and (ii) is done.
A 1-norm support vector machine stepwise (SVMS) algorithm is proposed for the hidden neurons selection of wavelet networks (WNs). In this new algorithm, the linear programming support vector machine (LPSVM) is employed to pre-select the hidden neurons, and then a stepwise selection algorithm based on ridge regression is introduced to select hidden neurons from the pre-selection. The main advantages of the new algorithm are that it can get rid of the influence of the ill conditioning of the matrix and deal with the problems that involve a great number of candidate neurons or a large size of samples. Four examples are provided to illustrate the efficiency of the new algorithm.
Magnetic flux leakage techniques are used extensively to detect and characterize defects in natural gas transmission pipelines. This paper presents a novel approach for training a multiresolution, hierarchical wavelet basis function (WBF) neural network for the three-dimensional characterization of defects from magnetic flux leakage signals. Gaussian radial basis functions and Mexican hat wavelet frames are used as scaling functions and wavelets respectively. The centers of the basis functions are calculated using a dyadic expansion scheme and a k-means clustering algorithm. The results indicate that significant advantages over other neural network based defect characterization schemes could be obtained, in that the accuracy of the predicted defect profile can be controlled by the resolution of the network. The feasibility of employing a WBF neural network is demonstrated by predicting defect profiles from both simulation data and experimental magnetic flux leakage signals.
We present an original initialization procedure for the parameters of feedforward wavelet networks, prior to training by gradient-based techniques. It takes advantage of wavelet frames stemming from the discrete wavelet transform, and uses a selection method to determine a set of best wavelets whose centers and dilation parameters are used as initial values for subsequent training. Results obtained for the modeling of two simulated processes are compared to those obtained with a heuristic initialization procedure, and the effectiveness of the proposed method is demonstrated.
Based on the recently published point symmetry distance (PSD) measure, this paper presents a novel PSD measure, namely symmetry similarity level (SSL) operator for K-means algorithm. Our proposed modified point symmetry-based K-means (MPSK) algorithm is more robust than the previous PSK algorithm by Su and Chou. Not only the proposed MPSK algorithm is suitable for the symmetrical intra-clusters as the PSK algorithm does, the proposed MPSK algorithm is also suitable for the symmetrical inter-clusters. In addition, two speedup strategies are presented to reduce the time required in the proposed MPSK algorithm. Experimental results demonstrate the significant execution-time improvement and the extension to the symmetrical inter-clusters of the proposed MPSK algorithm when compared to the previous PSK algorithm.
The selection of hyper-parameters in support vector machines (SVM) is a key point in the training process of these models when applied to regression problems. Unfortunately, an exact method to obtain the optimal set of SVM hyper-parameters is unknown, and search algorithms are usually applied to obtain the best possible set of hyper-parameters. In general these search algorithms are implemented as grid searches, which are time consuming, so the computational cost of the SVM training process increases considerably. This paper presents a novel study of the effect of including reductions in the range of SVM hyper-parameters, in order to reduce the SVM training time, but with the minimum possible impact in its performance. The paper presents reduction in parameter C, by considering its relation with the rest of SVM hyper-parameters (γ and ε), through an approximation of the SVM model. On the other hand, we use some characteristics of the Gaussian kernel function and a previous result in the literature to obtain novel bounds for γ and ε hyper-parameters. The search space reductions proposed are evaluated in different regression problems from UCI and StatLib databases. All the experiments carried out applying the popular LIBSVM solver have shown that our approach reduces the SVM training time, maintaining the SVM performance similar to when the complete range in SVM parameters is considered.
In this paper, an efficient method is proposed to design fuzzy wavelet neural network (FWNN) for function learning and identification by tuning fuzzy membership functions and wavelet neural networks. The structure of FWNN is based on the basis of fuzzy rules including wavelet functions in the consequent parts of rules. In order to improve the function approximation accuracy and general capability of the FWNN system, an efficient genetic algorithm (GA) approach is used to adjust the parameters of dilation, translation, weights, and membership functions. By minimizing a quadratic measure of the error derived from the output of the system, the design problem can be characterized by the proposed GA formulation. Moreover, the solution is directly obtained without any need for complicated computations. The performance of our approximation is superior to that of existing methods. Several numerical design examples are likewise presented to demonstrate the design flexibility and usefulness of this presented approach.
Recurrent wavelet neural network (RWNN) has the advantages in its dynamic responses and information storing ability. This
paper develops a recurrent wavelet neural backstepping control (RWNBC) scheme for multiple-input multiple-output (MIMO) mechanical
systems. This proposed RWNBC comprises a neural controller and a smooth compensator. The neural controller using an RWNN is
the principal tracking controller utilized to mimic an ideal backstepping control law; and the parameters of RWNN are online
tuned by the derived adaptation laws from the Lyapunov stability theorem. The smooth compensator is designed to dispel the
approximation error introduced by the neural controller, so that the asymptotic stability of the closed-loop system can be
guaranteed. Finally, two MIMO mechanical systems, a mass-spring-damper system and a two-inverted pendulum system, are performed
to verify the effectiveness of the proposed RWNBC scheme. From the simulation results, it is verified that the proposed RWNBC
scheme can achieve favorable tracking performance without any chattering phenomenon.
Properly designing a wavelet neural network (WNN) is crucial for achieving the optimal generalization performance. In this paper, two different approaches were proposed for improving the predictive capability of WNNs. First, the types of activation functions used in the hidden layer of the WNN were varied. Second, the proposed enhanced fuzzy c-means clustering algorithm—specifically, the modified point symmetry-based fuzzy c-means (MSFCM) algorithm—was employed in selecting the locations of the translation vectors of the WNN. The modified WNN was then applied to heterogeneous cancer classification using four different microarray benchmark datasets. The comparative experimental results showed that the proposed methodology achieved an almost 100% classification accuracy in multiclass cancer prediction, leading to superior performance with respect to other clustering algorithms. Subsequently, performance comparisons with other classifiers were made. An assessment analysis showed that this proposed approach outperformed most of the other classifiers.
In this paper, a genetic algorithm-based approach is proposed to determine a desired sampling-time range which guarantees minimum phase behaviour for the sampled-data system of an interval plant preceded by a zero-order hold (ZOH). Based on a worst-case analysis, the identification problem of the sampling-time range is first formulated as an optimization problem, which is subsequently solved under a GA-based framework incorporating two genetic algorithms. The first genetic algorithm searches both the uncertain plant parameters and sampling time to dynamically reduce the search range for locating the desired sampling-time boundaries based on verification results from the second genetic algorithm. As a result, the desired sampling-time range ensuring minimum phase behaviour of the sampled-data interval system can be evolutionarily obtained. Because of the time-consuming process that genetic algorithms generally exhibit, particularly the problem nature which requires undertaking a large number of evolution cycles, parallel computation for the proposed genetic algorithm is therefore proposed to accelerate the derivation process. Illustrated examples in this paper have demonstrated that the proposed GA-based approach is capable of accurately locating the boundaries of the desired sampling-time range.
By utilizing some of the important properties of wavelets like denoising, compression, multiresolution along with the concepts of fuzzy logic and neural network, new two fuzzy wavelet neural networks (FWNNs) are proposed for approximating any arbitrary non-linear function, hence identifying a non-linear system. The output of discrete wavelet transform (DWT) block, which receives the given inputs, is fuzzified in the proposed two methods: one using compression property and other using multiresolution property. We present a new type of fuzzy neuron model, each non-linear synapse of which is characterized by a set of fuzzy implication rules with singleton weights in their consequents.It is shown that noise and disturbance in the reference signal are reduced with wavelets and also the variation of somatic gain, the parameter that controls the slope of the activation function in the neural network, leads to more accurate output. Identification results are found to be accurate and speed of their convergence is fast. Next, we simulate a control system for maintaining the output at a desired level by using the identified models. Self-learning FNN controller has been designed in this simulation. Simulation results show that the controller is adaptive and robust.
Properly designing a wavelet neural network (WNN) is crucial for achieving the optimal generalization performance. In this paper, in order to improve the predictive capability of WNNs, the types of activation functions used in the hidden layer of the WNN were varied. The modified WNNs were then applied in approximating a benchmark piecewise function. Subsequently, performance comparisons with other developed methods in studying the same benchmark function were made. An assessment analysis showed that this proposed approach outperformed the rest. The efficiency of the modified WNNs was explored through a real-world application problem-specifically, the prediction of time-series pollution data at Texas of United States. The comparative experimental results showed that integrating different wavelet families into the hidden layer of WNNs leads to superior performance.
It is expected that solar energy plays an important role in the strategy of sustainable energy before long. In the case of various solar energy applications, accurate forecast of solar irradiation is increasingly required in recent years. Thanks to the progress of artificial intelligence, its application to various engineering fields vitalizes many conventional techniques. Forecast of solar irradiation is one of the techniques that benefit a lot from the progress, e.g., the progress of artificial neural networks (ANNs). Comparatively, various irradiation forecast models based on ANN perform much better in accuracy than many conventional prediction models. However, a fact could not be neglected that most of such existing ANN-based models have not yet satisfied researchers and engineers in forecast precision so far, and the generalization capability of these networks needs further improving. Combining the prominent dynamic characteristics of recurrent neural network (RNN) with the enhanced ability of wavelet neural network (WNN) in mapping nonlinear functions, a diagonal recurrent wavelet neural network (DRWNN) is newly established in this paper so as to carry out fine forecasting of the hourly global solar irradiance. Some additional steps, e.g., using fuzzy technique to apply historical information of cloud cover to sample data sets for network training and the forecasted cloud cover in weather program to network input for the irradiation forecasting, are adopted to help enhancing forecast precision. Besides, a specially scheduled 2-phase-training algorithm is adopted. As an example, an hourly irradiance forecast is completed using the sample data set in Shanghai, and comparisons between irradiation models show that the DRWNN model is definitely more accurate.
In the organosolv pulping of the oil palm fronds, the influence of the operational variables of the pulping reactor (viz. cooking temperature and time, ethanol and NaOH concentration) on the properties of the resulting pulp (yield and kappa number) and paper sheets (tensile index and tear index) was investigated using a wavelet neural network model. The experimental results with error less than 0.0965 (in terms of MSE) were produced, and were then compared with those obtained from the response surface methodology. Performance assessment indicated that the neural network model possessed superior predictive ability than the polynomial model, since a very close agreement between the experimental and the predicted values was obtained.
A model which takes advantage of wavelet-like functions in the functional form of a neural network is used for function approximation. The scale parameters are mainly used, neglecting the usual translation parameters in the function expansion. Two training operations are then investigated. The first one consists of optimizing the output synaptic weights and the second one on optimizing the scale parameters hidden inside the elementary tasks. Building upon previously published results, it is found that if (p+1) scale parameters merge during the learning process, derivatives of order p will emerge spontaneously in the functional basis. It is also found that for those tasks which induce such mergings, the function approximation can be improved and the training time reduced by directly implementing the elementary tasks and their derivatives in the functional basis. Attention has been also devoted to the role transfer functions, number of iterations, and formal neurons number may play during and after the learning process. The results complement previously published results on this problem.
Clustering large data sets is a central challenge in gene expression analysis. The hybridization of synthetic oligonucleotides to arrayed cDNAs yields a fingerprint for each cDNA clone. Cluster analysis of these fingerprints can identify clones corresponding to the same gene. We have developed a novel algorithm for cluster analysis that is based on graph theoretic techniques. Unlike other methods, it does not assume that the clusters are hierarchically structured and does not require prior knowledge on the number of clusters. In tests with simulated libraries the algorithm outperformed the Greedy method and demonstrated high speed and robustness to high error rate. Good solution quality was also obtained in a blind test on real cDNA fingerprints.
Three phenolic compounds, i.e. phenol, catechol and 4-acetamidophenol, were simultaneously determined by voltammetric detection of its oxidation reaction at the surface of an epoxy-graphite transducer. Because of strong signal overlapping, Wavelet Neural Networks (WNN) were used in data treatment, in a combination of chemometrics and electrochemical sensors, already known as the electronic tongue concept. To facilitate calibration, a set of samples (concentration of each phenol ranging from 0.25 to 2.5mM) was prepared automatically by employing a Sequential Injection System. Phenolic compounds could be resolved with good prediction ability, showing correlation coefficients greater than 0.929 when the obtained values were compared with those expected for a set of samples not employed for training.
Inspired by the theory of multiresolution analysis (MRA) of
wavelet transforms and fuzzy concepts, a fuzzy wavelet network (FWN) is
proposed for approximating arbitrary nonlinear functions. The FWN
consists of a set of fuzzy rules. Each rule corresponding to a
sub-wavelet neural network (WNN) consists of single-scaling wavelets.
Through efficient bases selection, the dimension of the approximated
function does not cause the bottleneck for constructing FWN. Especially,
by learning the translation parameters of the wavelets and adjusting the
shape of membership functions, the model accuracy and the generalization
capability of the FWN can be remarkably improved. Furthermore, an
algorithm for constructing and training the fuzzy wavelet networks is
proposed. Simulation examples are also given to illustrate the
effectiveness of the method
In the magnetic flux leakage (MFL) method of nondestructive testing commonly used to inspect ferromagnetic materials, a crucial problem is signal inversion, wherein the defect profiles must be recovered from measured signals. This paper proposes a neural-network-based inversion algorithm to solve the problem. Neural networks (radial-basis function and wavelet-basis function) are first trained to approximate the mapping from the signal to the defect space. The trained networks are then used iteratively in the algorithm to estimate the profile, given the measurement signal. The paper presents the results of applying the algorithm to simulated MFL data.