Figure - available from: Annals of Operations Research
This content is subject to copyright. Terms and conditions apply.
The sensitivity curves of SVM, RVM, and RRVM obtained from a the simulated data in Fig. 1 and b the contaminated Ripley data in Fig. 4
Source publication
The relevance vector machine (RVM) is a widely employed statistical method for classification, which provides probability outputs and a sparse solution. However, the RVM can be very sensitive to outliers far from the decision boundary which discriminates between two classes. In this paper, we propose the robust RVM based on a weighting scheme, whic...
Similar publications
This paper considers dense detection of out-of-distribution pixels. As a baseline, we start from recent image-wide approaches and adapt them for dense prediction at the pixel level. Subsequently, we propose and develop a novel approach based on a suitable discriminatively trained model. Thus we reduce our problem to dense classification of image pi...
In this paper, we introduce a new approach of semisupervised anomaly detection that deals with categorical data. Given a training set of instances (all belonging to the normal class), we analyze the relationship among features for the extraction of a discriminative characterization of the anomalous instances. Our key idea is to build a model that c...
Current language identification (LID) systems are based on an i-vector classifier followed by a multi-class recognition back-end. Identification accuracy degrades considerably when LID systems face open-set data. In this study, we propose an approach to the problem of out of set (OOS) data detection in the context of open-set language identificatio...
Multi-view clustering, which explores complementary information between multiple distinct feature sets, has received considerable attention. For accurate clustering, all data with the same label should be clustered together regardless of their multiple views. However, this is not guaranteed in existing approaches. To address this issue, we propose...
Citations
... A variant of RVM is proposed in paper [40] known as variational relevance vector machine. This method is RVM with variational inference. ...
... The amount of model parameters can reach millions (Salakhutdinov and Hinton 2007) and such models optimization requires multiple days (Sutskever et al. 2014). The hyperparameter values can influence significantly the quality of models (Hwang and Jeong 2018;Nystrup et al. 2018). These two facts make the hyperparameter optimization problem very important. ...
The paper investigates hyperparameter optimization problem. Hyperparameters are the parameters of model parameter distribution. The adequate choice of hyperparameter values prevents model overfit and allows it to obtain higher predictive performance. Neural network models with large amount of hyperparameters are analyzed. The hyperparameter optimization for models is computationally expensive. The paper proposes modifications of various gradient-based methods to simultaneously optimize many hyperparameters. The paper compares the experiment results with the random search. The main impact of the paper is hyperparameter optimization algorithms analysis for the models with high amount of parameters. To select precise and stable models the authors suggest to use two model selection criteria: cross-validation and evidence lower bound. The experiments show that the models optimized using the evidence lower bound give higher error rate than the models obtained using cross-validation. These models also show greater stability when data is noisy. The evidence lower bound usage is preferable when the model tends to overfit or when the cross-validation is computationally expensive. The algorithms are evaluated on regression and classification datasets.
... Moreover, the regularization coefficient is adjusted automatically during the estimation of hyper parameters. As an extension of SVM, RVM has become the research focus in recent years [24][25][26]. Nguyen employed RVM for Kinect gesture recognition and compared it with SVM [24]. ...
... It has been shown in literature that RVM can be very sensitive to outliers far from the decision boundary that discriminates between two classes. To solve this problem, Hwang proposed a robust RVM based on a weighting scheme that is insensitive to outliers [26]. Experimental results from synthetic and real data sets verified its effectiveness. ...
Partial Discharge (PD) pattern recognition plays an important part in electrical equipment fault diagnosis and maintenance. Feature extraction could greatly affect recognition results. Traditional PD feature extraction methods suffer from high-dimension calculation and signal attenuation. In this study, a novel feature extraction method based on Ensemble Empirical Mode Decomposition (EEMD) and Sample Entropy (SamEn) is proposed. In order to reduce the influence of noise, a wavelet method is applied to PD de-noising. Noise Rejection Ratio (NRR) and Mean Square Error (MSE) are adopted as the de-noising indexes. With EEMD, the de-noised signal is decomposed into a finite number of Intrinsic Mode Functions (IMFs). The IMFs, which contain the dominant information of PD, are selected using a correlation coefficient method. From that, the SamEn of selected IMFs are extracted as PD features. Finally, a Relevance Vector Machine (RVM) is utilized for pattern recognition using the features extracted. Experimental results demonstrate that the proposed method combines excellent properties of both EEMD and SamEn. The recognition results are encouraging with satisfactory accuracy.
... The eigenvector is just the projection space what is needed [16]. ...
Relevance vector machine (RVM) is a machine learning technique that uses Bayesian inference to obtain parsimonious solutions for regression and probabilistic. Compared to support vector machines (SVM), the Bayesian formulation of the RVM avoids the set of free parameters of the SVM. However, the classification accuracy of RVM is not high when apply to hyperspectral data. A novel classification method based on RVM is presented in this paper. The method combine principal component analysis (PCA) and linear discriminant analysis (LDA) to reduce the dimensionality of hyperspectral data. Firstly, PCA is used to the first dimension reduction and obtain nonsingular intra-class scatter matrix. Secondly, LDA is applied to the second dimension reduction and reduce the amounts of computation great. Finally, the relevance vector machine model is applied to remote sensing image classification. The hyperspectral data of 1992 Indian Pines has been used in this paper. The experimental results show that the proposed method not only improves overall accuracy than RVM and PCA-RVM, but also extend the ratio between inter-class distances and intra-class distances.
In semiconductor manufacturing, virtual metrology (VM) is a method of predicting physical measurements of wafer qualities using in-process information from sensors on production equipment. The relevance vector machine (RVM) is a sparse Bayesian kernel machine that has been widely used for VM modeling in semiconductor manufacturing. Missing values from equipment sensors, however, preclude training an RVM model due to missing kernels from incomplete instances. Moreover, imputation for such kernels can lead to a loss of model sparsity. In this work, we propose a restricted RVM (RRVM) that selects its basis functions from only complete instances to handle incomplete data for VM. We conduct the experiments using toy data and real-life data from an etching process for wafer fabrication. The results indicate the model's competitive prediction accuracy with massive missing data while maintaining model sparsity.
The learning problem from imbalanced data set poses a major challenge in data mining community. Conventional machine learning algorithms show poor performance in dealing with the classification problems of imbalanced data set since they are originally designed to work with balanced class distribution. In this paper, we propose a new over-sampling technique, which uses the real-value negative selection (RNS) procedure to generate artificial minority data with no requirement of actual minority data available. The generated minority data with rare actual minority data if available are combined with the majority data as input to a bi-class classification approach for learning. In the experiments, we demonstrate the effectiveness of RNS in avoiding the problems often encountered by the existing over-sampling methods such as the generation of noisy instances and almost duplicated instances in the same clusters. Moreover, the extensive experimental results on the different imbalanced datasets from UCI repository and real-world imbalanced datasets show that when dealing with the classification of imbalanced datasets, the proposed hybrid approach can achieve better performance in terms of both G-Mean and F-Measure evaluation metrics as compared to the other existing imbalanced dataset classification techniques.
