Robert Duin

Delft University of Technology | TU · Section of Pattern Recognition and Bioinformatics

The question is discussed from where the patterns arise that are recognized in the world. Are they elements of the outside world, or do they originate from the concepts that live in the mind of the observer? It is argued that they are created during observation, due to the knowledge on which the observation ability is based. For an experienced obse...

Classification in the dissimilarity space has become a very active research area since it provides a possibility to learn from data given in the form of pairwise non-metric dissimilarities, which otherwise would be difficult to cope with. The selection of prototypes is a key step for the further creation of the space. However, despite previous effo...

A significantly faster algorithm is presented for the original kNN mode seeking procedure. It has the advantages over the well-known mean shift algorithm that it is feasible in high-dimensional vector spaces and results in uniquely, well defined modes. Moreover, without any additional computational effort it may yield a multi-scale hierarchy of clu...

Multiscale information provides an opportunity to improve the outcomes of data analysis processes. However, if the multiscale information is not properly summarized in a compact representation, this may lead to problems related to high dimensional data. In addition, in some situations, it is convenient to define dissimilarities directly for the mul...

The non-linear scaling of given dissimilarities, by raising them to a power in the (0,1) interval, is often useful to improve the classification performance in the corresponding dissimilarity space. The optimal value for the power can be found by a grid search across a leave-one-out cross validation of the classifier: a procedure that might become...

When characterizing teams of people, molecules, or general graphs, it is difficult to encode all information using a single feature vector only. For these objects dissimilarity matrices that do capture the interaction or similarity between the sub-elements (people, atoms, nodes), can be used. This paper compares several representations of dissimila...

We consider general non-Euclidean distance measures between real world objects that need to be classified. It is assumed that objects are represented by distances to other objects only. Conditions for zero-error dissimilarity based classifiers are derived. Additional conditions are given under which the zero-error decision boundary is a continues f...

The majority of traditional classification ru les minimizing the expected probability of error (0-1 loss) are inappropriate if the class probability distributions are ill-defined or impossible to estimate. We argue that in such cases class domains should be used instead of class distributions or densities to construct a reliable decision function....

Multiscale information provides an opportunity to improve the outcomes of data analysis processes. However, if the multiscale information is not properly summarized in a compact representation, this may lead to problems related to high dimensional data. In addition, in some situations, it is convenient to define dissimilarities directly for the mul...

The dissimilarity representation for designing pattern recognition systems is analyzed for its ability to build new knowledge from examples using an anti-essentialist approach. It is argued that it may find universals (pattern classifiers) from particulars (training set of examples) but that the resulting knowledge can just be applied but not acces...

Many computer vision and pattern recognition problems may be posed as the analysis of a set of dissimilarities between objects. For many types of data, these dissimilarities are not euclidean (i.e., they do not represent the distances between points in a euclidean space), and therefore cannot be isometrically embedded in a euclidean space. Examples...

Showing the nearest neighbor is a useful explanation for the result of an automatic classification. Given, expert defined, distance measures may be improved on the basis of a training set. We study several proposals to optimize such measures for nearest neighbor classification, explicitly including non-Euclidean measures. Some of them may directly...

How to select the prototypes for classification in the dissimilarity space remains an open and interesting problem. Especially, achieving scalability of the methods is desirable due to enormous amounts of information arising in many fields. In this paper we pose the question: are genetic algorithms good for scalable prototype selection? We propose...

An unsupervised method for selecting training data is suggested here. The method is tested by applying it to hyperspectral land-use classification. The data set is reduced using an unsupervised band selection method and then clustered with a nonparametric cluster technique. The cluster technique provides centers of the clusters, and those are the s...

Missing values can occur frequently in many real world situations. Such is the case of multi-way data applications, where objects are usually represented by arrays of 2 or more dimensions e.g. biomedical signals that can be represented as time-frequency matrices. This lack of attributes tends to influence the analysis of the data. In classification...

The selection of prototypes for the dissimilarity space is a key aspect to overcome problems related to the curse of dimensionality and computational burden. How to properly define and select the prototypes is still an open issue. In this paper, we propose the selection of clusters as prototypes to create low-dimensional spaces. Experimental result...

Traditional pattern recognition techniques often assume that the data sets used for training and testing follow the same distribution. However, this assumption is usually not true for many real world problems as data from the same classes but different domains, e.g., data are collected under different conditions, may show different characteristics....

A widely used approach to cope with asymmetry in dissim-ilarities is by symmetrizing them. Usually, asymmetry is corrected by applying combiners such as average, minimum or maximum of the two directed dissimilarities. Whether or not these are the best approaches for combining the asymmetry remains an open issue. In this paper we study the performan...

This paper establishes a link between two supervised learning frameworks, namely multiple-instance learning (MIL) and learning from only positive and unlabelled examples (LOPU). MIL represents an object as a bag of instances. It is studied under the assumption that its instances are drawn from a mixture distribution of the concept and the non-conce...

In multiple-instance learning (MIL), an object is represented as a bag consisting of a set of feature vectors called instances. In the training set, the labels of bags are given, while the uncertainty comes from the unknown labels of instances in the bags. In this paper, we study MIL with the assumption that instances are drawn from a mixture distr...

Multi-spectral video endoscopy provides considerable potential for early stage cancer detection. Previous multi-spectral image acquisition systems were of limited use for endoscopy due to (i) the necessary spatial scanning of push-broom approaches or (ii) the impractical long switching times of liquid crystal tunable filters. Recent technological a...

In the literature, there are several criteria for validation of a clustering partition. Those criteria can be external or internal, depending on whether we use prior information about the true class labels or only the data itself. All these criteria assume a fixed number of clusters k and measure the performance of a clustering algorithm for that k...

In many pattern recognition applications, object structure is essential for the discrimination purpose. In such cases, researchers often use recognition schemes based on template matching which lead to the design of non-Euclidean dissimilarity measures. A vector space derived from the embedding of the dissimilarities is desirable in order to use ge...

One typically expects classifiers to demonstrate improved performance with increasing training set sizes or at least to obtain their best performance in case one has an infinite number of training samples at ones's disposal. We demonstrate, however, that there are classification problems on which particular classifiers attain their optimum performa...

Clustering by mode seeking is most popular using the mean shift algorithm. A less well known alternative with different properties on the computational complexity is kNN mode seeking, based on the nearest neighbor rule instead of the Parzen kernel density estimator. It is faster and allows for much higher dimensionalities. We compare the performanc...

In this study, we propose a computational diagnosis system for detecting the colorectal cancer from histopathological slices. The computational analysis was usually performed on patch level where only a small part of the slice is covered. However, slice-based classification is more realistic for histopathological diagnosis. The developed method com...

For many applications, a straightforward representation of objects is by multi-dimensional arrays e.g. signals. However, there are only a few classification tools which make a proper use of this complex structure to obtain a better discrimination between classes. Moreover, they do not take into account context information that can also be very bene...

When asymmetric dissimilarity measures arise, asymmetry correction methods such as averaging are used in order to make the ma-trix symmetric. This is usually needed for the application of pattern recognition procedures, but in this way the asymmetry information is lost. In this paper we present a new approach to make use of the asym-metry informati...

Structures and features are opposite approaches in building representations for object recognition. Bridging the two is an essential problem in pattern recognition as the two opposite types of information are fundamentally different. As dissimilarities can be computed for both the dissimilarity representation can be used to combine the two. Attribu...

Textures often show multiscale properties and hence multiscale techniques are considered useful for texture analysis. Scale-space theory as a biologically motivated approach may be used to construct multiscale textures. In this paper various ways are studied to combine features on different scales for texture classification of small image patches....

It is well-known that the mean squared error (MSE) is an inappropriate measure for the diierence between two images in many applications. For one such an application , edge-preserving smoothing, an alternative w as developed which t a k es both goals into account: the preservation or sharpening of edges and the smoothing of regions. In this paper,...

Human experts constitute pattern classes of natural objects based on their observed appearance. Automatic systems for pattern recognition may be designed on a structural description derived from sensor observations. Alternatively, training sets of examples can be used in statistical learning procedures. They are most powerful for vectorial object r...

Recently, generalized dissimilarity representations have shown their potential for small sample size problems. In generalizations by feature lines, instead of dissimilarities with objects, we have dissimilarities with feature lines. One drawback of such generalization is the high amount of generated lines that increases computational costs and may...

We consider the problem of localizing renal cancer cell nuclei in Tissue Micro Array (TMA) images. We address this problem in three steps. An initial image processing-based procedure finds potential candidate nuclei, while the subsequent phase employs a trained classifier to prune the candidate cell nuclei found in the first.

In image classification, multi-scale information is usually combined by concatenating features or selecting scales. Their main drawbacks are that concatenation increases the feature dimensionality by the number of scales and scale selection typically loses the information from other scales. We propose to solve this problem by the dissimilarity repr...

In the dissimilarity representation approach, objects are represented by their dissimilarities with respect to a representation set, rather than by features. Up to now, the representation or prototype set has usually been selected from the training data, limiting the different aspects that can be captured, especially when the training data set is s...

Multispectral endoscopy images provide potential for early stage cancer detection. This paper considers this relatively novel imaging technique and presents a supervised method for cancer detection using such multispectral data. The data under consideration include different types of cancer. This poses a challenge for the detection as different can...

An automated histology analysis is proposed for classification of local image patches of colon histopathology images into four principle classes: normal, cancer, adenomatous and inflamed classes. Shape features based on stroma, lumen and imperfectly segmented nuclei are combined with texture features for classification. The classification is analyz...

This paper introduces a novel classification algorithm named MAP-DID. This algorithm combines a maximum a posteriori (MAP) approach using the well-known Gaussian Mixture Model (GMM) method with a term that forces the various Gaussian components within each class to have a common structure. That structure is based on higher-order statistics of the d...

A flexible description of images is offered by a cloud of points in a feature space. In the context of image retrieval such clouds can be represented in a number of ways. Two approaches are here considered. The first approach is based on the assumption of a normal distribution, hence homogeneous clouds, while the second one focuses on the boundary...

This paper presents an empirical evaluation on a dissimilarity measure strategy by which dissimilarity-based classifications (DBCs) [10] can be efficiently implemented. In DBCs, classifiers are not based on the feature measurements of individual objects, but rather on a suitable dissimilarity measure among the objects. In image classification tasks...

The patterns in collections of real world objects are often not based on a limited set of isolated properties such as features. Instead, the totality of their appearance constitutes the basis of the human recognition of patterns. Structural pattern recognition aims to find explicit procedures that mimic the learning and classification made by human...

Representation of objects by multi-dimensional data arrays has become very common for many research areas e.g. image analysis, signal processing and chemometrics. In most cases, it is the straightforward representation obtained from sophisticated measurement equipments e.g. radar signal processing. Although the use of this complex data structure co...

The representation of objects by multi-dimensional arrays is widely applied in many research areas. Nevertheless, there is a lack of tools to classify data with this structure. In this paper, an approach for classifying objects represented by matrices is introduced , based on the advantages and success of the combination strategy, and particularly...

Due to the wide variety of fusion techniques available for combining multiple classifiers into a more accurate classifier, a number of good studies have been devoted to determining in what situations some fusion methods should be preferred over other ones. However, the sample size behavior of the various fusion methods has hitherto received little...

One of the problems in semi-supervised land classification tasks lies in improving classification results without increasing the number of pixels to be labeled. This would be possible if, instead of increasing the amount of data we increased the reliability of the data. We suggest to replace the random selection by a unsupervised clustering based s...

When objects cannot be represented well by single feature vectors, a collection of feature vectors can be used. This is what is done in Multiple Instance learning, where it is called a bag of instances. By using a bag of instances, an object gains more internal structure than when a single feature vector is used. This improves the expressiveness of...

Classification of spectral data has raised a growing interest in may research areas. However, this type of data usually suffers from the curse of dimensionality. This causes most statistical methods and/or classifiers to not perform well. A recently proposed alternative which can help avoiding this problem is the Dissimilarity Representation, in wh...

In chemometrics, spectral data are typically represented by vectors of features in spite of the fact that they are usually plotted as functions of e.g. wavelengths and concentrations. In the representation, this functional information is thereby not reflected. Consequently, some characteristics of the data that can be essential for discrimination b...

The representation of objects by multidimensional arrays is widely applied in many research areas. Nevertheless, there is a lack of tools to classify data with this structure. In this paper, an approach for classifying objects represented by matrices is introduced, based on the advantages and success of the combination strategy, and particularly in...

Detecting edges in multispectral images is difficult because different spectral bands may contain different edges. Existing approaches calculate the edge strength of a pixel locally, based on the variation in intensity between this pixel and its neighbors. Thus, they often fail to detect the edges of objects embedded in background clutter or object...

Kernel combination is meant to improve the performance of single kernels and avoid the difficulty of kernel selection. The most common way of combining kernels is to compute their weighted sum. Usually, the kernels are assumed to exist in independent empirical feature spaces and therefore were combined without considering their relationships. To ta...

In this article, a novel approach to schizophrenia classification using magnetic resonance images (MRI) is proposed. The presented method is based on dissimilarity-based classification techniques applied to morphological MRIs and diffusion-weighted images (DWI). Instead of working with features directly, pairwise dissimilarities between expert deli...

A semi-supervised pixel classification scheme for hyperspectral satellite images is presented. The scheme includes a previous band selection step followed by a clustering process to select modes of interest that will be labeled by an expert. Then pixel classification is performed resulting in a segmentation and classification of the fields appearin...

A common way to model multi-class classification problems is by means of Error-Correcting Output Codes (ECOC). Given a multi-class problem, the ECOC technique designs a codeword for each class, where each position of the code identifies the membership of the class for a given binary problem.A classification decision is obtained by assigning the lab...

This book presents an introduction to new and important research in the images processing and analysis area. It is hoped that this book will be useful for scientists and students involved in many aspects of image analysis. The book does not attempt to cover all of the aspects of Computer Vision, but the chapters do present some state of the art exa...

Classification, Parameter Estimation and State Estimation is a practical guide for data analysts and designers of measurement systems and postgraduates students that are interested in advanced measurement systems using MatLab. "Prtools" is a powerful MatLab toolbox for pattern recognition and is written and owned by one of the coauthors, B. Duin of...

The aim of this paper is to present a dissimilarity measure strategy by which a new philosophy for pattern classification pertaining to dissimilarity-based classifications (DBCs) can be efficiently implemented. In DBCs, classifiers are not based on the feature measurements of individual patterns, but rather on a suitable dissimilarity measure among...

In this paper, we propose to classify medical images using dissimilarities computed between collections of regions of interest. The images are mapped into a dissimilarity space using an image dissimilarity measure, and a standard vector space-based classifier is applied in this space. The classification output of this approach can be used in comput...

The dissimilarity representation has demonstrated advantages in the solution of classification problems. Meanwhile, the representation of objects by multi-dimensional arrays is necessary in many research areas. However, the development of proper classification tools that take the multi-way structure into account is incipient. This paper introduces...

In the process of designing pattern recognition systems one may choose a representation based on pairwise dissimilarities between objects. This is especially appealing when a set of discriminative features is difficult to find. Various classification systems have been studied for such a dissimilarity representation: the direct use of the nearest ne...

In this paper, we propose to solve multiple instance learning problems using a dissimilarity representation of the objects. Once the dissimilarity space has been constructed, the problem is turned into a standard supervised learning problem that can be solved with a general purpose supervised classifier. This approach is less restrictive than kerne...

The aim of this paper is to find an answer to the question: What is the difference between dissimilarity-based classifications(DBCs) and other kernelbased classifications(KBCs)? In DBCs [11], classifiers are defined among classes; they are not based on the feature measurements of individual objects, but rather on a suitable dissimilarity measure am...

Multi-way data analysis is a multivariate data analysis technique having a wide application in some fields. Nevertheless, the development of classification tools for this type of representation is incipient yet. In this paper we study the dissimilarity representation for the classification of three-way data, as dissimilarities allow the representat...

In text categorization (TC), which is a supervised technique, a feature vector of terms or phrases is usually used to represent the documents. Due to the huge number of terms in even a moderate-size text corpus, high dimensional feature space is an intrinsic problem in TC. Random subspace method (RSM), a technique that divides the feature space to...

Seismic events in and around volcanos, like tremors, earth quakes, ice quakes and strokes of lightning, are usually observed by multiple stations. The question rises whether classifiers trained for one seismic station can be used for classifying observations by other stations, and, moreover, whether a combination of station signals improves the cla...

Dissimilarities can be a powerful way to represent objects like strings, graphs and images for which it is difficult to find good features. The resulting dissimilarity space may be used to train any classifier appropriate for feature spaces. There is, however, a strong need for dimension reduction. Straightforward procedures for prototype selection...

The ways distances are computed or measured enable us to have different representations of the same objects. In this paper we want to discuss possible ways of merging different sources of information given by differently measured dissimilarity representations. We compare here a simple averaging scheme [1] with dissimilarity forward selection and ot...

Instead of solving complex pattern recognition problems using a single complicated classifier, it is often beneficial to leverage our prior knowledge and decompose the problem into parts. These may be tackled using specific feature subsets and simpler classifiers resulting in a hierarchical system. In this paper, we propose an efficient and scalabl...

We propose to approach the detection of patients affected by schizophrenia by means of dissimilarity-based classification techniques applied to brain magnetic resonance images. Instead of working with features directly, pairwise distances between expert delineated regions of interest (ROIs) are considered as representations based on which learning...

In many real world data applications, objects may have missing attributes. Conventional techniques used to classify this kind of data are represented in a feature space. However, usually they need imputation methods and/or changing the classifiers. In this paper, we propose two classification alternatives based on dissimilarities. These techniques...

Combining different distance matrices or dissimilarity representations can often increase the performance of individual ones. In this work, we experimentally study on the performance of combining Euclidean distances and its relationship with the non-Euclideaness produced from combining Euclidean distances. The relationship between the degree of non...

This study presents a system for detecting and scoring of a knee disorder, namely, osteoarthritis (OA). Data used for training and recognition is mainly data obtained through computerized gait analysis, which is a numerical representation of the mechanical measurements of human walking patterns. History and clinical characteristics of the subjects...

Regularities in the world are human defined. Patterns in the observed phenomena are there because we define and recognize them as such. Automatic pattern recognition tries to bridge human judgment with measurements made by artificial sensors. This is done in two steps: representation and generalization. Traditional object representations in pattern...

Many computer vision and pattern recognition problems may be posed by defining a way of measuring dissimilarities between patterns. For many types of data, these dissimilarities are not Euclidean, and may not be metric. In this paper, we provide a means of embedding such data. We aim to embed the data on a hypersphere whose radius of curvature is d...

Selecting a set of good and diverse base classifiers is essential for building multiple classifier systems. However, almost all commonly used procedures for selecting such base classifiers cannot be directly applied to select structural base classifiers. The main reason is that structural data cannot be represented in a vector space. For graph-base...

Gait analysis is used for non-automated and automated diagnosis of various neuromuskuloskeletal abnormalities. Automated systems are important in assisting physicians for diagnosis of various diseases. This study presents preliminary steps of designing a clinical decision support system for semi-automated diagnosis of knee illnesses by using tempor...

This article studies the possibility of detecting dementia in an early stage, using nonrigid registration of MR brain scans in combination with dissimilarity-based pattern recognition techniques. Instead of focussing on the shape of a single brain structure, we take into account the shape differences within the entire brain. Imaging data was obtain...

Two challenges of face recognition at a distance are the uncontrolled illumination and the low resolution of the images. One approach to tackle the first limitation is to use longwave infrared face images since they are invariant to illumination changes. In this paper we study classification performances on 3 different representations: pixelbased,...

The classification of unknown samples is among the most common problems found in chemometrics. For this purpose, a proper representation of the data is very important. Nowadays, chemical spectral data are analyzed as vectors of discretized data where the variables have not connection, and other aspects of their functional nature e.g. shape differen...

Recently, to increase the classification accuracy of dissimilarity-based classifications (DBCs), Kim and Duin [5] proposed a method of simultaneously employing fusion strategies in representing features (representation step) as well as in designing classifiers (generalization step). In this multiple fusion strategies, however, the resulting dissimi...

Hidden Markov models (HMMs) have been successfully applied to a wide range of sequence modeling problems. In the classification context, one of the simplest approaches is to train a single HMM per class. A test sequence is then assigned to the class whose HMM yields the maximum a posterior (MAP) probability. This generative scenario works well when...

Generative kernels represent theoretically grounded tools able to increase the capabilities of generative classification through a discriminative setting. Fisher Kernel is the first and mostly-used representative, which lies on a widely investigated mathematical background. The manufacture of a generative kernel flows down through a two-step serial...

This paper proposes an innovative spectral feature extraction (SFE) method called prototype space (PS) feature extraction (PSFE) based only on class spectra. The main novelties of the proposed SFE lie in the following: representing channels in a new space called PS, where they are characterized in terms of reflection properties of classes; and prop...

Although Multi-response Linear Regression (MLR) has been proposed as a trainable combiner to fuse heterogeneous base-level classifiers into an ensemble classifier, thus far it has not yet been evaluated ex-tensively. In this paper, we employ learning curves to investigate the relative performance of MLR for solving multi-class classification proble...

Edge detection in hyperspectral images is an in- trinsic difficult problem as the gray value intensity images related to single spectral bands may show dif- ferent edges. The few existing approaches are either based on a straight forward combining of these in- dividual edge images, or on finding the outliers in a region segmentation. We propose as...

For pattern recognition problems where a small set of relevant objects should be retrieved from a (very) large set of irrelevant objects, standard evaluation criteria are often insufficient. For these situations often the precision-recall curve is used. An often-employed scalar measure derived from this curve is the mean precision, that estimates t...

In general, classifying graphs with labelled nodes (also known as labelled graphs) is a more difficult task than classifying graphs with unlabelled nodes. In this work, we decompose the labelled graphs into unlabelled subgraphs with respect to the labels, and describe these decomposed subgraphs with the travelling matrices. By utilizing the travell...

The meta-learner MLR (Multi-response Linear Regression) has been proposed as a trainable combiner for fusing heterogeneous base-level classifiers. Although it has interesting properties, it never has been evaluated extensively up to now. This paper employs learning curves to investigate the relative performance of MLR for solving multi-class classi...

In the problem of one-class classification one of the classes, called the target class, has to be distinguished from all other possible objects. These are considered as non-targets. The need for solving such a task arises in many practical applications, e.g. in machine fault detection, face recognition, authorship verification, fraud recognition or...

Even though, under representational restrictions, the nearest feature rules and the dissimilarity-based classifiers are feasible alternatives to the nearest neighbor method; individually, they may not be sufficiently powerful if a very small set of prototypes is required, e.g. when it is computationally expensive to deal with larger sets of prototy...

The representations of real world objects based on distances (dissimilarities) has proven to be more suit-able than the classic feature-based ones for many pat-tern recognition problems. Measuring objects to obtain features is indeed needed to represent them, but that process can be costly, therefore selecting a reduced set of features (prototypes...

Classification problems can be found in any research area, and one of its most essential facts is trying to have a representation of the data where can be resumed as much useful information as possible. In the specific case of chemical spectral data, although they are typically plotted as functions of wavelengths, product concentration, etc. they t...

The recognition of faces at a distance has several challenges. One is the uncontrolled illumination, an-other is the low resolution of the images. One ap-proach to tackle the first limitation is to use longwave infrared (LWIR) face images since they are invariant to illumination changes. In this paper we studied the application of dissimilarity rep...

Error correcting output coding is a well known technique to decom- pose a multi-class classification problem into a group of two-class problems which can be faced by using a combination of binary classifiers. Each of them is trained on a different dichotomy of the classes. The way the set of classes is mapped on this set of dichotomies may essentia...

In Multiple Instance Learning (MIL) problems, objects are represented by a set of feature vectors, in contrast to the standard pattern recognition problems, where objects are represented by a single feature vector. Numerous classifiers have been proposed to solve this type of MIL classification problem. Unfortunately only two datasets are standard...