Figure 4 - uploaded by Mengjie Zhang
Content may be subject to copyright.
Source publication
Multi-class object classi cation is an important eld of re-search in computer vision. In this paper basic linear genetic program-ming is modi ed to be more suitable for multi-class classi cation and its performance is then compared to tree-based genetic programming. The directed acyclic graph nature of linear genetic programming is exploited. The e...
Citations
... If ( ) ≥ 0 and ( ) < 0 for all ≠ then belongs to class Thus, multi-tree GP decomposes a -class classification problem into binary classification problems which can be solved simultaneously by trees with a single threshold value of 0. Fig. 15 represents an example of multi-tree GP for multi-class classification. representation of the evolved program [30]. ...
... Given an instance, an LGP program outputs values from the output registers. The class with the highest value is assigned to the instance [25,30]. An example of LGP for multi-class classification is given in Fig. 16. ...
... The example shows that LGP can co-evolve sub-programs (or registers) for classes. Furthermore, a register can be re-used to define other registers, which can reduce the number of original features in comparison with using the tree-based representation [30]. Besides defining the set of registers, users also define an instruction set for LGP, which consists of two main instruction types: operations (such as arithmetic/Boolean operations) and conditional branches (i.e., if statements). ...
Classification is a supervised machine learning process that categories an instance based on a number of features. The process of classification involves several stages, including data preprocessing (such as feature selection and feature construction), model training and evaluation. Evolutionary computation has been widely applied to all these stages to improve the performance and explainability of the built classification models, where term for this research area is Evolutionary Classification. This chapter introduces the fundamental concepts of evolutionary classification, followed by the key ideas using evolutionary computation techniques to address existing classification challenges such as multi-class classification, unbalanced data, explainable/interpretable classifiers and transfer learning.
... For linear GP, features and operations form a many-to-many directed acyclic graph, in which each feature is loaded into predefined registers and register's value can be used in multiple operators [101]. However, graph encoding becomes inefficient on high-dimensional feature sets since the complexity of graph traversal exacerbates the difficulty of FC. ...
Over recent years, there has been a rapid development of deep learning (DL) in both industry and academia fields. However, finding the optimal hyperparameters of a DL model often needs high computational cost and human expertise. To mitigate the above issue, evolutionary computation (EC) as a powerful heuristic search approach has shown significant merits in the automated design of DL models, so-called evolutionary deep learning (EDL). This paper aims to analyze EDL from the perspective of automated machine learning (AutoML). Specifically, we firstly illuminate EDL from DL and EC and regard EDL as an optimization problem. According to the DL pipeline, we systematically introduce EDL methods ranging from data preparation, model generation, to model deployment with a new taxonomy (i.e., what and how to evolve/optimize), and focus on the discussions of solution representation and search paradigm in handling the optimization problem by EC. Finally, key applications, open issues and potentially promising lines of future research are suggested. This survey has reviewed recent developments of EDL and offers insightful guidelines for the development of EDL.
... Teller et al. [197] applied an arbitrary directed graph to represent all features and operators, where each possible high-level feature can be represented as a subgraph of this directed graph. For linear GP, features and operations form a many-to-many directed acyclic graph, in which each feature is loaded into predefined registers and register's value can be used in multiple operators [57]. However, graph encoding becomes inefficient on high-dimensional feature sets since the complexity of graph traversal exacerbates the difficulty of feature construction. ...
Over recent years, there has been a rapid development of deep learning (DL) in both industry and academia fields. However, finding the optimal hyperparameters of a DL model often needs high computational cost and human expertise. To mitigate the above issue, evolutionary computation (EC) as a powerful heuristic search approach has shown significant merits in the automated design of DL models, so-called evolutionary deep learning (EDL). This paper aims to analyze EDL from the perspective of automated machine learning (AutoML). Specifically, we firstly illuminate EDL from machine learning and EC and regard EDL as an optimization problem. According to the DL pipeline, we systematically introduce EDL methods ranging from feature engineering, model generation, to model deployment with a new taxonomy (i.e., what and how to evolve/optimize), and focus on the discussions of solution representation and search paradigm in handling the optimization problem by EC. Finally, key applications, open issues and potentially promising lines of future research are suggested. This survey has reviewed recent developments of EDL and offers insightful guidelines for the development of EDL.
... Nordin [46] already observed that his machine code system evolved more compact solutions, and Fogelberg and Zhang [16] show that LGP evolves shorter and simpler solutions. Controlling the degree of intermediate results reuse was critical for CGP and LGP on digital circuits tasks (Sotto et. ...
... Several works claim that graph representations provide inherent advantages over trees and demonstrate improved performance of graph GP methods over standard GP [4,6,10,23,27]. Given that graphs can naturally represent inactive code (code that is not connected to the main program outputs) which can be freely mutated without changing a given program's fitness, some publications show an improved performance of CGP due to neutral genetic drift [22,30,32,36]. Furthermore, graphs present automatic code reuse, that is, the result of a sub-expression can serve as an argument for more than one node, and this can make solutions more compact [6,10]. ...
... Given that graphs can naturally represent inactive code (code that is not connected to the main program outputs) which can be freely mutated without changing a given program's fitness, some publications show an improved performance of CGP due to neutral genetic drift [22,30,32,36]. Furthermore, graphs present automatic code reuse, that is, the result of a sub-expression can serve as an argument for more than one node, and this can make solutions more compact [6,10]. ...
... In [6], LGP was able to outperform GP on symbolic regression, digital circuits synthesis, and classification tasks. In [10], LGP produced better programs than GP for classification benchmarks, both in terms of performance and understandability. ...
Graph representations promise several desirable properties for genetic programming (GP); multiple-output programs, natural representations of code reuse and, in many cases, an innate mechanism for neutral drift. Each graph GP technique provides a program representation, genetic operators and overarching evolutionary algorithm. This makes it difficult to identify the individual causes of empirical differences, both between these methods and in comparison to traditional GP. In this work, we empirically study the behaviour of Cartesian genetic programming (CGP), linear genetic programming (LGP), evolving graphs by graph programming and traditional GP. By fixing some aspects of the configurations, we study the performance of each graph GP method and GP in combination with three different EAs: generational, steady-state and $$(1+\lambda )$$ ( 1 + λ ) . In general, we find that the best choice of representation, genetic operator and evolutionary algorithm depends on the problem domain. Further, we find that graph GP methods can increase search performance on complex real-world regression problems and, particularly in combination with the ( $$1 + \lambda$$ 1 + λ ) EA, are significantly better on digital circuit synthesis tasks. We further show that the reuse of intermediate results by tuning LGP’s number of registers and CGP’s levels back parameter is of utmost importance and contributes significantly to better convergence of an optimization algorithm when solving complex problems that benefit from code reuse.
... In the second approach, defining a proper threshold interval for each class requires domain knowledge which vary among different experts. A number of researchers have investigated different GP program representations for the third approach, such as Linear GP (LGP) [76], Modi-GP [214] and Probability based GP [186] for the task of multi-class classification. Generally, it is difficult to design a good GP program structure which can effectively and efficiently perform multi-class classification. ...
... Although treebased representation is the most common type of representation for GP individuals, it is not the only one. A number of researchers have investigated different types, such as Linear GP (LGP) [76], Cartesian GP (CGP) [127], Grammar-guided GP [165], and Modi-GP [214]. ...
Skin image classification involves the development of computational methods for solving problems such as cancer detection in lesion images, and their use for biomedical research and clinical care. Such methods aim at extracting relevant information or knowledge from skin images that can significantly assist in the early detection of disease. Skin images are enormous, and come with various artifacts that hinder effective feature extraction leading to inaccurate classification. Feature selection and feature construction can significantly reduce the amount of data while improving classification performance by selecting prominent features and constructing high-level features. Existing approaches mostly rely on expert intervention and follow multiple stages for pre-processing, feature extraction, and classification, which decreases the reliability, and increases the computational complexity. Since good generalization accuracy is not always the primary objective, clinicians are also interested in analyzing specific features such as pigment network, streaks, and blobs responsible for developing the disease; interpretable methods are favored. In Evolutionary Computation, Genetic Programming (GP) can automatically evolve an interpretable model and address the curse of dimensionality (through feature selection and construction). GP has been successfully applied to many areas, but its potential for feature selection, feature construction, and classification in skin images has not been thoroughly investigated.
The overall goal of this thesis is to develop a new GP approach to skin image classification by utilizing GP to evolve programs that are capable of automatically selecting prominent image features, constructing new high-level features, interpreting useful image features which can help dermatologist to diagnose a type of cancer, and are robust to processing skin images captured from specialized instruments and standard cameras. This thesis focuses on utilizing a wide range of texture, color, frequency-based, local, and global image properties at the terminal nodes of GP to classify skin cancer images from multiple modalities effectively.
This thesis develops new two-stage GP methods using embedded and wrapper feature selection and construction approaches to automatically generating a feature vector of selected and constructed features for classification. The results show that wrapper approach outperforms the embedded approach, the existing baseline GP and other machine learning methods, but the embedded approach is faster than the wrapper approach. %Insights of the evolved programs reveal that GP selects highly significant features that can help dermatologists make a diagnosis.
This thesis develops a multi-tree GP based embedded feature selection approach for melanoma detection using domain specific and domain independent features. It explores suitable crossover and mutation operators to evolve GP classifiers effectively and further extends this approach using a weighted fitness function. The results show that these multi-tree approaches outperformed single tree GP and other classification methods. They identify that a specific feature extraction method extracts most suitable features for particular images taken from a specific optical instrument.
This thesis develops the first GP method utilizing frequency-based wavelet features, where the wrapper based feature selection and construction methods automatically evolve useful constructed features to improve the classification performance. The results show the evidence of successful feature construction by significantly outperforming existing GP approaches, state-of-the-art CNN, and other classification methods.
This thesis develops a GP approach to multiple feature construction for ensemble learning in classification. The results show that the ensemble method outperformed existing GP approaches, state-of-the-art skin image classification, and commonly used ensemble methods. Further analysis of the evolved constructed features identified important image features that can potentially help the dermatologist identify further medical procedures in real-world situations.
... Several works claim that graph representations provide inherent advantages over trees and demonstrate improved performance of graph GP methods over standard GP [4,6,8,18,20]. Given that graphs can naturally represent inactive code (code that is not connected to the main program outputs) which can be freely mutated without changing a given program's fitness, some publications show an improved performance of CGP due to neutral genetic drift [17,22,24,27]. ...
... Given that graphs can naturally represent inactive code (code that is not connected to the main program outputs) which can be freely mutated without changing a given program's fitness, some publications show an improved performance of CGP due to neutral genetic drift [17,22,24,27]. Furthermore, graphs present automatic code reuse, that is, the result of a sub-expression can serve as an argument for more than one node, and this can make solutions more compact [6,8]. ...
... In [6], LGP was able to outperform GP on symbolic regression, digital circuits synthesis, and classification tasks. In [8], LGP produced better programs than GP for classification benchmarks, both in terms of performance and understandability. ...
A number of alternative representations have been proposed for Genetic Programming (GP). Linear Genetic Programming (LGP) and Cartesian Genetic Programming (CGP) use a linear genotype that can be interpreted as Directed Acyclic Graphs (DAGs), whereas Evolving Graphs by Graph Programming (EGGP) evolves graphs directly. Each of these variants uses its own set of genetic operators and evolutionary algorithms (EAs). Thus, it is difficult to point a direct cause for a difference in performance between these methods. The goal of the present work is to study how the representations of LGP, CGP, and EGGP differ from one another, in terms of performance, and to asses the role of the genetic operators and EAs that are used. With this purpose, we test each technique, including GP, using three different EAs: generational, steady-state, and 1+lambda, on symbolic regression and digital circuits benchmarks. We find that the generational EA generally works better for regression problems, and the 1+lambda EA presents a clear advantage on digital circuits problems. Moreover, the 1+lambda EA is able to perform much better in combination with graphs, thus making this alternative representation more suited to evolving digital circuits, whereas it presented no clear advantage on the regression problems.
... Since its introduction, GP has been used to tackle many image-related problems such as feature extraction [22], [23], classification [24], [25], object detection [26], [27], image segmentation [28], [29], image registration [30], and image processing [31]. Ebner and Zell [32] employed GP to automatically evolve an interest point detector. ...
The goodness of the features extracted from the instances and the number of training instances are two key components in machine learning, and building an effective model is largely affected by these two factors. Acquiring a large number of training instances is very expensive in some situations such as in the medical domain. Designing a good feature set, on the other hand, is very hard and often requires domain expertise. In computer vision, image descriptors have emerged to automate feature detection and extraction; however, domain-expert intervention is typically needed to develop these descriptors. The aim of this paper is to utilise Genetic Programming to automatically construct a rotation-invariant image descriptor by synthesising a set of formulae using simple arithmetic operators and first-order statistics, and determining the length of the feature vector simultaneously using only two instances per class. Using seven texture classification image datasets, the performance of the proposed method is evaluated and compared against eight domain-expert hand-crafted image descriptors. Quantitatively, the proposed method has significantly outperformed, or achieved comparable performance to, the competitor methods. Qualitatively, the analysis shows that the descriptors evolved by the proposed method can be interpreted.
... Structuring programs as sequences of instructions has many advantages over structuring them as trees. LGP has been shown to significantly outperform TGP on machine learning tasks such as multiclass classification [4, 8]. LGP performs well on multiclass classification problems because of the power of its flexible program structure. ...
Motivated by biological inspiration and the issue of code disruption, we develop a new form of LGP called Parallel LGP (PLGP). PLGP programs consist of n lists of instructions. These lists are executed in parallel, after which the resulting vectors are combined to produce program
output. PGLP limits the disruptive effects of crossover and mutation, which allows PLGP to significantly outperform regular
LGP.
... Normally these socalled calculation registers are initialized with a constant value each time a program is executed on a fitness case. The instruction set defines the particular programming language that is evolved [5]. Two-operand instructions may either possess two indexed variable (registers) operands or one indexed variable and a constant. ...
This paper presents a comparative analysis of linear genetic programming and artificial neural network methods to solve classification tasks. Usually classification tasks have data sets containing a large number of attributes and records, and more than two classes that will be processed using, for example, created classification rules. As a result, by using classical method to classify a large number of records, a high classification error value will be obtained. The artificial neural networks are often used to solve classification task, mostly obtaining good results. The linear genetic programming is a new direction of evolution algorithms that is not widely researched and its application areas are not well defined. However, some advantages of linear genetic programming are based on genetic operators whose structure does not require complicated calculations. During this work approximately 400 experiments were conducted with linear genetic programming and artificial neural network methods, using various data sets with different quantity of records, attributes and classes. Based on the results received, conclusions on possibilities of using the methods of linear genetic programming and artificial neural networks in classification problems were drawn, and suggestions for improving their performance were proposed.
