Table 1 - uploaded by Riccardo Poli
Content may be subject to copyright.
Source publication
Theoretical models of Turing complete linear genetic programming (GP) programs suggest the fraction of halting programs is
vanishingly small. Convergence results proved for an idealised machine, are tested on a small T7 computer with (finite) memory,
conditional branches and jumps. Simulations confirm Turing complete fitness landscapes of this type...
Context in source publication
Similar publications
The electron transport in current-biased superconducting nano-bridges is determined by the motion of the quantum vortex confined in the internal disorder landscape. Here we consider a simple case of a single or two neighbouring linear defects crossing a nano-bridge. The strong anharmonicity of the vortex motion along the defect leads, upon RF-excit...
Citations
... This is not always easy or exact because what has to be measured is the size of the destination space which matches the desired goals. We cannot presume that there is only one possible outcome that fits the goal, even though in biology (as well as in computation) the solution space is usually quite miniscule compared to the possibility space (Abel and Trevors 2005;Langdon 2006;Montañez 2017). ...
Teleological causes have been generally disfavored in biological explanations because they have been thought to lack rigor or act as stand-ins for non-teleological processes which are simply not yet understood sufficiently. Teleological explanations in biology have been limited to only teleonomic causes, which are teleological causes that are due to codes or similarly reified mechanisms. However, advances in the conceptualization of teleological and teleonomic causa-tion have allowed for more quantitative analyses of both. Additionally , although teleonomy has been historically excluded from potential causes of evolution, new research has shown that teleonomy actually plays a significant role in evolution. Combining these advances with advances in computability theory and information theory have allowed for a more rigorous and quantitative analysis of the capabilities and limitations of teleonomy in evolution.
... The halting problem presents a stumbling block for the effective evolution of recursive programs. Research has shown that in Turing-complete fitness landscapes (linear programs with finite memory, conditional branches and jumps) the proportion of halting programs falls towards zero with increasing program size [23,25]. This confirms the belief that Turing-complete program spaces are hard to search. ...
... Overall, our findings are in accordance with simulation results published in [25], which confirmed that in Turing-complete fitness landscapes the proportion of halting programs falls towards zero with increasing program size. The present study concerns small programs of maximum tree-size of 60 nodes. ...
... We expect that this finding will generalise to larger tree-sizes as well. Specifically, we expect the number of halting programs to rise exponentially with size, but to get increasingly rare as a fraction of all programs as in [25]. Figure 5 illustrates scatter plots of error versus edit-distance resulted from random walks with evolved solutions of ascending size highlighted in italics in Table 5. Error refers to the value of the loss function. ...
Recursion is a powerful concept that enables a solution to a problem to be expressed as a relatively simple decomposition of the original problem into sub-problems of the same type. We survey previous research about the evolution of recursive programs in tree-based Genetic Programming. We then present an analysis of the fitness landscape of recursive programs, and report results on evolving solutions to a range of problems. We conclude with guidelines concerning the choice of fitness function and variation operators, as well as the handling of the halting problem. The main findings are as follows. The distribution of fitness changes initially as we look at programs of increasing size but once some threshold has been exceeded, it shows very little variation with size. Furthermore, the proportion of halting programs decreases as size increases. Recursive programs exhibit the property of weak causality; small changes in program structure may cause big changes in semantics. Nevertheless, the evolution of recursive programs is not a needle-in-a-haystack problem; the neighbourhoods of optimal programs are populated by halting individuals of intermediate fitness. Finally, mutation-based variation operators performed the best in finding recursive solutions. Evolution was also shown to outperform random search.
... An important issue that must be considered when dealing with the evolution of unrestricted programs is whether they halt-or not [14]. Whenever Turing-complete programs with arbitrary control flow are evolved, a possibility arises that computation will turn out to be unending. ...
We describe Fertile Darwinian Bytecode Harvester (FINCH), a methodology for evolving Java bytecode, enabling the evolution of extant, unrestricted Java programs, or programs in other languages that compile to Java bytecode. Our approach is based upon the notion of compatible crossover, which produces correct programs by performing operand stack-based, local variables-based, and control flow-based compatibility checks on source and destination bytecode sections. This is in contrast to existing work that uses restricted subsets of the Java bytecode instruction set as a representation language for individuals in genetic programming. We demonstrate FINCH's unqualified success at solving a host of problems, including simple and complex regression, trail navigation, image classification, array sum, and tic-tac-toe. FINCH exploits the richness of the Java virtual machine architecture and type system, ultimately evolving human-readable solutions in the form of Java programs. The ability to evolve Java programs will hopefully lead to a valuable new tool in the software engineer's toolkit.
... An important issue that must be considered when dealing with the evolution of unrestricted programs is whether they halt—or not (Langdon and Poli, 2006). Whenever Turing-complete programs with arbitrary control flow are evolved, a possibility arises that computation will turn out to be unending. ...
The established approach in genetic programming (GP) involves the definition of functions and terminals appropriate to the
problem at hand, after which evolution of expressions using these definitions takes place. We have recently developed a system,
dubbed FINCH (Fertile Darwinian Bytecode Harvester), to evolutionarily improve actual, extant software, which was not intentionally written for the purpose of serving as a GP representation in particular, nor for evolution in general. This is in contrast to existing
work that uses restricted subsets of the Java bytecode instruction set as a representation language for individuals in genetic
programming. The ability to evolve Java programs will hopefully lead to a valuable new tool in the software engineer’s toolkit.
KeywordsJava bytecode-automatic programming-software evolution-genetic programming
... By "unstructured loop" I mean a loop in which the looping mechanism is not cleanly separated from the computation which is performed within the loop body. As an illustration of the problem, Langdon and Poli (2006) have shown that for Von Neumann architectures (a subset of Turing-Complete machines whose loops are nearly all unstructured), the number of programs that even finish (much less do anything helpful) drops to a statistically insignificant percentage of the possible search space as the program size increases. ...
The concept of irreducible complexity is a great intuitive concept which has historically lacked a well-defined theoretical basis. This paper will show how computational theory gives an adequate account for what irreducible complexity is, what irreducible complexity’s possible biological implications are, and how irreducible complexity can be useful to molecular biologists and creation biologists.
... To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. GECCO'08, July [12][13][14][15][16]2008, Atlanta, Georgia, USA. Copyright 2008 ACM 978-1-60558-130-9/08/07…$5.00. ...
... Also, similar results were derived for: a) cyclic CPU (increment, decrement and NOP instructions), b) bit flip computer, (flip bit and NOP), c) any non-reversible computer, d) any reversible computer, e) CCNOT (Toffoli gate) computer, f) quantum computers, g) the 'average' computer and h) AND, NAND, OR, NOR expressions. In addition, recently, these results have been extended to Turing complete machine code programs [12,17]. ...
The crossover bias theory for bloat (18) is a recent result which pre- dicts that bloat is caused by the sampling of short, unÞt programs. This theory is clear and simple, but it has some weaknesses: (1) it implicitly assumes that the population is large enough to allow sampling of all relevant program sizes (although it does explain what to expect in the many practical cases where this is not true, e.g., because the population is small); (2) it does not explain what is meant by its assumption that short programs are unÞt. In this paper we discuss these weaknesses and propose a reÞned version of the crossover bias theory that clariÞes the relationship between bloat and Þnite populations, and explains what features of the Þtness landscape cause bloat to occur. The theory, in particu- lar, predicts that smaller populations will bloat more slowly than large rones .Additionally ,th etheor ypredict stha tbloa twil lonly be observed in problems where short programs are less Þt than longer ones when looking at samples created by Þtness-based im- portance sampling, i.e. samplings of the search space in which Þt- ter programs have a higher probability of being sampled (e.g., the Metropolis-Hastings method). Experiments with two classical GP benchmarks fully corroborate the theory.
... In some ways the search space of linear GP is easier to analyse than that of trees [223,225,226,227,244]. For example, we have used the T7 and T8 architectures (in simulation) for several large scale experimental and mathematical analysis of Turing complete GP [238,247,232,246]. For these reasons, it makes sense to consider linear "machine" code GP, for example, in Java. ...
1 This paper introduces genetic programming (GP) – a set of evolutionary computation techniques for getting computers to automatically solve problems without having to tell them explicitly how to do it. Since its inception, GP has been used to solve many practical problems, producing a number of human competitive results and even patentable new inventions. We start with a gentle introduction to the basic representation, initialisation and operators used in GP, complemented by a step by step description of their use for the solution of an illustrative problem. We then progress to discuss a variety of alternative representations for programs and more advance specialisations of GP. A multiplicity of real-world applications of GP are then presented to illustrate the scope of the technique. For the benefits of more advanced readers, this is followed by a series of recommendations and suggestions to obtain the most from a GP system. Although the paper has been written with beginners and practitioners in mind, for completeness we also provide an overview of the theoretical
... For these reasons, we are interested in extending the analysis of program search spaces to the case of Turing-complete languages. In a recent paper [7], we started extending our results to Turing complete linear GP machine code programs. We analysed the formation of the first loop in the programs and whether programs ever leave that loop. ...
... Finally, to further corroborate our results we developed a first Markov chain model of program execution and halting. Given its complexity and the limitations of space available, however, in [7] we only provided a one-page summary of its structure and results. The results confirmed both the simulations and the " √ L " scaling laws. ...
... The results confirmed both the simulations and the " √ L " scaling laws. Since [7], we further developed, validated and refined our Markov chain model, developing ways of iterating its transition matrix efficiently. In this paper we present the new model in detail and we discuss its implications for genetic programming research. ...
We focus on the halting probability and the number of instructions executed by programs that halt for Turing-complete register
based machines. The former represents the fraction of programs which provide useful results in a machine code genetic programming
system. The latter determines run time and whether or not the distribution of program functionality has reached a fixed-point.
We describe a Markov chain model of program execution and halting which accurately fits empirical data allowing us to efficiently
estimate the halting probability and the numbers of instructions executed for programs including millions of instructions.
We also discuss how this model can be applied to improve GP practice.
... In this paper we discuss a method to " de-quantize " the algorithm. We have been motivated by some recent approximate solutions to the Halting Problem obtained by Köhler, Schindelhauer and M. Ziegler [13] and experimental work [4, 14]. 1 Different approaches were proposed by Hamkins and Miasnikov [15], and D'Abramo [12]. ...
Since many real-world problems arising in the fields of compiler optimisation, automated software engineering, formal proof systems, and so forth are equivalent to the Halting Problem--the most notorious undecidable problem--there is a growing interest, not only academically, in understanding the problem better and in providing alternative solutions. Halting computations can be recognised by simply running them; the main difficulty is to detect non-halting programs. Our approach is to have the probability space extend over both space and time and to consider the probability that a random $N$-bit program has halted by a random time. We postulate an a priori computable probability distribution on all possible runtimes and we prove that given an integer k>0, we can effectively compute a time bound T such that the probability that an N-bit program will eventually halt given that it has not halted by T is smaller than 2^{-k}. We also show that the set of halting programs (which is computably enumerable, but not computable) can be written as a disjoint union of a computable set and a set of effectively vanishing probability. Finally, we show that ``long'' runtimes are effectively rare. More formally, the set of times at which an N-bit program can stop after the time 2^{N+constant} has effectively zero density.
... In some ways the search space of linear GP is easier to analyse than that of trees [223,225,226,227,244]. For example, we have used the T7 and T8 architectures (in simulation) for several large scale experimental and mathematical analysis of Turing complete GP [238,247,232,246]. For these reasons, it makes sense to consider linear "machine" code GP, for example, in Java. ...
The goal of having computers automatically solve problems is central to artificial intelligence, machine learning, and the
broad area encompassed by what Turing called ‘machine intelligence’ [384]. Machine learning pioneer Arthur Samuel, in his
1983 talk entitled ‘AI: Where It Has Been and Where It Is Going’ [337], stated that the main goal of the fields of machine
learning and artificial intelligence is:
“to get machines to exhibit behavior, which if done by humans, would be assumed to involve the use of intelligence.”
