Solving even-parity problems using traceless genetic programming

Abstract

A genetic programming (GP) variant called traceless genetic programming (TGP) is proposed in this paper. TGP is a hybrid method combining a technique for building the individuals and a technique for representing the individuals. The main difference between TGP and other GP techniques is that TGP does not explicitly store the evolved computer programs. Two genetic operators are used in conjunction with TGP: crossover and insertion. TGP is applied for evolving digital circuits for the even-parity problem. Numerical experiments show that TGP outperforms standard GP with several orders of magnitude.

Full text

Solving Even-Parity Problems using
Traceless Genetic Programming
Mihai Oltean
Department of Computer Science
Faculty of Mathematics and Computer Science
Babes¸-Bolyai University, Kog
˘
alniceanu 1
Cluj-Napoca, 3400, Romania
Email: moltean@cs.ubbcluj.ro
Abstract—A new Genetic Programming (GP) variant called
Traceless Genetic Programming (TGP) is proposed in this paper.
TGP is a hybrid method combining a technique for building
the individuals and a technique for representing the individuals.
The main difference between TGP and other GP techniques
is that TGP does not explicitly store the evolved computer
programs. Two genetic operators are used in conjunction with
TGP: crossover and insertion. TGP is applied for evolving digital
circuits for the even-parity problem. Numerical experiments
show that TGP outperforms standard GP with several orders
of magnitude.
I. INTRODUCTION
Traceless Genetic Programming (TGP)
1
is a GP [2], [3]
variant as it evolves a population of computer programs. TGP
is a hybrid method combining a technique for building the
individuals and technique for representing the individuals.
The main difference between the TGP and GP is that TGP
does not explicitly store the evolved computer programs. TGP
is useful when the trace (the way in which the results are
obtained) between the input and output is not important. In
this way the space used by traditional techniques for storing
the entire computer programs (or mathematical expressions in
the simple case of symbolic regression) is saved.
We choose to apply the proposed TGP technique to the
even-parity problems because according to Koza [2] these
problems appear to be the most difficult Boolean functions
to be detected via a blind random search.
Evolutionary techniques have been extensively used for
evolving digital circuits [1], [2], [3], [8], [4], [5], [6], [7],
[11], due to their practical importance. The case of even-
parity circuits was deeply analyzed [2], [3], [8], [7] due to
their simple representation.
Special techniques have been proposed in order to improve
in standard GP: Automatically Defined Functions [3], Sub-
machine code GP [9] and Sub-symbolic node representation
[10].
Standard GP was able to solve up to even-5 parity. Using the
proposed TGP we are able to easily solve up to even-8 parity
problem. Numerical experiments show that TGP outperforms
standard GP with several order of magnitude.
1
The source code for TGP is available at www.tgp.cs.ubbcluj.ro.
The paper is organized as follows: In section II the Traceless
Genetic Programming technique is described. The parity prob-
lem is briefly presented in section III. In section IV several
numerical experiments for solving the parity problems are
performed.
II. TRACELESS GENETIC PROGRAMMING
In this section the proposed TGP technique is described.
TGP is a hybrid method combining a technique for building
the individuals and a technique for representing the individu-
als.
A. Prerequisite
The quality of a GP individual is usually computed using a
set of fitness cases [2], [3]. For instance, the aim of symbolic
regression is to find a mathematical expression that satisfies a
set of m fitness cases.
We consider a problem with n inputs: x
1
, x
2
, ...x
n
and
one output f. The inputs are also called terminals [2]. The
function symbols that we use for constructing a mathematical
expression are F = {+, −, ∗, /, sin}.
Each fitness case is given as a (n + 1) dimensional array
of real values:
v
k
1
, v
k
2
, v
k
3
, ..., v
k
n
, f
k
where v
k
j
is the value of the j
th
attribute (which is x
j
) in the
k
th
fitness case and f
k
is the output for the k
th
fitness case.
Usually more fitness cases are given (denoted by m) and
the task is to find the expression that best satisfies all these
fitness cases. This is usually done by minimizing the quantity:
Q =
m
X
k=1
|f
k
− o
k
|,
where f
k
is the target value for the k
th
fitness case and o
k
is
the actual (obtained) value for the k
th
fitness case.
B. Individual representation
Each TGP individual represents a mathematical expression
evolved so far, but the TGP individual does not explicitly
store this expression. Each TGP individual stores only the
obtained value, so far, for each fitness case. Thus a TGP

individual is:
(o
1
, o
2
, o
3
, ..., o
m
)
T
,
where o
k
is the current value for the k
th
fitness case. Each
position in this array (a value o
k
) is a gene. As we said
it before behind these values is a mathematical expression
whose evaluation has generated these values. However, we
do not store this expression. We store only the values o
k
.
Remark
The structure of an TGP individual can be easily enhanced
for storing the evolved computer program (mathematical ex-
pression). Storing the evolved expression can provide a more
easy way to analyze the results of the numerical experiments.
However, in this paper, we do not store the trees associated
with the TGP individuals.
C. Initial population
The initial population contains individuals whose values
have been generated by simple expressions (made up a
single terminal). For instance, if an individual in the initial
population represent the expression:
E = x
1
,
then the corresponding TGP individual is represented as:
C = (v
1
1
, v
2
1
, v
3
1
, ..., v
m
1
)
where v
k
j
has been previously explained.
The quality of this individual is computed using the equation
previously described:
Q =
m
X
i=1
¯
¯
v
k
1
− f
k
¯
¯
.
D. Genetic Operators
In this section the genetic operators used in conjunction with
TGP are described. TGP uses two genetic operators: crossover
and insertion. These operators are specially designed for the
proposed TGP technique.
1) Crossover: The crossover is the only variation operator
that creates new individuals. For crossover several individuals
(the parents) and a function symbol are selected. The offspring
is obtained by applying the selected operator for each of the
genes of the parents.
Speaking in terms of expressions, an example of TGP
crossover is depicted in Figure 1.
From Figure 1 we can see that the parents are subtrees of
the offspring.
The number of parents selected for crossover depends on
the number of arguments required by the selected function
symbol. Two parents have to be selected for crossover if
the function symbol is a binary operator. A single parent
needs to be selected if the function symbol is a unary operator.
Example 1
Let us suppose that the operator + is selected. In this case
two parents:
C
1
= (p
1
, p
2
, ..., p
m
)
T
and
C
2
= (q
1
, q
2
, ..., q
m
)
T
are selected and the offspring O is obtained as follows:
O = (p
1
+ q
1
, p
2
+ q
2
,..., p
m
+ q
m
)
T
.
Example 2
Let us suppose that the operator sin is selected. In this case
one parent:
C
1
= (p
1
, p
2
, ..., p
m
)
T
is selected and the offspring O is obtained as follows:
O = (sin(p
1
), sin(p
2
),..., sin(p
m
))
T
.
2) Insertion: This operator inserts a simple expression
(made up of a single terminal) in the population. This operator
is useful when the population contains individuals representing
very complex expressions that cannot improve the search.
By inserting simple expressions we give a chance to the
evolutionary process to choose another direction for evolution.
E. TGP Algorithm
Due to the special representation and due to the newly
proposed genetic operators, TGP uses a special generational
algorithm which is given below:
The TGP algorithm starts by creating a random population
of individuals. The evolutionary process is run for a fixed num-
ber of generation. At each generation the following steps are
repeated until the new population is filled: With a probability
p
insert
generate an offspring made up of a single terminal
(see the Insertion operator). With a probability 1-p
insert
select
two parents using a standard selection procedure. The parents
are recombined in order to obtain an offspring. The offspring
enters the population of the next generation.
The standard TGP algorithm is depicted in Figure 2.
F. Complexity of the TGP Decoding Process
A very important aspect of the GP techniques is the time
complexity of the procedure used for computing the fitness of
the newly created individuals.
The complexity of that procedure for the standard GP is:
O(m ∗ g),
where m is the number of fitness cases and g is average
number of nodes in the GP tree.

Fig. 1. An example of TGP crossover.
Fig. 2. Traceless Genetic Programming Algorithm.
By contrast, the TGP complexity is only
O(m)
because the quality of a TGP individual can be computed
by traversing it only once. The length of a TGP individual is
m.
Due to this reason we may allow TGP programs to run g
times more generations in order to obtain the same complexity
as the standard GP.
III. PARITY PROBLEM
The Boolean even-parity function of k Boolean arguments
returns T (True) if an even number of its arguments are T.
Otherwise the even-parity function returns NIL (False) [2].
In applying TGP to the even-parity function of k arguments,
the terminal set T consists of the k Boolean arguments d
0
,
d
1
, d
2
, ... d
k−1
. The function set F consists of four two-
argument primitive Boolean functions: AND, OR, NAND,
NOR. According to [2] the Boolean even-parity functions
appear to be the most difficult Boolean functions to be detected
via a blind random search.
The set of fitness cases for this problem consists of the 2
k
combinations of the k Boolean arguments. The fitness of an
TGP chromosome is the sum, over these 2
k
fitness cases, of
the Hamming distance (error) between the returned value by
the TGP chromosome and the correct value of the Boolean
function. Since the standardized fitness ranges between 0 and

2
k
, a value closer to zero is better (since the fitness is to be
minimized).
Several techniques have been used in the past for solving
the parity problems [2], [3], [8].
IV. NUMERICAL EXPERIMENTS
Several numerical experiments using TGP are performed in
this section using the even-parity problems. General parameter
of the TGP algorithm are given in Table I.
TABLE I
GENERAL PARAMETERS OF THE TGP ALGORITHM FOR SOLVING PARITY
PROBLEMS.
Parameter Value
Insertion probability 0.05
Selection Binary Tournament
Function set {gates} {AND, OR, NAND, NOR}
Terminal set Problem inputs
Number of runs 100
For assessing the performance of the TGP algorithm the
computational effort an the probability of success metrics [2]
are used.
The method used to assess the effectiveness of an algorithm
has been suggested by Koza [2]. It consists of calculating the
number of chromosomes, which would have to be processed
to give a certain probability of success. To calculate this figure
one must first calculate the cumulative probability of success
P (M, i), where M represents the population size, and i the
generation number. The value R(z) represents the number of
independent runs required for a probability of success (given
by z) at generation i. The quantity I(M, z, i) represents the
minimum number of chromosomes which must be processed
to give a probability of success z, at generation i. The formulae
are given by the equation (1), (2) and (3). Ns(i) represents
the number of successful runs at generation i, and N
total
,
represents the total number of runs:
P (M, i) =
Ns(i)
N
total
. (1)
R(z) = ceil
½
log(1 − z)
log(1 − P (M, i)
¾
. (2)
I(M, i, z) = M · R(z) · i. (3)
In the tables and graphs of this section z takes the value
0.99.
A. Even-3 parity
The number of fitness cases for this problem is 2
3
= 8. For
solving the even-3 parity we use a population of 50 individuals
evolved for 200 generations. Other TGP parameters are given
in Table I.
The effort and the probability of success of the TGP
algorithm are depicted in Figure 3.
Fig. 3. The cumulative probability of success and the computational effort
for the even-3 parity problem. Results are averaged over 100 runs.
The minimum effort is 33,750 and it was obtained at
generation 131. We want to compare the result obtained by
TGP with that obtained by standard GP.
In [2] GP was used for solving the even-3 parity prob-
lem using a population of 4000 individuals evolved for 51
generations. The results indicated that 80,000 individuals are
sufficient to be processed in order to obtain a solution for this
problem [2]. One of the obtained solutions is a tree with 45
nodes.
As we noted in section II-F, the complexity of computing
the fitness of the TGP individuals is g times lower (g is the
number of nodes in a GP tree) than the complexity of decoding
GP individuals.
Due to this reason we have to divide the TGP effort (33,750)
by 45 (the number of nodes in a GP tree for the even-3 parity
problem). Thus, the actual TGP effort is 750 which is with
2 orders of magnitude better than the result obtained by GP.
Note that a perfect comparison between GP and TGP cannot
be made due to their different individual representation.
B. Even-4 parity
The number of fitness cases for this problem is 2
4
= 16.
For solving the even-4 parity we use a population of 100 in-
dividuals evolved for 500 generations. Other TGP parameters
are given in Table I.
The effort and the probability of success of the TGP
algorithm are depicted in Figure 4.
The minimum effort is 240,000 and it was obtained at
generation 480.
The effort spent by GP for solving the parity problem is
1,276,000. This number was obtained using a population of
4000 individuals [2]. One of the solutions evolved by GP has
149 nodes.

Fig. 4. The cumulative probability of success and the computational effort
for the even-4 parity problem. Results are averaged over 100 runs.
If we want to compare the efforts spent by GP and TGP we
have to divide the TGP effort (240,000) by 149 (number of
nodes in a GP tree). Thus, we obtain the number 1610 which
is with almost 3 orders of magnitude better than the result
obtained by GP.
C. Even-5 parity
The number of fitness cases for this problem is 2
5
=
32. For solving the even-5 parity we use a population of
500 individuals evolved for 1000 generations. Other TGP
parameters are given in Table I.
The effort and the probability of success of the TGP
algorithm are depicted in Figure 5.
The effort is 2,417,500 and it was obtained at generation
967.
For this problem standard GP with a population of 8000
individuals obtained a solution in the 8
th
run [2]. No other
statistics were given for this problem.
D. Even-6 parity
The number of fitness cases for this problem is 2
6
=
64. For solving the even-6 parity we use a population of
1000 individuals evolved for 2500 generations. Other TGP
parameters are given in Table I.
The effort and the probability of success of the TGP
algorithm are depicted in Figure 6.
The minimum effort is 29,136,000 and it was obtained at
generation 2428.
E. Even-7 parity
The number of fitness cases for this problem is 2
7
=
128. For solving the even-7 parity we use a population of
2000 individuals evolved for 5000 generations. Other TGP
parameters are given in Table I.
Fig. 5. The cumulative probability of success and the computational effort
for the even-5 parity problem. Results are averaged over 100 runs.
Fig. 6. The cumulative probability of success and the computational effort
for the even-6 parity problem. Results are averaged over 100 runs.
The effort and the probability of success of the TGP
algorithm are depicted in Figure 7.
The minimum effort is 245,900,000 and it was obtained at
generation 4918.
F. Even-8 parity
The even-8 parity problem is the most difficult problem
analyzed in this paper. The number of fitness cases for this
problem is 2
8
= 256. For solving the even-8 parity we use a
population of 5000 individuals evolved for 10000 generations.
Other TGP parameters are given in Table I. Only 10 runs
are performed for this problem. In the 9
th
run we obtained a
solution.

Fig. 7. The cumulative probability of success and the computational effort
for the even-7 parity problem. Results are averaged over 100 runs.
V. SUMMARIZED RESULTS
Summarized results of applying Traceless Genetic Program-
ming for solving even-parity problems are given in Table II.
TABLE II
SUMMARIZED RESULTS FOR SOLVING THE EVEN-PARITY PROBLEM USING
TGP. SECOND COLUMN INDICATES THE POPULATION SIZE USED FOR
SOLVING THE PROBLEM. THE COMPUTATIONAL EFFORT IS GIVEN IN THE
3
rd
COLUMN. THE NUMBERS IN THE 4
th
COLUMN INDICATE THE
GENERATION WHERE THE MINIMUM EFFORT WAS OBTAINED.
Problem Pop Size Effort Generation
even-3 50 33,750 131
even-4 100 240,000 480
even-5 500 2,417,500 967
even-6 1000 29,136,000 2428
even-7 2000 245,900,000 4918
Table II shows that TGP is able to solve the even-parity
problems very well. Genetic Programming without ADF was
able to solve instances up to even-5 parity problem within a
reasonable time frame and using a reasonable population. Note
again that a perfect comparison between GP and TGP cannot
be made due to their different individual representation.
Table II also shows that the effort required for solving
the problem increases with one order of magnitude for each
instance.
In order to see the effectiveness and the simplicity of the
TGP algorithm we give, in Table III, the time needed for
solving these problems using a PIII computer at 850 MHz.
Table III shows that TGP is very fast. 92 seconds are needed
to obtain a solution for the even-7 parity problem. Note that
the standard GP was never used for solving the even-7 parity
problem due to the expensive computational time.
TABLE III
THE AVERAGE TIME FOR OBTAINING A SOLUTION FOR THE EVEN-PARITY
PROBLEM USING TGP.
Problem Time (seconds)
even-3 0.2
even-4 0.9
even-5 3.2
even-6 19.3
even-7 92.5
VI. LIMITATIONS OF THE PROPOSED APPROACH
There could be a problem with the length of the program
evolved by TGP. The number of gates in offspring is the sum
of the number of gates in parents + 1. In the case of binary
operators the number of gates in a TGP chromosome might
increase exponentially.
This problem could be avoided if the selection process takes
into account the number of gates of the chosen individuals.
VII. CONCLUSIONS
A new evolutionary technique called Traceless Genetic
Programming has been proposed in this paper. TGP uses a
new individual representation, new genetic operators and a
specific evolutionary algorithm.
TGP has been used for evolving digital circuits for the
even parity problems. Numerical experiments have shown that
TGP was able to evolve very fast a solution for up to even-
8 parity problem. Note that the standard GP evolved (within
a reasonable time frame) a solution for up to even-5 parity
problem.
VIII. FURTHER WORK
Further effort will be spent for improving the proposed
Traceless Genetic Programming technique. For instance, in
[9] a Sub-machine code technique was used for improving the
performance ot the GP technique. This kind of improvement
can be applied for TGP too.
In [10] an extended, unbiased set of 16 gates was used
for solving the even-parity problems. Numerical experiments
shown [10] that GP was able to solve up to even-22 parity
instance using the considered set of gates. Further numerical
experiments with TGP will include the use of the extended set
of all possible 16 gates with 2 inputs.
REFERENCES
[1] C. Coello, E. Alba, G. Luque and A. Aguire, ”Comparing different
Serial and Parallel Heuristics to Design Combinational Logic Circuits”,
in Proceedings of 2003 NASA/DoD Conference on Evolvable Hardware,
J. Lohn, R. Zebulum, J. Steincamp, D. Keymeulen, A. Stoica, and M.I.
Ferguson, Eds. IEEE Press, 2003, pp. 3-12.
[2] J. R. Koza, Genetic Programming: On the Programming of Computers
by Means of Natural Selection, MIT Press, Cambridge, MA, 1992.
[3] J. R. Koza, Genetic Programming II: Automatic Discovery of Reusable
Subprograms, MIT Press, Cambridge, MA, 1994.
[4] J. F. Miller, P. Thomson, ”Aspects of Digital Evolution: Evolvability
and Architecture”, in Proceedings of the Parallel Problem Solving from
Nature V, A. E. Eiben, T. B
¨
ack, M. Schoenauer and H-P Schwefel,
Eds. Springer-Verlag, 1998, pp. 927-936.

[5] J. F. Miller, P. Thomson, T. Fogarty, ”Designing Electronic Circuits
using Evolutionary Algorithms. Arithmetic Circuits: A Case Study”,
in Genetic Algorithms and Evolution Strategies in Engineering and
Computer Science, D. Quagliarella, J. Periaux, C. Poloni and G. Winter,
Eds. UK-Wiley, 1997, pp. 105-131.
[6] J. F. Miller, ”An Empirical Study of the Efficiency of Learning Boolean
Functions using a Cartesian Genetic Programming Approach”, in Pro-
ceedings of the 1
st
Genetic and Evolutionary Computation Conference,
W. Banzhaf, J. Daida, A. E. Eiben, M. H. Garzon, V. Honavar, M.
Jakiela, and R. E. Smith, Eds. vol. 2, San Francisco, CA: Morgan
Kaufmann, 1999, pp. 1135-1142.
[7] J. F. Miller, D. Job, V. K. Vassilev, ”Principles in the Evolutionary
Design of Digital Circuits - Part I”, Genetic Programming and Evolvable
Machines, vol. 1. pp. 7-35, Kluwer Academic Publishers, 2000.
[8] M. Oltean, ”Solving Even-parity problems with Multi Expression Pro-
gramming”, in Proceedings of the 5
th
International Workshop on
Frontiers in Evolutionary Algorithm, K. Chen (et. al) Eds. 2003, pp.
315-318.
[9] R. Poli, W. B. Langdon, ”Sub-machine Code Genetic Programming”,
in Advances in Genetic Programming 3, L. Spector, W. B. Langdon,
U.-M. O’Reilly, P. J. Angeline, Eds. Cambridge:MA, MIT Press, 1999,
chapter 13.
[10] R. Poli, J. Page, ”Solving High-Order Boolean Parity Problems with
Smooth Uniform Crossover, Sub-Machine Code GP and Demes”, Jour-
nal of Genetic Programming and Evolvable Machines, Kluwer, pp. 1-21,
2000.
[11] A. Stoica, R. Zebulum, X. Guo, D. Keymeulen, M. Ferguson, V. Duong,
”Silicon Validation of Evolution Designed Circuits”, in Proceedings of
the 2003 NASA/DoD Conference on Evolvable Hardware, J. Lohn, R.
Zebulum, J. Steincamp, D. Keymeulen, A. Stoica, and M.I. Ferguson,
Eds. IEEE Press, 2003, pp. 21-26.