# Using NEAT to evolve solutions to XOR¶

To use NEAT you must import the following.

In [1]:
from neat3 import config, population, chromosome, genome, visualize
from neat3.nn import nn_pure as nn
import pickle


### Configuring NEAT¶

In order to run NEAT, you'll need to configure a number of paramters. There is a configuration file, provided below, where these parameters are set.

The [phenotype] section defines the type of neural networks that will be evolved. You need to ensure that the number of input nodes and output nodes match the requirements of your problem domain. Typically you will start with zero hidden nodes, and let NEAT add them if necessary to achieve higher fitness. The nn_activation defines what activation function will be used. The function exp is a sigmoid and will only return values in the range [0,1]. The function tanh will return values in the range [-1,1]. Typically for robot control problems we will use tanh.

The [genetic] section defines the size of the population and the maximum fitness threshold. Both of these should be tuned for your specific problem domain. When deciding on a pop_size, keep in mind that the larger the population, the longer the evolution process will take. However, larger populations tend to yield better results. You'll need to decide the best tradeoff of size vs speed. The max_fitness_threshold is used by NEAT to decide whether to stop evolution early. If this threshold is ever achieved, NEAT will suspend evolution, otherwise it will continue until the specified number of generations is reached.

The other sections tune how NEAT operates. I tend to leave these at the default settings, but feel free to experiment with them.

In [2]:
%%file configXOR

[phenotype]
input_nodes         = 2
output_nodes        = 1
max_weight          = 30
min_weight          = -30
feedforward         = 1
nn_activation       = exp
hidden_nodes        = 0
weight_stdev        = 0.9

[genetic]
pop_size              = 50
max_fitness_threshold = 1

# Human reasoning
prob_mutatebias       = 0.2
bias_mutation_power   = 0.5
prob_mutate_weight    = 0.9
weight_mutation_power = 1.5
elitism               = 1

[genotype compatibility]
compatibility_threshold = 3.0
compatibility_change    = 0.0
excess_coeficient       = 1.0
disjoint_coeficient     = 1.0
weight_coeficient       = 0.4

[species]
species_size        = 10
survival_threshold  = 0.2
old_threshold       = 30
youth_threshold     = 10
old_penalty         = 0.2
youth_boost         = 1.2
max_stagnation      = 15

Overwriting configXOR


### Define a fitness function¶

Create a fitness function that takes a chromosome as input and as a side-effect sets the chromosome's fitness. It does not return anything. A fitness function should be defined such that higher values equate with better fitness.

For this task, we want to define fitness to match how well the network solves the XOR problem. We first calculate the sum-squared error of a network on the XOR task. Good networks will have low error, so we will define fitness as 1 - sqrt(avgError). In order to evaluate a NEAT chromosome, you must first convert the chromosome into a phenotype (a network). Then you can activate the network and test its output values.

In [3]:
import math
INPUTS = [[0, 0], [0, 1], [1, 0], [1, 1]]
OUTPUTS = [0, 1, 1, 0]

def eval_individual(chromo):
net = nn.create_ffphenotype(chromo)
error = 0.0
for i, inputs in enumerate(INPUTS):
net.flush()
output = net.sactivate(inputs)
error += (output[0] - OUTPUTS[i])**2
chromo.fitness = 1 - math.sqrt(error/len(OUTPUTS))


### Define a population evaluation function¶

NEAT requires a population evaluation function that takes a population as input. A population is defined to be a list of chromosomes. This function will be called once for every generation of the evolution, and it will evaluate the fitness of every member of the population.

In [4]:
def eval_population(population):
for chromo in population:
eval_individual(chromo)


### Evolve¶

The evolve function takes the number of generations to run. Each generation, NEAT will print out statistics about the population, including the current best fitness and the average fitness.

In [5]:
def evolve(n):
chromosome.node_gene_type = genome.NodeGene

# Tell NEAT that we want to use the above function to evaluate fitness
population.Population.evaluate = eval_population

# Create a population (the size is defined in the configuration file)
pop = population.Population()

# Run NEAT's genetic algorithm for at most 30 epochs
# It will stop if fitness surpasses the max_fitness_threshold in config file
pop.epoch(n, report=True, save_best=True, name="XOR")

# Plots the evolution of the best/average fitness
visualize.plot_stats(pop.stats)

# Visualizes speciation
visualize.plot_species(pop.species_log)

In [6]:
evolve(30)

 ****** Running generation 0 ******

Population's average fitness: 0.39660 stdev: 0.06082
Best fitness: 0.4998555915 - size: (0, 2) - species 1 - id 30
Species length: 1 totalizing 50 individuals
Species ID       : [1]
Each species size: [50]
Amount to spawn  : [50]
Species age      : [0]
Species no improv: [0]

****** Running generation 1 ******

Population's average fitness: 0.40501 stdev: 0.06442
Best fitness: 0.4998555915 - size: (0, 2) - species 1 - id 30
Species length: 1 totalizing 50 individuals
Species ID       : [1]
Each species size: [50]
Amount to spawn  : [50]
Species age      : [1]
Species no improv: [0]

****** Running generation 2 ******

Population's average fitness: 0.39970 stdev: 0.07279
Best fitness: 0.4998555915 - size: (0, 2) - species 1 - id 30
Species length: 3 totalizing 50 individuals
Species ID       : [1, 2, 3]
Each species size: [47, 2, 1]
Amount to spawn  : [17, 14, 20]
Species age      : [2, 0, 0]
Species no improv: [1, 0, 0]
Removing 1 excess indiv(s) from the new population

****** Running generation 3 ******

Population's average fitness: 0.38665 stdev: 0.07965
Best fitness: 0.4999850268 - size: (1, 4) - species 1 - id 183
Species length: 2 totalizing 50 individuals
Species ID       : [1, 2]
Each species size: [41, 9]
Amount to spawn  : [26, 24]
Species age      : [3, 1]
Species no improv: [2, 0]

****** Running generation 4 ******

Population's average fitness: 0.37976 stdev: 0.08801
Best fitness: 0.4999850268 - size: (1, 4) - species 1 - id 183
Species length: 2 totalizing 50 individuals
Species ID       : [1, 2]
Each species size: [26, 24]
Amount to spawn  : [26, 24]
Species age      : [4, 2]
Species no improv: [3, 1]

****** Running generation 5 ******

Population's average fitness: 0.39275 stdev: 0.09239
Best fitness: 0.4999964553 - size: (1, 4) - species 1 - id 259
Species length: 2 totalizing 50 individuals
Species ID       : [1, 2]
Each species size: [25, 25]
Amount to spawn  : [27, 23]
Species age      : [5, 3]
Species no improv: [0, 0]

****** Running generation 6 ******

Population's average fitness: 0.43390 stdev: 0.09431
Best fitness: 0.5179961340 - size: (1, 5) - species 1 - id 320
Species length: 3 totalizing 50 individuals
Species ID       : [1, 2, 4]
Each species size: [47, 2, 1]
Amount to spawn  : [18, 19, 13]
Species age      : [6, 4, 0]
Species no improv: [0, 0, 0]

****** Running generation 7 ******

Population's average fitness: 0.42348 stdev: 0.09208
Best fitness: 0.5179961340 - size: (1, 5) - species 1 - id 320
Species length: 3 totalizing 50 individuals
Species ID       : [1, 4, 5]
Each species size: [35, 11, 4]
Amount to spawn  : [18, 12, 19]
Species age      : [7, 1, 0]
Species no improv: [0, 0, 0]
Selecting 1 more indiv(s) to fill up the new population

****** Running generation 8 ******

Population's average fitness: 0.44845 stdev: 0.07554
Best fitness: 0.5179961340 - size: (1, 5) - species 1 - id 320
Species length: 2 totalizing 50 individuals
Species ID       : [1, 4]
Each species size: [38, 12]
Amount to spawn  : [26, 24]
Species age      : [8, 2]
Species no improv: [0, 0]

****** Running generation 9 ******

Population's average fitness: 0.45446 stdev: 0.07062
Best fitness: 0.5943586243 - size: (1, 5) - species 1 - id 437
Species length: 2 totalizing 50 individuals
Species ID       : [1, 4]
Each species size: [26, 24]
Amount to spawn  : [25, 25]
Species age      : [9, 3]
Species no improv: [0, 0]

****** Running generation 10 ******

Population's average fitness: 0.43392 stdev: 0.08821
Best fitness: 0.5943586243 - size: (1, 5) - species 1 - id 437
Species length: 3 totalizing 50 individuals
Species ID       : [1, 4, 6]
Each species size: [24, 25, 1]
Amount to spawn  : [13, 17, 19]
Species age      : [10, 4, 0]
Species no improv: [1, 0, 0]
Selecting 1 more indiv(s) to fill up the new population

****** Running generation 11 ******

Population's average fitness: 0.44828 stdev: 0.09289
Best fitness: 0.5943586243 - size: (1, 5) - species 1 - id 437
Species length: 4 totalizing 50 individuals
Species ID       : [1, 4, 6, 7]
Each species size: [13, 18, 18, 1]
Amount to spawn  : [12, 14, 14, 9]
Species age      : [11, 5, 1, 0]
Species no improv: [2, 0, 1, 0]
Selecting 1 more indiv(s) to fill up the new population

****** Running generation 12 ******

Population's average fitness: 0.45460 stdev: 0.08749
Best fitness: 0.6254678735 - size: (1, 4) - species 1 - id 579
Species length: 4 totalizing 50 individuals
Species ID       : [1, 4, 6, 7]
Each species size: [12, 28, 1, 9]
Amount to spawn  : [11, 14, 14, 11]
Species age      : [12, 6, 2, 1]
Species no improv: [3, 0, 2, 0]

****** Running generation 13 ******

Population's average fitness: 0.46742 stdev: 0.08110
Best fitness: 0.6254678735 - size: (1, 4) - species 1 - id 579
Species length: 4 totalizing 50 individuals
Species ID       : [1, 4, 6, 7]
Each species size: [11, 13, 14, 12]
Amount to spawn  : [11, 13, 13, 13]
Species age      : [13, 7, 3, 2]
Species no improv: [0, 1, 3, 0]

****** Running generation 14 ******

Population's average fitness: 0.45172 stdev: 0.08333
Best fitness: 0.6254678735 - size: (1, 4) - species 1 - id 625
Species length: 6 totalizing 50 individuals
Species ID       : [1, 4, 6, 7, 8, 9]
Each species size: [11, 13, 12, 11, 1, 2]
Amount to spawn  : [7, 9, 8, 8, 9, 9]
Species age      : [14, 8, 4, 3, 0, 0]
Species no improv: [1, 0, 4, 1, 0, 0]

****** Running generation 15 ******

Population's average fitness: 0.48658 stdev: 0.07023
Best fitness: 0.6254678735 - size: (1, 4) - species 1 - id 625
Species length: 7 totalizing 50 individuals
Species ID       : [1, 4, 6, 7, 8, 9, 10]
Each species size: [7, 12, 7, 9, 6, 8, 1]
Amount to spawn  : [7, 7, 7, 7, 7, 7, 8]
Species age      : [15, 9, 5, 4, 1, 1, 0]
Species no improv: [0, 0, 5, 2, 1, 1, 0]

****** Running generation 16 ******

Population's average fitness: 0.47677 stdev: 0.08117
Best fitness: 0.6254678735 - size: (1, 4) - species 1 - id 720
Species length: 7 totalizing 50 individuals
Species ID       : [1, 4, 6, 7, 8, 9, 10]
Each species size: [7, 12, 7, 7, 2, 7, 8]
Amount to spawn  : [7, 6, 7, 7, 8, 7, 7]
Species age      : [16, 10, 6, 5, 2, 2, 1]
Species no improv: [0, 1, 6, 3, 2, 2, 1]
Selecting 1 more indiv(s) to fill up the new population

****** Running generation 17 ******

Population's average fitness: 0.47432 stdev: 0.06924
Best fitness: 0.6254678735 - size: (1, 4) - species 1 - id 765
Species length: 6 totalizing 50 individuals
Species ID       : [1, 4, 6, 7, 9, 10]
Each species size: [7, 15, 7, 10, 4, 7]
Amount to spawn  : [7, 8, 8, 9, 9, 8]
Species age      : [17, 11, 7, 6, 3, 2]
Species no improv: [1, 0, 7, 0, 0, 2]
Selecting 1 more indiv(s) to fill up the new population

****** Running generation 18 ******

Population's average fitness: 0.49630 stdev: 0.06692
Best fitness: 0.6260981886 - size: (1, 4) - species 1 - id 851
Species length: 5 totalizing 50 individuals
Species ID       : [1, 4, 6, 7, 10]
Each species size: [7, 9, 8, 18, 8]
Amount to spawn  : [11, 9, 11, 10, 9]
Species age      : [18, 12, 8, 7, 3]
Species no improv: [0, 0, 8, 1, 3]

****** Running generation 19 ******

Population's average fitness: 0.47224 stdev: 0.07451
Best fitness: 0.6260981886 - size: (1, 4) - species 1 - id 851
Species length: 6 totalizing 50 individuals
Species ID       : [1, 4, 6, 7, 10, 11]
Each species size: [11, 8, 11, 10, 9, 1]
Amount to spawn  : [6, 8, 9, 9, 8, 9]
Species age      : [19, 13, 9, 8, 4, 0]
Species no improv: [1, 1, 9, 0, 4, 0]
Selecting 1 more indiv(s) to fill up the new population

****** Running generation 20 ******

Population's average fitness: 0.48621 stdev: 0.05207
Best fitness: 0.6260981886 - size: (1, 4) - species 1 - id 892
Species length: 7 totalizing 50 individuals
Species ID       : [1, 4, 6, 7, 10, 11, 12]
Each species size: [6, 8, 9, 9, 8, 9, 1]
Amount to spawn  : [6, 7, 7, 8, 8, 8, 5]
Species age      : [20, 14, 10, 9, 5, 1, 0]
Species no improv: [2, 2, 10, 0, 5, 1, 0]
Selecting 1 more indiv(s) to fill up the new population

****** Running generation 21 ******

Population's average fitness: 0.46624 stdev: 0.08291
Best fitness: 0.6260981886 - size: (1, 4) - species 1 - id 892
Species length: 8 totalizing 50 individuals
Species ID       : [1, 4, 6, 7, 10, 12, 13, 14]
Each species size: [7, 15, 7, 4, 7, 5, 4, 1]
Amount to spawn  : [6, 6, 5, 6, 7, 6, 7, 7]
Species age      : [21, 15, 11, 10, 6, 1, 0, 0]
Species no improv: [3, 3, 11, 1, 6, 0, 0, 0]

****** Running generation 22 ******

Population's average fitness: 0.45428 stdev: 0.08585
Best fitness: 0.6260981886 - size: (1, 4) - species 1 - id 892
Species length: 8 totalizing 50 individuals
Species ID       : [1, 4, 6, 7, 10, 12, 13, 14]
Each species size: [6, 7, 5, 12, 6, 6, 1, 7]
Amount to spawn  : [6, 6, 6, 6, 6, 7, 7, 6]
Species age      : [22, 16, 12, 11, 7, 2, 1, 1]
Species no improv: [4, 4, 0, 2, 7, 0, 0, 1]

****** Running generation 23 ******

Population's average fitness: 0.47589 stdev: 0.07391
Best fitness: 0.6260981886 - size: (1, 4) - species 1 - id 892
Species length: 7 totalizing 50 individuals
Species ID       : [1, 4, 6, 7, 12, 13, 14]
Each species size: [6, 14, 6, 12, 7, 1, 4]
Amount to spawn  : [6, 6, 7, 7, 8, 8, 7]
Species age      : [23, 17, 13, 12, 3, 2, 2]
Species no improv: [5, 5, 1, 3, 0, 0, 2]
Selecting 1 more indiv(s) to fill up the new population

****** Running generation 24 ******

Population's average fitness: 0.45183 stdev: 0.08799
Best fitness: 0.6260981886 - size: (1, 4) - species 1 - id 1064
Species length: 9 totalizing 50 individuals
Species ID       : [1, 4, 6, 7, 12, 13, 14, 15, 16]
Each species size: [6, 7, 6, 7, 7, 8, 7, 1, 1]
Amount to spawn  : [4, 5, 6, 5, 6, 6, 6, 4, 7]
Species age      : [24, 18, 14, 13, 4, 3, 3, 0, 0]
Species no improv: [6, 6, 0, 4, 1, 1, 3, 0, 0]
Selecting 1 more indiv(s) to fill up the new population

****** Running generation 25 ******

Population's average fitness: 0.44886 stdev: 0.09305
Best fitness: 0.6260981886 - size: (1, 4) - species 1 - id 1064
Species length: 9 totalizing 50 individuals
Species ID       : [1, 4, 6, 7, 12, 13, 14, 15, 16]
Each species size: [4, 6, 9, 6, 12, 6, 5, 1, 1]
Amount to spawn  : [6, 5, 5, 6, 7, 6, 7, 4, 4]
Species age      : [25, 19, 15, 14, 5, 4, 4, 1, 1]
Species no improv: [7, 7, 1, 5, 2, 2, 4, 1, 1]

****** Running generation 26 ******

Population's average fitness: 0.45017 stdev: 0.10080
Best fitness: 0.6260981886 - size: (1, 4) - species 1 - id 1064
Species length: 8 totalizing 50 individuals
Species ID       : [1, 4, 6, 7, 12, 13, 15, 16]
Each species size: [6, 12, 5, 6, 7, 6, 4, 4]
Amount to spawn  : [7, 6, 7, 7, 7, 8, 5, 5]
Species age      : [26, 20, 16, 15, 6, 5, 2, 2]
Species no improv: [8, 8, 2, 6, 3, 3, 0, 2]
Removing 2 excess indiv(s) from the new population

****** Running generation 27 ******

Population's average fitness: 0.46011 stdev: 0.08903
Best fitness: 0.6260981886 - size: (1, 4) - species 1 - id 1191
Species length: 9 totalizing 50 individuals
Species ID       : [1, 4, 6, 7, 12, 13, 15, 16, 17]
Each species size: [7, 5, 7, 7, 7, 8, 5, 3, 1]
Amount to spawn  : [6, 5, 5, 5, 6, 6, 6, 4, 7]
Species age      : [27, 21, 17, 16, 7, 6, 3, 3, 0]
Species no improv: [9, 9, 3, 7, 4, 4, 0, 3, 0]

****** Running generation 28 ******

Population's average fitness: 0.48232 stdev: 0.08515
Best fitness: 0.6260981886 - size: (1, 4) - species 1 - id 1191
Species length: 9 totalizing 50 individuals
Species ID       : [1, 4, 6, 7, 12, 13, 15, 16, 17]
Each species size: [6, 6, 5, 5, 6, 6, 6, 4, 6]
Amount to spawn  : [6, 5, 5, 5, 6, 6, 6, 4, 6]
Species age      : [28, 22, 18, 17, 8, 7, 4, 4, 1]
Species no improv: [10, 10, 4, 8, 0, 5, 0, 4, 1]
Selecting 1 more indiv(s) to fill up the new population

****** Running generation 29 ******

Population's average fitness: 0.46633 stdev: 0.08473
Best fitness: 0.6260981886 - size: (1, 4) - species 1 - id 1191
Species length: 10 totalizing 50 individuals
Species ID       : [1, 4, 6, 7, 12, 13, 15, 16, 17, 18]
Each species size: [7, 5, 5, 5, 6, 6, 5, 4, 6, 1]
Amount to spawn  : [4, 5, 5, 5, 5, 6, 5, 4, 5, 6]
Species age      : [29, 23, 19, 18, 9, 8, 5, 5, 2, 0]
Species no improv: [11, 11, 5, 9, 1, 6, 0, 5, 2, 0]


### Evaluating the results of evolution¶

NEAT will save several plots, one showing the best and average fitness over time, and another showing how NEAT's population speciated over time. These plots will have .svg extensions and can be opened from a terminal window using eog. Try viewing these plots.

In addition, NEAT will save the best chromosome present in the population at the end of each generation. To test a particular chromosome file, you can use the following function. This will also generate a new .svg file that includes a visualization of the network's topology. Try viewing this plot which will have a filename that begins with phenotype.

In [7]:
def eval_best(chromo_file):
fp = open(chromo_file, "rb")
fp.close()
print(chromo)
visualize.draw_net(chromo, "_"+ chromo_file)

print('\nNetwork output:')
brain = nn.create_ffphenotype(chromo)
for i, inputs in enumerate(INPUTS):
output = brain.sactivate(inputs)
print("%1.5f \t %1.5f" %(OUTPUTS[i], output[0]))

In [8]:
eval_best("XOR_best_chromo_29")

Nodes:
Node  1  INPUT, bias  0, response 4.924273
Node  2  INPUT, bias  0, response 4.924273
Node  3 OUTPUT, bias 1.56438882, response 4.924273
Node  4 HIDDEN, bias 0.95296582, response 4.94640357
Connections:
In  1, Out  3, Weight +0.40704, Enabled, Innov 1
In  2, Out  3, Weight +0.67050, Disabled, Innov 2
In  2, Out  4, Weight -2.18989, Enabled, Innov 3
In  4, Out  3, Weight -1.88029, Enabled, Innov 4
In  1, Out  4, Weight +3.32298, Enabled, Innov 5
Node order: [4]

Network output:
0.00000 	 0.18645
1.00000 	 0.99954
1.00000 	 0.61036
0.00000 	 0.61043


### Experiments¶

Run 10 experiments on the XOR problem using a popultion of size 50. For each experiment, record whether NEAT evolved a successful solution, and the number of hidden nodes it generated.

Run 10 more experiments on the XOR problem using a different population size (you can try smaller or larger, whichever you prefer). Record the same information as before, and compare the results.

### Use git to commit, add, and push¶

Save this notebook before moving on.