In this lab you will experiment with an evolutionary computation
method called NEAT (Neuro Evolution of Augmenting Topologies)
developed by Kenneth O. Stanley. NEAT operates on both the weights and
the structure of the neural network, allowing connections and nodes to
be added as potential mutations. We will be using a python package
that implements NEAT.
In Lab 1 we used the back-propagation algorithm to learn the weights of a fixed architecture neural network. Now we can use NEAT to evolve both the weights and the architecture of a neural network. We will begin exploring NEAT by using XOR as a testbed.
% python evolveXOR.pyThis will produce a number of messages detailing the progress of the evolutionary process. For each generation you will see the average fitness, best fitness, number of species, and information about each species. If evolution found a particular network that surpassed the maximum fitness threshold it will terminate early. Otherwise it will complete the entire 30 epochs.
Look at these graphs. Note the best fitness achieved and in what generation it was found.
% python evaluateXOR.py best_chromo_n
% xv phenotype_best_chromo_n.svgLook at the visualization of the best network's architecture.
We know that neural networks can more easily learn linearly separable functions. XOR is an example of a function that is not linearly separable, while AND is an example of a function that is linearly separable. Let's see if NEAT can more easily find solutions to the AND problem.
The NEAT paper we are discussing this week uses a very time consuming fitness test intended to create a co-evolutionary arms race. We will experiment with a simpler scenario in this lab. Rather than a robot duel, we will focus on a single robot foraging for food. We will place the robot in a world with 10 randomly located lights, representing food, and allow it to explore and eat as long as it has energy to move. We will test the robot with three different random placements of the food. The network has three input units representing the left and right light sensor values as well as an energy warning value. As its energy level gets low the warning value increases. The network has two output units representing the motor speeds of the left and right wheels of a simulated Pioneer robot. The fitness value is based on the total number of steps the robot survives in the three trials.
% python evolveEnergy.pyThis program uses the pyrobot simulator, but does not display it in order to speed up the fitness evaluations. This will still take some time to complete. The evolution is set to run for just 11 generations, using a population of 20 individuals. A more reasonable test would be to use a larger population (at least 30-50 individuals) and more generations.
% python evaluateEnergy.py best_chromo_n log_fileThe second parameter is the name of a log file (of your choice) where details of the evaluation will be saved. Observe the behaviors of the best networks from each generation and look at the log files that are produced. Are robots from later generations able to survive longer than their predecessors? Does NEAT complexify the networks in order to achieve this increased rate of survival? What kinds of behaviors are developed?
There are many different ways of implementing a food foraging task. Each of the choices made in the above program could affect the types of behavior that resulted. Here are just some of the issues that must be considered:
Look carefully at the files given for this task and list each of the choices that were made for each issue given above.
Then in the directory cs81/labs/2/ modifiedTask create your own version of the food foraging task that makes some different choices. In the text file experimentalLog keep a record of the options you tried (even if they failed). Be sure to explain each idea you explored, how you tested its effectiveness (list key parameter settings such as population size, maximum number of generations, etc.), and the results.
For example, you might decide that testing each network in random
environments is too noisy, and that a better test would be to use a
fixed environment similar to the one in the robot duel. Re-write the
code to do this, run it, and report on what happens.
When you are done run handin81 to turn in your completed lab work.