Cogs1 Spring 2007
Lab 5: Evolving Robot Brains
Due by 11:30pm on Wednesday, March 7


In this lab you will be observing the evolution of a neural network brain to control a robot. The evolving robot, which is green, is situated in a simple enclosed world with an avoiding robot, which is red. The avoiding robot is running a fixed control program that moves it straight forward unless it senses an obstacle, and then it will turn away from the obstacle. If the avoiding robot is ever jammed against an obstacle, it will try random movements to free itself.

The behavior of the evolving robot is scored by a fitness function that is based on four components:

Whenever the evolving robot is not moving or there is no visible red object, then the fitness for that time step will be zero. Otherwise the fitness is maximized when the translation speed is 1, there is a red object centered in the evolving robot's camera image, and the red object is as close as possible to the evolving robot. Since the only red object in this world is the avoiding robot, the evolving robot should discover behaviors that allow it to follow the avoiding robot around the environment.

The evolving robot has a neural network brain that consists of three fully connected layers: input (size 6), hidden (size 4), and output (size 2). The activations on the input layer represent the current sensor values: the minimum value of all of the front sonars, the minimum value of all of the back sonars, the range to the nearest red object, and the location of the nearest red object. The activations on the output layer represent the translation and rotation speed of the robot's next movement.

You will be using a genetic algorithm to find appropriate weights for the neural network brain of the evolving robot. For this particular experiment, the genetic algorithm consists of a population of 10 possible sets of weights for this brain. Initially these weights will be given random values. On each generation, each set of weights will be tested on the evolving robot for 250 time steps using the fitness measure described above. The weights that receive the best fitness scores will tend to be selected for inclusion in the next generation. Selected weights may also undergo crossover and mutation. This evolutionary process will continue for 5 generations.

Getting Started

  1. To get copies of the files you'll need for this lab, open a terminal window and type:
  2. To move down to the appropriate directory, type:
    cd cogs1/labs/5
  3. The entire evolutionary process may take 20-30 minutes to complete. To start the evolutionary process, type (the last character is an ampersand):
    python &
    This will put the robots into a starting configuration, load a new set of weights onto the evolving robot (colored green), and start both robots running. After 250 steps, you will see a printed message showing the fitness score for that particular set of weights. Then the next set of weights will be tested in the same way. After the entire population has been tested, you will see a printed message summarizing the generation just completed. In the initial generations, many of the weights will result in very poor behavior. The evolving robot may immediately crash into a wall and never move again. However, as the process continues, you should begin to see more interesting behaviors emerge.
  4. When you see the message:
    Done evolving at generation 5
    You should type:
  5. Typically in artificial evolution, you should see the average fitness of the population increase over time. You should also see the best fitness within the population increasing over time, possibly with some plateaus. Once the evolutionary process has ended, you can check these statistics by plotting a graph of the average and best fitness values by generation:
    xgraph test.avg
  6. Because we are using such a small population, these trends may not hold in our experiment.

  7. During the evolutionary process, the best weights from each generation were saved into files. We can now reload these weights into the robot and observe the resulting behaviors at more length. The following command will re-test the best weights from generation 1.
    python test-gen1.wts &
    Once you've seen enough, type endpyrobot to stop. To try a different set of weights just replace the "1" in the above command with the appropriate number. Sometimes it may take up to 30 seconds for the computer to release the ports you'll need to reconnect to the simulator. If you see a message Waiting on PyrobotSimulator, just type endpyrobot and wait a few more seconds before trying again.

What to turn in for this lab

Email me your summary of the experiment.