In this lab you will observe a robot as it moves through a two-room environment. On each time step it will check its sonar sensors and create a vector representing the minimum sonar values on its left, front, right, and back. It will pass this vector of sensor values to an unsupervised categorization method. You will record the kinds of categories that are formed. You will experiment both with the categorization method's parameters and with the robot's mode of exploration to see how they affect the number and type of categories that are discovered.

You will use a RAVQ, which was developed by Fredrik Linaker and Lars Niklasson, as the unsupervised categorization method. Given a time series of vectors, the RAVQ generates a set of model vectors that represent consistent categories of vectors within that time series. Resource allocating means that the number of categories is not fixed, but is dynamically determined as the unsupervised learning proceeds.

The RAVQ consists of two main components:

1. Input buffer
2. Set of model vectors

When the RAVQ begins, the buffer must be initialized by filling it with the first n inputs, where n is the buffer size. Then a moving average can be calculated. After the buffer is full, at each step the current input is added to the buffer and the oldest input in the buffer is deleted, maintaining the size of the buffer at n. A moving average vector is calculated by averaging all the inputs currently in the buffer. Then the RAVQ determines whether the current moving average is a member of an existing category or if it qualifies as a new model vector. To do so the moving average must meet two criteria. It must be a good representation of the inputs and it must be unique enough when compared to the existing set of model vectors.

There are three key RAVQ parameters that must be set before learning begins. Each of these parameters will affect the number of categories that will be created.

1. Buffer size
   This value determines the number of input vectors that are stored in the buffer. The size should reflect the probable rate of change within the environment. A small buffer size may lead to spurious categories that are based on noise. A large buffer size will cause the moving average to be quite stable and may lead to very few categories.
2. Epsilon: Stability criterion
   This value determines how close the moving average must be to the input buffer vectors to qualify as a possible new model vector. A small epsilon means that the vectors that make up the moving average must be nearly identical in order to be considered as a potential category. A large epsilon means that the vectors that make up the moving average could be quite dissimilar and still be considered as a potential category.
3. Delta: Mismatch reduction requirement
   This value determines how different a moving average vector must be from all existing model vectors to justify creating a new model vector. A small delta means that the moving average need not be very different from existing model vectors in order to create a new category. A large delta means that the moving average must be significantly different from the existing model vectors to create a new category.

If you'd like to see the implementation of the RAVQ go to:

/usr/local/pyrobot/brain/ravq.py

• Do an update81 to get the starting point files for this lab.

• Move to the current lab directory and open the file categorizeFollow.py. Read through the code and be sure that you understand how it works.

• To start the categorization process while wall following, type:

  python categorizeFollow.py &
  

  This will open up a pyrobot window. Before you begin, use the mouse to grab the lower right corner of the pyrobot window and drag it down to make it bigger. Then press the Run button. This will run the robot for 600 steps. It will print a message whenever the current category found by the RAVQ has changed.

  At the end of 600 steps it will print out all of the RAVQ's current categories. For each model vector, it reports how many input vectors were mapped to this category and how much variability there is within the current history of examples from this category (this is the incompatibility).

• For each category generated by the RAVQ, in the file ravqLog write down the model vector and a description of what it represents. Remember that the RAVQ's input is based on 4 sonar sensor values: left, front, right, and back. Also recall that sonar sensors reflect distances to an obstacle. So small values indicate that an obstacle is quite close and large values indicate that there is open space in front of that particular sensor.
• Move the robot back to its approximate starting position. Be sure to set its heading to be towards the bottom of the screen. At the command line of the pyrobot window type:

  self.counter = 0
  

  Now press the Run button again. Whenever a new category is reported, stop the robot and compare it's current location to your description of that category. Do they seem to coincide?
• The RAVQ paper we read this week reported that for a similar world three categories were formed for wall, corner, and corridor. If this does not match the results of your experiment, try varying the parameter settings of the RAVQ to reproduce their results. Keep in mind that sonar sensors (which we are using) and IR sensors (which they used) are not equivalent. IR sensors have a much more limited range. In the file ravqLog, report on the parameters settings you found that produce the most similar results to the paper.

• Copy the file categorizeFollow.py to a file called categorizeWander.py. Write a controller for the robot that will wander around and explore the environment in a more open-ended way. For instance you could have the robot turn and move towards the direction with the most open space. Based on how we know a RAVQ works, using consistency over time, you should design a behavior that is smooth rather than jittery.
• Once you are satisfied with your wandering behavior, allow the robot to categorize the world as it wanders. Report the results in the ravqLog file. Be sure to include the RAVQ parameters settings, the number of categories that were formed, and some examples of categories that were different than the categories formed while wall following.
• At the end of the ravqLog discuss your results. Do the kinds of categories formed by the RAVQ provide a good foundation for additional learning? Are there important distinctions in the environment that the RAVQ cannot adequately recognize?

When you are done run handin81 to turn in your completed lab work.