Cogs1 Spring 2007
Lab 6: Unsupervised Categorization of Robot Sensors
Due by 11:30pm on Wednesday, March 28
Introduction
The Resource Allocating Vector Quantizer (RAVQ) was developed by
Fredrik Linaker and Lars Niklasson. Given a data set of vectors, the
RAVQ generates a set of model vectors that are meant to represent
typical categories of vectors within the data set. Resource
allocating in this case means that the number of categories is not
fixed, but is dynamically determined as the unsupervised learning
proceeds.
The RAVQ consists of three main parts:
- Input buffer
- Moving average of vectors in the input buffer
- Set of model vectors
When the RAVQ begins, the buffer must be initialized by filling it
with the first n inputs, where n is the buffer
size. Then a moving average can be calculated. After the buffer is
full, at each step the current input is added to the buffer and the
oldest input in the buffer is deleted, maintaining the size of the
buffer at n. A moving average vector is calculated by
averaging all the inputs currently in the buffer. Then the RAVQ
determines whether the current moving average is a member of an
existing category or if it qualifies as a new model vector. To do so
the moving average must meet two criteria. It must be a good
representation of the inputs and it must be unique enough when
compared to the existing set of model vectors.
There are three key RAVQ parameters that must be set before learning
begins. Each of these parameters will affect the number of categories
that will be created.
- Buffer size
This value determines the number of input
vectors that are stored in the buffer. The size should reflect the
probable rate of change within the environment. A small buffer size
may lead to spurious categories that are based on noise. A large
buffer size will cause the moving average to be quite stable and may
lead to very few categories.
- Epsilon
This value determines how close the moving
average must be to the input buffer vectors to qualify as a possible
new model vector. A small epsilon means that the vectors that make up
the moving average must be nearly identical in order to be considered
as a potential category. A large epsilon means that the vectors that
make up the moving average could be quite dissimilar and still be
considered as a potential category.
- Delta This value determines how different a moving average
vector must be from all existing model vectors to justify creating a
new model vector. A small delta means that the moving average need
not be very different from existing model vectors in order to create a
new category. A large delta means that the moving average must be
significantly different from the existing model vectors to create a
new category.
In this lab we will observe a robot running a basic wander and avoid
brain. On each time step it will check its sensors and create a vector
representing the minimum sonar values on its left, front-left, front,
front-right, and right. It will pass this vector of sensor values to
a RAVQ. The RAVQ will be trying to learn appropriate categories for
this environment and will report any time the current category
changes. After a period of learning, the robot will stop and print
out the current set of RAVQ categories. Then we will analyze the
categories it found, trying to describe them verbally. Next we will
reposition the robot back to the starting point and observe its
behavior again. We will also try modifying the RAVQ parameters to
gain a better understanding of how these settings affect the types of
categories that are formed.
Getting Started
-
To get copies of the files you'll need for this lab, open a terminal
window and type:
update-cogs1
-
To move down to the appropriate directory, type:
cd cogs1/labs/6
- To start the categorization process, type:
python basicRAVQ.py &
This will open up a pyrobot window. Before you begin, use the mouse
to grab the lower right corner of the pyrobot window and drag it down
to make it bigger. Then press the Run button. This will run
the robot for 200 steps. It will print each step number and in
addition show you when the current category found by the RAVQ has
changed. At the end of 200 steps it will print out all of the RAVQ's
current categories.
-
Press the Stop button. For each category generated by the
RAVQ, write down a verbal description of what it represents. Remember
that the RAVQ is using 5 sonar sensor values: left, front-left, front,
front-right, and right. Also recall that sonar sensors reflect
distances to an obstacle. So small values indicate that an obstacle
is quite close and large values indicate that there is open space in
front of that particular sensor.
-
Move the robot back to its approximate starting position. Be sure to
set its heading to be towards the top of the screen. At the command
line of the pyrobot window type:
self.counter = 0
This will reset the counter. Now press the Run button again.
Whenever a new category is reported, stop the robot and compare it's
current location to your verbal description of that category. Do they
seem to coincide?
-
Now let's try to modify the parameter settings and see how the number
of categories changes. Click on the brain's filename in the pyrobot
window basicRAVQ.py. This will bring up an edit window
containing the program. At the top of the file you'll see several
lines like the following:
self.bufferSize = 5
self.epsilon = 1.0
self.delta = 0.3
Change the parameter settings here. Then save the file in the edit
window. Go back to the pyrobot window and press the Reload
Brain button. Then press the Run button to see the
results. Try buffer sizes of 1 and 20. Try epsilons of 0.1 and 4.0.
Try deltas of 0.1 and 0.9. Only change one parameter at a time,
keeping the others at their initial values. Record the number of
model vectors created in each case.
-
Based on your findings from experimenting with the parameters, change
all three parameter settings to try to get the most possible model
vectors.
What to turn in for this lab
Email me your answers to the following questions.
- Using the original parameter settings, train the RAVQ for 200
time steps. Write down a summary of each model vector. Then reset
the robot back to its approximate starting position and reset the
counter back to 0. Re-run the robot and report on whether the
categories it encounters on this run match your descriptions. Does it
find additional categories on the second run? If so, in what ways are
the different from the existing categories?
- Summarize the results of your experiments with the parameter
settings. Did the description of the parameters at the top of this
page match your findings? What setting of the parameters led to the
most model vectors being created?
- In this lab we've seen that we can use a RAVQ to categorize the
various states that a non-adaptive robot encounters as it moves around
its world. However, we'd ultimately like to use these categories in
an adaptive robot. Discuss how a categorization method, like the
RAVQ, might be useful to a learning robot. Don't worry about the
details of any specific learning algorithm, just think about the big
picture.