INTRODUCTION

This assignment gives you the chance to formulate a machine learning problem and use a neural network to solve it. You may work with a data set containing images of people's faces, or you may work with natural language, using Elman's experiments as the basis for your own experiments.

I strongly encourage you to work with a partner on this project. You will have a week and a half to complete the project.

You will turn in a 4-6 page paper describing in detail your learning tasks, your learning methods (including parameter settings), how you represented the data for input, your experiments, your results, and any conclusions you can draw from this. You must include a visualization of the network's weights or its internal representations to help you illustrate what the network has learned.

If you'd like to learn how to use latex, check out the sample paper in:

/home/meeden/public/latex-example/

GENERAL GUIDELINES

• Before you begin, review sections 4.6.5 - 4.7.3 of Mitchell's textbook.

• When designing your experiment be sure to use a training set and a validation set so as to avoid overfitting.

• Define your learning task and then through trial and error determine the best network architecture for accomplishing it. We'd like the smallest hidden layer size for which the task can be consistently learned.

• If you'd like to use cluster analysis or principal components analysis in your project, there are tools available in:

  /home/meeden/public/network-analysis
  

  Once you have trained a neural network in Conx, you can save the hidden layer activations as follows:

  n.setLearning(0)            #turn learning off
  n.setOrderedInputs(1)       #turn off the default random ordering for inputs
  hid = n.getLayer('hidden')  #get a pointer to the hidden layer
  hid.setLog("hiddens")       #set a log file for the hidden layer
  n.sweep()                   #sweep through the inputs in order
  hid.closeLog()              #close the log file for the hidden layer
  

  You'll also want to create a parallel file of labels that describe what the hidden layer activations represent.

OPTION 1: IMAGES

OPTION 2: NATURAL LANGUAGE

In Elman's 1990 paper Finding structure in time he describes a number of experiments focusing on natural language tasks. For this project, you may try to reproduce either the experiment on discovering word boundaries or the experiment on discovering the lexical classes of words. However, your experiment should add a new variation to what Elman originally attempted.