CS129/PSYCH129
Computational Models of Language

Spring 2001, Mondays 1:15-4:00
Trotter 305 (discussion) and Trotter 117 (lab)

Ed Kako, Papazian 204, extension 8454 (ekako1)
Lisa Meeden, Sproul 1, extension 8565, (meeden@cs)

Index

Course Description

Books & Readings

Evaluation

Schedule

Course description

Language is among the most complex and robust of all human cognitive capabilities. One way to understand this capacity better is by using computers to model it. In this course we'll focus primarily on various attempts to model language using connectionist (neural network) architectures. The course is made-up of two components: laboratory and discussion. Every week, students will have the opportunity to experiment with various models. Readings relevant to the models to be covered that week will also be assigned and be used as the basis for seminar-style discussions.

Books & Readings

• Pinker (1994). The Language Instinct, HarperCollins Publishers, New York, NY.
• McLeod, Plunkett, and Rolls (1998). Introduction to Connectionist Modeling of Cognitive Processes, Oxford University Press, New York, NY.
• A collection of articles provided in reading packets.

Evaluation

1. Class Participation & Reaction Notes: 20%
   Your reactions to each week's readings will be due by 8:00am Monday mornings via email. Send them to meeden@cs.swarthmore.edu. These notes will be posted to the course web page by 10am. You should read them before class time and be ready to respond to at least one question or comment raised by someone else. Put your reactions into html format so that they can be posted without any editing. Use this as a basic template. These notes should not be summaries of the papers, instead they should be the product the reading process (e.g., questions that were raised, points that were not clear, connections to previous material you've read, etc). You are expected to be an active participant in class discussions and should come prepared for class every week.
2. Home work: 20%
   One of the goals for this course is to provide students with a framework to get hands-on experience working with various connectionist models. Although we will do some of that work in-class, much that experience will have to be gained through homework.
3. Midterm Project: 20%
   There will be a mid-semester paper, a write-up of one of the computational simulations available in Introduction to Connectionist Modelling of Cognitive Processes. Assigned February 26. Due March 23.
4. Term Project: 40%
   Each student will be asked to design a project involving a computational approach to language. During the first half of the semester you should be thinking about possible project topics. These projects may be carried-out by pairs of students. Each student will be asked to present their project to the class at the end of the semester as well as turn in a written report.

Schedule

• Pinker (1994). The Language Instinct, HarperCollins Publishers, New York, NY. Chapters 1-4.
• McLeod, Plunkett, and Rolls (1998). Introduction to Connectionist Modeling of Cognitive Processes, Oxford University Press, New York, NY. Chapters 1-3.
• Reactions

• Pinker (1994). The Language Instinct, HarperCollins Publishers, New York, NY. Chapters 5-9.
• McLeod, Plunkett, and Rolls (1998). Introduction to Connectionist Modeling of Cognitive Processes, Oxford University Press, New York, NY. Chapter 4-5.
• Reactions

• Nakisa and Plunkett (1998). Evolution of a rapidly learned representation for speech. Language and Cognitive Processes, 13:2/3, 105-207.
• Protopapas (1999). Connectionist modeling of speech perception. Psychological Bulletin, 125:4, 410-436.
• Reactions

• Brent (1999). Speech segmentation and word discovery: A computational perspective. Trends in Cognitive Science, 3:8, 294-301.
• Christiansen, Allen, and Seidenberg (1998). Learning to segment speech using multiple cues: A connectionist model. Language and Cognitive Process, 13:2/3, 221-268.
• Norris (1994). Shortlist: A connectionist model of continuous speech recognition.
• Cognition, 52, 189-234.
• Perruchet and Vinter (1998). PARSER: A model for word segmentation. Journal of Memory and Language, 29, 346-363.
• Reactions

• Gasser and Smith (1998). Learning nouns and adjectives: A connectionist account. Language and Cognitive Processes, 13:2/3, 221-268.
• McLeod, Plunkett, and Rolls (1998). Introduction to Connectionist Modeling of Cognitive Processes, Oxford University Press, New York, NY. Chapter 9. Your midterm project will be based on one of the models in this chapter.
• Reactions

• Rumelhart and McClelland (1987). Learning the past tenses of English verbs: Implicit rules or parallel distributed processing? In MacWhinney editor, Mechanisms of Language Acquisition, Lawrence Erlbaum Associates, Hillsdale, NJ, 195-248.
• Pinker and Prince (1988). On language and connectionism: Analysis of a parallel distributed processing model of language acquisition. Cognition 28, 73-193.
• Reactions

• Marcus, Brinkman, Clahsen, Wiese, and Pinker (1995). German inflection: The exception proves the rule. Cognitive Psychology, 29, 189-256.
• Hahn and Nakisa (2000). German inflection: Single route or dual route? Cognitive Psychology, 41, 313-360.
• Reactions

• Elman (1990). Finding structure in time. Cognitive Science, 14, 179-211.
• McLeod, Plunkett, and Rolls (1998). Introduction to Connectionist Modeling of Cognitive Processes, Oxford University Press, New York, NY. Chapter 7.
• Reactions

• Allen and Seidenberg (1999). The emergence of grammaticality in connectionist networks. In MacWhinney editor, Mechanisms of Language Acquisition, Lawrence Erlbaum Associates, Hillsdale, NJ, 115-151.
• Christiansen & Chater (1999). Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, 23:2, 157-205.
• Reactions

• Dell, Cheng, and Griffin (1999). Connectionist models of language production: Lexical access and grammatical encoding. Cognitive Science, 23:4, 517-542.
• Plaut (1999). A connectionist approach to word reading and acquired dyslexia: Extension ot sequential processing. Cognitive Science, 23:4, 543-568.
• Reactions

  Be prepared to give a brief summary of your final project proposal for the class.

• Nowak, Plotkin, and Jansen (2000). The evolution of syntactic communication. Nature, 404.
• Pinker (2000). Survival of the clearest. Nature.
• Reactions