Please be aware that elements on this page will change throughout the semester, particularly the course schedule. It is your responsibility to review this page periodically for updates.

Course Basics

Instructor: Ben Mitchell, Sci 249

Lecture: TTh 2:40pm-3:55pm, Science Center 181

Lab:

Lab Section A

Wednesday 1:15pm-2:45pm

Science Center 240

Lab Section B

Wednesday 3:00pm-4:30pm

Science Center 240

Office hours:

Please note that the times listed here are ones I’ll definitely be available, but you are welcome to stop by my office any time the door is open. Additionally, you can always contact me to schedule a meeting at a different time.

Tuesday

1:30pm-2:30pm

Friday

2:00pm-3:30pm

Waitlist Procedure

For Spring 2023, please use the following waitlist procedure to stay active in the course and to be considered for open positions:

  1. See the general department lottery & waitlist information page

  2. If you haven’t already, be sure to fill out the Department waitlist request form.

  3. Stay current with the course lectures and labs by attending lecture and a lab for week 1. The Instructor(s) will stay in contact to provide updates. There are no guarantees that following the waitlist procedure will result in obtaining a seat; we will replace students that drop based on the department lottery priorities and with students who follow the above steps and who can fit the opened lab spot into their schedule.

Required Course Textbook

We will utilize A Course in Machine Learning by Hal Daume III as our primary text. It is an online, free ebook. We will also include additional reading material on the course schedule, most of which will also be available online for free.

Note that this book is only available in PDF form; if you want a hard copy, you should be able to get printouts made using TAP funds. Please let me know if you have any difficulties with this.

Additional References

These are all excellent books that I have read. However, some are geared more towards graduate students and researchers, so I did not choose them for our primary course textbook; we will be doing supplementary readings from some of them, but those will be freely available online. If you are looking to get deeper into the material, here are some suggestions.

Note that all of these books should be available on reserve in the library as well.

  • Elements of Statistical Learning by Trevor Hastie, Robert Tibshirani, and Jerome Friedman. We will be using this book (available for free online) for several supplementary readings on topics that CIML doesn’t cover in as much detail as I would like.

  • Machine Learning by Tom Mitchell ( Amazon link). Historically this is the gold standard; however, it is too expensive to be the required textbook, particularly given that its age means it does not cover several important topics. You may be able to find used versions for a reasonable price; there is also a reserved copy in the Cornell Library.

  • Introduction to Machine Learning by Ethem Alpaydin (Amazon link). If you are looking to purchase a hard copy, this is the one I recommend as it is reasonably priced and a good textbook. It is also available as an ebook available for free through the library’s ProQuest account.

  • Pattern Recognition and Machine Learning by Charles Bishop

  • Machine Learning: A Probabilistic Perspective by Kevin Murphy

  • Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville

Course Description

Welcome to CPSC 66. Machine learning is the study of algorithms that "learn" by finding patterns in examples provided to them. This course will introduce you to various frameworks (e.g., supervised learning) and associated algorithms for these frameworks (e.g., decision trees). The major aim of this course, however, is to develop an understanding of the entire machine learning pipeline rather than focus on the algorithm du jour. We will also spend a significant amount of time inspecting core concepts (e.g., generalization) from statistical and theoretical perspectives. With each topic, we will consider both the practical and open research questions at the heart of the field. You will be expected to implement solutions through lab assignments, but also digest and discuss readings that build off of lecture topics.

To enroll in this course you must have completed CPSC 35. There are no other requirements, though familiarity with the basics of linear algebra, probability, discrete math, and calculus will be useful. The course will also cover a good deal of probability theory, but much of this can be picked up from provided reading. This course is designated as a natural sciences and engineering practicum (NSEP) and qualifies as a Group 3: Applications course for the CS major/minor requirements.

Course Learning Goals

By the end of the course, you will understand:

  • several machine learning frameworks, including supervised learning, unsupervised learning, and hybrid approaches

  • various algorithms for the frameworks we explore, including the variation in data representation

  • how to choose and apply an appropriate framework and algorithm for a new problem

  • practical considerations for data, including data preprocessing, feature engineering, and resource constraints

  • the core concept of generalization, and the associated theoretical tools for inspecting both our data and models

  • theoretical and empirical evaluation of performance

Student Responsibilities

I have outlined the skills and objectives this course promises to provide you. For these promises to be upheld, you will need to commit to the policies outlined below. To succeed you should:

  • Attend class and lab The primary introduction to course material is through class lecture. Additionally, we often do learning exercises during class, which give you immediate experience with the material we are covering. Class and lab attendance is mandatory. While I am more than happy to help with any material in office hours, priority will be given to students who attend and participate in lecture. Office hours are not to make up for missed lecture.

  • Participate actively in learning process. Showing up is necessary, but not sufficient, for success in the course. To fully develop your analytical skills, you are expected to participate in class discussion. This includes asking questions during lecture portions and engaging your peers during short class exercises. Studies show active involvement is the number one determinant of student success.

  • Prepare for lecture You are expected to have done pre-reading before each lecture as well as reviewing notes from the prior meeting. If you have not done so, you will be unprepared for the daily concept quizzes that begin lecture as well as subsequent group discussions.

  • Start the lab assignments early If you get in the habit of doing this, you will be much better off. As the labs get longer and more difficult, starting early will give you plenty of time to mull over the lab problems even when you aren’t actively writing your solution.

  • Practice, practice, practice The only effective way to learn the material and pass the exams is to consistently do the labs, and to practice example problems presented in class and in the book. Forming study groups to go over practice problems and to review lecture and reading notes is a great way to prepare for exams.

  • Seek help early and often Because course material builds on previous material, it is essential to your success in this class that you keep up with the course material. There are a lot of sources of help: ask questions during lecture; ask your classmates (make sure you have read the Academic Integrity section for restrictions); get help during lab sessions; and come to office hours.

Resources

Class Resources

Course Work

Assessment (Grading)

This is subject to change but is a rough estimate

  • 30% Labs (3 to 4 total)

  • 5% Homework assignments (3 to 4, lightly graded)

  • 25% Final Project (with writeup and presentation)

  • 10% Participation

  • 30% Exams (2 Midterms, no final)

Class Participation

As discussed in responsibilities, your participation involves:

  • Required attendance to lecture and lab

  • Active participation in lecture

  • Active engagement in the class discussion groups

  • Concept quizzes and/or class prompts if and when they are used

Working With Partners

For partnered lab assignments, you should follow these guidelines:

  • The expectation is that you and your partner are working together side by side in the lab for most, if not all, of the time you work on partnered lab assignments.

  • You and your partner should work on all aspects of the project together: initial top-down design, incremental testing and debugging, and final testing and code review.

  • When working together, follow the practices of pair programming (where one of you types and one of you watches and assists, ensuring you swap roles periodically, taking turns doing each part).

  • If meeting in person is not possible, pair programming can be done via screen-sharing and voice chat (e.g. Zoom); working together remotely isn’t as good as working together in person, but it’s better than working alone.

  • There may be short periods of time where you each go off and implement some small part independently. However, you should frequently come back together, talk through your changes, push and pull each other’s code from the git repository, and test your merged code together.

  • You should not delete or significantly alter code written by your partner when he or she is not present. If there is a problem in the code, then meet together to resolve it.

  • If there is any issue with the partnership, contact the professor.

Taking time to design a plan for your solution together and to doing incremental implementation and testing together may seem like it is a waste of time, but in the long run it will save you a lot of time by making it less likely that you have design or logic errors in your solution, and by having a partner to help track down bugs and to help come up with solutions to problems.

Partnerships where partners work mostly independently rarely work out well and frequently fail to produce complete, correct and robust solutions. More importantly, they don’t result in both partners learning well (which is ultimately the point of doing the work in the first place). Partnerships where partners work side-by-side for all or most of the time tend to work out very well, with both partners learning (and accomplishing) more than they would alone.

You and your partner are both equally responsible for initiating scheduling times when you can meet to work together, and for making time available in your schedule for working together.

Policies

Academic Integrity

The general ethos of this policy is that actions which shortcut or avoid the learning process are forbidden, while actions which promote learning are encouraged. Studying lecture materials together, for example, provides an additional avenue for learning and is encouraged. Using a classmate’s solution, however, is prohibited because it avoids the learning process entirely. If you have any questions about what is or is not permissible, please contact your instructor.

Academic honesty is required in all your work. Under no circumstances may you hand in work done with or by someone else, or produced by an algorithm, without making the source of the work explicitly clear. Discussing ideas and approaches to problems with others on a general level is encouraged, but you should never share your solutions with anyone else nor allow others to share solutions with you. You may not examine solutions belonging to someone else, nor may you let anyone else look at or make a copy of your solutions. This includes, but is not limited to, obtaining solutions from students who previously took the course or solutions that can be found online. You may not share information about your solution in such a manner that a student could reconstruct your solution in a meaningful way (such as by dictation, providing a detailed outline, or discussing specific aspects of the solution). You may not share your solutions even after the due date of the assignment.

In your solutions, you are permitted to include material which was distributed in class, material which is found in the course textbook, and material developed by or with an assigned partner. In these cases, you should always include detailed comments indicating on which parts of the assignment you received help and what your sources were.

When working on tests, exams, or similar assessments, you are not permitted to communicate with anyone about the exam during the entire examination period (even if you have already submitted your work). You are not permitted to use any resources to complete the exam other than those explicitly permitted by course policy. (For instance, you may not look at the course website during the exam unless explicitly permitted by the instructor when the exam is distributed.)

Failure to abide by these rules constitutes academic dishonesty and will lead to a hearing of the College Judiciary Committee. According to the Faculty Handbook:

Because plagiarism is considered to be so serious a transgression, it is the opinion of the faculty that for the first offense, failure in the course and, as appropriate, suspension for a semester or deprivation of the degree in that year is suitable; for a second offense, the penalty should normally be expulsion.

This policy applies to all course work, including but not limited to code, written solutions (e.g. proofs, analyses, reports, etc.), exams, and so on. This is not meant to be an enumeration of all possible violations; students are responsible for seeking clarification if there is any doubt about the level of permissible communication.

Absence / Late Work

For lab assignments, each student may use up to a total 5 late days across the entire semester, no questions asked. Late days do not apply to homework, quizzes, or other forms of assessment. A late day is a considered a full 24-hours (i.e., 15 minutes late is the same as 23 hours late) and late days are counted against all lab partners.

To use your late days, you must make a private Ed post to Instructors after you have completed the lab and pushed to your repository. You do not need to inform anyone ahead of time, and you do not need to provide a reason. When you use late days, you should still expect to work on the newly-released lab during the following lab section meeting. The professor will always prioritize answering questions related to the current lab assignment during lab meetings and office hours. In the rare case in which only one partner has unused late days, the partnership can use the late days, barring a consistent pattern of abuse.

If you feel that you need an extension on an assignment or that you are unable to attend class for two or more meetings due to a medical condition (e.g., extended illness, concussion, hospitalization) or other emergency, you must contact the dean’s office and your instructors. Faculty will coordinate with the deans to determine and provide the appropriate accommodations. Note that for illnesses, the College’s medical excuse policy, states that you must be seen and diagnosed by the Worth Health Center if you would like them to contact your class dean with corroborating medical information.

Academic Accommodations

If you believe you need accommodations for a disability or a chronic medical condition, please contact Student Disability Services via email at studentdisabilityservices@swarthmore.edu to arrange an appointment to discuss your needs. As appropriate, the office will issue students with documented disabilities or medical conditions a formal Accommodations Letter. Since accommodations require early planning and are not retroactive, please contact Student Disability Services as soon as possible. For details about the accommodations process, visit the Student Disability Service Website

You are also welcome to contact me privately to discuss your academic needs. However, all disability-related accommodations must be arranged, in advance, through Student Disability Services.

To receive an accommodation for a course activity you must have an official Accommodations Letter and you need to meet with me to work out the details of your accommodation at least two weeks prior to any activity requiring accommodations.

Final Project

Detailed information about the final project can be found here:

Class Schedule

This is a tentative schedule. It will be updated as we go. We recommend that you review Tips for reading CS texbooks to help you determine what to focus on and how to get the most out of required readings.

WEEK DAY ANNOUNCEMENTS TOPIC ASSIGNMENT
1

Jan 17

 

Introduction to Machine Learning

Lab 0 - Python and Jupyter Warmup

Jan 19

 

Nearest-Neighbor Classifiers

2

Jan 24

 

Decision Trees

Lab 1 - KNN imprlementation

Jan 26

Drop/add ends (Jan 27)

3

Jan 31

 

Ensemble Learning Methods

Lab 2 - Decision Trees/Ensembles implementation

Feb 02

 
4

Feb 07

 

Linear and Logistic Regression

Linaer Algebra Primer

Linear Regression:

  • Ch 3 - 3.1.1 and 3.2 - 3.2.1 (inclusive), ISL

Logistic Regression:

Feb 09

 
5

Feb 14

 

Regularization; Bias-Variance Tradeoff

Lab 3 - Linear & Logistic Regression implementation

Feb 16

 
6

Feb 21

 

Unsupervised Learning; Dimensionality Reduction; Semi-Supervised Learning

Feb 23

 

Support Vector Machines and Kernels; Probabilistic Models

7

Feb 28

 

Anomaly Detection; Recommender Systems

midterm

Mar 02

 

Midterm 1

 

Mar 07

Spring Break

Mar 09

8

Mar 14

 

Evaluation Methodology & Practical Considerations

Lab 4 - Algorithm Evaluation & Comparison

Mar 16

 
9

Mar 21

 

Real-world Data and Applications

Lab 5 - Deep Learning?

Mar 23

CR/NC and Withdraw deadline (Mar 24)

Deep Learning

Readings:

10

Mar 28

 

Final Project

Mar 30

 
11

Apr 04

 

Apr 06

 
12

Apr 11

 

Apr 13

 
13

Apr 18

 

Apr 20

 
14

Apr 25

 

Midterm 2

Apr 27

 

Course Wrapup