Lab 3: Exploratory Analysis and Data Cleaning
Due Wednesday, September 23rd by 11:59pm

Learning Goals

• Obtain, curate, and explore real-world, messy data
• Identify and handle missing values
• Create meaningful visualizations to understand data distributions
• Engineer new features from existing data
• Prepare data for use in developing models that aim to represent the data, including using machine learning approaches
• Learn about methods of specifying operations for large amounts of data at once

Lab partner survey

This lab will be done solo, but later labs will be done with partners. To help us arrange pairings, please fill out this form before you start work on this week's lab: https://forms.gle/iXeYf4bxQB63UcV98.

Starting point code

To access the starter code, log on to the Swarthmore Github Enterprise site: github.swarthmore.edu, and find your Git repository for this lab, which should be called something like "CS33-F26/lab03-USERNAME", only with your own actual username in place of 'USERNAME'. Open the repo by clicking on the name, then click the green "Code" button, make sure "SSH" is selected, and copy the URL which should be "git@github.swarthmore.edu:" followed by your repo name.

Next, open a terminal, navigate to your CS33 directory, type git clone, then paste the URL for your repo and run the command. This should create a new directory with a copy of the lab files. Run ls in this new directory to make sure the files are there; in this case, make sure you have a file called imdb_data_cleaning.ipynb.

Running Jupyter

$ source /usr/swat/bin/cs33env
(cs33-f26) $ jupyter-lab 

Completing the Lab

Once you have the lab repo and have opened jupyter-lab, navigate to the lab directory and open imdb_data_cleaning.ipynb

All the instructions for the lab are included in the Jupyter Notebook. The Notebook format allows for mixing instructions, example code, explanations, solution code, and output, all in the same document.

Submitting

When you are finished, use git to add, commit, and push your work. In this case, all your solutions should be contained within imdb_data_cleaning.ipynb. Remember to also add/commit/push on a regular basis; you should get into the habit of pushing changes to Git at the end of each session working on a lab.

Remote access for Jupyter

It's possible to run Jupyter remotely over SSH and connect to it from your local web browser, but you need to perform a few extra steps to make this work:

1. Open a command terminal on your local computer
2. Start a remote session using SSH, but specify a port to connect with the -L flag:
   
   ssh username@cslab.cs.swarthmore.edu -L 6633:localhost:6633
   
   Be sure to use your own username. Note that if two users try to use the same port number on the same host you'll have a conflict, so pick a random 4-digit number above 2000 to use for yourself. Be sure to use the same number on both sides of the word 'localhost'.
3. Start the cs33env as usual.
4. Tell jupyter lab to use the port you picked when you start it:
   
   jupyter-lab --port=6633 --no-browser
   
   Be sure you use the same port number as you did when you started SSH. The last bit tells it not to try to launch a web browser on the remote host.
5. Make sure that jupyter was actually able to open the port you asked for; if it fails, it will pick a different one...which won't help you at all. If this happens, log out and try to SSH back in using a different port number.
6. Copy one of the URLs and paste it into the location bar in a web browser on your local computer.

At this point, you should see jupyter lab and be able to interact with it as usual; all your code will be exectued on the remote CS machine, so you don't need to install or configure anything on your personal computer.