CS 45: Infrastructure Setup

In this course, lab assignments will frequently ask you to make modifications to the Linux kernel. Thus, you will need to setup:

An environment to build Linux kernels, and
A (virtual) system where you can test your kernels.

This guide will walk you though both steps. Since they each need lots of storage space, you'll need to work in /local, which is the local hard disk of the machine you're using. Storing things on /local has two major implications that you need to account for:

The data on /local is stored on a single disk (unlike your home directory, which is split across multiple disks for redundancy). When you're done working, you should save your important lab files elsewhere (e.g., push them to github) to avoid data loss in the event of a disk failure. When you're done working on your VM, you should shut it down nicely: "sudo shutdown -h now"

The /local partition on each machine is only available on that machine. This means that if you want to move and work a different machine, you'll need to set up a new VM at the new location. You could also copy your VM image from /local on one machine to /local on another with scp.

Kernel Compilation - From Scratch

At the start of each lab assignment, you'll need to build a kernel from scratch. It will take approximately 10 minutes to compile, depending on which machine you use. At the end, you'll get nice .deb packages that you can install.

Create a directory in /local to use for your kernel builds. I suggest /local/kernels-[username], so for me, that would be /local/kernels-kwebb:
```
mkdir /local/kernels-username  # Fill in your username
cd /local/kernels-username
```
Download the Linux kernel sources. The apt package manager will make this easy:
```
apt-get source linux-image-4.4.0-109-generic
```
Move into the newly created linux-4.4.0 directory and download a configuration that will reduce the build time:
```
cd linux-4.4.0
wget https://www.cs.swarthmore.edu/~kwebb/cs45/s18/labs/.config
```
By default, this will suffix your kernel's name with "-cs45-lab". You should change that to give your kernel a unique name so that you can identify it:
```
make menuconfig
```
Then navigate to the General Setup -> local version setting. I would suggest naming it based on the current lab number you're working on.

Prep the build by running some provided scripts:

chmod a+x debian/rules
chmod a+x debian/scripts/*
chmod a+x debian/scripts/misc/*
fakeroot debian/rules clean

Build it (this will take a while):

export CONCURRENCY_LEVEL=4
time fakeroot make-kpkg --revision=1.0 kernel_image kernel_headers

You can adjust the revision number however you'd like. When it's finished, you should have two new .deb files one directory up:

   $ ls -l ../*.deb
   -rw-r--r-- 1 kwebb users  7815612 Jan 15 12:12 ../linux-headers-4.4.98-cs45-lab_1.0_amd64.deb
   -rw-r--r-- 1 kwebb users 14928660 Jan 15 12:11 ../linux-image-4.4.98-cs45-lab_1.0_amd64.deb

We'll install those .debs once the VM is up and running.

Environment - VM

For security and reliability reasons, you'll be testing your kernels on a virtual machine. That way, if your kernel fails, you can simply reboot the VM and there's no risk of damaging your CS files.

To get started, you'll need to import a copy of the starter VM image:

Run virtualbox in a terminal or open it from within your favorite graphical menu.
Go to File->Preferences, set your "Default Machine Folder" to "/local", and then close the preferences window.
Go to File->Import Appliance. Choose: /local/CS45-VM.ova and push next once.
You should now be seeing "Appliance settings". Edit the name to include your username. For example, CS45-kwebb.
Click import and wait a minute for it to complete.

After you've completed these steps, you should see the your VM in the list of VMs available to start. Go ahead and turn it on. The username and password are both cs45.

Before going any further, you should use the passwd command to set a new password to protect your VM. Open a terminal and run passwd.

Kernel Compilation - Incremental Changes

Because it's so slow, building the entire kernel from scratch every time you make a change would be maddening. It's a nice fallback in case something goes wrong, but otherwise you probably just want to rebuild the files that you've changed (plus anything that depends on those files). NOTE: If you make changes to a very important header file that gets included all over the place (e.g., sched.h), you'll likely end up rebuilding most of the kernel.

From the linux-4.4.0 directory, you can rebuild the kernel image with:

make -j4 bzImage

Note that if you change .h files and your userspace programs need to see those changes (rare), you'll need to copy them over to your VM. While you can copy them by hand, it is often easier to just rebuild the kernel_headers package, remove the old one, and install the new version.

Kernel Installation

Installing a .deb packaged kernel:

First, you'll need to copy the .deb files to your VM. The easiest way to do this is to use scp, which works like cp for transferring files over a network. It uses encryption, so the 's' stands for 'secure'. From the VM:

# Replace username with your username and hostname with the name of the machine you used to build your kernel.
scp username@hostname.cs.swarthmore.edu:/local/kernels-username/*.deb .    #Don't miss the trailing dot.

You can now install the .deb packages using dpkg:

sudo dpkg -i package_name.deb

If you've already an earlier version of .deb packages with the same name, you'll need to uninstall the old ones first:

sudo dpkg -r package_name

Reboot the VM to use the new kernel.

Installing an incrementally built bzImage kernel:

You'll need to copy two files from the kernel build directory to your VM. From the VM:

# Replace username with your username and hostname with the name of the machine you used to build your kernel.
scp username@hostname.cs.swarthmore.edu:/local/kernels-username/linux-4.4.0/arch/x86_64/boot/bzImage .    #Don't miss the trailing dot.
scp username@hostname.cs.swarthmore.edu:/local/kernels-username/linux-4.4.0/System.map .                  #Don't miss the trailing dot.

Now that the files are on the VM, you need to put them in /boot. You should overwrite the existing kernel that was created when you did the .deb file installation:

# Replace whatever with the name you gave to your kernel.
sudo mv -i bzImage /boot/vmlinuz-4.4.98-whatever
sudo mv -i System.map /boot/System.map-4.4.98-whatever
sudo update-grub