Introduction

This project requires a fair amount of time in designing, writing, and running user-level experiments, and very little time writing and testing your system call. I'd recommend leaving yourself at lease 1.5 weeks for designing, writing, and running experiments and for writing your final report.

Project Details

Part 1: Adding a new system call

int sys_pgfltstats(pid_t pid, int flag, pf_info_struct *info)

The pf_info_struct should contain fields for:

1. total number of minor page faults
2. total number of major page faults
3. number of processes included in these counts

If pid is:

• -1: then return total count statistics for all processes in the system.
• positive then look at the flag field and do the following:
  1. flag is PGFLTSTAT_PROC: then return count statistics for the single process with the matching pid
  2. flag is PGFLTSTAT_OWNER: then return the total count statistics for all process in the system with the same owner as the specified pid.

You can use fields in task_struct for obtaining these, or you could use (or add) values to the fault handling part of the kernel for obtaining them. Make sure to check for error conditions and return appropriate error values as in the previous assignments. Remember that header file contents (prototypes and definitions) that should only be visible inside the kernel (not at user level) should be inside #ifdef __KERNEL, and #endif preprocessor directives.

You can grab my starting point code for lab2 to use as a starting point for your system call implementation (~newhall/public/cs45/lab2/).

You can use vmstat, and /proc/pid/ files to help you determine if your system call is implemented correctly (look at vmstat and proc (man 5 proc) man pages for more info).

Kernel code you may want to read:

• page fault handling routines in do_page_fault() in arch/i386/mm/fault.c and mm/memory.c
• page replacement code in vmscan.c, which begins with try_to_free_pages.

Part 2: Experiments Testing the VM system and a Written Report

For this part, you will use your system call (and other utilities) to collect information about page faults and other performance data for different programming loads. I want you to (through experiments) determine what page replacement algorithm Linux likely implements. I'm sure you can figure out what Linux's page replacement policy is just by reading Linux documentation and/or reading Linux source code. However, the point of this assignment is to see if you can verify experimentally what you know (or suspect) about Linux's page replacement policy.

Think about running experiments to answer the questions about how the policy works for different types of workloads, and what that tells you about the policy. In designing your experiments, you should write user-level test programs that access memory in different patterns, and run different experiments using them to collect performance data to help you answer questions like:

1. When does Linux's page replacement policy work well (what types of workloads)?
2. When does it not work well?
3. How well does it work for a mixed workload?
4. etc.

To get started, you will likely want to read kernel code and other Linux documentation to help you develop some initial hypotheses. You can test negative hypotheses as well as positive ones (i.e. "I don't think Linix implements the X policy, and I'm going to test this by ..."), but you should have some positive hypotheses as well; there is a difference between negative hypotheses that can help support your positive ones, and bad hypotheses (and bad experiments) that don't really show anything.

Next,think about how to design experiments to test your hypotheses:

• You will want to write some user-level test programs that allocate memory and access it repeatedly in some pattern (you need to allocate enough memory to cause page faults).
• Use these test programs to design and run experiments to help support (or refute) your hypotheses.

Look at the details in the Report section below to help guide your hypothesis and experiments design and testing.

To write test programs that use enough memory to trigger paging, look at the /proc/meminfo file. This give the total amount of free memory (MemFree, in Kbytes). If you are running an experiments, make sure the the process(es) allocate enough memory to trigger page replacement (they need to allocate enough virtual memory space so that it won't all fit into free physical memory). However, be careful that you do not allocate too much memory, otherwise, you can run out of swap space and the kernel will start killing process. The amount of free swap space can be found in /proc/meminfo (SwapFree, in Kbytes).

Report

In particular, your report should have the following:

1. Introduction. Describe at a high-level what you are doing and why, and describe what your high-level hypothesis is (i.e. "We started out guessing that Linux's page replacement policy is X, but our experiments will show that it is in fact, more like Y" or "we start out making no assumptions about Linux's page replacement policy, and our experiments will answer the question of is it more like X, Y or Z.").

2. Short paragraph describing how you implemented your system call to get get per-process, per-owner, and system wide page fault counts? Also, describe other Unix utilities you used to collect performance data for your experiments.

3. Main part: describe your experiments and results: For each experiment, you should:
   1. Clearly state what your hypothesis is
      (i.e. "Application that does X should be a bad/good case for Linux's page replacement policy because ...)
   2. Explain how your experiment is testing that hypothesis
      (We are testing this hypothesis by running the application N times on and collecting X,Y, and Z metrics using our system call, vmstat, time,... With these measures, we can see whether or not ...).
   3. Present your Results.
   4. Explain your results!
      (Our results show that ... This does/does not match our expected result, because ... (if it doesn't, think about some other tests you could run to explain why it doesn't, or look for data you do have to explain it)
   Quality of Experiments is much, much better than quantity; a few well thought out and well done experiments is sufficient.

4. Conclusion. It should include a statement of the major result(s) of your experiments (which policy(ies) is Linux's most like? and in what way(s)?). Also, tell me what you found to be the most difficult part of the hypothesis testing phase of this assignment.

Some useful system calls and utilities for testing purposes

• sleep(val) make program block for val seconds
• usleep(val) make program block for val micro seconds
• clone create a kernel level thread
• fork you know what this does

• your pgfltstats system call
• your proc_info system call from lab2
• time a.out times the execution of the a.out file
• vmstat: prints out statistics about the VM system. This may help you in verifying that your system call is implemented correctly (vmstat alone is not enough to do your testing...you need your system call). 'vmstat -n 1' prints out vmstat statistics every 1 second.
• watch -n X [command]: every X seconds execute command. For example, to print out the contents of the stat file for process 22832 every 1 second I'd do the following:

  	watch -n 1 cat /proc/22832/stat
  

• /proc: file system interface to processes. You can read (and write) variables in a process' address space (I'd recommend never writing).
  • in /proc/sys is the /proc interface to the kernel. By cat'ing files in these subdirectories, you can read kernel statistics.
  • in /proc/[pid] directories are the /proc interface to individual processes in the system. Again you can cat the contents of files in these directories to get per-process statistics.
  • /proc/meminfo

Latex

/home/newhall/public/latex_examples

For this assignment, the example in the paper subdirectory is likely a good starting point. In the paper subdirectory is an README file with information on how to create a .dvi .ps and .pdf file from the latex source file (example.tex). In addition, there is a makefile that contains rules for building all of these. The example.tex file, contains examples of importing postscript figure files, creating sections and subsections, using different fonts, creating lists, and referencing figures within a document. Probably the easiest thing to do is to grab the contents of this directory as a starting point for your report, then edit example.tex with your report.

What to Hand in

1. Code: using cs45handin, submit a tar ball with the following:
   1. README file with (1) your names, (2) how many late days you have used on this assignment, and how many you have used total, (3) a description of how to run your test programs (show example command lines).
   2. copies of all .c and .h files you added or modified in the kernel to implement your system call
   3. copies of all your test programs used in experiments, and the test program you used to test your system call for correctness.
2. Report: turn in a hardcopy of your report in class on the same day that you submit your tar ball. Do not send me an electronic version of your report; I want a print out. If you turn in your report later than the beginning of class on the due date, then your assignment is late (even if your tar ball was submitted before the due date).

Demo