Interpreter Project

Part 6: Meta-Circularity

All parts due: NOON on Sunday, May 12th

Quick link: Summary page (from Part 5; no summary for Part 6)

Wrapping up Part 5: Test and handin37 and update37

Be sure that the code you have written so far can pass the tests provided for you in the i-tests file from Part 5. You should add your own tests cases to make sure that your interpreter is working properly.

If you are still working on Part 5, please let me know if you need help.

Also, be sure that you have turned in your completed version of Part 5 using handin37.

As in previous steps, use update37 to download Part 6, and use the accompanying merge program to merge to your code from Part 5 with new code provided in Part 6. Note that there aren't any tests provided in Part 6, but the code provided is quite a bit more than you've had in previous parts.

Introduction

This is the final section of the interpreter. You will begin (and possibly complete) the process of making your interpreter meta-circular.

There will also be Extension Three which you can implement for extra credit; however, you will need to have completed Extension One in order to complete Extension Three.

Be sure to read the final section which describes the files you should turn in when you have completed all sections of the project.

Meta-Circularity

Your interpreter is sophisticated enough to be able to execute Racket programs of substantial complexity. Aside from a few examples (most notably, the stream examples), your interpreter can evaluate all the code you have written in this course so far. In fact, with just a few more modifications, it's possible to run the interpreter itself in the interpreter. Interpreters that can do this are called "meta-circular" interpreters. Of course, the resulting "meta-interpreter" can do the same thing, yielding a "meta-meta interpreter", which of course can do the same thing, yielding...

Adding all the necessary primitives

The first step to take in order to make your interpreter become meta-circular is to ensure that every primitive and special form that you have used in writing this project has been implemented in your interpreter. Below is a list of the most common primitives and special forms you may have used (excluding map and apply). However, depending on your particular implementation, you may have used others which you must also implement.

not, void, void?, string?, char?, pretty-print
add1, sub1
cadr, cdar, caddr, cadar, et cetera
list-ref, pair?, read

You do not need to implement all of the functions listed above: only the ones that you use in implementing your interpreter; however, given how easy it is to add them, there isn't much harm -- with the exception of map and apply which will not work properly if you use Racket's versions. See the next sections for details.

You will need to add char? and you will also need to make sure that your interpreter knows that chars, like numbers and strings, are self-evaluating. Add a test in i-eval that checks if the expression is a character (using char?) and if it is, return the character. You can test this by entering characters in the intepreter. The character A is entered by typing #\A.

Note that pretty-print is not a Racket primitive, but we will pretend that it is for our interpreter without difficulties. Importantly, and and or, which you almost certainly used in this project, are not primitives. You will need to add the and and or special forms as described in an upcoming section.

Finally, there are a number of primitives you have likely never even heard of before, but which are now part of your program because I have included them in the code I gave you in Part 6 as part of implementing include (which is described in another section). The primitives you will need to add are:

string-ref
string-length
string-append
read
read-line
eof-object?
open-input-file
open-input-string

Implementing map

Racket's primitive map requires that the first argument be a Racket procedure (which Racket displays as something like #<procedure>). However, when our interpreter sees a procedure, it turns it into a closure, not a Racket procedure. For this reason, unlike with other primitives, we cannot directly use Racket's map primitive. Instead, we must implement map ourselves.

We will only implement the basic version of map which takes two arguments, a procedure and a single list, and provides as output a new list where each element is the result of applying the procedure to each item in the list. For example:

INTERPRETER> (map (lambda (n) (+ n 1)) '(1 2 3 4 5))
(2 3 4 5 6)
I have provided you with eval-map, however it calls a helper function, eval-map-helper which you have to write.

Note: While you might not care about the difference, doing what we just did for map -- and what we're about to do below for apply and eval -- means that in our interpreter, map, apply and eval are special forms, whereas in Racket, they are primitives. As a result, expressions such as the following will work in Racket, but not in your interpreter:

(map eval '((+ 1 2) (* 3 4) (/ 6 2))) ; => (3 12 3)
((lambda (x) (x + '(1 2 3))) apply)   ; => 6
It is completely unnecessary for you to worry about this difference as it pertains to this project, but if you are interested in trying to fix that, come talk to me when you've finished everything else.

Implementing apply

Completing the implementation of apply is required for meta-circularity since it was used in defining apply-primitive-procedure. The implementation of apply is trivial.

We can't use Racket's primitive apply for the same reason that we couldn't use Racket's version of map: namely, that Racket's version expects Racket procedures, not our closures.

The syntax for apply is: (apply proc list-of-args) where proc is a procedure and args is a list of arguments to pass to the procedure. The result of the above statement is to call the function proc with the arguments stored in the list-of-args. For example, (apply cons '(a b)) will give you (a . b). You may wish to try a few examples in Racket first, just to make sure you understand how apply is working.

You'll certainly notice the similarities between the Racket procedure apply and the procedure i-apply you wrote: both take a procedure and a list of arguments to apply the procedure to. This should indicate to you that to get apply working in your interpreter, you are going to want to make a single call to i-apply. Your only job, therefore, is to figure out what parameters you should be passing.

Implementing eval (optional)

Most of you have probably not used Racket's eval function in writing your interpreter, and you are not required to complete the implementation of eval unless you used it. Note that you almost certainly should not have used eval in your interpreter so far. If you did, you may want to ask me about it. You are welcome to implement eval but it is not required. (It has only one slightly tricky part.)

Implementing and and or

Did you even notice that these didn't work in your interpreter? The implementation for eval-and and eval-or are nearly identical. Write one first, then copy-and-paste and make the necessary small modifications.

Note that both functions can handle an arbitrary number of predicates, anything that isn't #f is considered true, and that both functions always return the result of the last tested item.

(define x 5)
(define y 20)

(or (= x 3) (< y 10) (+ x y) (< y x)) ; => 25  -- no, that's not a typo
(and (= x 5) (< y 30) (+ x y) (< x y)) ; => #t
(and) ; => #t
Remember that both are short-circuiting, which means that if they stop once the result is known. For example (and (= 3 4) (display "hello")) will not display "hello".

Actually implementing meta-circularity

Believe it or not, you don't need to write any more code to get meta-circularity to work. However, the code you already wrote might not work as well you thought it did -- and you may need to go back and alter a few things. To make things worse, debugging a meta-circular interpreter is quite difficult.

One way to test whether or not your interpreter is meta-circular is to copy and paste all of your code, excluding any lines that begin with a # sign (e.g. "#lang racket"), or begin with (require ...), (trace ...), or (provide ...) from interpreter.rkt onto the INTERPRETER> prompt. (Be sure you press Enter after you paste your code or DrRacket won't even begin evaluating what you've pasted!) Unfortunately, doing this is EXTREMELY slow -- probably due to a bug in DrRacket -- and for some strange reason, this takes a while.

The other way to test it, and the way that I will test it after you have handed it in, is to simply include your file into your interpreter. The eval-include function will read any file, evaluating each statement one-by-one, and return void. To help you in debugging problems, the eval-include function prints out the result of evaluating each statement in the file you are including. (Note: If you'd rather mimic Racket's behavior a little more closely, you can comment out the line (i‑print result) (newline) in the eval-include function and omit the printing of each statement as it is evaluated.) I've written all of the code for including files, you just need to add the following line to your i-eval function:

      ((include? exp) (eval-include exp env))

To help you try it out, I have included a small file in the i-6 directory. First run your interpeter, then at the INTERPRETER> prompt, type (include "test.rkt"). The test program will define two functions which compute factorial: a recursive process solution, factorial-r, and an interative process solution, factorial-i. When each function is included, it's name will be displayed. Once it is included, try out each function:

(factorial-r 5)
(factorial-i 5)

Assuming that things worked, it's time to try including your interpreter into itself. At the INTERPRETER> prompt, type (include "interpreter.rkt"). You will see each function name printed out as it is included into your interpreter. It may not work the first time... or the second time... or the third time... or... Getting your interpreter to read in all of its own code can be challenging.

When you get the INTERPRETER> prompt after including the interpreter.rkt file, your interpreter has actually included all of its own code into itself... which means you can now type (repl) and run your interpreter inside of itself...

If, after typing (repl) you get the INTERPRETER> prompt, that prompt is actually from your interpreter running inside of itself!

Now it's time to see if your interpreter running inside of itself works. Many times, you can get to the INTERPERTER> prompt but some simple things (like primitives) don't work. So, try running some simple tests of your interpreter running inside itself. When you are confident it is working, try some more difficult tests.

When you are really sure you think it's working, it's time to try including your interpreter into your interpreter which is already running in your interpreter. Type (include "interpreter.rkt") and press enter. This will now include your interpreter code into your meta-circular interpreter. When it is complete (it should take 2-5 minutes depending on the speed of your machine, with the bulk of the time running the setup-env procedure), and you are presented with the INTERPRETER> prompt, it means you can just run (repl) again, and now you are running your interpreter in your interpreter in your interpreter, in DrRacket. Test it out on something really simple, like (+ 3 4). It will take about one or two minutes to give you an answer.

Finally, if you are daring, and you have a lot of time to kill, you can include your interpreter into itself again. Expect to wait a very long time. I'd recommend going to lunch, taking a walk, and going to dinner, maybe even going to sleep, and waking up the next morning to see if it finished. I started mine running and came back 4 hours later and it was still going... it can be very very slow. However, if you get a new prompt, type (repl) to run the meta-meta-meta-meta-interpreter and expect to wait a very long time for anything to happen. Theoretically you could keep doing this forever, but I doubt you'd want to try -- or wait.


What to turn in now that you are done

You will not receive full credit for the interpreter unless:

  1. ... you used handin37 to turn in each of the 6 individual parts by noon, Sunday, May 12th. No late assignments will be accepted. To make sure you have handed in each of the individual parts, run handin37 and then choose option 'v' to view the files you have turned in.

  2. ... you selected a partner (or NONE if you worked alone)

  3. ... your interpreter.rkt has all debugging statements removed and you have no functions which are traced. It may be helpful to comment your code if you believe I may have a difficult time understanding functions you have written. Add comments only where you feel they are necessary.

  4. ... your code is organized so that similar portions go together. (Unless you moved things around, it should already be like that.)

  5. ... your program executes! You will be penalized substantially if the file you turn in produces errors simply trying to execute the code. In other words, if I press the execute button, your program should not produce any errors.

  6. ... you require the user to type (repl) to start the interpreter, rather than including that as the last line in your file.

  7. ... your i-tests.rkt file includes the tests for all of the portions of the interpreter (both my tests and your tests). You do not need to have any explicit tests that show that meta-circularity works, but you are welcome to include them if you can figure out how to do write such tests.

  8. ... you write a "README" file (1 page is adequate) that gives an overview of how your interpreter works and explains any special features you have added to your language. This README file should be a text file you create in a text editor, not Microsoft Word (or other word processing) document.

  9. ... you make sure that if sections of the interpreter don't work, you comment this adequately and keep your interpreter from crashing. In addition, you should describe them in the README report. If there are required features that you didn't implement, you need to state that there.