Note: there is amusing CS212 graffiti (which RDZ cannot officially condone). Check out the web page.

You will need to brush up on the ENVIRONMENT MODEL to understand this lecture.

Metacircular Evaluator - Writing an interpreter for Scheme in Scheme

• Disadvantages: not of much value in practice, because you can't run it unless you already have Scheme 
• Advantages: 
  • Better understanding of how the language works 
  • See that languages are not magic, just another program! 
  • You can change the language (that the interpreter evaluates) -- PS6 
• Environment Model in action 
• Similar to the expression evaluator that we did in PS3, but more complex expressions (most of Scheme)

This is an INTERPRETER - You give it a program, and it "walks over" the program and does stuff, running pieces of it. 

It is not a COMPILER (coming soon!) - You give it a program, and it translates it into another program in a lower level language (machine code).

Interpreters are generally: 

• Easier to build, understand, and get right 
• MUCH slower

In PS6, we will build an interpreter based on what is covered in the next few lectures and recitations.

We've been looking at techniques for implementing computational processes as ABSTRACTIONS:

2 ---> [square] ---> 4

We're going to look at a rather unusual kind of box today, a UNIVERSAL MACHINE -- an evaluator for Scheme.

* It can do *anything* that any other machine can.

You give it a DESCRIPTION of any computational process, and it acts like that process:

       +----------------------------------------------------+
       |                                                    |
 2 --->| (lambda (x)(* x x)) ---> [EVALUATOR]               | ---> 4
       |                                                    |
       +----------------------------------------------------+

Note that the input to the evaluator is a LIST (a DESCRIPTION of a function to compute) not a Scheme procedure!

Given a description of squaring -- Or anything else! -- act like that.

* This is a pretty strange kind of box: It reconfigures itself so it can act like any other box.

* The processor one inside a computer is one -- programmed in the native machine code for the processor.

Scheme-expr ---> [Metacircular Evaluator] ---> values

It's called a Metacircular Evaluator because: 

• We fancy words
• It's a Scheme interpreter written in Scheme.

That means there are TWO SEPARATE Scheme LANGUAGES here! 

• Scheme, the implementation language 
• Metacircular Scheme, the one we're writing (a subset).  We call this one "Tiny-Scheme".

WARNING: * They look exactly the same - until we start tinkering with Tiny-Scheme at least * But don't get them confused!

Tiny-Scheme won't handle all Scheme expressions, some things are too involved.

NOTE, there are *many* layers of evaluation/compilation here:

The top level of a Scheme interpreter is a read-eval-print loop which does four things

1. Prints the prompt 

2. READS an expression 

3. EVALUATES the expression 

4. PRINTS the result.

(define (ts:read-eval-print-loop)
  (echon "TS==> ")
  (let* ((exp   (ts:read))
         (value (ts:eval exp *ts:top-level-env*)))
    (unless (void? value)
      (write value) (newline))
    (ts:read-eval-print-loop)))

The reader, READ, builds list structures out of characters. You used READ in PS3. 

If you type an input 

• it is initially just a sequence of characters, not symbols, lists, etc. 
• The reader turns these characters into list structure. 
• There's a whole big technology of readers we don't have time to discuss. 
  • Uses finite state machines (which we did in PS4)

Recall from the environment model that top level expressions are evaluated w.r.t. the global environment which here is called *ts:top-level-env*

What does ts:eval do?

Remember the environment model: 

• To evaluate a compound expression other than a special form, evaluate the subexpressions and apply the value of the first to the rest. 
• To evaluate a primitive expression, either look up the variable value or if it is self evaluating return its value.

It's a dispatch on the type of expression, implemented using generic functions.

;;; Evaluate an expression in an environment.
(defgeneric (ts:eval (exp <top>) (env <env>)))

Most kinds of objects evaluate to themselves.

;;; Default method: return the expression as the value.  Thus, numbers,
;;; booleans, strings, etc. will evaluate to themselves.
(defmethod (ts:eval (exp <top>) (env <env>))
  exp)

Evaluating symbols causes their associated value to be looked up.

;;; We evaluate a symbol by looking it up in the environment.
(defmethod (ts:eval (var <symbol>) (env <env>))
  (let ((bv (env-lookup-binding env var)))
    (if bv
      (binding-value bv)
      (unbound-var env var))))

Evaluating a compound expression (a list structure)
calls ts:eval-combination

;;; The exp is of the form (e1 e2 e3 ... en) -- extract the first
;;; expression and use eval-combination to evaluate.
(defmethod (ts:eval (combination <pair>) (env <env>))
  (ts:eval-combination (head combination) (tail combination) env))

;;; Evaluate a combination expression of the form
;;; (operator e1 e2 e3 ... en) where args is the list of expressions
;;; e1 e2 e3 ... en.
(defgeneric (ts:eval-combination operator sub-exps (env <env>)))

;;; The default method evaluates the operator, and the sub-expressions
;;; and then applies the operator value to the sub-expression values.
;;; Notice that application does not take an environment as an argument.
(defmethod (ts:eval-combination operator sub-exps
                                (env <env>))
  (ts:apply (ts:eval operator env)
            (map (lambda (e) (ts:eval e env)) sub-exps)))

What does ts:apply do?

Remember the environment model: 

• To apply a procedure, evaluate the body of the procedure in a new environment that is built by extending the procedure's environment 
• To apply a special form, use a particular rule for that special form 
• To apply a primitive, do the computation for that primitive

;;; Apply a primitive or closure value to a list of argument values.
(defgeneric (ts:apply value args))

For a primitive

Evaluate the arguments in the environment in which the primitive is being called. Apply the Scheme function that performs the work of the primitive.

;;; We use Swindle to apply a primitive operator such as +, *, - to
;;; arguments or in fact, any Swindle procedure.
(defmethod (ts:apply (operator <procedure>) args)
  (apply operator args))

For a regular function (user-defined methods)

We evaluate the arguments and then evaluate the body in the extended environment

;;; To apply a closure, zip the formal arguments and actual arguments
;;; into a list of bindings, create a new frame with those bindings, and
;;; link the environment of the the closure in to create a new
;;; environment.  Then evaluate the body of the closure in the new
;;; environment.
(defmethod (ts:apply (operator <closure>) args)
  (let ((env (make <nested-env>
                   :previous-env   (closure-env operator)
                   :frame-bindings (zip-bindings (closure-args operator)
                                                 args))))
    (ts:eval (closure-body operator) env)))

That's all very nice, but how do we actually represent something like an environment or a procedure object??

* Internal representations in the interpreter!

ENVIRONMENT: sequence of frames, each frame is a set of bindings

Binding: variable name and value

;;; A binding has a symbol and a value.
(defclass <binding> ()
  (binding-symbol :type     <symbol>
                  :initarg  :binding-symbol
                  :accessor binding-symbol)
  (binding-value  :initarg  :binding-value
                  :accessor binding-value))

;;; A frame is a list of bindings.
(defclass <frame> ()
  (frame-bindings :type      <list> ; of bindings
                  :initarg   :frame-bindings
                  :initvalue null
                  :accessor  frame-bindings))

;;; A top-level environment is just a frame.
(defclass <env> (<frame>))

;;; A nested environment is an environment (i.e., frame) and a previous
;;; environment.
(defclass <nested-env> (<env>)
  (previous-env :type     <env>
                :initarg  :previous-env
                :accessor previous-env))

The procedure expand-environment, which is used to create a new frame and link it in, is a good illustration of how the environment data structures work:

;;; A utility function for combining a list of symbols (formal
;;; parameters) and a list of values (actual parameters) into bindings.
(define (zip-bindings formals actuals)
  (cond ((and (null? formals) (null? actuals)) null)
        ((null? formals) (error "too many arguments"))
        ((null? actuals) (error "too few arguments"))
        (else (cons (make <binding>
                          :binding-symbol (head formals)
                          :binding-value  (head actuals))
                    (zip-bindings (tail formals) (tail actuals))))))

PROCEDURES: Recall that they have three parts: 

>>> Draw a double-circle procedure object <<< 

1. Parameter list 

2. Body 

3. Environment where created

So, let us do the obvious thing:

;;; A closure has a list of argument symbols, a body expression, and an
;;; environment.
(defclass <closure> ()
  (closure-args :type     <list> ; of symbols
                :initarg  :closure-args
                :accessor closure-args)
  (closure-body :initarg  :closure-body
                :accessor closure-body)
  (closure-env  :type     <env>
                :initarg  :closure-env
                :accessor closure-env))

;;; Evaluation of special form (lambda (x1 ... xn) e) -- we create a
;;; closure and return it as the value of the combination.
(defmethod (ts:eval-combination (operator = 'lambda) sub-exps
                                (env <env>))
  (make <closure>
        :closure-args (first sub-exps)  ; (x1 ... xn)
        :closure-body (second sub-exps) ; e
        :closure-env  env))

This is where the symbol "lambda" gets its power.  (You can amuse yourself by trying to figure out if you can use lambda as the name of a local variable in tiny-scheme.)

Let's try it out.

TS==> ((lambda (x) (+ x 3)) 2)

1. The top level REPL calls ts:eval with this expression and the global environment.

2. In ts:eval, this is a combination (i.e., a list structure) so we call ts:eval-combination on its head, its tail, and the environment.  

• The head is the list (lambda (x) (+ x 3))
• The tail is (2)
• The env is the global environment

ts:eval-combination will evaluate the first argument, map the function "eval in env" over the second argument, and then call ts:apply.

2.1 ts:eval gets called with (lambda (x) (+ x 3)) and the global env

2.2 This calls ts:eval-combination with arguments:

• lambda
• ((x) (+ x 3))
• global env

2.3 The method for (operator = 'lambda) gets invoked, and we create a <closure> with the right parts.

3. Tiny-Scheme-apply gets called with its first argument being the procedure object created above, and its second argument being the list of evaluated arguments (2) [I've skipped what happens when we map eval over the list of arguments, since it's pretty dull.]

This calls ts:eval on the body of the procedure, with the environment extended by evaluated argument list using zip-bindings.  In particular, we add the new frame with the binding x:2.  Call this environment E.

4. ts:eval gets called with (+ x 3) in this new environment. This is a list structure (combination) so the arguments get evaluated in this new environment E.

4.1 We look up the value of + in E and have to go to the global environment.  There we find a <procedure> i.e. {add}.

4.2 We look up the value of x in E and get 2. We evaluate 3 and get 3.

5. ts:apply gets called to apply this <procedure> to the list (2 3)

6. ts:apply calls apply, which returns 5