CS 312 Lecture 6
The Substitution Model of Evaluation

In this lecture, we examine how ML programs evaluate more closely, building a more formal and precise description of the evaluation process. The name of this model of evaluation comes from the fact that substitutions are performed at certain steps during the evaluation (namely, at function applications, let bindings, and case statements). We say that the substitution model defines the semantics of the ML language, i.e. the meaning of programs written in ML.

When we type an expression at the SML prompt, the system prints out an answer showing the result of that expression:

- (1+2) * 4;
val it = 12 : int
- if 2 < 1 then "hello" else "good"^"bye";
val it = "goodbye" : string

An ML program is a term (expression); running the program means evaluating the term to a value. In the above examples, the term (1+2)*4 evaluates to integer value 12, and the term if 2 < 1 then "hello" else "good"^"bye" evaluates to the string value "goodbye".

Functional programs make it easier to understand and reason about computation than imperative ones. In an imperative (non-functional) language like Java, there is a notion of "current state" (the values of variables) and a "current statement" that modifies the current state. In contrast, the evaluation of functional programs can be described without a notion of state or sequence of instructions to execute. Instead, the evaluation can be thought as rewriting the expression repeatedly until you obtain a value. This is similar to evaluating mathematical expressions. For example, take expression (1+2)*3. You first evaluate sub-expression 1+2, getting a new expression 3*3. Then you evaluate 3*3. ML evaluation works the same way. As each point in time, the ML evaluator takes the left-most expression that is not a value and rewrites (or reduces) it to some simpler expression. Eventually the whole expression is a value and then evaluation stops: the program is done. Or maybe the expression never reduces to a value, in which case you have an infinite loop.

Hence, the substitution model of execution describes how functional (ML) programs execute as follows:

Rewrite the term or expression step-by-step until the expression is a value. That value is the result of the evaluation.
At each step, pick the left-most sub-expression that can be reduced. Reduce that sub-expression and rewrite the entire expression.

In the rest of the lecture we will precisely define the reduction rules for a subset of ML.

A subset of ML

Let us consider expressions defined by the following grammar:

e ::= c                            (* constants *)
    | x                            (* variables *)
    | unop e                       (* unary operations, ~, not, etc. *)
    | e1 binop e2                  (* binary operations, +, *, <, etc. *)
    | (e1,..,en)                   (* tuples  *)
    | X(e1,..,en)                  (* datatype constructors *)
    | if e then e1 else e2         (* if expressions *)
    | e1(e2)                       (* function applications *)
    | fn x => e                    (* anonymous functions *) 
    | let val x:t = e1 in e2 end   (* let expressions *) 
    | case e of p1=>e1|..|pn=>en   (* case expressions *)

The above definition is recursive, so expressions can be nested arbitrarily deep. Essentially, you can have expression trees that are nested arbitrarily deep.

What are the values in this language? Constants (including int, real, char, bool, and string constants) are clearly values. They cannot be further reduced and they can represent the final answer when evaluating an expression. In addition, the following are also values: anonymous functions, as well as tuples and constructed datatypes, provided that all al of their components are values themselves. Hence, the grammar for values is as follows:

v ::= c                            (* constants *)
    | (v1,..,vn)                   (* tuples  *)
    | X(v1,..,vn)                  (* datatypes *)
    | fn x => e                    (* anonymous functions *)

For instance, ((1,true), Cons(fn x:int => x+1, Nil)) is a value. Note that the function body in the anonymous function is an expression, not a value. The body of the function will be evaluated only when the function is called, after parameters have been bound to the argument values. You can think of anonymous function values as being just a pointers to the functions' code.

At this point we only need to describe the reduction rules. Using those, we will then be able to evaluate programs by mechanically applying reduction rules.

Reduction Rules

All of the reduction rules are shown below:

[Rule 1]  unop v            →  v'   (where v' = unop v)
[Rule 2]  v1 binop v2 →  v'   (where v' = v1 binop v2)
[Rule 3]  if true then e1 else e2   →  e1
[Rule 4]  if false then e1 else e2 →  e2
[Rule 5]  let val x:t = v in e end → e{v/x}
[Rule 6]  (fn x:t => e) (v) → e{v/x}
[Rule 7]  case v of p₁=>e₁ | .. | p_n=>t_n   → e_i{v/p_i}  (where p_i matches v)

We next discuss each of the rules in turn.

Evaluating expressions

The first two rules show how to evaluate unary and binary expressions:

[Rule 1]  unop v            →  v'   (where v' = unop v)
[Rule 2]  v1 binop v2 →  v'   (where v' = v1 binop v2)

Note that each rule requires that the operands of the expressions be fully evaluated, i.e., they must be values. The reduction roughly corresponds to executing one machine instruction, for instance an addition or a multiplication. For expressions whose arguments are not fully reduced, we must find sub-expressions that can be reduced first.

An example of applying this rules:

not (3 < 4)
→  not true   (by Rule 2)
→  false      (by Rule 1)

Evaluating if-then-else

If the evaluator runs into an if expression, the first thing it does is try to reduce the conditional expression to either true or false. Then it can apply one of the two rules here. Before applying the rule, the test condition must be fully evaluated, i.e., it must be a constant. Since it must also be a boolean and the only boolean constants are true and false, we have two rules, one for true and one for false. Note that the expressions in the branches need not be evaluated before applying the reduction:

[Rule 3]  if true then e1 else e2   →  e1
[Rule 4]  if false then e1 else e2 →  e2

For example, consider the term if 2=3 then "hello" else "good" ^ "bye". This term evaluates as follows:

if 2=3 then "hello" else "good" ^ "bye"
→ if false then "hello" else "good" ^ "bye"
→ "good" ^ "bye"
→ "goodbye"

Notice that the term "good"^"bye" isn't evaluated to produce the string value "goodbye" until the If term is removed. This is because if is lazy about evaluating its then and else clauses. If it weren't lazy, it wouldn't work very well.

Evaluating the let term

The rule for evaluating a let expression "let val x = e1 in e" end is as follows. It requires that expression e1 (that x is bound to) be first reduced to a value v. At that point, the evaluation of the let construct proceeds via substitution (hence the name of this evaluation model): the reduction eliminates the "let" and substitutes each free occurrence of x in e by v. The rule is:

[Rule 5]  let val x:t = v in e end → e{v/x}

where e{v/x} stands for "e with v substituted for all occurrences of x". For example, here is an evaluation using let :

let val x = 1+4 in x*3  
→  let val x = 5 in x*3  
→  5*3  
→  15

Notice that the variable x is only substituted once there is a value (5) to substitute. That is, SML eagerly evaluates the binding for the variable. Most languages (e.g., Java) work this way. However, in a lazy language like Haskell, the term 1+4 would be substituted for x instead. This could make a difference if evaluating the term could create an exception, side effect, or an infinite loop.

In the above language, each let construct defines one single binding. One could easily extend the language with let constructs with multiple binding at once: that would just be equivalent to a set of nested let constructs, one for each binding; its evaluation would just require evaluating each of the nested let's in turn.

Substitution

When we wrote “e with v substituted for all occurrences of x” above, we missed an important but subtle issue. The term e may contain occurrences of x whose binding occurrence is not this binding x:t = v. Those occurrences should not be substituted. For example, consider evaluation of the expression:

let val x:int = 1
    val f = fn x:int => x
    val y:int = x+1
in
  fn(a:string) => x*2
end

The next step of evaluation replaces the red occurrences of x with 1, because these occurrences have the first declaration as their binding occurrence. Notice that the two occurrences of x inside the function f, which are respectively a binding and a bound occurrence, are not replaced. Thus, the result of this rewriting step should be:

let val f = fn x:int => x
    val y:int = 1+1
in
  fn(a:string) => 1*2
end

Hence, the substitution e{v/x} stands for "e with v substituted for all free (or unbound) occurrences of x". Bound occurrences of x in e are not substituted.

Here are some more examples of substitution:

x{2/x} = 2 x{2/y} = x (fn(y:int)=>x) {"hi"/x} = (fn(y:int)=>"hi") f(x) { fn(y:int)=>y / f } = (fn(y:int)=>y)(x)

Evaluating functions

Function calls are an interesting case. When a function is called, SML does a similar substitution: it substitutes the values passed as arguments into the body of the function. The rule for evaluating functions is:

[Rule 6]  (fn x:t => e) (v) → e{v/x}

As let constructs, function calls also require substitutions. In fact, by comparing rules 5 and 6, one can see that expressions "let val x:t = v in e end" and "(fn x:t => e) (v)" are actually the same thing!

Consider the following example:

(fn (r: real) =>
  if r < 0.0 then ~r else r)(2.0+1.0)  
→ (fn (r: real) =>
  if r < 0.0 then ~r else r)(3.0)  
→ if 3.0 < 0.0 then ~3.0 else 3.0
→ if false then ~3.0 else 3.0 
→ 3.0

In our simple language, we only use "val" declarations in let constructs. We can easily extend the language with "fun" declarations for non-recursive functions, since those are equivalent to "val" bindings of anonymous functions. For instance the declaration:

let fun abs(r: real):real = if r < 0.0 then ~r else r
in ...

is equivalent to:

let val abs = fn r: real => if r < 0.0 then ~r else r
in ...

Hence, the "fun" keyword is really just syntactic sugar for binding an anonymous function. The evaluation of fun declarations proceeds just as for val declarations:

let fun abs(r: real):real = if r < 0.0 then ~r else r
in abs(2.0 + 1.0) end

→ (fn (r: real) => if r < 0.0 then ~r else r)(2.0 + 1.0)
    (* replace occurrences of abs in let body with anonymous function *)

→ ...

Cases and Patterns

One of the features that makes ML fairly unique is the ability to write complex patterns containing binding occurrences of variables. Pattern matching in ML causes these variables to be bound in such a way that the pattern matches the supplied value. This is can be a very concise and convenient way of binding variables. We can generalize the notation for substitution to patterns. For a pattern p, containing zero or more identifiers (say x, y, etc) and a value v, we write e{v/p} to denote "expression e with all unbound occurrences of variables appearing in the pattern p replaced by the values obtained when p is matched against v" . For instance, consider matching the pattern p = x::y with the value v = [1,2,3]. This means matching x with 1, and y with [2,3]. Substituting p means substituting free occurrences of both x and y: e{v/p} means e{1/x}{[2,3]/y}.

With this generalized notion of substitution for patterns, we can define the rule for case statements as follows:

[Rule 7]  case v of p₁=>e₁ | .. | p_n=>t_n      →   e_i{v/p_i}  (where p_i matches v)

That is, once the expression we're matching is evaluate to an expression v, then the above rule finds the matching case and reduces the entire case expression to the expression e_i in the matching case, substituting the matching pattern p_i with v.

Some caveats

This is a model for how SML evaluates. The truth is that SML terms are compiled into machine code that executes much more efficiently than rewriting would. But that is much more complex to explain, and not that important for our purposes. The goal here is to allow us as programmers to understand what the program is going to do. We can do that much more clearly in terms of term rewriting than by thinking about the machine code, or for that matter in terms of the transistors in the computer and the electrons inside them. This evaluation model is an abstraction that hides complexity you don't need to know about. Understanding how programs execute in terms of these lower levels of abstraction is the topic of other courses, like CS 314 and CS 412.

Some aspects of the model should not be taken too literally. For example, you might think that function calls take longer if an argument variable appears many times in the body of the function. It might seem that calls to function f are faster if it is defined as fun f(x) = x*3 rather than as fun f(x) = x+x+x because the latter requires three substitutions. Actually the time required to pass arguments to a function typically depends only on the number of arguments. Chances are the definition on the right is at least as fast as that on the left.

The model as given also has one significant weakness: it doesn't explain how recursive functions work. The problem is that a function is in scope within its own body. Mutually recursive functions are also problematic, because each mutually recursive function is in scope within the bodies of all the others.

CS 312 Lecture 6 The Substitution Model of Evaluation