\documentclass{article} \usepackage{611-lecture}
\usepackage{amsfonts}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{bbm}
\usepackage{pifont}
\usepackage{amsbsy}
\usepackage[dvips]{graphicx}
\usepackage[dvips]{color}
\usepackage[scanall]{psfrag}
\usepackage{fancyheadings}
\usepackage{pstcol}
\usepackage{pst-grad}
\usepackage{pst-text}
\usepackage{wrapfig}



%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\lecture{4} %% Lecture number
\title{Equivalence, Reductions and Normal Forms}   %% Title of lecture
\author{Ymir Vigfusson, Daria Sorokina}  %% name of scribe
\date{02 September 2005}    %% Date of lecture
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{document}

\maketitle

%\section{Administrativa}
%\begin{itemize}
%\item No class on Monday (Labor day)
%\item Wednesday: Radu Rugina
%\item Problem Set 1 is due on Wednesday, the paper component is due on 5PM, and
%\texttt{lambda.sml} is due on midnight.
%\end{itemize}

\section{Term equivalence}
When are two terms equal to one another? 
Consider, for example, $\lambda x.x$ and $\lambda x . \lambda y . x$. 
Are these two terms equal?
It turns out that defining a precise mathematical model for 
determining equivalence of two $\lambda$-terms is tricky.
In fact Alonzo Church received the Turing-award for explaining how to 
define functions in Lambda-calculus.
One way of attacking the problem is to try all possible arguments and 
compare the results. However, we cannot be sure whether
we get a result or not. Let us formulate this.

We say that an expression $e$ _terminates_ or _converges_ if there is 
a finite sequence of expressions such that
\[ e \rightarrow e' \rightarrow e'' \rightarrow \cdots \rightarrow v \]
where $v$ is a value. This is denoted by $e \Downarrow v$.
Otherwise, when $e \rightarrow e' \rightarrow e'' \rightarrow \cdots$
and we never get anywhere, we say that $e$ _diverges_. This is denoted by 
$e \Uparrow$.

There are infinitely many divergent terms.
An example of one is $\Omega$ which was defined in the last lecture. 
In some way they are equal to one another, they all go off to infinity.
We are now ready to give a better notion of equality.

\subsection{Equality of terms}
Two terms are _equal_ if:
\begin{itemize}
\item they behave the same in every context, or
\item they cause divergence in every context.
\end{itemize}

If two terms converge, we should naturally require that they
converge to the same value. But we do not need this requirement explicitly,
because if they converge to different values, there exists a context
in which they behave differently.

%For example, we add integers to the language and use them as base values and consider the following:
%, we could add the requirement that convergent terms converge to the same value. But we don't actually need that. For example:
%\[
%"if" e = 1 "then" \Omega "else" 0\\
%"if" [\cdot ] = 1 "then" \Omega "else" 0
%\]
%XXX: define the = being used here
%and we have two expressions $e_1 \rightarrow 1$ and $e_2 \rightarrow 2$
%is a context where one expression diverges and the other converges (to 0).

In an attempt to formalize the previous definition, consider the
following notation.

\begin{itemize}
\item $C[\cdot]$ is a context, an expression with a hole.
\item $C[e]$ is $C$ with the hole replaced by $e$
\end{itemize}

We then define equality in the following manner.
\[
e_1 = e_2 \textrm{ if for all } C[\cdot] \textrm{ we have } C[e_1] \Downarrow \iff C[e_2] \Downarrow.
\]

Sounds simple in the mathematical sense, but this is hard to compute. 
We need to test all different contexts, and so we would need to enumerate 
them all. 
We may also want to talk about whether open terms are equal to one another, and there we need to be more careful.

We need a conservative test of whether two terms are equal. However, 
we must realize that we can never tell precisely whether two terms are 
equal to one another, else we would have solved the halting problem.
But how close can we get?

How about the following definition? 
Let $e_1$ and $e_2$ be expressions.
If 
\begin{eqnarray*}
&& e_1 \rightarrow e_1' \rightarrow \cdots \rightarrow v_1, \\
&&e_2 \rightarrow e_2' \rightarrow \cdots \rightarrow v_2,  
%\text{and } && v_1 = v_2 \text{ then } e_1 = e_2.
\end{eqnarray*}
and $v_1 = v_2$ then $e_1 = e_2$.

In other words, if we can show that two expressions evaluate to the 
same thing, then the expressions are equal.
This is especially useful for compiler optimization.
The previous definition raises the following questions.
\begin{itemize}
\item What should be the right notion of equality in this sense?
\item What kind of rewrite rules (i.e.~$e_1 \rightarrow e_2$) should we allow? 
\end{itemize}


\section{Rewrite rules}

The order of the following rewrite rules is somewhat arbitrary.

\subsection{$\beta$-reduction}

A _$\beta$-reduction_ is the following rule:
\[
(\lambda x.e_1) e_2 \xrightarrow{\beta} e_1 \{ e_2 / x\}.
\]
For example, 
\[
(\lambda x.\underbrace{(\lambda y.y)x}_{\beta \text{ redex}})
\xrightarrow{\beta} \lambda x.x
\]
%However, according to Call-By-Value 
%According to CBV, $(\lambda x.\underbrace{(\lambda y.y)x)}_{\beta redex}$ is a value and we are done. 

\subsection{$\alpha$-equivalence}


In $\lambda x.x z$ the name of the bound variable $x$ doesn't really matter. 
This term should really be the same as e.g.~$\lambda y.y z$. 
Renamings like that are known as _$\alpha$-conversions_ or _$\alpha$-renamings_.
In other words: $\lambda x.x z$ and $\lambda y.y z$ are \emph{$\alpha$-equivalent} to one another.
Note that this defines an equivalence relation on the set of terms, so 
$e_1 =_\alpha e_2$ is well defined.

Let's introduce the notation of 
$FV(e)$ as the set of free variables of $e$.
In general we have
\[
\lambda x.e =_\alpha \lambda x'.e\{x'/x\}.
\]
But what if $x'$ is already being used in e? Then instead we must define
\[
\lambda x.e =_\alpha \lambda x'.e\{x'/x\} \text { if } x' \notin FV(e).
\]
When writing a $\lambda$-interpreter, the job of looking for $\alpha$-renamings doesn't 
seem all that practical. However, we can e.g.~use them to improve our earlier definition of equality:

If 
\[
e_1 \rightarrow e_1' \rightarrow \cdots \rightarrow v_1, \text{ and}
\]
\[
e_2 \rightarrow e_2' \rightarrow \cdots \rightarrow v_2,
\]
and $v_1 =_\alpha v_2$ then $e_1 = e_2$.


\begin{figure}[!hbt]
\begin{center}
\includegraphics[scale=0.6,angle=270]{lambda}
\caption{Stoy-diagram for $\lambda x.(\lambda y.y)x$}
\end{center}
\end{figure}
We can create a _Stoy diagram_ for a closed term in the following manner.
Instead of writing $\lambda x.(\lambda y.y)x$, we write $\lambda \cdot.(\lambda \cdot.\cdot)\cdot$ 
and connect variables that are the same by edges.
Then $\alpha$-equivalent terms have the same Stoy-diagram.

\subsection{$\eta$-reduction}
%Another notion of equality.

Think about $e$ and $\lambda x.ex$.
If these two things represent functions, then you apply them to something,
say $e'$, what's going to happen? They are the same, because
$e e'$ and $(\lambda x.ex)e' \xrightarrow{\beta} e e'$, so 
they $\beta$-reduce to the same thing.
We must be sure that $x$ does not appear inside of $e$.

Formally:
\[
(\lambda x.e_1 x)e_2 \xrightarrow{\beta} e_1 e_2 \text{ if } x \notin FV(e_1)
\]

This we call an _$\eta$-reduction_, that is
\[
\lambda x.ex \xrightarrow{\eta} e
\text{ if } x \notin FV(e).
\]
In other words, if we apply two expressions to the same thing, they reduce to the same thing.

This reverse operation, i.e.~an _$\eta$-expansion_, 
 is practical as well. It was e.g.~used to create $\Omega$ in the last lecture.
%\begin{eqnarray*}
%(\lambda x . e_1) e_2 &\xrightarrow[\eta] & e_1 (e_2 / x)
%(\lambda x . e_1) e_2 &\xrightarrow[\eta] & e_1 (e_2 / x)
%\end{eqnarray*}


%To summarize:
%\begin{itemize}
%\item
%_$\eta$-reduction:_ If we apply two expressions to the same thing, they reduce to the same thing.
%\item
%_$\eta$-expansion:_ The reverse operation. It was used to create 
%$\Omega$ in the last lecture.
%Generally, an $\eta$-expansion is used to delay divergence by trapping 
%expressions inside $\lambda$ terms. In practice, you will 
%use $\eta$-expansions more than $\eta$-reductions.
%$\eta$ aren't exactly meaning conserving
%\end{itemize}

But there is a problem, what if $e$ diverges?
We need another condition:

\[
e \Uparrow \iff \lambda x.ex \Uparrow
\]

%\lambda x.ex \xrightarrow{\eta} e
%\text{ if } x \notin FV(e).
%$(\lambda x.ex)e' \rightarrow_\beta e e'$ if $x \notin FV(e)$ and (CHECK)

\section{Normal forms}

What if the rewrite rules do more than one thing?  If we do not fix the ordering
in which rewrite rules should be done, such as in CBV, you can have a choice between rewrite rules.

For example, consider that
\[
e_1 \rightarrow e_1' \rightarrow \cdots \rightarrow v_1 
\]
and
\[
e_2 \rightarrow e_2' \rightarrow \cdots \rightarrow v_2.
\]
But what if you can have $e_1' \rightarrow v_3$ ? Then $e_1$ and $e_2$ are not
necessarily equal.
This is not a problem, because under $\beta$-reduction or 
$\beta + \eta$ reduction rewriting is _confluent_. This is also
known as the _diamond property_ or _Church-Rosser_ property.

What is confluence? The Church-Rosser theorem says that the $\lambda$-calculus 
 that regardless of order of rewrite rules, we will arrive at a unique 
final result if one exists. 
This final form is called the _normal form_. 
However, because it is possible for an expression not to terminate, it 
does not necessarily have a normal form.

\[
\begin{matrix}
   & & e' & & \\
   & \nearrow & & \searrow & \\
e  & &    & & v \text{ (normal form)}\\
   & \searrow & & \nearrow & \\
   & & e'' & & 
\end{matrix}
\]

Note that under Call By Value, there is at most one rule possible at each time,
so you will not get the above diamond form.

It may not be apparent, but there is a catch. We may not always pick the 
\emph{easiest} rewrite rule to expedite termination, in fact, we may get stuck 
in a loop or a _trace_ when rewriting. However, the Church-Rosser theorem 
provides a guarantee that it is possible to get unstuck, provided that the
end result exists. Let's formalize this.
We will denote by an arrow closure, that is $e_1 \rightarrow^* e_2$, that we perform zero or more rewrite rules to get from $e_1$ to $e_2$.

Suppose $e_1 \rightarrow^* e_2$ and $e_1 \rightarrow^* e_2'$.
More formally, the Church-Rosser theorem states that 
there exists an expression $e_3$ such that $e_2 \rightarrow^* e_3$ and $e_2' \rightarrow^* e_3$.
This tells us that if we hit a trace when rewriting a reducible expression (we get stuck), we 
can always get unstuck and hit the value.
(A value is something you cannot do any more rewrites on, except for $\alpha$-reductions)

%(This doesn't work always because we could have divergence.)

However, we could also get stuck by doing an infinite number of rewrites.
For example:
\[
(\lambda xy.y) \Omega \xrightarrow{\beta} \lambda y.y
\]
Yet in this case, if we choose evaluate the right-most argument, we get
\[
(\lambda xy.y) \Omega \xrightarrow{\beta} (\lambda xy.y) \Omega \rightarrow_\beta \cdots
\]
and we're fine.
There is an ordering to prevent such traces, a so-called _normal order_ 
where you reduce the _leftmost_ redex. 
By using normal order, we are guaranteed to get to a normal form if one exists.
This corresponds to Call By Name evaluation. 
Another ordering is the _applicative order_ where one reduces arguments to the 
normal form, and then applies $\beta$-reductions. This corresponds to
Call By Value evaluation.\\

\textbf{Example:} 
In C the evaluation order of arguments to functions is implementation-specific.
Also, C does not specify in what order operands work. Hence C is not confluent.

To demonstrate this, in C, $(x=1) + x$ can be evaluated as 2 if you start by 
evaluating the 
left argument to the $+$ operand. If you start with the right argument,
it evaluates to $x+1$. This depends on the implementation.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\end{document}
