insertion sort:

fun insert(e:int, l:int list):int list =
  case l of
     [] => [e]
   | x::xs => if (e < x) then e::l else x::(insert (e,xs))

fun isort' (l1: int list, l2:int list) =
   case l1 of
     [] => l2
   | x::xs => isort'(xs, insert(x, l2))

fun isort(l:int list) = isort'(l, [])

The code above is a version of insertion sort, and below we will attempt to
prove that isort works for all integer list. To prove this, we will first
prove that 1) insert(e,l) returns a sorted list that contains all the
elements of l and e, and 2) that isort'(l1, l2) returns a sorted list
containing all the elements lf l1 and l2.

In the contex of the proof, we will say that the definition of sorted is the
following: for a non-nil list x::xs, for all elements e being members of xs,
x <= e and xs is sorted. The nil list is considered sorted.

First, the proof for insert.

Proof by indution on length of l, assume that l is a sorted list:

P(n) = for a sorted list l of length n and any element e, insert(e,l) returns a
sorted list l' which contains all the elements of l and also contains n

base case: length(l) = 0, by definition of list, the only list with length =
to 0 is the empty(nil) list.  Therefore,

insert(n, []) =>  (evaluation of function application)

case [] of 
   [] => [n]
 | x::xs => ... =>  (pattern matching)

[n]

[n] contains all elements of [] and n, and is also sorted. Therefore, the
base case holds.

IH: P(n) true for n = k, k > 0. And therefore we have a sorted list l' s.t.
length(l') = n, and that insert(e,l') => l'' s.t. l'' is sorted and contains
all the elements in l' and also includes e.

IS: We now want to prove that P(n) holds for for n=k+1. From the definition
of the list, we can say that for a list l of length k+1, we can say that l
== h::tl, where length(tl) = n.

Now we evaluate as follows:

insert(e, l) => (function evaluation and replacing l with h::tl)

case h::tl of 
  [] => [e]
| x::xs =>  if (e < x) then n::l else x::(insert(e,xs)) => (pattern matching)

if (e < h) then e::l else h::(insert(e,tl)) 

Now we have two possible results that depend on the value of the expression
"e < h", which can have two values, true and false.

If e < h = true, then insert(e,l) = e::l. since l is sorted and e < h, e is
less than all the elements in l. Therefore, the list e::l is a sorted list
and P(n+1) holds for this branch.

If e < h = false, then insert(e,l) = h::(insert(e,tl)). Since h::tl is a
sorted list by assumption, we know that 1) xs is a sorted list, and 2) that
h < all elements in tl. Since h is also < e, then the list h::(insert(e,tl))
is sorted iff insert(e,tl) returns a sorted list. And since length(tl) = n,
by IH insert(e,tl) returns a sorted list that contains all the elements of
tl and e. Therefore, we can say that h::(insert(e,tl)) is a sorted list that
contains all the elements of h::tl and e, and therefore P(n+1) holds for
this branch.

Since P(n+1) holds for all branches of evaluation, we can therefore say that
P(n) holds for all n > 0.


Proving isort' works:


fun isort' (l1: int list, l2:int list) =
   case l1 of
     [] => l2
   | x::xs => isort'(xs, insert(x, l2))

We will now prove by induction on the length of l1 that isort' works, or
more formally, we will prove the following property:

P(n) = for a list l1 of length n, and an sorted list l2 of arbitrary length,
isort'(l1, l2) = l, where l is sorted and contains all the elements of l1
and l2.

As shown in P(n), we will be inducting on the length of the list l1.

Base case: n = 0, again by definition of lists if length(l) = 0 then l is
the empty(nil) list. l2 is an arbitrary sorted list.

isort' ([], l2) => (evaluation of function application)

case [] of
  [] => l2
| x::xs => isort'(xs, insert(x, l2)) => (pattern matching)

l2

l2 is a sorted list by assumption, and also contains all the elements of l2
and [], therefore P(0) holds, and we have proven the base case.

IH: P(n) holds for n = k, k > 0. And therefore we have a list l1 of length
k, and an sorted list l2 of arbitrary length, isort'(l1, l2) = l, where l is
sorted and contains all the elements of l1 and l2.

IS: We now want to prove that P(n) holds for for n=k+1. From the definition
of the list, we can say that for a list l1 of length k+1, we can say that l1
== h::tl, where length(tl) = n. We can also say that we have an arbitrary
sorted list l2.

isort'(l1, l2) => (evaluation and replacing l1 with h::tl)

case h::tl of
  [] => l2
| x::xs => isort'(xs, insert(x, l2)) => (pattern matching)

isort'(tl, insert(h, l2)) => (lemma for insert, see (a))

isort'(tl, l') 

(a) We can use the insert lemma here that we proved above because we know
that l2 is a sorted list, and so we can say that insert(h,l2) returns a list
l' which contains all the elements of h and l2.

We know that length(tl) = n, and so by the IH isort'(tl, l') = l'', where
l'' is a sorted list containing all the elements of tl and l'. l' contains
all the elements of h and l2, and so we can say that l'' contains all the
elements of h::tl and l2. This proves P(n+1), and so we can say that P(n)
holds for all n > 0.

We have now proven that insert works, and using that proved that isort'
works. Using the isort' lemma, we will now prove that isort works.

fun isort(l:int list) = isort'(l, [])

We wish to prove that isort(l) = l', where l' is a sorted list and contains
all the elements of l. We don't need to do induction here, as we can prove
correctness directly.  For arbitrary l, we can step the evaluation as
follows:

isort(l) => (evaluation)

isort'(l, [])

And since we know that isort'(l, []) = l' and l' is a sorted list containing
all the elements of l and [] (by lemma), we can say that isort(l) returns a
sorted list l' which contains all the elements of l.

QED


Another inductive proof of correctness


For those of you in the 12:20 section, here is the remainder of the proof
for the findLoop function. For others, here is just another nice induction
proof to practise on.
This was a question in the 2002sp final (designed by Prof. Myers).

A word of warning: the second part of this proof is relatively heavy in
mathematical content. Personally I like proving things using Math rather
than wordy arguments. A version of the proof with less mathematical content
is linked off the 2002fa Sample exams, Q5.

Anyway, to the problem. Imagine there is a way to make lists in ML that
either end with a NIL (the usual kinds of lists), or the end in a loop. i.e.

            +-----------------+
            V                 |
a --> b --> c --> ... --> s --+

(the list ends in a loop from s back to c)

Also, for simplicity, consider a,b,c etc are all distinct.

Now consider the following code.

fun findLoop(l) = let
    (* Assumes: l2 can be reached from l1
     * i.e l1 is of the form x1::x2::...::xn::l2
     *)
    fun findLoop'(l1, l2) =
        case (l1,l2) of
            (h::t, a::b::xs) = if h = a then true
                               else findLoop'(t,xs)
           | _ => false
in
    findLoop(l,l)
end

To show that this works, lets first show that if the list does not have
loops, then findLoop' returns false.

If the list l has no loops, then length(l1) and length(l2) are well defined.
Let length(l1) = n, length(l2) = m. And we know m < n. We want to induct on
the difference n - m.

Base case: n - m = l. 0 <= n,m <= l. Hence n-m can be atmost l. When n - m =
l, this implies n = l and m = 0. When m is 0, l2 = [] hence the second case
is hit and we return false.

Induction hypothesis: findLoop(l1,l2) works for all l1, l2 such that n - m >
k.
(Notice the inequality sign. We have proven the base case for n-m = l.
Eventually we want to show that when l1=l2=l, i.e. n-m=0, the function
works. Hence given l, we prove it for l-1, given (l,l-1), we prove it for
l-2, and so on. This is fine since the values for the induction variable are
required to change such that they are always going away from the base case
relying only on values between themself and the base case.)

Induction Step: n - m = k
In the general case, we hit the first pattern.
Since the list has no loops, we know that h != a (since the elements are
unique).
The algorithm the calls itself recursively with lists of length n'=n-1 and
m'=m-2.
n' - m' = (n - m) + 1 = k + 1. By the Induction Hypothesis, we know findLoop
returns false if the two lists passed have no loops and l >= n - m >= k + 1.
Hence we are done. QED.

-----

Now the case if the list does have a loop. For simplicity sake, consider a
list that forms a loop i.e. the last element links back to the first. If the
list did have a tail like in the example above, it can be split into two
parts -- the tail and the loop, and a similar argument can be applied.

P(n) = So given a loop of length N, we need to show that the algorithm
returns true. Induction of n where n is such that
l1 = x1::x2::...::xn::l2
(The intuition here is that the is second list is travelling faster around
the loop and at some point, it'll loop back and catch up with the first one.
When that happens, we know that the second list is an integral multiple of n
steps in front of the first list. So let our base case be that when l2 is 0
(mod n) steps in front of l1 and l1 is a part of the loop, the algorithm
returns true)

Base case: n = 0 (mod N)
Since l1 is in the loop, and the loop is of length n, by travelling n steps
along the loop, we will end up on the same node. Hence h = a is true and the
algorithm return true. (Note we have in essence proved for multiple base
cases 0, N, -N, 2N, -2N, so on)

Induction Hypothesis: P(i) holds for all -N <= i <= k.
(Notice the use of the -N base case)

Induction Step: P(k+1)
If k+1 = 0 (mod N) -- P(k+1) is proved by a base case.
(This is the case when N = 1 since everything is 0 (mod 1))

If not, we know that h = a is false and hence the algorithm calls itself
with t,xs where xs is k+2 steps ahead of t. k+2 = -N+(k+2) (mod N). But -N
<= (k+2)-N <= k and hence by the Induction Hypothesis, we know the recursive
call will compute the correct answer. QED.