PS6 part 1 due TODAY. PS6 part 2 due 12/14.
Return: available on 12/15 from Scott Coldren.
Course evals!
Final exam:
Thursday, December 16 3:00-5:30pm
Olin 255 (I hope you can find the room!)
Closed book, closed notes.
Think of it as a big prelim. Everything is on it except the continuations lecture and the Y-operator lecture. For example, you will definitely have an induction problem and a modify-the-evaluator problem.
Review session: Time, place TBA (on the web). Will be the evening of
12/14.
Today’s lecture theme: applying things to themselves!
********************************************************************
In this lecture we explore the far reaches of what computers can
(and cannot) do. We also prove the single deepest result in
Computer Science, Turing's Halting Theorem (it's related to Goedel's
incompleteness proof, among other things), and talk about bigger and
smaller infinities. At the end, if there is time we'll relate it all
back to compiler optimizations (it’s not all theory).
* We've been showing you lots of powerful programming tools.
* You might think that computer can solve any problem, given the right
program.
* If you've thought a bit about it, you might realize there could be
problems with solving, say, social or philosophical or religious
problems. So you might only think that they could solve all
mathematical problems.
But even that isn't true.
Today we'll look at some things that CANNOT be computed.
* And prove that they can't be computed.
* No matter
- what program,
- what language,
- what machine,
- how long you wait
- ANYTHING.
We will look at TWO reasons:
1. There are more mathematical functions than programs.
* Lots more.
* There are infinitely many of both,
* But there is a *bigger* infinity of functions
2. We'll even show you a function which it would be contradictory to
compute (an explicit counterexample to the claim that everything is
computable).
Some desirable compiler optimizations are uncomputable. So this isn't
just a math problem (although it is at heart mathematics).
Here's what we'll do:
1. There are *countably* many programs. (We can give all of them
integers.)
2. There are *uncountably* many functions. (It's impossible to give
them all integers.)
3. That means that there must be some function which isn't
programmable.
4. Then we'll show you a particular uncomputable function.
----------------------------------------------------------------------
First we're going to show that the set of all Scheme programs is
*countable*.
Let's say that a set S is *countable* if there is a way to number the
elements of the set. For each integer there is a unique element of S,
and every element of S has an integer.
More formally,
S is countable if there is an onto function f: N-->S.
(f is onto if, for each s in S, there is some number n with f(n)=s)
(in other words, f "covers" S)
For example, N is countable: f(n) = n
Z = {..., -3, -2, -1, 0, 1, 2, 3, ...} is countable:
f(2n) = n
f(2n+1) = -n
The *rationals* are countable.
1/1 1/2 1/3 1/4
2/1 2/2 2/3 2/4
3/1 3/2 3/3 3/4
4/1 4/2 4/3 4/4
Number these in a diagonal zigzag:
1 2 4 7
3 5 8
6 9
10
etc...
f(n) = the n'th rational in that ordering. You can figure it out.
----------------------------------------------------------------------
There are countably many programs:
* Any program is a finite string of ASCII characters.
* We can count the finite strings over an alphabet.
1 - a
2 - b
...
26 - z
27 - aa
28 - ab
etc.
* Now, not all of these are legal Scheme programs,
* but all Scheme programs are in this list.
* So, there are clearly only countably many Scheme programs.
But there are an uncountable number of functions.
In fact, we will just consider a small subset of all functions,
and see that that is uncountable, S = {f: integers->#T/#F}
Any element s of S can be thought of as an infinite sequence, showing the
output of s for each each integer
INPUTS
1 2 3 4 5 6
odd? T F T F T F ...
prime? F T T F T F ...
etc.
Suppose that S were countable. Then there is some mapping f:N-->S
which is onto. In other words, we could arrange the elements of S
into a sequence so that f(1) is the first element of the sequence,
f(2) is the second, etc, and EVERY element of S is in this sequence (f
is onto).
We'll show that there is no such f. More precisely, we'll show that
for any such supposed list, it is possible to construct an element of
S that isn't in the list (hence, f is not onto).
We can write the effects of f as a table:
INPUTS
1 2 3 4
f(1) T F T F
f(2) F F F F
f(3) F F F T
f(4) T T F T .....
.
.
.
But for any such table, one can construct a function r that does not
appear in this table (i.e, there is no integer i such that f(i) = r).
How? We *diagonalize*
r will be an element of S, hence a function from the integers to #t/#f.
What is the value of r applied to j? The opposite of the value of f(j)
applied to j! [See? Apply things to themselves!]
INPUTS
1 2 3 4
r = F T T F ...
Now, r cannot appear in the table. It isn't f(1) because f(1) and r give
different values when applied to 1. It isn't f(2) because f(2) and r give
different values when applied to 2. Etc!
So, there are uncountably many functions, and countably many programs.
We can't compute all functions.
Note that this procedure for constructing r will work for *any* choice of f.
* It's not just that we left r(f) off that list --
* For *any* list f' with r(f) on it,
there is some other number r(f') left off.
----------------------------------------------------------------------
Note: a function from the integers to #T/#F has a name (it's a real number!)
So the real numbers are NOT countable!
Cantor discovered this around the turn of the century and flipped out...
and turned logic and set theory on their heads.
Let I = (0,1) = the Open Unit Interval (not including 0 or 1).
Suppose that I were countable. For any f:N-->I, we'll construct
an element of I that isn't covered (hence, f is not onto).
Write down a big oo x oo table of the digits of the numbers.
f(1) = 0 . 3 8 5 9 4 0 5 5
f(2) = 0 . 2 3 6 5 3 5 5 5
f(3) = 0 . 4 3 4 4 3 3 3 4
f(4) = 0 . 2 3 4 1 4 2 3 4
...
Now, can construct a real that is not in the table in the same manner
as above, pick the digits on the diagonal and make sure that the new
real differs from the i-th element of the list in the i-th digit.
The real numbers are UNCOUNTABLE --
* There is NO onto function from the integers to the reals
* There are A LOT MORE reals than integers.
- AMAZINGLY more.
This is called a diagonalization argument, btw.
----------------------------------------------------------------------
Why would you ever want to compute one of these functions that can't be
computed ?
Well, many interesting questions concerning programs are uncomputable.
Here's a real useful one:
(safe? prog arg) ---> #T if the function call (prog arg)
halts with an answer
#F if it doesn't.
This would be very, very useful, e.g., in finding infinite loops or programs
that get errors.
But there's a problem. Consider the following program using safe?
(define safify
(lambda (p)
(if (safe? p p)
(not (p p))
#f)))
Note that give that safe? exists, it must be that safify is always
safe on any argument, because it uses safe? to check whether the
computation is safe, and only does the computation if it is.
Therefore safify must always halt with an answer.
For example,
(safify (lambda (x) 4))
--> (not (lambda (x) 4) (lambda (x) 4)))
--> (not 4) --> #f
What about (safify safify)?
(safify safify) is by definition safe, so what value does it have?
(safify safify) = (not (safify safify)) by substitution.
But this is impossible, there's no Scheme value that is equal to
the negation of itself -- this is a contradiction.
All we assumed was the existence of safe?, so there CANNOT be a
procedure `safe?'. This is Turing's Halting Theorem.
An interesting thing that all the neat tricks we've taught you can't
possibly help with.
----------------------------------------------------------------------
There are *lots* of useful uncomputable functions. For example,
suppose we have in the compiler
(let ((t1 (f))
(t2 (g))
(t3 (+ t1 t2)))
t3)
Now we'd like to go common subexpression one better: if F and G
compute the same value, then we can optimize out the call to G (ie,
bind t2 to t1).
So how can we tell if two functions F,G compute the same value?
Sounds easy, and various special cases are.
(lambda () 3) vs (lambda () (+ 1 2))
Can we do it in general? NO. It's uncomputable!
----------------------------------------------------------------------
The two major themes of CS212 are reasoning about programs and efficiency. Uncomputability, in a sense, unifies both of them.