Lecture 28: Turing machines

Turing machines

Turing machines are a small generalization of finite automata. Like DFA, a Turing machine has a finite set of states, and upon observing a character, changes to a new state. However, the Turing machine is also able to write information back on top if the input, and may move its position to the left as well as the right. At may also move past the end of its input, so that it can keep additional state as it computes. Because running off the end of the input doesn't cause the machine to stop, the machine needs to indicate when to stop processing. In particular, a TM can run forever.

Definitions: Let \(L \subseteq Σ^*\) be a language. If there exists a TM M such that for all \(x \in L\), \(M\) halts and accepts \(x\), and for all \(x \notin L\), \(M\) halts and rejects \(x\), then \(M\) decides \(L\) (and \(L\) is said to be "decidable"). If there is a machine \(M\) such that for all \(x\) in \(L\), \(M\) halts and accepts \(x\), but if \(x \notin L\), \(M\) either halts and rejects or \(M\) runs forever, then we say \(M\) recognizes \(L\) (and \(L\) is said to be "recognizable").

Example: Recognizing \(\{0^n1^n \mid n \in \mathbb{N}\}\)

To show that Turing Machines are more strictly more powerful than DFA, and to give a flavor of how TMs work, let's build a TM that decides the language \(\{0^n1^n \mid n \in \mathbb{N}\}\).

Here's the general strategy: if the string is empty, we will accept. If it is not empty, we will erase a 0 from the beginning of the string, erase a 1 from the end of the string, and then return to the beginning of the string and repeat. If at any point we can't do that, we will reject the string. Here is the machine:

Turing machine (TeX)

Turing machine (TeX)

We start in \(q_0\); if we see a blank character, then the string is empty, so we accept. If we see a 1, then the string starts with a 1; it can't be in the language, so we reject. If we see a 0, we replace it with a blank, move to the right, and change to state \(q_1\).

State \(q_1\)'s job is to skip over all the other 0's. If we see a blank, that means the string ends in 0, so we reject. If we see a 0, we leave it and keep going; if we see a 1, then we move to state \(q_2\).

State \(q_3\)'s job is to skip over all the 1's. If we see a 0, then there is some 0 after some 1, so the string should be rejected. If we see a blank, then we've gotten to the end of the string. We step back to the left (so that we are looking at the last 1), and transition to state \(q_3\); from there we replace the 1 with a blank, and then go to state \(q_4\).

\(q_4\)'s job is to rewind back to the beginning of the string: it keeps looping until it finds a blank character, then it moves one space back to the right, so that it is looking at the first character of the string. We then transition back to state \(q_0\) and start the process again.

Interesting facts about Turing machines

We don't have time to go into depth on Turing machines, but let me state several facts about Turing machines without proving them.

The halting problem

It is useful to know at least one unsolvable problem. This lets you prove other things are unsolvable by reducing them to the unsolvable problem. Here we will show the halting problem is solvable. In other classes you will make arguments like "if I could build a machine to check that this grammar is unambiguous, then I could use that machine to solve the halting problem; the halting problem is unsolvable, ergo, the ambiguity question is unsolvable as well"

Definition: The halting problem is the language \[L_{HP} = \{"M,x"\mid \text{$M$ on input $x$ halts}\}\]

Informally, a machine \(M_{HP}\) that decides the halting problem would be a machine that looks at the "source code" of another machine, and an input \(x\), and tells you whether \(M\) runs forever on input \(x\) or not.

Claim: The halting problem is recognizable.

Proof: Let \(M_{HP}\) do the following on input "\(M,x\)": first, run the universal Turing machine \(U\) to simulate \(M\) on input \(x\). If \(U\) halts and accepts, accept. If \(U\) halts and rejects, still accept (because \(M\) would have halted). If \(U\) runs forever, \(M\) will be running it forever; this means that \(M_{HP}\) halts and accepts any string \("M,x"\) where \(M\) halts on input \(x\), and on any other string, \(M_{HP}\) either runs forever or rejects (in particular, it runs forever).

Claim: The halting problem is undecidable.

Proof: This means there is no program that always halts and says yes on input "M,x" if and only if \(M\) on input \(x\) would halt. Proof by contradiction. Suppose there were some machine \(M_{HP}\) that decides the halting problem. We can build a diabolical machine \(M_D\) that does the following on input \(x\):

  1. Run \(M_{HP}\) on input "x,x". This interprets \(x\) as both the source code of a machine \(M_x\) and as the input. We are guaranteed that \(M_{HP}\) will halt, so we always get to the next step.
  2. If \(M_{HP}\) accepted, go into an infinite loop.
  3. If \(M_{HP}\) rejects, halt and accept.

Now, what happens if we run \(M_D\) on its own source code? Does it halt? If it does, that means that \(M_{HP}\) on input "\(M_D,M_D\)" will accept; which means \(M_D\) goes to step 2 and goes into an infinite loop. But that's a contradiction, because we said \(M_D\) halts. But if it doesn't halt, then \(M_{HP}\) will reject "\(M_D,M_D\); therefore \(M_D\) will go to step 3, and halt. But this is also a contradiction, because we assumed \(M_D\) will not halt. In either case, we get a contradiction, which means our initial assumption (that \(M_{HP}\) exists) must have been false.