reading: automata constructions, 1.1
Definition of a recognizable language
Proof that the union of recognizable languages is recognizable
Counting proof that there are unrecognizable languages
Definition: A language is a set of strings (i.e. a subset of \(Σ^*\))
Definition: A language \(L\) is DFA-recognizable if there exists a deterministic finite automaton \(M\) with \(L = L(M)\).
Last lecture, we constructed a machine with the intent that it would recognize the union of the languages of two given machines. This construction is covered in the reading; here we include the details of the proof of correctness.
We were given machines \(M_1 = (Q_1, Σ, δ_1, q_{01}, A_1)\) and \(M_2 = (Q_2, Σ, δ_2, q_{02},A_2)\) with the same alphabet \(Σ\). We constructed a machine \(M = (Q,Σ,δ,q_0,A)\) by defining
\[\begin{aligned} Q &:= Q_1 \times Q_2 \\ q_0 &:= (q_{01},q_{02}) \\ A &:= \{(q_1,q_2) \mid q_1 \in A_1 \text{ or } q_2 \in A_2\} \\ δ((q_1,q_2),a) &:= \left(δ_1(q_1,a), δ_2(q_2,a)\right) \end{aligned}\]
The intent is that \(L(M) = L(M_1) \cup L(M_2)\). The idea behind the construction is that if \(M\) processes \(x\) and ends in state \((q_1,q_2)\) then \(M_1\) would be in state \(q_1\) and \(M_2\) would be in state \(q_2\). In other words,
subclaim: for all \(x\), \(\hat{δ}(q_0,x) = \left(\hat{δ}_1(q_{01},x), \hat{δ}_2(q_{02},x)\right)\).
note: although it looks similar, the subclaim is different from the definition of \(δ\); \(δ\) gives us the individual transitions in the machine, while the subclaim says that the transition function behaves as expected while processing the entire string \(x\).
The proof that the subclaim implies the claim is given in the reading. Here, we give the inductive proof of the subclaim.
The proof is by induction on the structure of \(x\). Let \(P(x)\) be the statement "\(\hat{δ}(q_0,x) = \left(\hat{δ}_1(q_{01},x), \hat{δ}_2(q_{02},x)\right)\)." We must show \(P(ε)\) and \(P(xa)\) assuming \(P(x)\).
To prove \(P(ε)\), we must show that \(\hat{δ}(q_0,ε) = \left(\hat{δ}_1(q_{01},ε), \hat{δ}_2(q_{02},ε)\right)\). Plugging in the definition of \(\hat{δ}\), \(\hat{δ}_1\), and \(\hat{δ}_2\) shows that this is the same as showing \(q_0 = \left(q_{01}, q_{02}\right)\) but this is just the definition of \(q_{01}\).
To prove \(P(xa)\), first inductively assume \(P(x)\). Now, we have
\[\begin{aligned} \hat{δ}(q_0,xa) &= δ\left(\hat{δ}(q_0,x), a\right) && \text{by definition of $\hat{δ}$} \\ &= δ\left(\left(\hat{δ}_1(q_{01},x), \hat{δ}_2(q_{02},x)\right), a\right) && \text{by $P(x)$} \\ &= \left(δ_1(\hat{δ}_1(q_{01},x), a), δ_2(\hat{δ}_2(q_{02},x), a)\right) && \text{by definition of $δ$ above} \\ &= \left(\hat{δ}_1(q_{01},xa), \hat{δ}_2(q_{02},xa)\right) && \text{by definition of $\hat{δ}_1$ and $\hat{δ}_2$} \end{aligned}\]
This is exactly the statement of \(P(xa)\), so we have shown \(P(xa)\).
Discussion: This construction follows the same outline as the construction given in the last lecture.
Decide what each state should "mean"; i.e. if a string ends in a given state, what do we know about the string? For example, "if \(\hat{δ}(q_0,x) = q_{eo}\) then \(x\) has an even number of 0's and an odd number of 1's," or "if \(\hat{δ}(q_0,x) = (q_1,q_2)\) then \(x\) would bring \(M_1\) to state \(q_1\) and \(M_2\) to state \(q_2\)"
Prove that \(\hat{δ}\) does indeed match this specification. In both cases, we proved this by induction on the structure of \(x\), and the proofs mostly involved plugging in definitions.
Plug this into the definition of \(L(M)\) to ensure that the two languages are actually equal.
A language (set of strings) is like a specification, and a DFA is like an implementation. The language defines which strings the machine should accept, and the DFA gives rules for processing a string and deciding whether to accept it or not.
Is every specification implementable? A simple counting argument says no. The set of languages is the set of sets of strings, i.e. \(2^{\Sigma^*}\). Since \(\Sigma^*\) is infinite, \(2^{\Sigma^*}\) is uncountable.
Finite automata, on the other hand, are built out of finite sets. One can represent a set of states as a string, an alphabet as a string, the transition function as a string (it's a finite table, because the domain is finite), the initial state and set of final states can also be encoded as a string. Therefore, there are fewer DFAs than strings; in other words, the set of DFAs is countable.
If every language was recognizable, then the function taking a machine to its language would be surjective; but this is impossible because the set of machines is countable but the set of languages is uncountable.
We will see examples of unrecognizable languages in a future lecture.
This argument applies to any programming language you can think of. A program is just a special string (its source code), so there are fewer programs than strings. There are only countably many strings. Since there are an uncountable number of languages, there must be problems that can't be solved in your favorite programming language (or indeed, in any programming language).