Summary notes for lecture 1
- What is probability? If the universe of possibilities is U and the set of things I care about is C, then the
probability that something from C will happen = (size of C)/(size of U). For example,
- if U = ordinary deck of playing cards (no jokers) and C = picture cards, then the probability that
a picture card will be selected at random from a deck of cards = (12)/(52) = 3/13.
- if U are all the three letter passwords and C = your three letter password, then the probability
that I'll guess your password is (1)/(26 cubed), ignoring non-letters and differences between
upper and lower case.
- What about probabilities when there's an infinite universe, and is it relevant? When playing with words, there
are only finitely many words. When dealing with music, there are in some sense only finitely many notes,
though simultanaety allows for more complex constructions than typically occurs with words. However, when
doing visual art, there's an infinitude of angles, patterns, colours, etc., so employing randomness even in
a controlled way will mean working with infinities. So are all inifinities the same?
- Let N be the set of natural numbers, ie, 0, 1, 2, 3, 4, 5, ... There are clearly infinitely many of
these, but for sake of argument let's call this size of inifinity A. (We often say that N is
'countable' for the obvious reason that we can count the numbers!)
- When comparing the sizes of two collections of things, say X and Y, we could try pairing things off
by grabbing one thing from X and one from Y, and say that if we run out of things in X before we've
finished Y then X <= Y. That if Y runs out first then X >= Y, and that if we can find a way to show
X <= Y and X >= Y then we must conclude that X and Y have the same size!
- Let Z be all the integers, ie, ..., -3, -2, -1, 0, 1, 2, 3, ... At first sight Z is twice the size
of N, so certainly N <= Z. However, consider the following matching game: pair 0 in N with 0 in Z,
1 in N with 1 in Z, 2 with 1, 3 with -2, 4 with 2, 5 with -3, 6 with 3, etc.. It's clear that we'll
never run out of things in N as we march slowly up through Z, so at least N >= Z, which coupled
with our earlier observation means that N and Z have the same size.
- Let L be the set of points on an infinite sheet of graph paper where the grid lines intersect. So
each of those points will have a pair of coordinates (x,y) defining its location, eg, (2,3) is at
2 along the x axis and 3 up with y axis. At first sight, since there are countably many integers,
there must be countably many pairs looking like (1,n) for n in Z. There must also be countably
many pairs (2,n), and countably many pairs (3,n), etc.. So the there are countably many sets of
coutably many pairs, which smells like countable squared, so certainly L >= N. However, let's
play the following matching game: pair 0 in N with (0,0) in L, pair 1 in N with (1,0) in L,
2 with (1,1), 3 with (0,1), 4 with (-1,1), 5 with (-1,0), 6 with (-1,-1), 7 with (0,-1),
8 with (1,-1), 9 with (2,0), 10 with (2,1), 11 with (2,2), 12 with (1,2), etc., spiralling around
and progressively outwards. Clearly we won't run out of counting numbers, so at least N >= L, but
coupling that with what we observed earlier means that N and L have the same size!
- Let Q be the set of all fractions (aka rational numbers). Since there are infinitely many rationals
between 0 and 1, infinitely many between 1 and 2, etc., it's very clear that Q >= N. However, it's
easy to link the fraction p/q in Q with a pair (p,q) in L; indeed, since (2p)/(2q) = p/q is just
one example of the many 'fractions' which actually have the same value, and since in our pairing
(2p,2q) is different from (p,q), it's not hard to see that Q <= L. Yet in the previous example we
saw that L has the same size as N, so then Q <= N, which coupled with our earlier observation means
that Q and N actually have the same size!!!!
- Perhaps all this means that all infinities are the same size. Actually this is not the case. Let R
be all the real numbers ... all possible numbers along the x axis. The previous example would make
us doubt the simplicity of saying that there are infinitely many real numbers between 0 and 1, between
1 and 2, etc, and so claiming that there are more real numbers than integers. Although that argument
won't work, we can try a very different flavour of reasoning. This takes the approach of challenge
and response, namely that 'I' challenge 'you' to deliver a pairing of N with R and then 'I' show 'you'
where you failed to deliver. Clearly I can't take the time to go through every conceivable way of
pairing N and R, destroying each proposed pairing one by one!! Instead, I show a way of taking any
proposed list, and show how to construct a number which you did not include in your list (even though
it 'should' have been there). Let a_ij be a digit, ie, an integer between 0 and 9. My challenge is
for 'you' to present a list pairing N with all the decimals between 0 and 1. Let's suppose that 'you'
give me your proposed list, namely 1 with 0.a_11 a_12 a_13 a_14 ... , 2 with 0.a_21, a_22 a_23 a_24 ... ,
3 with 0.a_31 a_32 a_33 a_34 ... , etc.. (This could look like 1 with 0.1234567, 2 with 0.32445648,
3 with 0.638824975, etc..) So 'you' claim that your list contains ALL the decimals between 0 and 1.
I'll now exhibit to you a number x that you missed, and I construct that number as follows. It looks
like 0.b_1 b_2 b_3 b_4 b_5 ... , as it must, since it has to be between 0 and 1. If I write (mod 10)
to mean the reaminder on dividing by 10, then 3(mod 10) = 3, 25(mod 10) = 5, etc.. So I build the
missing number by choosing b_1 = a_11 + 5 (mod 10), b_2 = a_22 + 5 (mod 10), b_3 = a_33 (mod 10), etc..
(So our concrete example in parentheses earlier would give x = 0.681...... since 6 = 1+5, 8 = 3+5, and
1 = 6+5 (mod 10).) Notice that x can't be the first number in your list (the first digit is wrong),
it can't be the second one (the second digit is wrong), it can't be the third one (the third digit is
wrong), etc.. Hence it can't be any of the numbers in your list, hence you failed to list ALL the
decimals between 0 and 1. Since this process will work no matter what list you give me, it must be
impossible to create such a list, so R is seriously larger than N (and Z and Q and L). We typically
say that R is uncountable, and use some Hebrew letters to denote the various sizes of infinity, so
the size of N (and Z and L and Q) is Aleph_0, and the size of R is Aleph_1.
So what this means for probabilities is that, for example, giving a 'number' at random from all the real
numbers ... the prob that the number will be an integer is (size Z) / (size R) = 0, since the infinite
set of integers is negligible compared with the infinited set of real numbers. Likewise the prob that
the number will be a fraction is (size Q) / (size R) = 0. The prob that the number will be irrational
is (size R - size Q) / (size R) = 1. From this we can see that prob = 0 does not mean impossible and
that prob = 1 doesn't mean certain, they just mean that the prob that they won't mean what they seem to
mean is negligible!
- Enough maths for day one! What about art? Well, we'll start with poetry. We can certainly try writing a
poem where each word is chosen at random from some bucket of words, but the result isn't likely to be
very interesting. If however we build in some structure to allow us to express some biases, then we might
have better success. There are many ways to do this, but we'll take parts of speach as an example. Let's
take 3 buckets, one holding 3 nouns, another with 4 verbs, and the last with 5 adjectives. Then build
a 3x3 grid holding probabilities.
N V A
N .1 .3 .3
V .5 .2 .1
A .4 .5 .6
We'll say that if we've just used a noun then we're 'in state N',
similarly for state V and state A, and we'll use the grid to help us transition into the next state. So
for example, if we're in state N then the probabilities that our next state will be N, V, or A are .1,
.5, or .4 respectively (ie, if we've just used a noun, then we have a 10% chance of the next word being
a noun, a 50% chance of it being a verb, and a 40% chance of it being an adjective). If then our three
buckets were N = {cat, balloon, fish}, V = {jump, fly, drift, sing}, A = {soft, blue, subtle, small, round},
it would be easy to imagine a resulting poem being
cat drift subtle subtle blue balloon
small fish sing soft round cat fly round
balloon jump soft
soft small fish blue
jump round fish cat subtle blue soft balloon
Such a grid is called a Markov transition matrix, and the procedure is called a stochastic process or
Markov chain.
A convenient trick to pick the needed random numbers is either to make a ten-sided polygon out of card and
use a toothpick to make it into a spinning top (or have, in our case above, three triagular tops with sides
having the right proportional lengths -- hence a biased top), or use a table such as the one below and get
a friend to give you integers between 0 and 9 at random:
N V A
0 N N N
1 V n n
2 v n n
3 v V V
4 v v A
5 v A a
6 A a a
7 a a a
8 a a a
9 a a a
Homework: make a fully random poem, and then choose an interesting structure (not necessarily parts of
speech, perhaps rhythm, or syllables, or consonant sounds) to use to build a Markov poem. You may do this
in any language, but do be sure to be able to describe your process (and your scrapped failures!) as well
as being prepared to read your poems aloud in class. Due next Tuesday.