Lecture 17: Bayes's rule, random variables

• Law of total probability, Bayes's rule

• Random variables

• Lecture slides 71 -- 91 and 95 -- 99

• Board image

Law of total probabilty

Claim (Law of total probability): If B₁, B₂, …, B_n are mutually exclusive and ∪B_i = S, then Pr(A)=∑Pr(A|B_i)Pr(B_i).

Proof: Use third axiom to write P(A)=∑Pr(A ∩ B_i); use defn. of Pr(A|B_i) to conclude Pr(A ∩ B_i)=Pr(A|B_i)Pr(B_i)

Bayes's rule

Claim (Bayes's rule): Pr(A|B)=Pr(B|A)Pr(A)/Pr(B).

Proof: Follows directly from definition of Pr(A|B) and Pr(B|A).

Alternate form: Pr(A|B)=Pr(B|A)Pr(A)/∑Pr(B|A_i)Pr(A_i)

Usually used with A₁ = A and $A_2 = \bar{A}$.

Random variables

Definition: A (real-valued) Random variable X on a sample space S is a function X : S → ℝ.

Definition: If f : ℝ → ℝ is any function and f is a random variable, then we write f(X) to denote the random variable f(X):S → ℝ given by f(X)(s)=f(X(s)). Examples: sin(X), X², X + Y and XY are all random variables. (X + Y)(s)=X(s)+Y(s).

Definition: If X is a random variable, then E(X)=∑_s ∈ SX(s)Pr({s}).

Lemma: This definition is equivalent to E′(X)=∑_{x ∈ ℝ}xPr(X = x).

Proof: To get from E to E′, group all of the terms that have the same X value. Details in lecture slides.

Distributions

Definition: The probability density function (or probability distribution function) of a random variable X is the function PDF_X : ℝ → ℝ given by PDF_X(x)=Pr(X = x).

Definition: The cumulative distribution function of X is given by CDF_X(x)=Pr(X ≤ x).