Lecture 17: Bayes's rule, random variables

Law of total probabilty

Claim (Law of total probability): If B1, B2, …, Bn are mutually exclusive and Bi = S, then Pr(A)=∑Pr(A|Bi)Pr(Bi).

Proof: Use third axiom to write P(A)=∑Pr(A ∩ Bi); use defn. of Pr(A|Bi) to conclude Pr(A ∩ Bi)=Pr(A|Bi)Pr(Bi)

Bayes's rule

Claim (Bayes's rule): Pr(A|B)=Pr(B|A)Pr(A)/Pr(B).

Proof: Follows directly from definition of Pr(A|B) and Pr(B|A).

Alternate form: Pr(A|B)=Pr(B|A)Pr(A)/∑Pr(B|Ai)Pr(Ai)

Usually used with A1 = A and $A_2 = \bar{A}$.

Random variables

Definition: A (real-valued) Random variable X on a sample space S is a function X : S → ℝ.

Definition: If f : ℝ → ℝ is any function and f is a random variable, then we write f(X) to denote the random variable f(X):S → ℝ given by f(X)(s)=f(X(s)). Examples: sin(X), X2, X + Y and XY are all random variables. (X + Y)(s)=X(s)+Y(s).

Definition: If X is a random variable, then E(X)=∑s ∈ SX(s)Pr({s}).

Lemma: This definition is equivalent to E′(X)=∑x ∈ ℝxPr(X = x).

Proof: To get from E to E, group all of the terms that have the same X value. Details in lecture slides.


Definition: The probability density function (or probability distribution function) of a random variable X is the function PDFX : ℝ → ℝ given by PDFX(x)=Pr(X = x).

Definition: The cumulative distribution function of X is given by CDFX(x)=Pr(X ≤ x).