Module 6: Elementary Probability - Computing with Events

• The probability that the first card is a club given that the second is a club.
• The probability that the first card is a diamond given that the second is a club.

Perhaps the most important term:

• An event is a subset of the sample space
            \(\Rightarrow\) Any subset is an event.

Correspondence between events and "word problems":

• Consider this problem: what is the probability of obtaining exactly 2 heads in three flips of a coin?
  • What is the sample space?
              \(\Omega \eql \{(H,H,H), (H,H,T), (H,T,H), (H,T,T), (T,H,H), (T,H,T), (T,T,H), (T,T,T)\}\)
  • What is the event of interest?
              Let \(E \eql \{(H,H,T), (H,T,H), (T,H,H)\} \)

• Example: what is the probability of obtaining an odd number in a die roll?
  • What is the sample space?
              \(\Omega \eql \{1, 2, 3, 4, 5, 6\} \)
  • What is the event of interest?
              Let \(E \eql \{1, 3, 5\} \)

• The "occurrence" of an event:
  • Consider a die roll and the event \(E = \{1, 3, 5\} \)
  • Suppose you roll the die and "3" shows up
              \(\Rightarrow\) Event \(E\) occurred (because one of its outcomes occurred).

Probability:

• More precisely, a probability measure.

• A probability measure on a sample space \(\Omega\) is:
  • A collection of numbers, one number for each event.
              Notation: \(\prob{E}\) is the "number" associated with event \(E\)
  such that the following "rules" are satisfied:
  • (Axiom 1)   \(\prob{\Omega} = 1\).
  • (Axiom 2)   \(0 \leq \prob{E} \leq 1\) for any event \(E\).
  • (Axiom 3)   For disjoint events \(A\) and \(B\), $$\prob{A \cup B} \eql \prob{A} + \prob{B}$$

• Alternative notation: \(\mbox{Pr}\{E\}\) (for event \(E\)).

• Example: Consider a single coin flip
  • Sample space \(\Omega=\{H,T\}\)
  • We'd have to specify a number for each possible event:
              \(\prob{H} = ?\)
              \(\prob{T} = ?\)
              \(\prob{\Omega} = 1\)     (always 1, by definition)
              \(\prob{\emptyset} = ?\)     (empty subset)
  • For example:
              \(\prob{H} = 0.6\)
              \(\prob{T} = 0.4\)
              \(\prob{\Omega} = 1\)
              \(\prob{\emptyset} = 0\)     (must be zero)

• Note: we have specified the numbers as a (complete) list.
            \(\Rightarrow\) One number for each event.

• How do we know the above measure is a valid measure (satisfies the axioms)?

  We can laboriously check:

  • Axiom 1: \(\prob{\Omega} = 1\)
              \(\Rightarrow\) Satisfied.
  • Axiom 2: \(0 \leq \prob{E} \leq 1\) for any event \(E\).
              \(\Rightarrow\) Satisfied for each event in list.
  • Axiom 3: For disjoint events \(A\) and \(B\), \(\prob{A\cup B} = \prob{A} + \prob{B}\)
              \(\Rightarrow\) We need to check for each possible combination of disjoint events.
              e.g., with \(A=\{H\}, B=\{T\}\) $$ \prob{A\cup B} \eql \prob{\{H,T\}} \eql \prob{\Omega} \eql 1 \eql 0.6 + 0.4 \eql \prob{A} + \prob{B} $$

Operations on events:

• The usual set operators apply to events, since events are sets.

• \(A' = \Omega - A\) = complement of \(A\).

• \(A\) and \(B\) = \(A \cap B\) = intersection of event \(A\) and event \(B\).

Other ways of specifying probability measures:

• For a die-roll, specify as follows:
            \(\prob{E} = \frac{|E|}{6}\), for any singleton event \(E\).
  • For the die-roll, the singleton events are $$ \{1\}, \;\;\;\; \{2\}, \;\;\;\; \{3\}, \;\;\;\; \{4\}, \;\;\;\; \{5\}, \;\;\;\; \{6\}, \;\;\;\; $$
  • Or, in shorter form, the six events \(\{i\}\) for \(i=1,2,\ldots,6\).
  • By assuming (or declaring) that $$ \prob{\{i\}} \eql \frac{1}{6} $$ we can build the rest of the probability measure.

• Note: this achieves two purposes
  • Compact representation (instead of full list of events).
  • Assumes Axiom 3 is satisfied
              \(\Rightarrow\) Axiom 3 is used by construction
              e.g., \(\prob{\{1,3,5\}} = \prob{\{1\}} + \prob{\{3\}} + \prob{\{5\}} = \frac{1}{6} + \frac{1}{6} + \frac{1}{6} = \frac{3}{6}\)

• Similarly, for a single card drawing:
            \(\prob{E} = \frac{|E|}{52}\), for any singleton event \(E=\{i\}\) (a particular card).
  • For a larger subset like \(E'=\{0,1,...,12\}\) $$\eqb{ \prob{E'} \eqt \prob{\{0,1,\ldots,12\}} \\ \eqt \prob{\{0\}} + \prob{\{1\}} + \ldots + \prob{\{12\}} \;\;\;\;\;\;\mbox{using Axiom 3} \\ \eqt \frac{1}{52} + \frac{1}{52} + \ldots + \frac{1}{52}\\ \eqt \frac{13}{52} }$$
  • Thus, \(\prob{E'} = \frac{|E'|}{52}\), for any event \(E'\)

• Notation: we will occasionally use an informal event description (in English) e.g.,
            \(\probm{draw a spade} = \frac{13}{52}\).

Simple and complex experiments:

• Consider two die rolls:
  • Sample space: $$\eqb{ \Omega & \eql & \{(1,1), (1,2), (1,3), (1,4), (1,5), (1,6),\\ & & (2,1), (2,2), (2,3), (2,4), (2,5), (2,6), \\ & & (3,1), (3,2), (3,3), (3,4), (3,5), (3,6), \\ & & (4,1), (4,2), (4,3), (4,4), (4,5), (4,6), \\ & & (5,1), (5,2), (5,3), (5,4), (5,5), (5,6), \\ & & (6,1), (6,2), (6,3), (6,4), (6,5), (6,6)\} }$$
  • Thus, there are \(2^{36}\) possible events.
  • The convenient way to specify: \(\prob{E} = \frac{|E|}{36}\), for each single event \(E\).
  • For example: \(\prob{(4,3)} = \frac{1}{36}\)
  • Then, one can calculate other probabilities.
  • Consider the event "same number on both dice".
  • This corresponds to the subset (event): $$ \{(1,1), \;(2,2), \;(3,3), \;(4,4), \;(5,5), \;(6,6)\}$$
  • Which can be written as the union of disjoint singleton events: $$ \{(1,1)\} \;\cup\; \{(2,2)\} \;\cup\; \{(3,3)\} \;\cup\; \{(4,4)\} \;\cup\; \{(5,5)\} \;\cup\; \{(6,6)\} $$
  • Therefore, since \(\prob{\{(i,i)\}} = \frac{1}{36}\) (by assumption) $$\eqb{ \probm{same number on both dice} & \eql & \prob{ \{(1,1), \;(2,2), \;(3,3), \;(4,4),\;(5,5), \;(6,6)\} } \\ & \eql & \prob{ \{(1,1)\} \;\cup\; \{(2,2)\} \;\cup\; \{(3,3)\} \;\cup\; \{(4,4)\} \;\cup\; \{(5,5)\} \;\cup\; \{(6,6)\} } \\ & \eql & \prob{\{(1,1)\}} \plss \prob{\{(2,2)\}}, \plss \prob{\{(3,3)\}}, \plss \prob{\{(4,4)\}} \plss \prob{\{(5,5)\}} \plss \prob{\{(6,6)\}} \\ & \eql & \frac{1}{36} \ops{+} \frac{1}{36} \ops{+} \frac{1}{36} \ops{+} \frac{1}{36} \ops{+} \frac{1}{36} \ops{+} \frac{1}{36}\\ & \eql & \frac{6}{36} }$$

• More complex experiments can be constructed out of simpler ones.

A minor tweak to Axiom 3:

• Since $$\prob{A \cup B} \eql \prob{A} + \prob{B}$$ it's true that $$\prob{A \cup B \cup C} \eql \prob{A} + \prob{B} + \prob{C}$$

• In general, $$ \prob{A_1 \cup A_2 \cup \ldots \cup A_n} \eql \prob{A_1} + \prob{A_2} + \ldots + \prob{A_n} $$

• But consider a sample space like this: \(\Omega = \{1, 2, 3, ...\}\) (countably infinite)
  • Let event \(A_i=\{i\}\).
  • Let event $$\eqb{ A & \eql & \mbox{ union of all } A_i's \\ & \eql & A_1 \cup A_2 \cup \ldots \\ & \eql & \bigcup_{i=1}^\infty A_i }$$
  • Then, the event \(A\) makes sense and must be accounted for.
  • Another example: the event $$ A' \eql A_1 \cup A_3 \cup A_5 \cup \ldots $$ (the odd numbers)

• Countable additivity axiom: $$\eqb{ \prob{A} & \eql & \prob{A_1} + \prob{A_2} + \ldots \\ & \eql & \sum_{i=1}^\infty \prob{A_i} }$$ Or $$ \prob{\bigcup_{i=1}^\infty A_i} \eql \sum_{i=1}^\infty \prob{A_i} $$
            \(\Rightarrow\) Axiom 3 needs to hold for countable unions of disjoint events.

The meaning of probability: the frequentist interpretation

• Consider an experiment, an event \(E\) and the number \(\prob{E}\).

• Suppose we are able to repeat the experiment any number of times.

• Let \(f(n) =\) number of times event \(E\) occurs in \(n\) repetitions (trials).

• Then, $$ \lim_{n\to\infty} \frac{f(n)}{n} \eql \prob{E} $$

6.6   Conditional probability

About conditional probability:

• The theory developed so far can be used to solve many simple word problems.

• However, it does not account for questions of this sort:
            Given that the outcome of a die-roll is odd, what then is the probability that the outcome is \(3\)?

• Unfortunately, there does not seem to be an event corresponding to the above item of interest.

        ...
        double numBuses = 0;
        while (arrivalTime < myArrivalTime) {
            busStop.nextBus ();
            arrivalTime = busStop.getArrivalTime ();
            numBuses ++;
        }
        ...