On Bayes' Rule
===================================

Question 1: Which Card Box Was Selected?
----------------------------------------

**Question:** One of two card boxes is selected.

- Box :math:`B_1` is selected with probability :math:`\frac{2}{5}` and contains 4 red cards and 1 black card.
- Box :math:`B_2` is selected with probability :math:`\frac{3}{5}` and contains 1 red card and 4 black cards.

One card is drawn from the selected box.

Let :math:`R` be the event that the card is red.

Find:

(a) :math:`P(R\mid B_1)` and :math:`P(R\mid B_2)`;

(b) the overall probability :math:`P(R)`;

(c) :math:`P(B_1\mid R)`; and

(d) compare :math:`P(B_1)` with :math:`P(B_1\mid R)` and explain what the observed red card tells us.

**Solution:**
~~~~~~~~~~~~~

Part (a): Probability of red within each box
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

If Box :math:`B_1` is known to have been selected, 4 of its 5 cards are
red.

Therefore,

.. math::

   P(R\mid B_1)
   =
   \frac{4}{5}.

If Box :math:`B_2` is known to have been selected, only 1 of its 5 cards is
red.

Therefore,

.. math::

   P(R\mid B_2)
   =
   \frac{1}{5}.

Thus,

.. math::

   \boxed{
   P(R\mid B_1)=\frac{4}{5},
   \qquad
   P(R\mid B_2)=\frac{1}{5}
   }.

These are conditional probabilities. They answer:

    If I know which box was selected, how likely is a red card?

Part (b): Find the overall probability of red
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

A red card can occur through two possible paths:

.. math::

   B_1\rightarrow R

or

.. math::

   B_2\rightarrow R.

For the first path,

.. math::

   P(B_1\cap R)
   =
   P(B_1)P(R\mid B_1)
   =
   \frac{2}{5}\times\frac{4}{5}
   =
   \frac{8}{25}.

For the second path,

.. math::

   P(B_2\cap R)
   =
   P(B_2)P(R\mid B_2)
   =
   \frac{3}{5}\times\frac{1}{5}
   =
   \frac{3}{25}.

The two boxes form a partition, so the two paths do not overlap.

Using total probability,

.. math::

   P(R)
   =
   P(B_1)P(R\mid B_1)
   +
   P(B_2)P(R\mid B_2).

Therefore,

.. math::

   P(R)
   =
   \frac{8}{25}
   +
   \frac{3}{25}
   =
   \frac{11}{25}.

Thus,

.. math::

   \boxed{P(R)=\frac{11}{25}}.

Part (c): Find :math:`P(B_1\mid R)`
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Now the condition is reversed.

We observed a red card and want to know which box was likely selected.

Bayes' rule gives

.. math::

   P(B_1\mid R)
   =
   \frac{
      P(B_1)P(R\mid B_1)
   }{
      P(R)
   }.

The numerator is the desired path:

.. math::

   P(B_1)P(R\mid B_1)
   =
   \frac{8}{25}.

The denominator is the probability of all paths leading to red:

.. math::

   P(R)=\frac{11}{25}.

Therefore,

.. math::

   P(B_1\mid R)
   =
   \frac{8/25}{11/25}
   =
   \frac{8}{11}.

Thus,

.. math::

   \boxed{P(B_1\mid R)=\frac{8}{11}\approx0.727}.

Part (d): Compare the prior and posterior probabilities
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Before observing the card,

.. math::

   P(B_1)=\frac{2}{5}=0.40.

After observing a red card,

.. math::

   P(B_1\mid R)
   =
   \frac{8}{11}
   \approx0.727.

Therefore, the probability of Box :math:`B_1` increases:

.. math::

   0.40
   \quad\longrightarrow\quad
   0.727.

Why?

A red card is much more likely from Box :math:`B_1`:

.. math::

   P(R\mid B_1)=\frac{4}{5}

compared with

.. math::

   P(R\mid B_2)=\frac{1}{5}.

So observing red gives evidence in favor of Box :math:`B_1`.

.. note::

   Bayes' rule reverses the direction of the condition.

   We begin with probabilities such as

   .. math::

      P(R\mid B_1)

   and use the observed event :math:`R` to find

   .. math::

      P(B_1\mid R).


Question 2: Which Die Was Used?
-------------------------------

**Question:** One of three dice is selected.

- Die :math:`D_1` is selected with probability :math:`\frac{1}{2}`, and :math:`P(6\mid D_1)=\frac{1}{6}`.
- Die :math:`D_2` is selected with probability :math:`\frac{3}{10}`, and :math:`P(6\mid D_2)=\frac{1}{3}`.
- Die :math:`D_3` is selected with probability :math:`\frac{1}{5}`, and :math:`P(6\mid D_3)=\frac{1}{2}`.

The selected die is rolled once and a 6 is observed.

Find:

(a) the probability of each path leading to a 6;

(b) the overall probability of rolling a 6;

(c) :math:`P(D_1\mid 6)`, :math:`P(D_2\mid 6)`, and :math:`P(D_3\mid 6)`; and

(d) identify the most likely die after observing the 6 and explain why the answer is not determined only by the largest value of :math:`P(6\mid D_i)`.

**Solution:**
~~~~~~~~~~~~~

Part (a): Calculate each path to a 6
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

For Die :math:`D_1`,

.. math::

   P(D_1\cap 6)
   =
   P(D_1)P(6\mid D_1)
   =
   \frac{1}{2}\times\frac{1}{6}
   =
   \frac{1}{12}.

For Die :math:`D_2`,

.. math::

   P(D_2\cap 6)
   =
   \frac{3}{10}\times\frac{1}{3}
   =
   \frac{1}{10}.

For Die :math:`D_3`,

.. math::

   P(D_3\cap 6)
   =
   \frac{1}{5}\times\frac{1}{2}
   =
   \frac{1}{10}.

Thus, the three path probabilities are

.. math::

   \boxed{
   \frac{1}{12},
   \qquad
   \frac{1}{10},
   \qquad
   \frac{1}{10}
   }.

Notice that Die :math:`D_3` has the largest probability of producing a 6
once it has been selected, but it also has the smallest starting probability
of being selected.

Part (b): Find the overall probability of rolling a 6
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

A 6 can occur through any of the three dice.

Using total probability,

.. math::

   P(6)
   =
   P(D_1)P(6\mid D_1)
   +
   P(D_2)P(6\mid D_2)
   +
   P(D_3)P(6\mid D_3).

Therefore,

.. math::

   P(6)
   =
   \frac{1}{12}
   +
   \frac{1}{10}
   +
   \frac{1}{10}.

Using a common denominator of 60,

.. math::

   P(6)
   =
   \frac{5}{60}
   +
   \frac{6}{60}
   +
   \frac{6}{60}
   =
   \frac{17}{60}.

Thus,

.. math::

   \boxed{P(6)=\frac{17}{60}}.

Part (c): Find the posterior probability of each die
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

For Die :math:`D_1`,

.. math::

   P(D_1\mid 6)
   =
   \frac{P(D_1)P(6\mid D_1)}{P(6)}
   =
   \frac{1/12}{17/60}.

Therefore,

.. math::

   P(D_1\mid 6)
   =
   \frac{5}{17}.

For Die :math:`D_2`,

.. math::

   P(D_2\mid 6)
   =
   \frac{1/10}{17/60}
   =
   \frac{6}{17}.

For Die :math:`D_3`,

.. math::

   P(D_3\mid 6)
   =
   \frac{1/10}{17/60}
   =
   \frac{6}{17}.

Thus,

.. math::

   \boxed{
   P(D_1\mid 6)=\frac{5}{17},
   \qquad
   P(D_2\mid 6)=\frac{6}{17},
   \qquad
   P(D_3\mid 6)=\frac{6}{17}
   }.

As a check,

.. math::

   \frac{5}{17}
   +
   \frac{6}{17}
   +
   \frac{6}{17}
   =
   1.

This must happen because exactly one of the three dice was selected.

Part (d): Which die is most likely?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

After observing the 6,

.. math::

   P(D_2\mid 6)
   =
   P(D_3\mid 6)
   =
   \frac{6}{17}.

So Dice :math:`D_2` and :math:`D_3` are tied as the most likely sources.

This may seem surprising because

.. math::

   P(6\mid D_3)=\frac{1}{2}

is larger than

.. math::

   P(6\mid D_2)=\frac{1}{3}.

However, Die :math:`D_2` had a larger starting probability:

.. math::

   P(D_2)=\frac{3}{10}

compared with

.. math::

   P(D_3)=\frac{1}{5}.

Bayes' rule uses **both** pieces of information:

- how likely the case was before the observation; and
- how likely the observation is within that case.

.. note::

   Do not choose a case only because it has the largest conditional
   probability :math:`P(A\mid B_i)`.

   Bayes' rule uses the path probability

   .. math::

      P(B_i)P(A\mid B_i).


Question 3: Standard Deck or Special Deck?
------------------------------------------

**Question:** Before drawing a card, one of two decks is selected.

- A standard 52-card deck is selected with probability :math:`\frac{9}{10}`.
- A special 20-card deck is selected with probability :math:`\frac{1}{10}`.

The standard deck contains 4 aces. The special deck contains 5 aces.

One card is drawn from the selected deck and is found to be an ace.

Let

- :math:`S` = the standard deck was selected;
- :math:`T` = the special deck was selected;
- :math:`A` = an ace was drawn.

Find:

(a) :math:`P(A\mid S)` and :math:`P(A\mid T)`;

(b) the overall probability :math:`P(A)`;

(c) :math:`P(T\mid A)`; and

(d) compare :math:`P(T)` with :math:`P(T\mid A)` and explain why the posterior probability is not simply :math:`P(A\mid T)`.

**Solution:**
~~~~~~~~~~~~~

Part (a): Probability of an ace from each deck
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

For the standard deck,

.. math::

   P(A\mid S)
   =
   \frac{4}{52}
   =
   \frac{1}{13}.

For the special deck,

.. math::

   P(A\mid T)
   =
   \frac{5}{20}
   =
   \frac{1}{4}.

Therefore,

.. math::

   \boxed{
   P(A\mid S)=\frac{1}{13},
   \qquad
   P(A\mid T)=\frac{1}{4}
   }.

An ace is more likely if the special deck is used.

Part (b): Find the overall probability of an ace
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

An ace can be obtained through either deck.

Using total probability,

.. math::

   P(A)
   =
   P(S)P(A\mid S)
   +
   P(T)P(A\mid T).

Substitute the values:

.. math::

   P(A)
   =
   \frac{9}{10}\times\frac{1}{13}
   +
   \frac{1}{10}\times\frac{1}{4}.

Therefore,

.. math::

   P(A)
   =
   \frac{9}{130}
   +
   \frac{1}{40}.

Using a common denominator of 520,

.. math::

   P(A)
   =
   \frac{36}{520}
   +
   \frac{13}{520}
   =
   \frac{49}{520}.

Thus,

.. math::

   \boxed{P(A)=\frac{49}{520}}.

Part (c): Find :math:`P(T\mid A)`
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

We observed an ace and want the probability that the special deck was used.

Bayes' rule gives

.. math::

   P(T\mid A)
   =
   \frac{
      P(T)P(A\mid T)
   }{
      P(A)
   }.

The desired path is

.. math::

   P(T)P(A\mid T)
   =
   \frac{1}{10}\times\frac{1}{4}
   =
   \frac{1}{40}
   =
   \frac{13}{520}.

Therefore,

.. math::

   P(T\mid A)
   =
   \frac{13/520}{49/520}
   =
   \frac{13}{49}.

Thus,

.. math::

   \boxed{P(T\mid A)=\frac{13}{49}\approx0.265}.

Part (d): Explain the effect of the starting probability
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Before seeing the card,

.. math::

   P(T)=\frac{1}{10}=0.10.

After observing an ace,

.. math::

   P(T\mid A)
   =
   \frac{13}{49}
   \approx0.265.

The probability of the special deck increases because an ace is more likely
under the special deck.

However,

.. math::

   P(T\mid A)\neq P(A\mid T).

In fact,

.. math::

   P(A\mid T)=\frac{1}{4}=0.25,

while

.. math::

   P(T\mid A)\approx0.265.

More importantly, Bayes' rule does not use only :math:`P(A\mid T)`.

It also uses the starting probability

.. math::

   P(T)=0.10.

The standard deck was much more likely to be selected initially:

.. math::

   P(S)=0.90.

Therefore, many aces can still come from the standard-deck path even though
an ace is less likely within that deck.

.. note::

   The observed event updates the starting probability.

   It does not erase the starting probability.


Question 4: One Head Versus Two Heads
-------------------------------------

**Question:** One of two coins is selected.

- Coin :math:`C_1` is selected with probability :math:`0.70` and is fair, so :math:`P(H\mid C_1)=0.50`.
- Coin :math:`C_2` is selected with probability :math:`0.30` and has :math:`P(H\mid C_2)=0.80`.

The selected coin is tossed twice. Assume that the tosses are independent once the coin has been selected.

Find:

(a) :math:`P(HH\mid C_1)` and :math:`P(HH\mid C_2)`;

(b) the overall probability :math:`P(HH)`;

(c) :math:`P(C_2\mid HH)`; and

(d) first find :math:`P(C_2\mid H)` after observing only one head, then compare the three probabilities :math:`P(C_2)`, :math:`P(C_2\mid H)`, and :math:`P(C_2\mid HH)`.

**Solution:**
~~~~~~~~~~~~~

Part (a): Probability of two heads under each coin
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

For Coin :math:`C_1`, each toss has probability :math:`0.50` of heads.

Because the tosses are independent once the coin is known,

.. math::

   P(HH\mid C_1)
   =
   (0.50)(0.50)
   =
   0.25.

For Coin :math:`C_2`,

.. math::

   P(HH\mid C_2)
   =
   (0.80)(0.80)
   =
   0.64.

Thus,

.. math::

   \boxed{
   P(HH\mid C_1)=0.25,
   \qquad
   P(HH\mid C_2)=0.64
   }.

Two heads are much more likely under Coin :math:`C_2`.

Part (b): Find the overall probability of two heads
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The event :math:`HH` can occur through either coin.

Using total probability,

.. math::

   P(HH)
   =
   P(C_1)P(HH\mid C_1)
   +
   P(C_2)P(HH\mid C_2).

Therefore,

.. math::

   P(HH)
   =
   (0.70)(0.25)
   +
   (0.30)(0.64).

Calculate the two paths:

.. math::

   (0.70)(0.25)=0.175

and

.. math::

   (0.30)(0.64)=0.192.

Thus,

.. math::

   P(HH)
   =
   0.175+0.192
   =
   0.367.

Therefore,

.. math::

   \boxed{P(HH)=0.367}.

Part (c): Find :math:`P(C_2\mid HH)`
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Bayes' rule gives

.. math::

   P(C_2\mid HH)
   =
   \frac{
      P(C_2)P(HH\mid C_2)
   }{
      P(HH)
   }.

Substitute the values:

.. math::

   P(C_2\mid HH)
   =
   \frac{(0.30)(0.64)}{0.367}
   =
   \frac{0.192}{0.367}.

Therefore,

.. math::

   \boxed{
   P(C_2\mid HH)
   =
   \frac{192}{367}
   \approx0.523
   }.

Before any tosses, Coin :math:`C_2` had only a 0.30 probability of being
selected.

After observing two heads, it becomes slightly more likely than Coin
:math:`C_1`.

Part (d): Compare the effect of one head and two heads
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

First find the overall probability of one head on the first toss:

.. math::

   P(H)
   =
   P(C_1)P(H\mid C_1)
   +
   P(C_2)P(H\mid C_2).

Therefore,

.. math::

   P(H)
   =
   (0.70)(0.50)
   +
   (0.30)(0.80)
   =
   0.35+0.24
   =
   0.59.

Now apply Bayes' rule:

.. math::

   P(C_2\mid H)
   =
   \frac{
      (0.30)(0.80)
   }{
      0.59
   }.

Thus,

.. math::

   P(C_2\mid H)
   =
   \frac{0.24}{0.59}
   =
   \frac{24}{59}
   \approx0.407.

We now have

.. math::

   P(C_2)=0.30,

.. math::

   P(C_2\mid H)\approx0.407,

and

.. math::

   P(C_2\mid HH)\approx0.523.

Therefore,

.. math::

   0.30
   \quad\longrightarrow\quad
   0.407
   \quad\longrightarrow\quad
   0.523.

One head increases the probability that Coin :math:`C_2` was selected.

A second head provides more evidence in the same direction because heads
are more likely under Coin :math:`C_2`.

.. note::

   Repeated observations can strengthen the update when the observations
   are more consistent with one possible case than another.


Question 5: An Odd Result from One of Two Dice
----------------------------------------------

**Question:** One of two dice is selected.

- Die :math:`D_1` is selected with probability :math:`\frac{2}{5}` and is fair.
- Die :math:`D_2` is selected with probability :math:`\frac{3}{5}` and is loaded so that the probability of an even result is :math:`\frac{5}{6}`.

The selected die is rolled once and an odd number is observed.

Let :math:`O` be the event that the result is odd.

Find:

(a) :math:`P(O\mid D_1)` and :math:`P(O\mid D_2)`;

(b) the overall probability :math:`P(O)`;

(c) :math:`P(D_1\mid O)`; and

(d) :math:`P(D_2\mid O)` and explain how observing an odd result changes the relative likelihood of the two dice.

**Solution:**
~~~~~~~~~~~~~

Part (a): Probability of an odd result under each die
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Die :math:`D_1` is fair.

Three of the six possible results are odd:

.. math::

   1,3,5.

Therefore,

.. math::

   P(O\mid D_1)
   =
   \frac{3}{6}
   =
   \frac{1}{2}.

For Die :math:`D_2`, we are given

.. math::

   P(\text{even}\mid D_2)=\frac{5}{6}.

Odd and even are complementary events.

Therefore,

.. math::

   P(O\mid D_2)
   =
   1-\frac{5}{6}
   =
   \frac{1}{6}.

Thus,

.. math::

   \boxed{
   P(O\mid D_1)=\frac{1}{2},
   \qquad
   P(O\mid D_2)=\frac{1}{6}
   }.

An odd result is three times as likely under Die :math:`D_1` as under
Die :math:`D_2`.

Part (b): Find the overall probability of an odd result
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

An odd result can occur through either die.

Using total probability,

.. math::

   P(O)
   =
   P(D_1)P(O\mid D_1)
   +
   P(D_2)P(O\mid D_2).

Substitute the values:

.. math::

   P(O)
   =
   \frac{2}{5}\times\frac{1}{2}
   +
   \frac{3}{5}\times\frac{1}{6}.

The two path probabilities are

.. math::

   \frac{2}{5}\times\frac{1}{2}
   =
   \frac{1}{5}

and

.. math::

   \frac{3}{5}\times\frac{1}{6}
   =
   \frac{1}{10}.

Therefore,

.. math::

   P(O)
   =
   \frac{1}{5}
   +
   \frac{1}{10}
   =
   \frac{3}{10}.

Thus,

.. math::

   \boxed{P(O)=\frac{3}{10}}.

Part (c): Find :math:`P(D_1\mid O)`
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Bayes' rule gives

.. math::

   P(D_1\mid O)
   =
   \frac{
      P(D_1)P(O\mid D_1)
   }{
      P(O)
   }.

Therefore,

.. math::

   P(D_1\mid O)
   =
   \frac{1/5}{3/10}
   =
   \frac{2}{3}.

Thus,

.. math::

   \boxed{P(D_1\mid O)=\frac{2}{3}}.

Part (d): Find :math:`P(D_2\mid O)` and interpret the update
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Using Bayes' rule,

.. math::

   P(D_2\mid O)
   =
   \frac{
      P(D_2)P(O\mid D_2)
   }{
      P(O)
   }.

Therefore,

.. math::

   P(D_2\mid O)
   =
   \frac{1/10}{3/10}
   =
   \frac{1}{3}.

Thus,

.. math::

   \boxed{P(D_2\mid O)=\frac{1}{3}}.

Before the roll,

.. math::

   P(D_1)=\frac{2}{5}=0.40

and

.. math::

   P(D_2)=\frac{3}{5}=0.60.

So Die :math:`D_2` was initially more likely to have been selected.

After observing an odd number,

.. math::

   P(D_1\mid O)=\frac{2}{3}

and

.. math::

   P(D_2\mid O)=\frac{1}{3}.

The observation reverses which die is more likely.

Why?

An odd result is much more compatible with Die :math:`D_1`:

.. math::

   P(O\mid D_1)=\frac{1}{2}

compared with

.. math::

   P(O\mid D_2)=\frac{1}{6}.

The observation therefore provides evidence in favor of Die
:math:`D_1`.

As a check,

.. math::

   P(D_1\mid O)+P(D_2\mid O)
   =
   \frac{2}{3}+\frac{1}{3}
   =
   1.

.. note::

   A Bayes' rule problem combines three ideas:

   1. multiply within each possible path;
   2. add all paths leading to the observed event;
   3. divide the desired path by the total probability of the observed
      event.