1.8 Total Probability
=====================

:bolditalic:`Key Terms`

:bolditalic:`Partition（分割）`
   A group of non-overlapping events that together cover the whole sample space.

:bolditalic:`Total probability（全機率）`
   A method for finding an overall probability by combining probabilities from all possible cases.

:bolditalic:`Case（情況）`
   One possible situation or branch in a partition.

:bolditalic:`Path probability（路徑機率）`
   The probability of one complete route in a probability tree.


1.8.2 Learning Outcomes
-----------------------

After completing this section, students should be able to:

- explain what a partition is;
- identify whether several events form a partition of a sample space;
- explain why the total probability rule works;
- use the total probability rule for several cases;
- use the two-case form with :math:`B` and :math:`B^c`;
- organize a total probability problem using a tree diagram;
- multiply probabilities along one path;
- add different paths that lead to the same final event;
- distinguish a conditional probability within one case from an overall probability;
- choose the correct steps for a basic total probability problem.

1.8.3 From Splitting One Event to Several Cases
-----------------------------------------------

In Section 1.5, we used the idea

.. math::

   P(A)
   =
   P(A\cap B)
   +
   P(A\cap B^c).

This works because :math:`B` and :math:`B^c` divide the sample space into
two non-overlapping cases.

Every outcome is either in :math:`B` or in :math:`B^c`.

Therefore, event :math:`A` can occur in one of two ways:

.. math::

   A\cap B

or

.. math::

   A\cap B^c.

The same splitting idea is shown visually below with the event labels
reversed. In the figure, :math:`A` and :math:`A^c` form the two possible
cases, while :math:`B` is the event being split.

.. image:: ../_static/figures/probability/figure_1_8_1_event_split_by_A_and_Ac.png
   :width: 92%
   :align: center
   :alt: Event B split into the two non-overlapping parts B intersection A and B intersection A complement.

Every outcome in :math:`B` must lie either inside :math:`A` or inside
:math:`A^c`. Therefore,

.. math::

   B
   =
   (B\cap A)
   \cup
   (B\cap A^c).

The two pieces do not overlap, so

.. math::

   P(B)
   =
   P(B\cap A)
   +
   P(B\cap A^c).

The letters are different from the equation above, but the structure is the
same: a complete event is divided into non-overlapping pieces according to
the possible cases.

This picture gives the basic geometric idea behind total probability. The
next step is to extend the same split from two cases to several cases.

The total probability idea extends this same reasoning from two cases to
several cases.

Suppose the possible cases are

.. math::

   B_1,B_2,\ldots,B_k.

To use total probability, these cases must form a **partition** of the
sample space.

1.8.4 Partition
---------------

A collection of events

.. math::

   B_1,B_2,\ldots,B_k

forms a partition of the sample space :math:`S` when two conditions hold.

First, the events do not overlap.

For different cases,

.. math::

   B_i\cap B_j=\varnothing,
   \qquad i\neq j.

This means that two different cases cannot occur at the same time.

Second, the cases together cover the whole sample space:

.. math::

   B_1\cup B_2\cup\cdots\cup B_k=S.

This means that one of the cases must occur.

.. note::

   **Partition**

   A partition is a collection of cases that is

   - non-overlapping; and
   - exhaustive.

   Therefore, exactly one case must occur.

Because the cases form the entire sample space,

.. math::

   P(B_1)+P(B_2)+\cdots+P(B_k)=1.

1.8.5 Example 1: Roll One Fair Die
----------------------------------

Suppose one fair die is rolled.

Define

.. math::

   B_1=\{1,2\},

.. math::

   B_2=\{3,4\},

and

.. math::

   B_3=\{5,6\}.

These events do not overlap.

For example,

.. math::

   B_1\cap B_2=\varnothing.

The same is true for every other pair.

Also,

.. math::

   B_1\cup B_2\cup B_3
   =
   \{1,2,3,4,5,6\}
   =
   S.

Therefore,

.. math::

   B_1,B_2,B_3

form a partition of the sample space.

Each die result belongs to exactly one of the three cases.

This is the basic structure needed for the total probability rule.

1.8.6 How an Event Is Split Across a Partition
----------------------------------------------

Now suppose another event :math:`A` can occur under any of the cases

.. math::

   B_1,B_2,\ldots,B_k.

Because exactly one case must occur, event :math:`A` can be separated into
the pieces

.. math::

   A\cap B_1,\quad
   A\cap B_2,\quad
   \ldots,\quad
   A\cap B_k.

Together,

.. math::

   A
   =
   (A\cap B_1)
   \cup
   (A\cap B_2)
   \cup
   \cdots
   \cup
   (A\cap B_k).

The figure below shows this same structure for four cases. It uses
:math:`E_1,E_2,E_3,E_4` for the partition and :math:`B` for the event of
interest.

.. image:: ../_static/figures/probability/figure_1_8_2_total_probability_partition.png
   :width: 92%
   :align: center
   :alt: Event B divided into four non-overlapping pieces by the partition E1, E2, E3, and E4.

The four case regions cover the whole sample space, so every point of
:math:`B` must belong to exactly one of them. The event is therefore split
into

.. math::

   B\cap E_1,\quad
   B\cap E_2,\quad
   B\cap E_3,\quad
   B\cap E_4,

and

.. math::

   B
   =
   (B\cap E_1)
   \cup
   (B\cap E_2)
   \cup
   (B\cap E_3)
   \cup
   (B\cap E_4).

The notation in the figure is equivalent to the notation used in this
section:

.. math::

   E_i\ \longleftrightarrow\ B_i,
   \qquad
   B\ \longleftrightarrow\ A.

Visually, no part of the event is counted twice because the partition cases
do not overlap, and no part is missed because the cases cover the whole
sample space.

These pieces do not overlap because the :math:`B_i` events do not overlap.

Therefore, their probabilities can be added:

.. math::

   P(A)
   =
   \sum_{i=1}^{k}P(A\cap B_i).

From the multiplication rule,

.. math::

   P(A\cap B_i)
   =
   P(B_i)P(A\mid B_i).

Substituting this into the previous expression gives the total probability
rule.

1.8.7 Total Probability Rule
----------------------------

If

.. math::

   B_1,B_2,\ldots,B_k

form a partition of the sample space, then

.. math::

   P(A)
   =
   \sum_{i=1}^{k}
   P(B_i)P(A\mid B_i).

.. note::

   **Total Probability Rule**

   .. math::

      P(A)
      =
      \sum_{i=1}^{k}
      P(B_i)P(A\mid B_i).

The structure is important.

For each case:

.. math::

   P(B_i)P(A\mid B_i)

is the probability of following that case and then reaching event
:math:`A`.

So the method is:

    multiply within each case

    then add the cases

Another useful way to write the idea is:

    case probability
    x
    probability of :math:`A` within that case
    =
    probability of that path to :math:`A`

Then add all paths that lead to :math:`A`.

1.8.8 Example 2: Choose One of Two Coins
----------------------------------------

One of two coins is chosen at random.

- :math:`C_1` is a fair coin.
- :math:`C_2` is a double-headed coin.

Each coin is selected with probability

.. math::

   P(C_1)=\frac{1}{2}

and

.. math::

   P(C_2)=\frac{1}{2}.

Let :math:`H` be the event of getting heads.

For Coin 1,

.. math::

   P(H\mid C_1)=\frac{1}{2}.

For Coin 2,

.. math::

   P(H\mid C_2)=1.

There are two possible ways to get heads.

**Path 1: Choose Coin 1, then get heads**

Using the multiplication rule,

.. math::

   P(C_1\cap H)
   =
   P(C_1)P(H\mid C_1).

Therefore,

.. math::

   P(C_1\cap H)
   =
   \frac{1}{2}\times\frac{1}{2}
   =
   \frac{1}{4}.

**Path 2: Choose Coin 2, then get heads**

.. math::

   P(C_2\cap H)
   =
   P(C_2)P(H\mid C_2).

Therefore,

.. math::

   P(C_2\cap H)
   =
   \frac{1}{2}\times1
   =
   \frac{1}{2}.

The two paths cannot occur together because only one coin is selected.

Therefore, add the path probabilities:

.. math::

   P(H)
   =
   P(C_1\cap H)
   +
   P(C_2\cap H).

Thus,

.. math::

   P(H)
   =
   \frac{1}{4}
   +
   \frac{1}{2}
   =
   \frac{3}{4}.

The answer :math:`3/4` is the **overall probability** of heads.

It is not the probability of heads for Coin 1 or Coin 2 separately.

The conditional probabilities

.. math::

   P(H\mid C_1)

and

.. math::

   P(H\mid C_2)

describe heads within each individual case.

Total probability combines these cases into one overall probability.

1.8.9 Why the Case Probabilities Matter
---------------------------------------

A common mistake is to add the conditional probabilities directly.

For the coin example, this would give

.. math::

   P(H\mid C_1)+P(H\mid C_2)
   =
   \frac{1}{2}+1.

This is not a valid total probability calculation.

The two conditional probabilities describe different cases.

Each must first be weighted by the probability that its case is selected.

The correct calculation is

.. math::

   P(H)
   =
   P(C_1)P(H\mid C_1)
   +
   P(C_2)P(H\mid C_2).

Thus,

.. math::

   P(H)
   =
   \frac{1}{2}\frac{1}{2}
   +
   \frac{1}{2}(1)
   =
   \frac{3}{4}.

The case probabilities tell us how often each conditional situation
contributes to the overall result.

1.8.10 Tree Diagram
-------------------

A tree diagram makes the total probability structure visible.

For the two-coin example:

::

                           H: 1/2
                         /
          C1: 1/2 ------+
                         \
                           T: 1/2


                           H: 1
                         /
          C2: 1/2 ------+
                         \
                           T: 0

There are two paths that lead to heads.

For the first path,

.. math::

   C_1\rightarrow H,

multiply along the path:

.. math::

   \frac{1}{2}\times\frac{1}{2}
   =
   \frac{1}{4}.

For the second path,

.. math::

   C_2\rightarrow H,

multiply along the path:

.. math::

   \frac{1}{2}\times1
   =
   \frac{1}{2}.

Then add the two paths:

.. math::

   P(H)
   =
   \frac{1}{4}
   +
   \frac{1}{2}
   =
   \frac{3}{4}.

.. note::

   **Tree rule**

   Multiply probabilities along one path.

   Add different paths that lead to the same final event.

The first operation is the multiplication rule.

The second operation is the addition rule for mutually exclusive paths.

1.8.11 Reading a Total Probability Tree
---------------------------------------

A total probability tree usually has two stages.

**Stage 1: Choose the case**

The first branches represent

.. math::

   B_1,B_2,\ldots,B_k.

Their probabilities must add to 1:

.. math::

   \sum_{i=1}^{k}P(B_i)=1.

**Stage 2: Event within each case**

From each case, the next branches show conditional probabilities such as

.. math::

   P(A\mid B_i)

and

.. math::

   P(A^c\mid B_i).

For each case,

.. math::

   P(A\mid B_i)+P(A^c\mid B_i)=1.

**Path probability**

A path such as

.. math::

   B_i\rightarrow A

has probability

.. math::

   P(B_i)P(A\mid B_i).

**Overall probability**

If several paths end at :math:`A`, add them:

.. math::

   P(A)
   =
   \sum_i P(B_i)P(A\mid B_i).

This gives the same total probability rule in visual form.

1.8.12 Example 3: Choose One of Two Card Piles
----------------------------------------------

One of two card piles is selected.

Pile :math:`B_1` is selected with probability

.. math::

   P(B_1)=\frac{1}{3}

and contains:

- 3 red cards;
- 1 black card.

Pile :math:`B_2` is selected with probability

.. math::

   P(B_2)=\frac{2}{3}

and contains:

- 1 red card;
- 3 black cards.

One card is drawn from the selected pile.

Let :math:`R` be the event that the card is red.

For the first pile,

.. math::

   P(R\mid B_1)=\frac{3}{4}.

For the second pile,

.. math::

   P(R\mid B_2)=\frac{1}{4}.

Now calculate the path through :math:`B_1`:

.. math::

   P(B_1)P(R\mid B_1)
   =
   \frac{1}{3}\times\frac{3}{4}
   =
   \frac{1}{4}.

The path through :math:`B_2` is

.. math::

   P(B_2)P(R\mid B_2)
   =
   \frac{2}{3}\times\frac{1}{4}
   =
   \frac{1}{6}.

Add the two ways to obtain a red card:

.. math::

   P(R)
   =
   \frac{1}{4}
   +
   \frac{1}{6}
   =
   \frac{5}{12}.

Notice that the two piles are not selected with equal probability.

This is why the values :math:`1/3` and :math:`2/3` must be included in the
calculation.

The conditional probability of red is higher in Pile 1, but Pile 2 is more
likely to be selected.

Total probability combines both pieces of information.

1.8.13 Two-Case Form
--------------------

A common partition contains only two cases:

.. math::

   B

and

.. math::

   B^c.

These two events are automatically non-overlapping:

.. math::

   B\cap B^c=\varnothing.

They also cover the whole sample space:

.. math::

   B\cup B^c=S.

Therefore, they form a partition.

The total probability rule becomes

.. math::

   P(A)
   =
   P(B)P(A\mid B)
   +
   P(B^c)P(A\mid B^c).

.. note::

   **Two-Case Total Probability Rule**

   .. math::

      P(A)
      =
      P(B)P(A\mid B)
      +
      P(B^c)P(A\mid B^c).

This form is useful whenever a problem naturally separates into

    case :math:`B`

and

    case not :math:`B`.

1.8.14 Why the Two-Case Form Matches Earlier Probability Rules
--------------------------------------------------------------

The two-case form is not a completely new idea.

Recall from Section 1.5 that

.. math::

   A
   =
   (A\cap B)
   \cup
   (A\cap B^c).

The two parts are mutually exclusive, so

.. math::

   P(A)
   =
   P(A\cap B)
   +
   P(A\cap B^c).

Using the multiplication rule,

.. math::

   P(A\cap B)
   =
   P(B)P(A\mid B)

and

.. math::

   P(A\cap B^c)
   =
   P(B^c)P(A\mid B^c).

Therefore,

.. math::

   P(A)
   =
   P(B)P(A\mid B)
   +
   P(B^c)P(A\mid B^c).

So total probability combines ideas already studied:

- splitting an event;
- mutually exclusive addition;
- conditional probability;
- multiplication along a path.

1.8.15 A Simple Method
----------------------

For a total probability problem, use the following steps.

**Step 1: Identify the possible cases.**

Write

.. math::

   B_1,B_2,\ldots,B_k.

Ask:

    What different cases can happen first?

**Step 2: Check that the cases form a partition.**

The cases must:

- not overlap; and
- cover all possible outcomes.

**Step 3: Write the probability of each case.**

Find

.. math::

   P(B_1),P(B_2),\ldots,P(B_k).

Check that they add to 1.

**Step 4: Find the probability of the final event within each case.**

Find

.. math::

   P(A\mid B_1),
   P(A\mid B_2),
   \ldots,
   P(A\mid B_k).

**Step 5: Multiply within each case.**

Calculate

.. math::

   P(B_i)P(A\mid B_i).

This is the probability of the path

.. math::

   B_i\rightarrow A.

**Step 6: Add all paths leading to the event.**

Use

.. math::

   P(A)
   =
   \sum_{i=1}^{k}P(B_i)P(A\mid B_i).

A useful summary is:

    identify the cases

    -> multiply along each path

    -> add the paths to the same event

1.8.16 Common Mistakes
----------------------

A common mistake is to use total probability when the cases do not form a
partition.

The cases must be non-overlapping and together cover the whole sample
space.

Another mistake is to add conditional probabilities directly.

For example,

.. math::

   P(A\mid B_1)+P(A\mid B_2)

is not generally equal to :math:`P(A)`.

Each conditional probability must be weighted by its case probability:

.. math::

   P(B_1)P(A\mid B_1)
   +
   P(B_2)P(A\mid B_2).

A third mistake is to forget one possible path to the final event.

If event :math:`A` can occur through several cases, every path to
:math:`A` must be included.

A fourth mistake is to add branch probabilities along one path.

Along one path, multiply.

Across different mutually exclusive paths that lead to the same event, add.

Another mistake is to forget that the second-stage probabilities are
conditional probabilities.

For a branch beginning with :math:`B_i`, the next probability is

.. math::

   P(A\mid B_i),

not simply :math:`P(A)`.

A final mistake is to assume that all cases are equally likely.

In the card-pile example,

.. math::

   P(B_1)=\frac{1}{3}

and

.. math::

   P(B_2)=\frac{2}{3}.

The different case probabilities must be used.

1.8.17 Summary
--------------

A **partition** divides the sample space into cases that are

- mutually exclusive; and
- exhaustive.

For a partition

.. math::

   B_1,B_2,\ldots,B_k,

exactly one case must occur.

The total probability rule is

.. math::

   P(A)
   =
   \sum_{i=1}^{k}
   P(B_i)P(A\mid B_i).

For each case,

.. math::

   P(B_i)P(A\mid B_i)

is the probability of the path

.. math::

   B_i\rightarrow A.

The main tree rule is:

    multiply along a path

    add paths leading to the same event

For two complementary cases,

.. math::

   P(A)
   =
   P(B)P(A\mid B)
   +
   P(B^c)P(A\mid B^c).

The main idea is:

    several possible cases -> one final event

    combine all paths -> overall probability

Before calculating:

1. identify the possible cases;
2. check that they form a partition;
3. write the probability of each case;
4. write the conditional probability of the final event in each case;
5. multiply within each path;
6. add all paths leading to the event.

.. 1.8.18 Short Review Questions
.. -----------------------------

.. 1. What is a partition?
.. 2. What does mutually exclusive mean?
.. 3. What does exhaustive mean?
.. 4. Why must exactly one case occur in a partition?
.. 5. State the total probability rule for :math:`k` cases.
.. 6. What does :math:`P(B_i)P(A\mid B_i)` represent?
.. 7. Why do we multiply probabilities along one path?
.. 8. Why do we add different paths that lead to the same event?
.. 9. In the two-coin example, what are the two paths that lead to heads?
.. 10. Why should conditional probabilities not be added directly?
.. 11. State the two-case total probability rule using :math:`B` and :math:`B^c`.
.. 12. Why do :math:`B` and :math:`B^c` form a partition?
.. 13. In a probability tree, what do the first-stage branches represent?
.. 14. In a probability tree, what do the second-stage probabilities represent?
.. 15. What are the main steps for solving a total probability problem?