1.7 Independence

Key Terms

Independent events（獨立事件）

Events for which knowing that one occurred does not change the probability of the other.

Dependent events（相依事件）

Events for which knowing that one occurred changes the probability of the other.

Pairwise independence（兩兩獨立）

Every pair of events in a group is independent.

Mutual independence（相互獨立）

All required combinations of events satisfy the independence conditions.

1.7.2 Learning Outcomes

After completing this section, students should be able to:

• explain independence in simple words;

• compare \(P(B\mid A)\) with \(P(B)\) to check independence;

• use \(P(A\cap B)=P(A)P(B)\) for independent events;

• explain when the general multiplication rule is still needed;

• distinguish independent events from mutually exclusive events;

• explain how replacement can affect independence;

• use complements of independent events;

• calculate probabilities for several independent events;

• distinguish pairwise independence from mutual independence.

1.7.3 Independent Events

In Section 1.6, conditional probability showed how probability can change when new information becomes available.

Independence is a special case where the new information causes no change.

Suppose \(A\) and \(B\) are two events.

If knowing that \(A\) occurred does not change the probability of \(B\), then \(A\) and \(B\) are independent.

Note

Independent events

Knowing that one event occurred does not change the probability of the other.

When \(P(A)>0\),

\[P(B\mid A)=P(B).\]

If the probability changes after the condition is known, the events are dependent.

So the main idea is:

independent -> probability does not change

dependent -> probability changes

1.7.4 Example 1: Toss a Fair Coin Twice

A fair coin is tossed twice.

Let

• \(A\) be the event that the first toss is heads;

• \(B\) be the event that the second toss is heads.

Before knowing the result of the first toss,

\[P(B)=\frac{1}{2}.\]

Now suppose we know that the first toss was heads.

The possible outcomes are then

\[HH,\ HT.\]

Among these two outcomes, one has heads on the second toss.

Therefore,

\[P(B\mid A)=\frac{1}{2}.\]

Thus,

\[P(B\mid A)=P(B).\]

Knowing the first toss does not change the probability of the second toss.

Therefore, \(A\) and \(B\) are independent.

This example shows the meaning of independence directly through conditional probability.

1.7.5 Checking Independence

From Section 1.6, the general multiplication rule is

\[P(A\cap B) = P(A)P(B\mid A).\]

If \(A\) and \(B\) are independent, then

\[P(B\mid A)=P(B).\]

Substituting this into the multiplication rule gives

\[P(A\cap B) = P(A)P(B).\]

Note

For independent events

\[P(A\cap B)=P(A)P(B).\]

Therefore, we can check independence in two common ways.

Conditional probability check

Compare

\[P(B\mid A)\]

with

\[P(B).\]

When \(P(A)>0\), equality shows that knowing \(A\) occurred does not change the probability of \(B\).

Product check

Compare

\[P(A\cap B)\]

with

\[P(A)P(B).\]

If they are equal, the events are independent.

For two events with positive probabilities, the conditional-probability and product checks give the same conclusion.

1.7.6 Example 2: Two Coin Tosses

Use the same events from Example 1:

• \(A\) = first toss is heads;

• \(B\) = second toss is heads.

We have

\[P(A)=\frac{1}{2}\]

and

\[P(B)=\frac{1}{2}.\]

Both events occur when the outcome is \(HH\).

Therefore,

\[P(A\cap B)=\frac{1}{4}.\]

Now calculate

\[P(A)P(B) = \frac{1}{2}\times\frac{1}{2} = \frac{1}{4}.\]

Since

\[P(A\cap B)=P(A)P(B),\]

the events are independent.

This gives the same conclusion as the conditional probability check.

Note

Use

\[P(A\cap B)=P(A)P(B)\]

only after independence has been established.

If independence is not known, use the general multiplication rule:

\[P(A\cap B)=P(A)P(B\mid A).\]

1.7.7 Independent and Mutually Exclusive Events

Independence and mutual exclusivity describe fundamentally different relationships between events.

Independent events

Knowing that one event occurred does not change the probability of the other. Independent events may occur together, so their intersection need not be empty.

Mutually exclusive events

Mutually exclusive events cannot occur together; their intersection is empty.

The difference can be visualized as follows.

In the left diagram, \(A\) and \(B\) overlap. This is compatible with independence because knowing one event occurred does not change the probability of the other. The probability of their intersection satisfies

\[P(A\cap B)=P(A)P(B).\]

In the right diagram, \(A\) and \(B\) do not overlap. Thus,

\[A\cap B=\varnothing\]

and

\[P(A\cap B)=0.\]

The key distinction is:

independent -> overlap may exist, but one event does not change the probability of the other

mutually exclusive -> overlap is impossible

If two mutually exclusive events both have positive probability, they cannot be independent. Once one occurs, the other becomes impossible, so its conditional probability changes to 0.

1.7.8 Example 3: Roll One Fair Die

One fair die is rolled.

Let

• \(C\) be the event of rolling a 1;

• \(D\) be the event of rolling a 6.

These events cannot occur together on a single roll.

Therefore,

\[C\cap D=\varnothing\]

and

\[P(C\cap D)=0.\]

However,

\[P(C)=\frac{1}{6}\]

and

\[P(D)=\frac{1}{6}.\]

So,

\[P(C)P(D) = \frac{1}{6}\times\frac{1}{6} = \frac{1}{36}.\]

Because

\[P(C\cap D)\neq P(C)P(D),\]

the events are not independent.

They are dependent.

We can also understand this using conditional probability.

If we know that \(C\) occurred, then rolling 6 is impossible:

\[P(D\mid C)=0.\]

But before knowing the result,

\[P(D)=\frac{1}{6}.\]

Conditioning on \(C\) changes the probability of \(D\) from \(1/6\) to 0, confirming that the events are dependent.

Note

Mutually exclusive events with positive probabilities are dependent.

1.7.9 With and Without Replacement

Replacement can affect whether events are independent.

Suppose two cards are drawn one after another.

Let

• \(A\) be the event that the first card is an ace;

• \(B\) be the event that the second card is an ace.

The key question is:

Does the first draw change the probability of the second draw?

1.7.10 Example 4: Draw Two Cards

With replacement

Before the first draw,

\[P(B)=\frac{4}{52}.\]

Suppose the first card is an ace and is then returned to the deck.

The deck again contains 52 cards and 4 aces.

Therefore,

\[P(B\mid A)=\frac{4}{52}.\]

Thus,

\[P(B\mid A)=P(B).\]

The first event does not change the probability of the second event.

Therefore, \(A\) and \(B\) are independent.

Without replacement

Again,

\[P(B)=\frac{4}{52}\]

before the first draw.

Now suppose the first card is an ace and is not returned.

Only 51 cards remain, and only 3 aces remain.

Therefore,

\[P(B\mid A)=\frac{3}{51}.\]

Since

\[\frac{3}{51}\neq\frac{4}{52},\]

the first event changes the probability of the second event.

Therefore, \(A\) and \(B\) are dependent.

Note

Check whether the probability changes

\[P(B\mid A)=P(B) \quad\longrightarrow\quad \text{independent}\]

\[P(B\mid A)\neq P(B) \quad\longrightarrow\quad \text{dependent}\]

Replacement often restores the original probabilities.

Without replacement often changes the probabilities.

However, the important distinction is not simply the phrases with replacement and without replacement.

The main question is always:

Does the first event change the probability of the next event?

1.7.11 Independence and Complements

If \(A\) and \(B\) are independent, then independence is also preserved when one or both events are replaced by their complements.

Therefore, the following pairs are also independent:

• \(A^c\) and \(B\);

• \(A\) and \(B^c\);

• \(A^c\) and \(B^c\).

This means we may use the independent-event product rule with these pairs.

For example,

\[P(A^c\cap B) = P(A^c)P(B)\]

and

\[P(A^c\cap B^c) = P(A^c)P(B^c).\]

Before multiplying, find the required complement probabilities.

1.7.12 Example 5: Use Complements

Suppose

\[P(A)=0.4\]

and

\[P(B)=0.3,\]

and suppose \(A\) and \(B\) are independent.

First,

\[P(A^c)=1-P(A)=1-0.4=0.6.\]

Also,

\[P(B^c)=1-P(B)=1-0.3=0.7.\]

Because \(A^c\) and \(B\) are independent,

\[P(A^c\cap B) = P(A^c)P(B) = (0.6)(0.3) = 0.18.\]

Because \(A^c\) and \(B^c\) are also independent,

\[P(A^c\cap B^c) = P(A^c)P(B^c) = (0.6)(0.7) = 0.42.\]

The important point is that independence is preserved when either event is replaced by its complement.

Engineering application: two components in parallel

Independence and complements are especially useful in system reliability. Consider two independent components connected in parallel between points \(a\) and \(b\).

The upper component operates with probability \(0.93\), and the lower component operates with probability \(0.87\). A parallel system works if at least one component works; it fails only when both components fail.

Let \(F_1\) and \(F_2\) denote failure of the two components. Then

\[P(F_1)=1-0.93=0.07\]

and

\[P(F_2)=1-0.87=0.13.\]

Assuming the component states are independent,

\[P(F_1\cap F_2) = P(F_1)P(F_2) = (0.07)(0.13) = 0.0091.\]

Therefore, the probability that the parallel system operates is

\[P(\text{system works}) = 1-P(F_1\cap F_2) = 1-0.0091 = 0.9909.\]

This example combines two ideas: independence allows us to multiply the failure probabilities, and the complement rule converts both fail into at least one works.

1.7.13 Several Independent Events

The product rule also extends to several independent events.

If

\[A_1,A_2,\ldots,A_k\]

are mutually independent, then

\[P(A_1\cap A_2\cap\cdots\cap A_k) = P(A_1)P(A_2)\cdots P(A_k).\]

Note

Several independent events

\[P(A_1\cap A_2\cap\cdots\cap A_k) = P(A_1)P(A_2)\cdots P(A_k).\]

This is the independent-event version of the multiplication rule.

The condition of independence is important.

Without independence, we must use conditional probabilities as in Section 1.6.

1.7.14 Example 6: Three Fair Coin Tosses

A fair coin is tossed three times.

Let

• \(A_1\) = heads on the first toss;

• \(A_2\) = heads on the second toss;

• \(A_3\) = heads on the third toss.

The three tosses are independent.

Each event has probability

\[P(A_i)=\frac{1}{2}.\]

Therefore,

\[P(A_1\cap A_2\cap A_3) = P(A_1)P(A_2)P(A_3).\]

Thus,

\[P(A_1\cap A_2\cap A_3) = \frac{1}{2} \times \frac{1}{2} \times \frac{1}{2} = \frac{1}{8}.\]

So, the probability of getting three heads is \(1/8\).

Engineering application: several independent components

A more complex reliability system can combine parallel and series structures.

Assume that all component operating states are independent. The first subsystem contains three components in parallel with operating probabilities

\[0.88,\qquad 0.90,\qquad 0.92.\]

It fails only if all three components fail. Therefore,

\[P(\text{first subsystem works}) = 1-(0.12)(0.10)(0.08) = 0.99904.\]

The second subsystem contains two components in parallel with operating probabilities \(0.94\) and \(0.96\). Thus,

\[P(\text{second subsystem works}) = 1-(0.06)(0.04) = 0.9976.\]

Finally, a component with reliability \(0.98\) is in series with the two parallel subsystems. The complete system works only if all three stages work. Using independence,

\[P(\text{system works}) = (0.99904)(0.9976)(0.98) \approx 0.9767.\]

This calculation uses complements for the parallel components and then multiplies the probabilities of the independent stages that must all operate.

Note

These reliability calculations require independent component states. If component failures are dependent, the product rules cannot be applied directly.

1.7.15 Pairwise and Mutual Independence

For three or more events, independence requires more precise terminology.

Suppose we have three events:

\[A,\ B,\ C.\]

Pairwise independence means that every pair is independent.

Therefore,

\[P(A\cap B)=P(A)P(B),\]

\[P(A\cap C)=P(A)P(C),\]

and

\[P(B\cap C)=P(B)P(C).\]

These checks look at the events two at a time.

Mutual independence is stronger.

For three events, all three pairwise conditions must hold, and we must also have

\[P(A\cap B\cap C) = P(A)P(B)P(C).\]

Note

Pairwise independence does not necessarily imply mutual independence.

Checking only the pairs is not enough to conclude that all three events are mutually independent.

For three events, mutual independence requires:

1. \(A\) and \(B\) are independent;

2. \(A\) and \(C\) are independent;

3. \(B\) and \(C\) are independent;

4. the three-way intersection also satisfies the product rule.

1.7.16 Checking for Independence

There are several ways to check independence.

If conditional probabilities are available

Compare

\[P(B\mid A)\]

with

\[P(B).\]

When \(P(A)>0\), equality means that conditioning on \(A\) did not change the probability of \(B\).

If intersection probabilities are available

Compare

\[P(A\cap B)\]

with

\[P(A)P(B).\]

If they are equal, the events are independent.

If sampling is with or without replacement

Ask whether the first selection changes the probability of the next selection.

If the events cannot occur together

They are mutually exclusive.

If both events have positive probabilities, they are not independent.

If several events are involved

Distinguish between pairwise independence and mutual independence.

The main idea is:

Note

Independence

\[P(B\mid A)=P(B).\]

Equivalently,

\[P(A\cap B)=P(A)P(B).\]

1.7.17 Common Mistakes

A common mistake is to think that independent events cannot occur together.

This is not correct.

For two independent coin tosses, the outcome \(HH\) shows that the two events first toss is heads and second toss is heads can occur together.

Another mistake is to confuse independent events with mutually exclusive events.

Mutually exclusive events cannot occur together.

Independent events do not change each other’s probabilities.

A third mistake is to use

\[P(A\cap B)=P(A)P(B)\]

without first establishing independence.

If independence is not known, use

\[P(A\cap B)=P(A)P(B\mid A).\]

Another mistake is to assume that replacement alone is the definition of independence.

Replacement often restores the original probability, but the correct test is whether the probability changes.

A fifth mistake is to use the original probability after a draw without replacement.

For example, after an ace is drawn and not replaced,

\[P(\text{second ace}\mid\text{first ace}) = \frac{3}{51},\]

not \(4/52\).

A final mistake is to check only pairwise independence for three events and then conclude that the three events are mutually independent.

Mutual independence also requires the three-way product condition.

1.7.18 Summary

Independent events do not change each other’s probabilities.

For two independent events,

\[P(B\mid A)=P(B).\]

Equivalently,

\[P(A\cap B)=P(A)P(B).\]

If independence has not been established, use the general multiplication rule:

\[P(A\cap B)=P(A)P(B\mid A).\]

Independent events and mutually exclusive events are different.

Mutually exclusive events satisfy

\[A\cap B=\varnothing.\]

Two mutually exclusive events with positive probabilities are dependent.

With replacement often restores the original probability.

Without replacement often changes the probability.

The key question is:

Does knowing one event occurred change the probability of the other?

If \(A\) and \(B\) are independent, then all pairs formed from \(A\), \(B\), and their complements remain independent.

For several mutually independent events,

\[P(A_1\cap A_2\cap\cdots\cap A_k) = P(A_1)P(A_2)\cdots P(A_k).\]

For three events, pairwise independence checks each pair.

Mutual independence requires the pairwise conditions and the three-way product condition.

The main decision process is:

Does the probability change?

No -> independent

Yes -> dependent