1.7 Independence ================ :bolditalic:`Key Terms` :bolditalic:`Independent events（獨立事件）` Events for which knowing that one occurred does not change the probability of the other. :bolditalic:`Dependent events（相依事件）` Events for which knowing that one occurred changes the probability of the other. :bolditalic:`Pairwise independence（兩兩獨立）` Every pair of events in a group is independent. :bolditalic:`Mutual independence（相互獨立）` All required combinations of events satisfy the independence conditions. 1.7.2 Learning Outcomes ----------------------- After completing this section, students should be able to: - explain independence in simple words; - compare :math:`P(B\mid A)` with :math:`P(B)` to check independence; - use :math:`P(A\cap B)=P(A)P(B)` for independent events; - explain when the general multiplication rule is still needed; - distinguish independent events from mutually exclusive events; - explain how replacement can affect independence; - use complements of independent events; - calculate probabilities for several independent events; - distinguish pairwise independence from mutual independence. 1.7.3 Independent Events ------------------------ In Section 1.6, conditional probability showed how probability can change when new information becomes available. Independence is a special case where the new information causes **no change**. Suppose :math:`A` and :math:`B` are two events. If knowing that :math:`A` occurred does not change the probability of :math:`B`, then :math:`A` and :math:`B` are independent. .. note:: **Independent events** Knowing that one event occurred does not change the probability of the other. When :math:`P(A)>0`, .. math:: P(B\mid A)=P(B). If the probability changes after the condition is known, the events are dependent. So the main idea is: independent -> probability does not change dependent -> probability changes 1.7.4 Example 1: Toss a Fair Coin Twice --------------------------------------- A fair coin is tossed twice. Let - :math:`A` be the event that the first toss is heads; - :math:`B` be the event that the second toss is heads. Before knowing the result of the first toss, .. math:: P(B)=\frac{1}{2}. Now suppose we know that the first toss was heads. The possible outcomes are then .. math:: HH,\ HT. Among these two outcomes, one has heads on the second toss. Therefore, .. math:: P(B\mid A)=\frac{1}{2}. Thus, .. math:: P(B\mid A)=P(B). Knowing the first toss does not change the probability of the second toss. Therefore, :math:`A` and :math:`B` are independent. This example shows the meaning of independence directly through conditional probability. 1.7.5 Checking Independence --------------------------- From Section 1.6, the general multiplication rule is .. math:: P(A\cap B) = P(A)P(B\mid A). If :math:`A` and :math:`B` are independent, then .. math:: P(B\mid A)=P(B). Substituting this into the multiplication rule gives .. math:: P(A\cap B) = P(A)P(B). .. note:: **For independent events** .. math:: P(A\cap B)=P(A)P(B). Therefore, we can check independence in two common ways. **Conditional probability check** Compare .. math:: P(B\mid A) with .. math:: P(B). When :math:`P(A)>0`, equality shows that knowing :math:`A` occurred does not change the probability of :math:`B`. **Product check** Compare .. math:: P(A\cap B) with .. math:: P(A)P(B). If they are equal, the events are independent. For two events with positive probabilities, the conditional-probability and product checks give the same conclusion. 1.7.6 Example 2: Two Coin Tosses -------------------------------- Use the same events from Example 1: - :math:`A` = first toss is heads; - :math:`B` = second toss is heads. We have .. math:: P(A)=\frac{1}{2} and .. math:: P(B)=\frac{1}{2}. Both events occur when the outcome is :math:`HH`. Therefore, .. math:: P(A\cap B)=\frac{1}{4}. Now calculate .. math:: P(A)P(B) = \frac{1}{2}\times\frac{1}{2} = \frac{1}{4}. Since .. math:: P(A\cap B)=P(A)P(B), the events are independent. This gives the same conclusion as the conditional probability check. .. note:: Use .. math:: P(A\cap B)=P(A)P(B) only after independence has been established. If independence is not known, use the general multiplication rule: .. math:: P(A\cap B)=P(A)P(B\mid A). 1.7.7 Independent and Mutually Exclusive Events ----------------------------------------------- Independence and mutual exclusivity describe fundamentally different relationships between events. **Independent events** Knowing that one event occurred does not change the probability of the other. Independent events may occur together, so their intersection need not be empty. **Mutually exclusive events** Mutually exclusive events cannot occur together; their intersection is empty. The difference can be visualized as follows. .. image:: ../_static/figures/probability/figure_1_7_1_independent_mutually_exclusive.png :width: 92% :align: center :alt: Comparison of independent events that may overlap and mutually exclusive events that do not overlap. In the left diagram, :math:`A` and :math:`B` overlap. This is compatible with independence because knowing one event occurred does not change the probability of the other. The probability of their intersection satisfies .. math:: P(A\cap B)=P(A)P(B). In the right diagram, :math:`A` and :math:`B` do not overlap. Thus, .. math:: A\cap B=\varnothing and .. math:: P(A\cap B)=0. The key distinction is: independent -> overlap may exist, but one event does not change the probability of the other mutually exclusive -> overlap is impossible If two mutually exclusive events both have positive probability, they cannot be independent. Once one occurs, the other becomes impossible, so its conditional probability changes to 0. 1.7.8 Example 3: Roll One Fair Die ---------------------------------- One fair die is rolled. Let - :math:`C` be the event of rolling a 1; - :math:`D` be the event of rolling a 6. These events cannot occur together on a single roll. Therefore, .. math:: C\cap D=\varnothing and .. math:: P(C\cap D)=0. However, .. math:: P(C)=\frac{1}{6} and .. math:: P(D)=\frac{1}{6}. So, .. math:: P(C)P(D) = \frac{1}{6}\times\frac{1}{6} = \frac{1}{36}. Because .. math:: P(C\cap D)\neq P(C)P(D), the events are not independent. They are dependent. We can also understand this using conditional probability. If we know that :math:`C` occurred, then rolling 6 is impossible: .. math:: P(D\mid C)=0. But before knowing the result, .. math:: P(D)=\frac{1}{6}. Conditioning on :math:`C` changes the probability of :math:`D` from :math:`1/6` to 0, confirming that the events are dependent. .. note:: Mutually exclusive events with positive probabilities are dependent. 1.7.9 With and Without Replacement ---------------------------------- Replacement can affect whether events are independent. Suppose two cards are drawn one after another. Let - :math:`A` be the event that the first card is an ace; - :math:`B` be the event that the second card is an ace. The key question is: Does the first draw change the probability of the second draw? 1.7.10 Example 4: Draw Two Cards -------------------------------- **With replacement** Before the first draw, .. math:: P(B)=\frac{4}{52}. Suppose the first card is an ace and is then returned to the deck. The deck again contains 52 cards and 4 aces. Therefore, .. math:: P(B\mid A)=\frac{4}{52}. Thus, .. math:: P(B\mid A)=P(B). The first event does not change the probability of the second event. Therefore, :math:`A` and :math:`B` are independent. **Without replacement** Again, .. math:: P(B)=\frac{4}{52} before the first draw. Now suppose the first card is an ace and is not returned. Only 51 cards remain, and only 3 aces remain. Therefore, .. math:: P(B\mid A)=\frac{3}{51}. Since .. math:: \frac{3}{51}\neq\frac{4}{52}, the first event changes the probability of the second event. Therefore, :math:`A` and :math:`B` are dependent. .. note:: **Check whether the probability changes** .. math:: P(B\mid A)=P(B) \quad\longrightarrow\quad \text{independent} .. math:: P(B\mid A)\neq P(B) \quad\longrightarrow\quad \text{dependent} Replacement often restores the original probabilities. Without replacement often changes the probabilities. However, the important distinction is not simply the phrases ``with replacement`` and ``without replacement``. The main question is always: Does the first event change the probability of the next event? 1.7.11 Independence and Complements ----------------------------------- If :math:`A` and :math:`B` are independent, then independence is also preserved when one or both events are replaced by their complements. Therefore, the following pairs are also independent: - :math:`A^c` and :math:`B`; - :math:`A` and :math:`B^c`; - :math:`A^c` and :math:`B^c`. This means we may use the independent-event product rule with these pairs. For example, .. math:: P(A^c\cap B) = P(A^c)P(B) and .. math:: P(A^c\cap B^c) = P(A^c)P(B^c). Before multiplying, find the required complement probabilities. 1.7.12 Example 5: Use Complements --------------------------------- Suppose .. math:: P(A)=0.4 and .. math:: P(B)=0.3, and suppose :math:`A` and :math:`B` are independent. First, .. math:: P(A^c)=1-P(A)=1-0.4=0.6. Also, .. math:: P(B^c)=1-P(B)=1-0.3=0.7. Because :math:`A^c` and :math:`B` are independent, .. math:: P(A^c\cap B) = P(A^c)P(B) = (0.6)(0.3) = 0.18. Because :math:`A^c` and :math:`B^c` are also independent, .. math:: P(A^c\cap B^c) = P(A^c)P(B^c) = (0.6)(0.7) = 0.42. The important point is that independence is preserved when either event is replaced by its complement. **Engineering application: two components in parallel** Independence and complements are especially useful in system reliability. Consider two independent components connected in parallel between points :math:`a` and :math:`b`. .. image:: ../_static/figures/probability/figure_1_7_2_parallel_reliability_system.png :width: 60% :align: center :alt: Two independent components connected in parallel with reliabilities 0.93 and 0.87. The upper component operates with probability :math:`0.93`, and the lower component operates with probability :math:`0.87`. A parallel system works if at least one component works; it fails only when both components fail. Let :math:`F_1` and :math:`F_2` denote failure of the two components. Then .. math:: P(F_1)=1-0.93=0.07 and .. math:: P(F_2)=1-0.87=0.13. Assuming the component states are independent, .. math:: P(F_1\cap F_2) = P(F_1)P(F_2) = (0.07)(0.13) = 0.0091. Therefore, the probability that the parallel system operates is .. math:: P(\text{system works}) = 1-P(F_1\cap F_2) = 1-0.0091 = 0.9909. This example combines two ideas: independence allows us to multiply the failure probabilities, and the complement rule converts ``both fail`` into ``at least one works``. 1.7.13 Several Independent Events --------------------------------- The product rule also extends to several independent events. If .. math:: A_1,A_2,\ldots,A_k are mutually independent, then .. math:: P(A_1\cap A_2\cap\cdots\cap A_k) = P(A_1)P(A_2)\cdots P(A_k). .. note:: **Several independent events** .. math:: P(A_1\cap A_2\cap\cdots\cap A_k) = P(A_1)P(A_2)\cdots P(A_k). This is the independent-event version of the multiplication rule. The condition of independence is important. Without independence, we must use conditional probabilities as in Section 1.6. 1.7.14 Example 6: Three Fair Coin Tosses ---------------------------------------- A fair coin is tossed three times. Let - :math:`A_1` = heads on the first toss; - :math:`A_2` = heads on the second toss; - :math:`A_3` = heads on the third toss. The three tosses are independent. Each event has probability .. math:: P(A_i)=\frac{1}{2}. Therefore, .. math:: P(A_1\cap A_2\cap A_3) = P(A_1)P(A_2)P(A_3). Thus, .. math:: P(A_1\cap A_2\cap A_3) = \frac{1}{2} \times \frac{1}{2} \times \frac{1}{2} = \frac{1}{8}. So, the probability of getting three heads is :math:`1/8`. **Engineering application: several independent components** A more complex reliability system can combine parallel and series structures. .. image:: ../_static/figures/probability/figure_1_7_3_mixed_reliability_system.png :width: 82% :align: center :alt: Reliability system with three components in parallel, followed by two components in parallel, followed by one component in series. Assume that all component operating states are independent. The first subsystem contains three components in parallel with operating probabilities .. math:: 0.88,\qquad 0.90,\qquad 0.92. It fails only if all three components fail. Therefore, .. math:: P(\text{first subsystem works}) = 1-(0.12)(0.10)(0.08) = 0.99904. The second subsystem contains two components in parallel with operating probabilities :math:`0.94` and :math:`0.96`. Thus, .. math:: P(\text{second subsystem works}) = 1-(0.06)(0.04) = 0.9976. Finally, a component with reliability :math:`0.98` is in series with the two parallel subsystems. The complete system works only if all three stages work. Using independence, .. math:: P(\text{system works}) = (0.99904)(0.9976)(0.98) \approx 0.9767. This calculation uses complements for the parallel components and then multiplies the probabilities of the independent stages that must all operate. .. note:: These reliability calculations require independent component states. If component failures are dependent, the product rules cannot be applied directly. 1.7.15 Pairwise and Mutual Independence --------------------------------------- For three or more events, independence requires more precise terminology. Suppose we have three events: .. math:: A,\ B,\ C. **Pairwise independence** means that every pair is independent. Therefore, .. math:: P(A\cap B)=P(A)P(B), .. math:: P(A\cap C)=P(A)P(C), and .. math:: P(B\cap C)=P(B)P(C). These checks look at the events two at a time. **Mutual independence** is stronger. For three events, all three pairwise conditions must hold, and we must also have .. math:: P(A\cap B\cap C) = P(A)P(B)P(C). .. note:: Pairwise independence does not necessarily imply mutual independence. Checking only the pairs is not enough to conclude that all three events are mutually independent. For three events, mutual independence requires: 1. :math:`A` and :math:`B` are independent; 2. :math:`A` and :math:`C` are independent; 3. :math:`B` and :math:`C` are independent; 4. the three-way intersection also satisfies the product rule. 1.7.16 Checking for Independence -------------------------------- There are several ways to check independence. **If conditional probabilities are available** Compare .. math:: P(B\mid A) with .. math:: P(B). When :math:`P(A)>0`, equality means that conditioning on :math:`A` did not change the probability of :math:`B`. **If intersection probabilities are available** Compare .. math:: P(A\cap B) with .. math:: P(A)P(B). If they are equal, the events are independent. **If sampling is with or without replacement** Ask whether the first selection changes the probability of the next selection. **If the events cannot occur together** They are mutually exclusive. If both events have positive probabilities, they are not independent. **If several events are involved** Distinguish between pairwise independence and mutual independence. The main idea is: .. note:: **Independence** .. math:: P(B\mid A)=P(B). Equivalently, .. math:: P(A\cap B)=P(A)P(B). 1.7.17 Common Mistakes ---------------------- A common mistake is to think that independent events cannot occur together. This is not correct. For two independent coin tosses, the outcome :math:`HH` shows that the two events ``first toss is heads`` and ``second toss is heads`` can occur together. Another mistake is to confuse independent events with mutually exclusive events. Mutually exclusive events cannot occur together. Independent events do not change each other's probabilities. A third mistake is to use .. math:: P(A\cap B)=P(A)P(B) without first establishing independence. If independence is not known, use .. math:: P(A\cap B)=P(A)P(B\mid A). Another mistake is to assume that replacement alone is the definition of independence. Replacement often restores the original probability, but the correct test is whether the probability changes. A fifth mistake is to use the original probability after a draw without replacement. For example, after an ace is drawn and not replaced, .. math:: P(\text{second ace}\mid\text{first ace}) = \frac{3}{51}, not :math:`4/52`. A final mistake is to check only pairwise independence for three events and then conclude that the three events are mutually independent. Mutual independence also requires the three-way product condition. 1.7.18 Summary -------------- Independent events do not change each other's probabilities. For two independent events, .. math:: P(B\mid A)=P(B). Equivalently, .. math:: P(A\cap B)=P(A)P(B). If independence has not been established, use the general multiplication rule: .. math:: P(A\cap B)=P(A)P(B\mid A). Independent events and mutually exclusive events are different. Mutually exclusive events satisfy .. math:: A\cap B=\varnothing. Two mutually exclusive events with positive probabilities are dependent. With replacement often restores the original probability. Without replacement often changes the probability. The key question is: Does knowing one event occurred change the probability of the other? If :math:`A` and :math:`B` are independent, then all pairs formed from :math:`A`, :math:`B`, and their complements remain independent. For several mutually independent events, .. math:: P(A_1\cap A_2\cap\cdots\cap A_k) = P(A_1)P(A_2)\cdots P(A_k). For three events, pairwise independence checks each pair. Mutual independence requires the pairwise conditions and the three-way product condition. The main decision process is: Does the probability change? No -> independent Yes -> dependent .. 1.7.19 Short Review Questions .. ----------------------------- .. 1. What does it mean for two events to be independent? .. 2. What does it mean for two events to be dependent? .. 3. How can :math:`P(B\mid A)` and :math:`P(B)` be used to check independence? .. 4. State the product condition for two independent events. .. 5. Why should :math:`P(A)P(B)` not be used automatically? .. 6. Are two fair coin tosses independent? Why? .. 7. What is the difference between independent and mutually exclusive events? .. 8. Why are mutually exclusive events with positive probabilities dependent? .. 9. Why does replacement often make repeated card draws independent? .. 10. Why does drawing without replacement often create dependence? .. 11. If :math:`A` and :math:`B` are independent, are :math:`A^c` and :math:`B` also independent? .. 12. How do you find :math:`P(A^c\cap B^c)` when :math:`A` and :math:`B` are independent? .. 13. How do you find the probability of several independent events occurring together? .. 14. What is pairwise independence? .. 15. What additional condition is needed for three events to be mutually independent?