3.5 Negative Binomial Distribution ================================== :bolditalic:`Key Terms` :bolditalic:`Negative binomial distribution（負二項分配）` A distribution for the number of Bernoulli trials required to obtain a fixed number of successes. :bolditalic:`Required successes（目標成功次數）` The fixed number :math:`r` of successes that must occur before the experiment stops. :bolditalic:`Waiting time（等待次數）` The number of trials required to reach the required success. :bolditalic:`Failure-count convention（失敗次數定義）` An alternative negative-binomial convention that counts failures before the :math:`r`-th success instead of total trials. 3.5.1 Learning Outcomes ----------------------- After completing this section, students should be able to: - explain how a negative binomial experiment is built from Bernoulli trials; - distinguish a negative binomial model from a binomial model; - define the random variable as the number of trials until the :math:`r`-th success; - derive the negative binomial PMF by fixing the final success; - calculate exact negative binomial probabilities; - explain why the geometric distribution is the case :math:`r=1`; - derive the negative binomial mean and variance using geometric waiting times; - connect negative-binomial cumulative probabilities to binomial probabilities; - recognize the alternative convention that counts failures rather than trials. 3.5.2 From Geometric to Negative Binomial ----------------------------------------- The geometric distribution asks: How many trials are needed to obtain the **first** success? The negative binomial distribution generalizes this question: How many trials are needed to obtain the :math:`r`-th success? Suppose the same Bernoulli trial is repeated independently with .. math:: p=P(\text{success}), \qquad q=1-p=P(\text{failure}). Let .. math:: r=\text{required number of successes} and define .. math:: X=\text{number of trials required to obtain the }r\text{-th success}. Then .. math:: X\in\{r,r+1,r+2,\ldots\}. The smallest possible value is :math:`r`, because obtaining :math:`r` successes requires at least :math:`r` trials. Unlike a binomial random variable, :math:`X` has no fixed upper limit. 3.5.3 Conditions for a Negative Binomial Experiment ---------------------------------------------------- A negative binomial model uses repeated Bernoulli trials and requires: .. note:: **Negative Binomial Conditions** 1. Each trial has two possible categories: success or failure. 2. The trials are independent. 3. The probability of success :math:`p` is constant from trial to trial. 4. The required number of successes :math:`r` is fixed. 5. Trials continue until the :math:`r`-th success occurs. The important change from the binomial distribution is what is fixed. For a binomial experiment, the number of trials is fixed. For a negative binomial experiment, the number of successes required is fixed while the number of trials is random. 3.5.4 Two Common Random-Variable Conventions -------------------------------------------- Negative binomial distributions are commonly defined in two different ways. The formulas differ only because the random variables count different things. In this handout, we use .. math:: X=\text{total number of trials required to obtain the }r\text{-th success}. Thus, .. math:: X=r,r+1,r+2,\ldots. Some books and software instead use .. math:: Y=\text{number of failures before the }r\text{-th success}. Then .. math:: Y=0,1,2,\ldots and the variables are related by .. math:: Y=X-r, \qquad X=Y+r. .. warning:: Always check what the random variable counts before using a negative binomial formula or software function. 3.5.5 Deriving the Negative Binomial PMF ---------------------------------------- Suppose the :math:`r`-th success occurs exactly on trial :math:`x`. The last trial is therefore not free to vary: trial :math:`x` **must be a success**. Among the first :math:`x-1` trials, we therefore need exactly :math:`r-1` successes and .. math:: (x-1)-(r-1)=x-r failures. A possible sequence for :math:`r=3` and :math:`x=5` is illustrated below. The first :math:`x-1` positions contain the first :math:`r-1` successes, while the final position is fixed as the required success. .. image:: ../_static/figures/discrete_distributions/figure_3_5_1_last_success_fixed.png :width: 78% :align: center :alt: Sequence of Bernoulli trials showing the final trial fixed as the r-th success. The number of ways to choose the locations of the first :math:`r-1` successes among the first :math:`x-1` trials is .. math:: \binom{x-1}{r-1}. Every valid sequence contains exactly :math:`r` successes and :math:`x-r` failures. Therefore, the probability of any one valid sequence is .. math:: p^r q^{x-r}. Multiplying the number of valid arrangements by the probability of one arrangement gives .. note:: **Negative Binomial PMF: total-trials convention** .. math:: \boxed{ P(X=x) = \binom{x-1}{r-1}p^r(1-p)^{x-r} } for .. math:: x=r,r+1,r+2,\ldots. The combination is :math:`\binom{x-1}{r-1}`, not :math:`\binom{x}{r}`, because the last trial has already been fixed as a success. 3.5.6 Main Example: Third Success on Trial 5 -------------------------------------------- Suppose independent trials have .. math:: p=0.40, \qquad q=0.60. Let :math:`X` be the total number of trials required to obtain the third success. Then .. math:: r=3. Find the probability that the third success occurs on trial 5. For :math:`X=5`, the fifth trial must be a success. Among the first four trials, exactly two must be successes. Therefore, .. math:: \binom{4}{2}=6 valid arrangements are possible. For any one arrangement, there are three successes and two failures, so .. math:: P(\text{one valid arrangement}) = (0.40)^3(0.60)^2. Hence, .. math:: P(X=5) = \binom42(0.40)^3(0.60)^2 .. math:: = 6(0.064)(0.36) = 0.13824. Thus, the probability that the third success occurs exactly on trial 5 is about :math:`13.8\%`. 3.5.7 Geometric Distribution as a Special Case ----------------------------------------------- Set .. math:: r=1. Then the negative binomial PMF becomes .. math:: P(X=x) = \binom{x-1}{0}p(1-p)^{x-1}. Because .. math:: \binom{x-1}{0}=1, we obtain .. math:: P(X=x) = (1-p)^{x-1}p, \qquad x=1,2,\ldots. This is the geometric PMF. Therefore, .. note:: The geometric distribution is the negative binomial distribution with :math:`r=1`. 3.5.8 Deriving the Mean and Variance ------------------------------------ The mean and variance have a useful interpretation through geometric waiting times. Let :math:`G_1` be the number of trials required to obtain the first success, :math:`G_2` the number of additional trials from the first success to the second success, and so on. For :math:`r=3`, the waiting process can be viewed as three geometric blocks. .. image:: ../_static/figures/discrete_distributions/figure_3_5_2_geometric_waiting_blocks.png :width: 82% :align: center :alt: Bernoulli trials partitioned into three geometric waiting blocks ending at successive successes. In general, .. math:: X=G_1+G_2+\cdots+G_r. Each :math:`G_i` is geometric with success probability :math:`p`, so .. math:: E(G_i)=\frac1p and .. math:: \operatorname{Var}(G_i) = \frac{1-p}{p^2}. Using linearity of expectation, .. math:: E(X) = E(G_1+G_2+\cdots+G_r) .. math:: = E(G_1)+E(G_2)+\cdots+E(G_r) .. math:: = r\left(\frac1p\right). Therefore, .. note:: **Mean** .. math:: \boxed{E(X)=\frac{r}{p}}. For the variance, the waiting blocks are independent because the Bernoulli trials are independent. Therefore, .. math:: \operatorname{Var}(X) = \operatorname{Var}(G_1+\cdots+G_r) .. math:: = \operatorname{Var}(G_1)+\cdots+\operatorname{Var}(G_r) .. math:: = r\left(\frac{1-p}{p^2}\right). Thus, .. note:: **Variance and Standard Deviation** .. math:: \boxed{ \operatorname{Var}(X) = \frac{r(1-p)}{p^2} } and .. math:: \boxed{ \sigma = \frac{\sqrt{r(1-p)}}{p} }. 3.5.9 Mean and Variance in the Main Example ------------------------------------------- For :math:`r=3` and :math:`p=0.40`, .. math:: E(X) = \frac{3}{0.40} = 7.5. The long-run average number of trials needed to obtain three successes is 7.5 trials. The variance is .. math:: \operatorname{Var}(X) = \frac{3(0.60)}{(0.40)^2} .. math:: = \frac{1.8}{0.16} = 11.25. The standard deviation is .. math:: \sigma = \sqrt{11.25} \approx 3.35. The PMF is right-skewed because long waiting times are possible, although their probabilities become progressively smaller. .. image:: ../_static/figures/discrete_distributions/figure_3_5_3_negative_binomial_pmf.png :width: 76% :align: center :alt: Negative binomial PMF for three required successes with success probability 0.40. The expected value is not the most likely value and does not need to be an integer. It represents a long-run average waiting time. 3.5.10 Cumulative Probabilities and the Binomial Connection ----------------------------------------------------------- The negative binomial distribution has a direct connection to the binomial distribution. The event .. math:: X\leq x means that the :math:`r`-th success has occurred by trial :math:`x`. Equivalently, there must be **at least :math:`r` successes among the first :math:`x` trials**. If .. math:: B\sim\operatorname{Binomial}(x,p), then .. math:: P(X\leq x) = P(B\geq r). Using the complement rule, .. note:: .. math:: \boxed{ P(X\leq x) = 1- \sum_{k=0}^{r-1} \binom{x}{k}p^k(1-p)^{x-k} }. Similarly, .. math:: P(X>x) = P(B\leq r-1). For the main example, :math:`r=3`, :math:`p=0.40`, and :math:`x=5`: .. math:: P(X\leq5) = P(B\geq3), \qquad B\sim\operatorname{Binomial}(5,0.40). Therefore, .. math:: P(X\leq5) = 1- \left[P(B=0)+P(B=1)+P(B=2)\right] .. math:: = 0.31744. Notice that :math:`P(X=5)=0.13824` and :math:`P(X\leq5)=0.31744` answer different questions. The first asks for the third success **exactly** on trial 5, whereas the second asks whether the third success has occurred **by** trial 5. 3.5.11 Failure-Count Form of the Negative Binomial -------------------------------------------------- Some references define .. math:: Y=X-r as the number of failures before the :math:`r`-th success. If :math:`Y=y`, then the total number of trials is .. math:: X=y+r. Substituting :math:`x=y+r` into the total-trials PMF gives .. math:: P(Y=y) = \binom{y+r-1}{r-1}p^r(1-p)^y, for .. math:: y=0,1,2,\ldots. Because :math:`Y=X-r`, subtracting the constant :math:`r` changes the mean but not the variance: .. math:: E(Y) = E(X)-r = \frac{r}{p}-r = \frac{r(1-p)}{p}, while .. math:: \operatorname{Var}(Y) = \operatorname{Var}(X) = \frac{r(1-p)}{p^2}. This convention difference is one of the most common sources of conflicting negative binomial formulas. 3.5.12 Binomial Versus Negative Binomial ---------------------------------------- The two models use the same Bernoulli-trial assumptions, but they stop the experiment differently. .. list-table:: :header-rows: 1 :widths: 34 33 33 * - Feature - Binomial - Negative binomial * - Fixed quantity - Trials :math:`n` - Required successes :math:`r` * - Random quantity - Number of successes - Number of trials * - Random variable - :math:`X=\text{successes in }n\text{ trials}` - :math:`X=\text{trials until }r\text{-th success}` * - Support - :math:`0,1,\ldots,n` - :math:`r,r+1,\ldots` * - Typical question - How many successes occur? - How long until enough successes occur? A useful diagnostic question is: **What is fixed: the number of trials or the number of successes?** 3.5.13 Effect of the Parameters ------------------------------- For a fixed :math:`r`, increasing :math:`p` makes successes occur more often. The distribution therefore shifts toward smaller values of :math:`X`, and both the mean and variance decrease. For a fixed :math:`p`, increasing :math:`r` increases the required number of successes. The expected number of trials therefore increases because .. math:: E(X)=\frac{r}{p}. The variance also increases linearly with :math:`r`: .. math:: \operatorname{Var}(X) = \frac{r(1-p)}{p^2}. 3.5.14 A Simple Method ---------------------- For a negative binomial problem: 1. Define what counts as a success. 2. Identify the success probability :math:`p`. 3. Check independence and constant :math:`p`. 4. Identify the required number of successes :math:`r`. 5. Define whether the random variable counts **total trials** or **failures**. 6. For an exact total-trials probability, use .. math:: P(X=x) = \binom{x-1}{r-1}p^r(1-p)^{x-r}. 7. For a ``by trial x`` question, translate the problem to a binomial cumulative probability if convenient. 8. Interpret the answer in terms of waiting for the required success. 3.5.15 Quick Practice --------------------- A basketball player makes each free throw with probability .. math:: p=0.75. Assume attempts are independent. Let :math:`X` be the number of attempts required to make the second basket. Find: (a) :math:`P(X=3)`; (b) :math:`E(X)`; (c) :math:`\operatorname{Var}(X)`. **Answer** Here, .. math:: r=2, \qquad p=0.75, \qquad q=0.25. For the second success to occur on attempt 3, exactly one success must occur among the first two attempts and the third attempt must be a success: .. math:: P(X=3) = \binom21(0.75)^2(0.25) = 0.28125. The mean is .. math:: E(X) = \frac{2}{0.75} \approx 2.67. The variance is .. math:: \operatorname{Var}(X) = \frac{2(0.25)}{(0.75)^2} \approx 0.889. 3.5.16 Common Mistakes ---------------------- Do not confuse negative binomial with binomial. In a binomial model, the number of trials is fixed. In a negative binomial model, the required number of successes is fixed. Do not use .. math:: \binom{x}{r} for the total-trials PMF. If the :math:`r`-th success occurs on trial :math:`x`, the final trial is already fixed as a success, leaving only :math:`x-1` positions for the first :math:`r-1` successes. Do not allow :math:`x