2. Introduction to Probability

The previous lesson introduced uncertainty in measurements, customer demand, and operating processes. It also introduced two directions of reasoning:

Probability:   model  ->  possible data

Statistics:    observed data  ->  model

This lesson now focuses on the probability direction. We begin with an assumed model and organize the possible outcomes before making a calculation.

The main learning routine is:

uncertainty
   -> probability model
   -> statistical experiment
   -> sample space
   -> event
   -> favorable outcomes
   -> probability by counting
   -> interpretation

This lesson focuses on the following ideas:

uncertainty and practical randomness;
statistical experiment;
random process;
sample space;
sample point;
event;
favorable outcomes;
equally likely outcomes;
probability by simple counting.

This lesson focuses on the basic concepts of probability. More advanced topics, such as permutations, combinations, factorials, addition rules, conditional probability, independence, total probability, or Bayes’ rule, will be covered in later lessons.

1. From Uncertainty to a Probability Model

Industrial Engineering deals with systems that are not always easy to predict. Measurements may contain errors, customer demand may change, and production processes may not behave in exactly the same way every time.

Concept: Uncertainty

Uncertainty means that we do not know an exact value, condition, or outcome.

Three common examples are:

Type of uncertainty	Example	Main decision question
Measurement uncertainty	Repeated measurements are different	Is the difference real or measurement error?
Demand uncertainty	Actual demand differs from the forecast	How much inventory should be prepared?
Process uncertainty	Processing and waiting times vary	How much capacity is needed?

Measurement Uncertainty

A product may have a fixed true value, but the measured value may not be exact. The instrument, operator, environment, calibration, or procedure may affect the result.

\[\text{measured value} = \text{true value} + \text{measurement error}\]

The key question is whether an observed difference comes from the product or from the measurement system.

Demand Uncertainty

A company must often prepare products before actual customer demand is known. The actual demand may be lower or higher than the forecast.

\[\text{actual demand} = \text{forecast} + \text{forecast error}\]

The key question is how much inventory should be prepared before demand is observed.

Process Uncertainty

Processing times, customer arrivals, machine failures, and transportation times may vary. These outcomes usually have causes, but it may be difficult to observe and model every cause.

The key question is how much capacity is needed when operating conditions are uncertain.

Principle: Industrial Engineering and uncertainty

Industrial Engineering uses probability and statistics to measure uncertainty, understand variation, evaluate risk, and support decisions.

Randomness as a Practical Model

A system does not need to be uncaused for us to model it as random.

A coin flip follows physical laws. Its outcome depends on force, angle, spin, height, air resistance, gravity, and surface contact. However, we usually do not know these conditions precisely enough to predict the final result.

Therefore, we model the coin flip as random.

Principle: Practical randomness

Random does not mean uncaused.

Random means that the exact outcome is not predictable enough for the present purpose.

The practical idea is:

We know what can happen,
but we do not know what will happen.

Probability and Statistics Use Different Directions

Probability and statistics are closely related, but they begin with different information.

The basic directions are:

\[\text{Probability: model}\rightarrow\text{possible outcomes}\]

\[\text{Statistics: observed data}\rightarrow\text{model}\]

For example, probability may assume that a coin is fair and ask what outcomes are likely. Statistics may observe many coin flips and ask whether the data support the fair-coin model.

Focus of this lesson

This lesson follows the probability direction:

\[\text{known probability model}\rightarrow\text{unknown outcome}\]

2. Probability Begins with an Uncertain Experiment

A probability problem begins when an outcome is uncertain before it occurs.

Suppose we flip one coin.

Before the coin is flipped, we do not know the result. The result may be heads or tails.

\[H=\text{heads}, \qquad T=\text{tails}\]

After the coin is flipped, only one result is observed.

      Flip one coin
            |
  +---------+---------+
  |                   |
  v                   v
Heads               Tails

This is a probability situation because we know what can happen, but we do not know what will happen.

Definition: Statistical experiment

A statistical experiment is a process that generates an outcome or a set of observations.

The basic structure is:

experiment  ->  outcome

Examples of simple statistical experiments include:

Principle: Uncertainty before observation

Before the experiment occurs, the outcome is uncertain.

After the experiment occurs, one outcome is observed.

Probability is useful because it helps us reason before the outcome is known.

Example: Flipping One Coin

Suppose we flip one coin.

Before the flip, both outcomes are possible:

+-----+-----+
|  H  |  T  |
+-----+-----+
   \     /
    \   /
     \ /
both are possible before the flip

After the flip, only one outcome is observed.

If the result is heads, then the observed outcome is:

\[H\]

If the result is tails, then the observed outcome is:

\[T\]

Interpretation

A coin flip is a probability problem because the possible outcomes are known, but the actual outcome is not known before the flip.

3. The Sample Space Lists What Can Happen

Before calculating probability, we need to list the complete set of possible outcomes.

This complete set is called the sample space.

Definition: Sample space

The sample space is the set of all possible outcomes of a statistical experiment.

It is usually denoted by \(S\).

Each element of the sample space is called a sample point.

Definition: Sample point

A sample point is one individual outcome in the sample space.

For one coin flip:

\[S=\{H,T\}\]

There are two sample points:

\[H \qquad \text{and} \qquad T\]

For one die roll:

\[S=\{1,2,3,4,5,6\}\]

There are six sample points:

\[1,\;2,\;3,\;4,\;5,\;6\]

Principle: Sample space before probability

A probability calculation should begin by specifying the sample space.

If the sample space is unclear, the event and probability calculation will also be unclear.

Example: One Coin Flip

Question:

Flip one fair coin. What is the sample space?

The experiment is:

\[\text{flip one coin}\]

The possible outcomes are heads and tails. Therefore:

\[S=\{H,T\}\]

Sketch:

Sample space for one coin flip

+-----+-----+
|  H  |  T  |
+-----+-----+

The number of sample points is:

\[|S|=2\]

Interpretation

Before flipping the coin, we do not know whether the result will be heads or tails. However, we know the complete set of possible outcomes.

Example: One Die Roll

Question:

Roll one fair die. What is the sample space?

The experiment is:

\[\text{roll one die}\]

The possible outcomes are:

\[1,\quad 2,\quad 3,\quad 4,\quad 5,\quad 6\]

Therefore:

\[S=\{1,2,3,4,5,6\}\]

Sketch:

Sample space for one die roll

+---+---+---+---+---+---+
| 1 | 2 | 3 | 4 | 5 | 6 |
+---+---+---+---+---+---+

The number of sample points is:

\[|S|=6\]

Interpretation

Before rolling the die, we do not know which face will appear, but we know the possible outcomes.

Example: Different Sample Spaces for the Same Die

The sample space should match the question being asked.

If we care about the exact die face, then the sample space is:

\[S_1=\{1,2,3,4,5,6\}\]

If we care only whether the result is even or odd, then a reduced sample space may be:

\[S_2=\{\text{even},\text{odd}\}\]

These two sample spaces describe the same die roll at different levels of detail.

Interpretation

The correct sample space depends on the purpose of the experiment and the question being asked.

4. Events Identify the Outcomes

The sample space lists all possible outcomes. Usually, however, we are interested in only some of those outcomes.

That subset is called an event.

Definition: Event

An event is a subset of the sample space.

If \(A\) is an event, then:

\[A\subseteq S\]

An event collects the sample points that satisfy a specified condition.

The sample space answers:

What can happen?

The event answers:

Which outcomes satisfy the condition of interest?

Principle: Event before probability

Probability is calculated for events.

Therefore, the event must be defined before computing \(P(A)\).

Example: Getting Heads in One Coin Flip

Question:

Flip one fair coin. Define the event that the result is heads.

The sample space is:

\[S=\{H,T\}\]

The event of getting heads is:

\[A=\{H\}\]

Sketch:

+-----+-----+
|  H  |  T  |
+-----+-----+
   ^
   |
event A = getting heads

The relationship between the event and the sample space is:

\[A\subseteq S\]

Interpretation

The event \(A\) contains the outcome that satisfies the condition “getting heads.”

Example: At Least One Head in Two Coin Flips

Question:

Flip one fair coin twice. Define the event that at least one head occurs.

First, list the sample space:

\[S=\{HH,HT,TH,TT\}\]

There are four sample points:

\[|S|=4\]

The event “at least one head” is:

\[B=\{HH,HT,TH\}\]

There are three favorable outcomes:

\[|B|=3\]

Sketch:

+------+------+------+------+
| HH   | HT   | TH   | TT   |
+------+------+------+------+
  ^      ^      ^
  |      |      |
  at least one head

The outcome \(TT\) is excluded because it has no heads.

Interpretation

The phrase “at least one head” means one or more heads. It excludes only the outcome with zero heads.

Example: At Least One Head versus Exactly One Head

Students often confuse “at least one” with “exactly one.”

For two coin flips:

\[S=\{HH,HT,TH,TT\}\]

The event “at least one head” is:

\[A=\{HH,HT,TH\}\]

The event “exactly one head” is:

\[B=\{HT,TH\}\]

Sketch:

Sample space:

+------+------+------+------+
| HH   | HT   | TH   | TT   |
+------+------+------+------+

At least one head:
+------+------+------+
| HH   | HT   | TH   |
+------+------+------+

Exactly one head:
       +------+------+
       | HT   | TH   |
       +------+------+

Warning

Translate the verbal condition into a set of outcomes before calculating probability.

5. Probability by Counting

Once the sample space and event are clear, probability can often be calculated by counting.

For this introductory lesson, we focus on equally likely outcomes.

Definition: Equally likely outcomes

Outcomes are equally likely when each outcome in the sample space has the same chance of occurring.

For example, in one fair coin flip:

\[P(H)=P(T)=\frac{1}{2}\]

For one fair die:

\[P(1)=P(2)=P(3)=P(4)=P(5)=P(6)=\frac{1}{6}\]

Rule: Probability by counting

If all outcomes in the sample space are equally likely, then:

\[P(A)=\frac{|A|}{|S|}\]

Here:

\(|S|\) is the number of outcomes in the sample space;
\(|A|\) is the number of favorable outcomes in event \(A\).

The counting idea is:

favorable outcomes
-------------------  =  probability
total outcomes

Principle: Condition for using the counting rule

The formula \(P(A)=|A|/|S|\) can be used directly only when the outcomes in \(S\) are equally likely.

Example: Probability of Heads

Question:

Flip one fair coin. What is the probability of getting heads?

Step 1. Identify the experiment:

\[\text{flip one fair coin}\]

Step 2. List the sample space:

\[S=\{H,T\}\]

Step 3. Define the event:

\[A=\{H\}\]

Step 4. Count total and favorable outcomes:

\[|S|=2, \qquad |A|=1\]

Step 5. Calculate:

\[P(A)=\frac{|A|}{|S|} = \frac{1}{2} = 0.5 = 50\%\]

Step 6. Interpret:

Interpretation

There is a 50 percent chance of getting heads in one fair coin flip.

Example: Probability of At Least One Head

Question:

Flip one fair coin twice. What is the probability of getting at least one head?

Step 1. Identify the experiment:

\[\text{flip one fair coin twice}\]

Step 2. List the sample space:

\[S=\{HH,HT,TH,TT\}\]

Step 3. Define the event:

\[B=\{HH,HT,TH\}\]

Step 4. Count total and favorable outcomes:

\[|S|=4, \qquad |B|=3\]

Step 5. Calculate:

\[P(B)=\frac{|B|}{|S|} = \frac{3}{4} = 0.75 = 75\%\]

Step 6. Interpret:

Interpretation

There is a 75 percent chance of getting at least one head in two fair coin flips.

Example: Why the Event Must Match the Question

Suppose the question asks for the probability of rolling a number greater than 4.

The correct event is:

\[A=\{5,6\}\]

If a student writes:

\[A=\{4,5,6\}\]

then the student has answered a different question: rolling a number greater than or equal to 4.

Warning

Small changes in wording can change the event and therefore change the probability.

6. Counting Sample Points in Multi-Stage Experiments

Some experiments happen in stages.

Examples:

flip one coin twice;
roll two dice;
choose one item, then choose another item.

For now, we only use simple multiplication to count sample points. We do not use permutations or combinations in this lesson.

Rule: Simple multiplication principle

If one stage has \(n_1\) possible outcomes and a second stage has \(n_2\) possible outcomes, then the two-stage experiment has:

\[n_1n_2\]

possible ordered outcomes.

The idea is:

For each outcome in stage 1,
stage 2 has its own possible outcomes.

Example: Two Coin Flips

Suppose we flip one coin twice.

The first flip has two possible outcomes:

\[H,\quad T\]

The second flip also has two possible outcomes:

\[H,\quad T\]

Therefore, the total number of ordered outcomes is:

\[2\times 2=4\]

The sample space is:

\[S=\{HH,HT,TH,TT\}\]

Sketch:

       First flip
     H            T
    / \          / \
   H   T        H   T
  /     \      /     \
HH      HT    TH      TT

Example: Two Dice

Question:

Roll two distinguishable fair dice. How many ordered outcomes are possible?

The first die has 6 possible outcomes:

\[n_1=6\]

The second die has 6 possible outcomes:

\[n_2=6\]

Therefore:

\[|S|=n_1n_2=6\times 6=36\]

Sketch:

First die outcome:  1   2   3   4   5   6
                    |   |   |   |   |   |
Second die choices: 6   6   6   6   6   6

Total ordered outcomes = 6 + 6 + 6 + 6 + 6 + 6 = 36

The ordered pair \((1,2)\) is different from \((2,1)\).

\[(1,2)\neq(2,1)\]

Interpretation

Rolling two dice gives 36 ordered outcomes because each result of the first die can be paired with each result of the second die.

Warning

In this lesson, we are only using simple multiplication to count sample points. We are not yet using permutations or combinations.

8. Common Mistakes in Simple Probability Problems

This section collects common mistakes that occur when students begin probability by counting.

Mistake 1: Starting with the Formula Too Early

A common mistake is to start with:

\[P(A)=\frac{|A|}{|S|}\]

before defining \(S\) and \(A\).

The better sequence is:

define the experiment
   -> list the sample space
   -> define the event
   -> count
   -> calculate
   -> interpret

Warning

Do not calculate before defining the event.

Mistake 2: Forgetting That Order Can Matter

For two coin flips:

\[HT\neq TH\]

The outcome \(HT\) means heads first, tails second.

The outcome \(TH\) means tails first, heads second.

They are different ordered outcomes.

Warning

In repeated experiments, pay attention to whether the order of outcomes matters.

Mistake 3: Confusing “At Least One” with “Exactly One”

For two coin flips:

\[S=\{HH,HT,TH,TT\}\]

The event “at least one head” is:

\[\{HH,HT,TH\}\]

The event “exactly one head” is:

\[\{HT,TH\}\]

These are different events.

Mistake 4: Using the Counting Formula Without Equal Likelihood

The formula:

\[P(A)=\frac{|A|}{|S|}\]

works directly only when outcomes are equally likely.

For a fair die, this assumption is reasonable.

For a biased die, it is not.

Warning

Counting outcomes is not enough when outcomes are not equally likely.

9. Final Summary: Key Messages and Takeaways

This lesson connected uncertainty in Industrial Engineering with probability, sample spaces, events, and simple counting.

The main routine is:

uncertainty
   -> statistical experiment
   -> sample space
   -> event
   -> favorable outcomes
   -> probability by counting
   -> interpretation

Probability begins with uncertainty. Before the experiment occurs, more than one outcome may be possible, and we do not know which outcome will occur. In Industrial Engineering, the same logic appears in measurement, demand, reliability, queueing, production, and logistics.

A statistical experiment is a process that generates an outcome or a set of observations.

A sample space lists all possible outcomes of the experiment. It is usually denoted by \(S\).

A sample point is one individual outcome in the sample space.

An event is a subset of the sample space. It contains the outcomes that satisfy the condition of interest.

Probability is calculated for events, not for vague situations.

For equally likely outcomes, probability can be calculated by counting:

\[P(A)=\frac{|A|}{|S|}\]

Here, \(|A|\) is the number of favorable outcomes, and \(|S|\) is the number of total outcomes.

The counting formula should be used directly only when all outcomes in the sample space are equally likely.

For multi-stage experiments, sample points can often be counted by multiplication. If stage 1 has \(n_1\) outcomes and stage 2 has \(n_2\) outcomes, then the two-stage experiment has:

\[n_1n_2\]

ordered outcomes.

Random means practically uncertain. It does not mean uncaused.

Probability and statistics reason in different directions:

\[\text{Probability: model}\rightarrow\text{outcomes}\]

\[\text{Statistics: data}\rightarrow\text{model}\]

Core Procedure

For simple probability problems, use this procedure:

Closing Message

The important skill is not memorizing formulas first.

The important skill is organizing the uncertain situation:

experiment first,
sample space second,
event third,
counting fourth,
interpretation last.

The calculation answers:

\[\text{How likely is it?}\]

The interpretation answers:

\[\text{What does it mean?}\]

Probability is disciplined reasoning and counting under uncertainty.