Chapter 10 Probability
Probability describes how likely an event is to occur. We express it as a number between 0 and 1. Larger probabilities mean that events are more likely to happen.
For example, if you flip a fair coin, the probability of getting heads is .5. We write this as:
\[ P(\text{heads})=.5. \]
The letter P means probability. You can also think of a probability as a percentage: .5 means a 50% chance of heads on a particular flip.
Of course, you can’t get half a heads. And if you flip the coin 100 times, you aren’t guaranteed exactly 50 heads. Instead, .5 describes the long-run proportion of heads. As we make more and more independent flips of a fair coin, the proportion of heads tends to get closer to .5.
10.1 Trials, outcomes, and events
We’ll use three words throughout this chapter:
- A trial is one performance of a procedure, such as flipping a coin once, rolling a die once, or drawing one card.
- An outcome is what happens on a trial. Heads and tails are the two possible outcomes of a coin flip. The numbers 1 through 6 are the possible outcomes of rolling a six-sided die.
- An event is an outcome or a collection of outcomes we’re interested in. For a die, “rolling a 6” is an event. So is “rolling a 1 or a 2.”
An event can therefore include just one possible outcome or several. For example, the event “rolling an even number” includes the outcomes 2, 4, and 6.
When we repeat a procedure, we’ll call this repeated trials or a sequence of trials. Three coin flips form a sequence of three trials. One possible sequence of outcomes is HHT: heads, heads, then tails. An event can describe the whole sequence, such as “getting exactly two heads in three flips.”
We’ll call trials independent when knowing the outcomes of previous trials doesn’t change the probabilities on the next trial. We’ll return to this important idea when we introduce the “and” rule.
10.2 Probability distributions
A probability distribution lists the probabilities of all the possible outcomes. For a fair coin, the distribution is very short:
| Outcome | Probability |
|---|---|
| Heads | 0.5 |
| Tails | 0.5 |
The probabilities of all the distinct possible outcomes must add up to 1:
\[ P(\text{heads})+P(\text{tails})=.5+.5=1. \]
Something on our list has to happen, and a single flip can’t be both heads and tails.
Rolling a fair six-sided die has six possible outcomes. Each has probability \(1/6\), so those probabilities add up to 1 too.
We can show these distributions as bar graphs. Each bar’s height gives the probability of that outcome.
Figure 10.1: Probability distributions for a fair coin and a fair six-sided die. Each bar gives the probability of one outcome. Within each distribution, the probabilities sum to 1.
When n outcomes are equally likely, each has probability \(1/n\). For example, a well-shuffled standard deck contains 52 cards, so the probability of drawing a particular card, such as the ace of spades, is \(1/52\).
The phrase equally likely matters. A die with six sides doesn’t have to be fair, and a coin with two sides doesn’t have to give heads and tails equally often. We use \(1/n\) only when the outcomes really have equal probabilities under our model.
The same ideas apply to people. Suppose, for an example, that 10% of a population is left-handed. If we randomly select one person from that population, \(P(\text{left-handed})=.1\). This probability comes from the specified population proportion, rather than from counting two possible handedness categories and assigning each probability .5.
10.3 The probability of something not happening
The event “not A” is called the complement of event A. Either A happens or it doesn’t, so their probabilities add up to 1. This gives us a useful rule:
\[ P(\text{not A})=1-P(A). \]
For a fair die, the probability of not rolling a six is:
\[ P(\text{not 6})=1-\frac{1}{6}=\frac{5}{6}. \]
We could get the same answer by adding the probabilities of rolling a 1, 2, 3, 4, or 5. Subtracting from 1 is often easier.
For our hypothetical population, if \(P(\text{left-handed})=.1\), then \(P(\text{not left-handed})=.9\). Calling this “not left-handed” avoids assuming that everyone must fit into just the left- and right-handed categories.
10.4 Combining probabilities: The “or” rule
Suppose we want the probability that A or B happens. Here, “or” includes either event happening, and also includes both happening if that’s possible.
If A and B are mutually exclusive, they can’t both happen on the same trial. We can then add their probabilities:
\[ P(\text{A or B})=P(A)+P(B) \qquad\text{when A and B are mutually exclusive.} \]
For example, a single roll of a die can’t be both 1 and 2. The probability of rolling a 1 or a 2 is therefore:
\[ P(\text{1 or 2})=\frac{1}{6}+\frac{1}{6}=\frac{2}{6}=\frac{1}{3}. \]
Notice how we created one event, “1 or 2,” by combining two outcomes. Combining them makes the event more likely than either outcome alone.
10.4.1 Drawing any heart
A standard deck contains four suits, with 13 cards in each suit. Each specific card has probability \(1/52\) of being drawn. What’s the probability of drawing any heart?
We could draw the ace of hearts, the two of hearts, and so on through the king of hearts. On one draw, these 13 outcomes are mutually exclusive. Add their probabilities:
\[ P(\text{heart})=\underbrace{\frac{1}{52}+\frac{1}{52}+\cdots+\frac{1}{52}}_{13\text{ cards}} =\frac{13}{52}=\frac{1}{4}. \]
10.4.2 When the events overlap
What if we want the probability of drawing a heart or an ace? These events aren’t mutually exclusive: the ace of hearts belongs to both.
Adding 13 hearts and 4 aces would count the ace of hearts twice. We need to subtract that overlap once:
\[ P(\text{heart or ace})=\frac{13}{52}+\frac{4}{52}-\frac{1}{52} =\frac{16}{52}\approx .3077. \]
This gives the general addition rule:
\[ P(\text{A or B})=P(A)+P(B)-P(\text{A and B}). \]
If A and B are mutually exclusive, their overlap is zero, and this reduces to the simpler rule we used earlier. Before adding probabilities, check whether you’re counting any outcomes twice.
10.5 Combining probabilities: The “and” rule
Now suppose we want the probability that A and B both happen. If the events are independent, we multiply their probabilities:
\[ P(\text{A and B})=P(A)\times P(B) \qquad\text{when A and B are independent.} \]
For independent fair coin flips, the probability of heads on the second flip is .5, regardless of what happened on the first flip. Getting heads on both flips therefore has probability:
\[ P(\text{first H and second H})=.5\times .5=.25. \]
One way to picture this is that half the first flips are heads. Of those sequences, half also have heads on the second flip. Half of a half is one-quarter.
10.5.1 Drawing two clubs with replacement
What is the probability of drawing a club twice in a row if we put the first card back and shuffle before drawing again?
On each draw, \(P(\text{club})=13/52=1/4\). Replacing the card keeps the second draw’s probabilities unchanged. The draws are independent, so:
\[ P(\text{club on both draws})=\frac{1}{4}\times\frac{1}{4} =\frac{1}{16}=.0625. \]
Notice that we used both rules: the “or” rule to combine the 13 possible clubs on a draw, and the “and” rule to combine the two independent draws.
## [1] 0.062510.5.2 When the trials aren’t independent
What changes if we don’t put the first card back?
If the first card is a club, there are only 12 clubs among the 51 remaining cards. The probability of a club on the second draw is now \(12/51\), rather than \(13/52\).
To get two clubs, we need a club first, followed by another club from the remaining deck:
\[ P(\text{two clubs without replacement})=\frac{13}{52}\times\frac{12}{51} \approx .0588. \]
We still multiply along that sequence, but the second probability must account for what happened on the first draw. This is a conditional probability: the probability of the second event given the first. We can’t multiply the original probabilities as if nothing had changed.
10.5.3 Independent and mutually exclusive mean different things
These two conditions belong to different rules:
| Condition | Meaning |
|---|---|
| Mutually exclusive | The events can’t both happen. |
| Independent | Knowing A occurred doesn’t change P(B). |
Heads and tails on the same flip are mutually exclusive. If you know the result is heads, the probability that this same flip is tails is zero. Those events are therefore not independent.
Heads on the first flip and heads on the second can both happen. For independent fair flips, knowing the first result doesn’t change the probability of heads on the second.
The coin also doesn’t have a memory. After five heads in a row, the next independent fair flip still has a .5 probability of heads. Tails isn’t “due,” and the coin doesn’t try to make up for its earlier outcomes.
10.6 Combining the rules: Rolling two dice
Suppose we roll two independent fair six-sided dice. What is the probability that their sum is seven?
Think of the dice as distinguishable, perhaps one red and one black. A 2 on the first die and a 5 on the second is a different outcome pair from a 5 on the first and a 2 on the second.
There are \(6\times6=36\) possible pairs. Each has probability:
\[ P(\text{a particular pair})=\frac{1}{6}\times\frac{1}{6}=\frac{1}{36}. \]
The diagram shows the sum for every pair. The highlighted cells give seven.
Figure 10.2: The 36 equally likely pairs of die outcomes. Each cell shows their sum. Six highlighted cells sum to seven.
The six pairs are (1,6), (2,5), (3,4), (4,3), (5,2), and (6,1). Only one pair can occur on a particular roll of the two dice, so these possibilities are mutually exclusive. Add their probabilities:
\[ P(\text{sum of 7})=\underbrace{\frac{1}{36}+\cdots+\frac{1}{36}}_{6\text{ pairs}} =\frac{6}{36}=\frac{1}{6}\approx .1667. \]
We used the “and” rule to calculate the probability of each pair and the “or” rule to combine the pairs that give a sum of seven.
10.7 Sequences of coin flips
What is the probability of getting eight heads in a row over eight independent fair coin flips?
All eight flips have to be heads. Multiply .5 eight times:
\[ P(\text{HHHHHHHH})=(.5)^8=\frac{1}{256}\approx .0039. \]
## [1] 0.00390625Now consider this particular sequence:
\[ \text{H H T H T T H H}. \]
It looks less remarkable than eight heads, but its probability is exactly the same. Each specified flip has probability .5, so this particular sequence also has probability \((.5)^8\).
Every particular sequence of eight independent fair flips has the same probability. A sequence can look more random without being more probable. We’re specifying the sequence in advance, rather than asking whether a run of heads occurs somewhere in a longer series of flips.
10.7.1 Exactly two heads in three flips
The probability of one particular sequence is different from the probability of an event that includes several sequences.
For example, suppose we flip a fair coin three times. What is the probability of exactly two heads?
There are eight possible sequences. Each has probability \((.5)^3=1/8\):
| Sequence | Number of heads | Probability | Exactly two heads? |
|---|---|---|---|
| HHH | 3 | 1/8 | No |
| HHT | 2 | 1/8 | Yes |
| HTH | 2 | 1/8 | Yes |
| HTT | 1 | 1/8 | No |
| THH | 2 | 1/8 | Yes |
| THT | 1 | 1/8 | No |
| TTH | 1 | 1/8 | No |
| TTT | 0 | 1/8 | No |
Three sequences satisfy our event: HHT, HTH, and THH. These possibilities are mutually exclusive, so:
\[ P(\text{exactly 2 heads})=P(\text{HHT})+P(\text{HTH})+P(\text{THH}) =\frac{1}{8}+\frac{1}{8}+\frac{1}{8}=\frac{3}{8}=.375. \]
Again, we used the “and” rule within a sequence and the “or” rule across the sequences that meet our condition.
10.7.2 Counting heads instead of listing sequences
We can group the sequences by how many heads they contain. This gives another probability distribution:
| Number of heads | Number of sequences | Probability |
|---|---|---|
| 0 | 1 | 1/8 |
| 1 | 3 | 3/8 |
| 2 | 3 | 3/8 |
| 3 | 1 | 1/8 |
The probabilities still add up to 1. But the counts are not equally likely: two heads can happen in three ways, while three heads can happen in only one way.
Figure 10.3: The probability distribution for the number of heads in three independent fair flips. The individual sequences are equally likely, but the counts are not.
What if we wanted at least two heads? That includes two heads or three heads:
\[ P(\text{at least 2 heads})=\frac{3}{8}+\frac{1}{8}=\frac{1}{2}. \]
The words exactly, at least, and more than matter. “Exactly two” includes only the count two. “At least two” includes two and three. “More than two” includes only three in this example.
This is the beginning of the binomial distribution, which describes the number of successes across independent trials with the same probability of success on each trial. In the next chapter, we’ll use it to handle larger numbers of trials without listing every sequence.
10.8 Choosing a probability rule
Before calculating, describe the event in words. Then ask:
- Am I finding a probability directly, or would its complement be easier?
- Does the event include several alternatives joined by or? If so, check whether they overlap before adding.
- Does it require several events joined by and? If so, check whether the probabilities change when earlier outcomes are known.
The important work is identifying what we’re counting and why the rule applies. R can take care of the arithmetic once we’ve done that.
10.9 Practice with R
For each question, first write the event in words and choose the rule. Then use R as a calculator. A fraction or an equivalent decimal is fine.
- What is the probability of rolling an even number on one fair die?
- What is the probability of drawing a heart or a spade from one well-shuffled standard deck?
- What is the probability of getting three tails in a row in three independent fair flips?
- What is the probability of drawing two aces with replacement and reshuffling? What if you don’t replace the first card?
- What is the probability of rolling a sum of two with two independent fair dice? Why is this smaller than the probability of a sum of seven?
- For three independent fair flips, what is the probability of at least one head? Try the complement instead of listing all the sequences.
10.10 Exam practice
Try these without R, AI, a calculator, or notes. Give a short reason for your choice before checking the answers.
10.10.1 Multiple choice
1. We roll a fair die once and are interested in an even number. Which statement uses the terms correctly?
A. Rolling the die is an event, and “even” is a trial.
B. Rolling the die is a trial; “even” is an event containing 2, 4, and 6.
C. There are three equally likely outcomes of rolling a six-sided die.
D. A trial must contain several rolls.
2. A fair coin has produced five heads in a row. What is the probability of heads on the next independent flip?
A. Less than .5, because tails is due.
B. .5.
C. More than .5, because heads is on a streak.
D. It depends on how long we wait before the next flip.
3. A student adds \(P(\text{heart})+P(\text{ace})\) to find the probability of drawing a heart or an ace. What needs correcting?
A. Multiply the probabilities instead.
B. Subtract the probability of the ace of hearts once.
C. Divide the sum by two.
D. Nothing; the events are mutually exclusive.
4. Which pair of events is mutually exclusive?
A. Heads on the first flip and heads on the second.
B. Drawing a heart and drawing an ace on the same draw.
C. Rolling a 1 and rolling a 2 on the same roll of one die.
D. Rolling an even number and rolling a number greater than 3 on one die.
5. Compare HHH and HHT as particular sequences of three independent fair flips.
A. HHT is more probable because it looks more random.
B. HHH is more probable because it uses only one kind of outcome.
C. Each sequence has probability \(1/8\).
D. Each sequence has probability \(1/2\).
6. For three independent fair flips, which event has the greatest probability?
A. The particular sequence HHT.
B. Exactly two heads.
C. At least two heads.
D. More than two heads.
10.10.2 Short answer
7. Explain why rolling a sum of seven with two fair dice is more likely than rolling a sum of two. Identify where the “and” and “or” rules enter.
8. The probability of a club on the first draw is \(13/52\). A student uses \((13/52)\times(13/52)\) for two clubs without replacement. Explain what is wrong and write the correct expression. You don’t have to evaluate it.
10.11 Answers and explanations
Attempt the questions before reading this section.
10.11.1 Practice with R
1. The outcomes 2, 4, and 6 are mutually exclusive. Add their probabilities: \(3/6=1/2\).
## [1] 0.52. Hearts and spades are mutually exclusive on one draw. \(13/52+13/52=26/52=1/2\).
## [1] 0.53. All three independent flips must be tails: \((.5)^3=1/8=.125\).
## [1] 0.1254. With replacement, multiply \((4/52)\times(4/52)=1/169\approx.0059\). Without replacement, a first ace leaves three aces among 51 cards: \((4/52)\times(3/51)=1/221\approx.0045\).
## [1] 0.00591716## [1] 0.0045248875. Only (1,1) gives a sum of two, so its probability is \((1/6)\times(1/6)=1/36\approx.0278\). Six pairs give a sum of seven.
## [1] 0.027777786. The complement of “at least one head” is “no heads,” or TTT. Subtract its probability from 1: \(1-(.5)^3=7/8=.875\).
## [1] 0.87510.11.2 Exam practice
1. B. The roll is a trial. An even number is an event containing three of the six possible outcomes.
2. B. Independence means the preceding outcomes don’t change the probability on the next flip. The probability remains .5.
3. B. The ace of hearts appears in both groups. Subtract its probability once to avoid counting it twice.
4. C. A single die can’t show both 1 and 2. The other pairs can both occur. For D, rolling a 4 or a 6 satisfies both descriptions.
5. C. Each specified sequence requires three outcomes, each with probability .5. Each therefore has probability \((.5)^3=1/8\).
6. C. The probabilities are \(1/8\), \(3/8\), \(4/8\), and \(1/8\), respectively. “At least two” includes both two and three heads.
7. Use the “and” rule to find the probability of a particular pair: \((1/6)\times(1/6)=1/36\). Only (1,1) gives a sum of two. Six mutually exclusive pairs give a sum of seven, so the “or” rule adds six copies of \(1/36\), giving \(6/36\).
8. The draws aren’t independent. After the first club is removed, there are 12 clubs among 51 remaining cards. The correct expression is \((13/52)\times(12/51)\).