by Matt Enlow
Probability problems are notorious for yielding surprising and counterintuitive results. One famous example -- or pair of examples -- is the following:
1. A couple has two children and the older child is a boy. If the probabilities of having a boy or a girl are both , what is the probability that the couple has two boys?
We already know that the older child is a boy. The probability of two boys is equivalent to the probability that the younger child is a boy, which is .
2. A couple has two children, one of which is a boy. If the probabilities of having a boy or a girl are both , what is the probability that the couple has two boys?
At first glance, this appears to be asking the same question. We might reason as follows: “We know that one is a boy, so the only question is whether the other one is a boy, and the chances of that being the case are . So again, the answer is .”
This makes perfect sense. It also happens to be incorrect.
There are several approaches we could take to straighten out this logical tangle, as well as many other tangles that arise in probability. One common tool is known as Bayes’ Theorem. Before we state the theorem, we will discuss conditional probability.
When calculating probabilities of certain events, we often obtain new information that may cause our calculations to change. For example, if I draw a single card from a standard -card deck and ask the probability that my card is a Heart, you would quickly say , as there are Hearts in the deck. However, if I then peek at the card and tell you that it is red, then we will have narrowed the possibilities for my card to possible cards, and the probability that my card is a Heart is now .
Let’s define the events and as “the card is a Heart”and “the card is red,” respectively. Our inital value of is the probability that my card is a Heart before we have any other information about it. This is called the prior probability of event , and is denoted . Once we take into account the knowledge that the card is red, we calculate that the posterior probability of the card being a Heart is . This probability is denoted , and read “the probability of given .”
What about the values of and ? It’s clear that , the prior probability that the card is red, is . To think about the posterior probability , we must ask, “What is the probability that the card is red, if we assume that it’s a Heart?” Phrased that way, the probability is clearly , or . Sometimes conditional probabilities are not this easy to calculate. For a given pair of events and , it is often the case that the value of one of or is much easier to calculate than the other. This is where Bayes’ Theorem comes to the rescue, by giving a relationship between these two conditional probabilities. There are several formulations of Bayes’ Theorem; we first state one of the simpler formulations.
￼For any two events and ,
The proof of Bayes' Theorem follows from the fact that both and are equivalent to , the probability of both events occurring.
We want to find the probability that the coin we flipped is unfair, given that we flipped it three times and got . So let’s define event as “the coin we flipped is unfair,” and event as “the outcome of 3 flips was .” We want to find . To use Bayes’ Theorem, we must first determine , , and .
is the probability that the coin we flipped is unfair, without taking into account any flipping that took place. We can assume that we were just as likely to select the fair coin as we were to select the unfair one, so . is the probability that the outcome is if the coin we flip is the unfair one. Since the unfair coin comes up Heads of the time, That leaves , the probability that the outcome is if we pick either coin and flip it three times. Here, we come across a little bump in the road. “Well, that depends,” we say. “If it’s the fair coin, the probability is .
But if it’s the unfair coin, we just calculated that the probability is . How can we calculate the probability if we don’t know whether or not the coin is fair?” We will use the fact that for any two sets and , the sets and partition . This implies
￼which gives us another formulation of Bayes’ Theorem:
￼￼￼￼￼We can now plug everything in:
￼￼￼￼￼￼￼￼￼Therefore, given the evidence of the flips, the coin we’ve been flipping is slightly more likely to be the unfair one.
￼ One common application of Bayes’Theorem is in analyzing the quality of diagnostic tests. Consider the following.
Before we implement Bayes’Theorem, take a guess as to what the answer might be. (You might require some convincing that the answer is not simply !)
Let’s define event as “the person has TB” and event as “the person tests positive for TB”. It is clear that the prior probability is equal to and .
What about , the probability that the person will test positive for TB given that the person has TB? This was given to us as . The other value we need is , the probability that the person will test positive for TB given that the person does not have TB. Since a person who does not have TB will test negative of the time, he or she will test positive of the time, and therefore . ￼￼￼ This implies
You might find this hard to believe— that fewer than of people who test positive for TB using this test actually have the disease. Even though the sensitivity and specificity of this test are both high, the extremely low incidence of TB in the population has a tremendous effect on the test’s positive predictive value— the proportion of people who test positive that actually have the disease. To see this for yourself, you might try answering the same question assuming that the incidence of TB in the population is in instead of in .
Let’s see how to use Bayes’ Theorem to solve the two problems at the beginning of this post:
1. A couple has two children, the older of which is a boy. What is the probability that they have two boys?
2. A couple has two children, one of which is a boy. What is the probability that they have two boys?
Let’s define three events, , , and , as follows:
Question 1 is asking for , and Question 2 is asking for . The first is computed using the simpler version of Bayes’ Theorem:
To find , we must determine , the prior probability that the couple has at least one boy. This is equal to . Therefore the desired probability is
There are probably simpler ways to come to an understanding of how these two questions are different. But Bayes’ Theorem is always an option, and an important tool to keep in our toolbox!
The Paradox of the Second Ace
The following pair of problems come from Greg Ross’ wonderful blog, Futility Closet.
(1) You are watching four people play bridge, where a hand begins by dealing each player 13 cards.
a. After a hand is dealt, you ask a player, “Do you have at least one Ace?” She says, “Yes.” What is the probability that she’s holding more than one Ace?
b. On a later hand, you ask another player, “Do you have the Ace of Spades?” She says, “Yes.” What is the probability that she’s holding more than one Ace?
The following problem is a favorite of Stanford professor and author Keith Devlin.
(2) A certain town has two taxi companies: Blue Birds, whose cabs are blue, and Night Owls, whose cabs are black. Blue Birds has taxis in its fleet, and Night Owls has . Late one night, there is a hit-and-run accident involving a taxi. The town's taxis were all on the streets at the time of the accident. A witness saw the accident and claims that a blue taxi was involved. ￼￼￼ At the request of the police, the witness undergoes a vision test under conditions similar to those on the night in question. Presented repeatedly with a blue taxi and a black taxi, in random order, he shows he can successfully identify the color of the taxi times out of . Which company is more likely to have been involved in the accident?
(3) In front of you are two identical-looking coins. One is fair, and the other comes up Heads of the time. You choose one of the coins, and flip it ten times, yielding . What is the probability that the coin you’ve been flipping is the unfair one?
(4) In front of you are two identical-looking coins. One is fair, and the other comes up Heads of the time. You flip one of the coins five times, yielding . You flip the other coin three times, yielding . Which one is more likely to be the unfair coin?
(5) In front of you are three identical-looking coins. Two are fair, and the third comes up Heads of the time. You flip one of the coins three times, yielding . You flip a second coin once, yielding H. Which coin is most likely to be the unfair one?
(6) You have three identical-looking coins in front of you. Two of them are fair, and the third comes up Heads of the time. You flip one of the coins twice, yielding . You flip a second coin once, yielding . You are permitted to perform one more flip, with any coin you choose, after which you will be asked to identify which coin you believe to be the unfair one. Which coin should you flip in order to maximize the likelihood of your guessing correctly?