The Law of Iterated Expectation states that the expected value of a random variable is equal to the sum of the expected values of that random variable conditioned on a second random variable. Intuitively speaking, the law states that the expected outcome of an event can be calculated using casework on the possible outcomes of an event it depends on; for instance, if the probability of rain tomorrow depends on the probability of rain today, and all of the following are known:
- The probability of rain today
- The probability of rain tomorrow given that it rained today
- The probability of rain tomorrow given that it did not rain today
the probability of rain tomorrow can be calculated by considering both cases (it rained today/it did not rain today) in turn. To use specific numbers, suppose that
- The probability of rain today is 70%
- If it rains today, it will rain tomorrow with probability 30%
- If it does not rain today, it will rain tomorrow with probability 90%
In this case, the probability of rain tomorrow is
The Law of Iterated Expectation is useful when the probability distribution of both a random variable and a conditional random variable is known, and the probability distribution of is desired. This occurs extremely often in practice, especially in economics and poker.
Let be random variables. Then
where is the conditional probability distribution of given .
If takes the outcomes , the law can be written in the more natural form
This means that the expected value of can be calculated from the probability distribution of and , which is often useful both in theory and practice.
For example, consider a star basketball player who scores (2 points) 80% of the time when unguarded, but only 40% of the time when guarded. Against the team's current opponent, the player will be guarded 70% of the time. Then, when the player shoots, the Law of Iterated Expectation says that:
so the player scores an average of 1.04 points every time he gains possession of the ball.
This law has practical application for the opposing team: for each defensive scheme, they can calculate how many points the opposing team will score (on average), so long as they know
- How often each player will be guarded
- How well each player shoots while guarded, and how well they shoot while unguarded
which allows the opponent to pick the best defensive scheme possible.
You're asking the Oracle to foretell when you would meet your soul mate.
Oracle: You'll have a 52% chance to meet your soul mate tomorrow. Whether you'll see her today will affect whether you'll see her tomorrow.
You: Then what is my chance of meeting her today?
Oracle: I shall not speak Heaven's truth. All I can tell you is that your chance of seeing her tomorrow will be doubled if you see her today, and the chance of not seeing her tomorrow will be tripled if you don't see her today.
What is the probability (in percentage) of seeing your soul mate today?
An important theorem that can simplify the reasoning is the law of joint distribution:
This theorem makes logical sense: the probability that events and both occur is the same as the probability that occurs, then occurs. The probability that occurs is simply , and the probability that subsequently occurs is . Equivalently, the order can be reversed, leading to the second equality.
Note that when are independent, this law becomes the more familiar .
This theorem is important because it allows the calculation of given and , which is useful as is used in the law of iterated expectation.
Horace turns up at school either late or on time. He is then either shouted at or not. The probability that he turns up late is If he turns up late, the probability that he is shouted at is . If he turns up on time, the probability that he is still shouted at for no particular reason is .
You hear Horace being shouted at. What is the probability that he was late?
This problem is not original.
This law can also be rearranged into Bayes' theorem, which states that:
which allows for the same calculations as above.
On planet Brilliantia, there are two types of creatures: mathematicians and non-mathematicians.
Mathematicians tell the truth of the time and lie only of the time, while non-mathematicians tell the truth of the time and lie of the time.
It is also known that there is a chance a creature from Brilliantia is a mathematician and a chance that it is a non-mathematician, but there is no way of differentiating from these two types.
You are visiting Brilliantia on a research trip. During your stay, you come across a creature who states that it has found a one line proof for Fermat's Last Theorem. Immediately after that, a second creature shows up and states that the first creature's statement was a true one.
If the probability that the first creature's statement was actually true is , for some coprime positive integers , find the value of .