Discrete Random Variables - Joint Probability Distribution

The joint probability distribution of two random variables is a function describing the probability of pairs of values occurring. For instance, consider a random variable \(X\) that represents the number of heads in a single coin flip, and a random variable \(Y\) that represents the number of heads in a different single coin flip. Then the joint distribution of \(X\) and \(Y\) is

\[\text{Pr}(X=x, Y=y) = \begin{cases} \frac{1}{4} & x = 0, y = 0 \\ \frac{1}{4} & x = 0, y = 1 \\ \frac{1}{4} & x = 1, y = 0 \\ \frac{1}{4} & x = 1, y = 1 \end{cases} \]

The joint distribution of two random variables \(X,Y\) is given by

\[\text{Pr}[X=x \text{ and } Y=y] = \text{Pr}(Y = y | X = x)\text{Pr}(X = x) = \text{Pr}(X = x | Y = y)\text{Pr}(Y = y)\]

where \(P(Y = y | X = x)\) is the probability that \(Y = y\) given that \(X = x\). When \(X,Y\) are independent, this value is equal to \(P(Y = y)\) (as the fact that \(X = x\) is irrelevant), and

\[\text{Pr}[X = x, Y = y] = \text{Pr}(X=x)\text{Pr}(Y=y)\]

This also allows the joint distribution to be used as an independence test: if the above relation holds for all \(x,y\), then \(X\) and \(Y\) are independent. Otherwise, they are not.

Notably, this relation immediately rearranges to Bayes' theorem, which is very important in conditional probability.

If \(X\) takes on the value \(x\), \(Y\) must take on some value, and so

\[\text{Pr}[X = x] = \sum_y \text{Pr}[X = x, Y = y] = \sum_y \text{Pr}[X = x|Y = y]\text{Pr}[Y = y]\]

which is related to the law of iterated expectation.

• Random variables