Discrete Random Variables - Joint Probability Distribution

The joint probability distribution of two random variables is a function describing the probability of pairs of values occurring. For instance, consider a random variable $X$ that represents the number of heads in a single coin flip, and a random variable $Y$ that represents the number of heads in a different single coin flip. Then the joint distribution of $X$ and $Y$ is

$$\text{Pr}(X=x, Y=y) = \begin{cases} \frac{1}{4} & x = 0, y = 0 \\ \frac{1}{4} & x = 0, y = 1 \\ \frac{1}{4} & x = 1, y = 0 \\ \frac{1}{4} & x = 1, y = 1 \end{cases}$$

The joint distribution of two random variables $X,Y$ is given by

$$\text{Pr}[X=x \text{ and } Y=y] = \text{Pr}(Y = y | X = x)\text{Pr}(X = x) = \text{Pr}(X = x | Y = y)\text{Pr}(Y = y)$$

where $P(Y = y | X = x)$ is the probability that $Y = y$ given that $X = x$. When $X,Y$ are independent, this value is equal to $P(Y = y)$ (as the fact that $X = x$ is irrelevant), and

$$\text{Pr}[X = x, Y = y] = \text{Pr}(X=x)\text{Pr}(Y=y)$$

This also allows the joint distribution to be used as an independence test: if the above relation holds for all $x,y$, then $X$ and $Y$ are independent. Otherwise, they are not.

Notably, this relation immediately rearranges to Bayes' theorem, which is very important in conditional probability.

If $X$ takes on the value $x$, $Y$ must take on some value, and so

$$\text{Pr}[X = x] = \sum_y \text{Pr}[X = x, Y = y] = \sum_y \text{Pr}[X = x|Y = y]\text{Pr}[Y = y]$$

which is related to the law of iterated expectation.

• Random variables