Axioms of Probability

In order to compute probabilities, one must restrict themselves to collections of subsets of the arbitrary space \(\Omega\) known as \(\sigma\)-algebras. Due to the Banach-Tarski paradox, it turns out that assigning probability measures to any collection of sets without taking into consideration the set's cardinality will yield contradictions. Thus, a special class of sets must be adhered to in order to correctly define the notion of a probability measure.

This section will develop specific types of set structures in which we can compute probabilities.

\(\sigma\)-algebras are by far the most important set structure defined here as they are the building blocks for defining probability measures.

A collection, \(\mathcal F\), of subsets of \(\Omega\), is a \(\sigma\)-algebra if

1) \(\mathcal F\) is closed under complements: if \( A \in \) \(\mathcal F\) \(\implies\) \(A^c \in \) \(\mathcal F;\)

2) \(\mathcal F\) is closed under countable unions: if \(A_n \in \) \(\mathcal F\) \(\forall i \in \mathbb N \) \(\implies\) \( \cup_{i \in \mathbb N} A \in \mathcal F. \)

A collection, \(\mathcal A\), of subsets of \(\Omega\), is an algebra if

1) \(\mathcal A\) is closed under complements: if \( A \in \) \(\mathcal A\) \(\implies\) \(A^c \in \) \(\mathcal A;\)

2) \(\mathcal a\) is closed under finite unions: if \(A_n \in \) \(\mathcal A\) \(\forall i \in \{1,...,n\} \) \(\implies\) \(\cup_i A \in \mathcal A. \)

Let \(\mathcal C\) be a collection of subsets of \(\Omega\). We call the smallest \(\sigma\)-algebra that contains or is generated by \(\mathcal C\) as \(\sigma(\mathcal C) = \bigcap _{\mathcal \alpha \in \mathcal A } \mathcal F_{\alpha},\) where each \(\mathcal F_{\alpha}\) is a \(\sigma\)-algebra that contains \(\mathcal C\).

Define \(2^\Omega\) as the set of all subsets of \(\Omega\). Since \(\forall A \subset \Omega \implies \) \(A \in 2^\Omega\), \(A^c \in \Omega\) since \(A \cup A^c = \Omega\) and thus \(A^c \in \Omega\). Let \(A_i \subset \Omega\ \forall i \in \mathbb N \), then each \(A_i \in 2^\Omega \). Therefore, \(\cup_{i \in \mathbb N}A_i \in 2^\Omega \) since \(\cup_{i \in \mathbb N}A_i \subset \Omega \). Thus, \(2^\Omega\) is a \(\sigma\)-algebra of subsets of \(\Omega\).

I claim that based on the definitions of a \(\sigma\)-algebra, for any set on \(\Omega\), we can always find a generating \(\sigma\)-algebra, \(\mathcal A\), such that \(\mathcal A \ne \emptyset \). Since \(2^\Omega\ \) is a \(\sigma\)-algebra such that \(2^\Omega\) is the set of all subsets of \(\Omega\), every \(\sigma\)-algebra of subsets of \(\Omega \) is contained in \(2^\Omega\). Therefore, we can always find a \(\sigma\)-algebra for an arbitrary set \(\Omega \). \(_\square\)

As it turns out, there are certain set structures that are a fair bit weaker than that of the \(\sigma\)-algebra but they are considerably useful when attempting to generate \(\sigma\)-algebras and prove particularly tricky results in probability.

We call a collection of subsets \(\mathcal M \) of \(\Omega\) a monotone class if the following hold:

1) \(\mathcal M \) is closed under increasing unions: if \(A_{n} \subset A_{n+1} \forall n ,\) then \(\bigcup_{i \in \mathbb N} A_{i} \in \mathcal M.\)

2) \(\mathcal M \) is closed under decreasing intersections: if \(A_{n+1} \subset A_{n} \forall n, \) then \(\bigcap_{i \in \mathbb N} A_{i} \in \mathcal M.\)

We call a collection \(\mathcal P \) of subsets of \(\Omega\) a \( \pi\)-system if \(\mathcal P \) is closed under finite intersection, i.e.

1) \(\forall A,B \in \)\(\mathcal P \) \(\implies A \cap B \in \mathcal P.\)

A collection \(\mathcal L\) is called a \( \lambda\)-system if

1) \(\Omega \in \mathcal L\)
2) \(A \in \mathcal L \implies A^c \in \mathcal L\)
3) \(A_i \in \mathcal L A_i \subset A_{i+1} \forall A_i \implies \bigcup_{n \in \mathbb N} A_{n} \in \mathcal L.\)

Let \(\mathcal L\) be a \( \lambda\)-system, then by the definition of a \( \lambda\)-system we can derive the following property:

2-1) If \(A, B \in \mathcal L \) such that \(A \subset B\), then \(B-A \in \mathcal L.\)

Let \(\mathcal L\) be a \( \lambda\)-system. Let \(A, B \in \lambda \) such that \(A \subset B\). By property 2 and 3 of \( \lambda\)-systems, since \(B \in \mathcal L \implies B^c \in \mathcal L\). Thus, \(A \cup B^c \in \mathcal L \) since \(A \cap B^c = \emptyset\). Thus, by property 2 again, we see that \(A \cup B^c \in \mathcal L \implies (A \cup B^c)^c \in \mathcal L \). By DeMorgan's law, \((A \cup B^c)^c = A^c \cap B = B-A \in \mathcal L\).

Now, assume 1 and 2-1. Since \(\Omega \in \mathcal L\) and \(A \subset \Omega \implies \Omega-A \in \mathcal L\) and \(\Omega - A = \Omega \cap A^c = A^c\). So \(A^c \in \mathcal L\), which gives property 2 of \(\mathcal L\). \(_\square\)

Lemma: If \(\mathcal L\) is a \(\pi\)-system and a \(\lambda\)-system, then \(\mathcal L\) is a \(\sigma\)-algebra.

Let \(\mathcal L\) be a \(\pi\)-system and a \(\lambda\)-system. Then, \(\Omega \in \mathcal L \) and \(A \in \mathcal L \implies A^c \in \mathcal L \). So, we need only to show that \(\mathcal L\) is closed under countable unions. Let \(A_n \in \mathcal L\), where \(A_n\)'s are not necessarily disjoint. Hence, we can form a partition of the \(A_n\)'s as follows: Let \(B_1 = A_1\), \(B_2 = A_2-A_1\), \(B_3 = A_3 - (A_1 \cup A_2) \), ... , \(B_n = A_n -(A_1 \cup A_2 \cup ... \cup A_{n-1})\). Now we have a disjoint partition of the \(A_n\)'s and since \(\mathcal L\) is also a \(\pi\)-system, \(\mathcal L\) is closed under intersection, so \(\cup_n B_n = \cup_n A_n \in \mathcal L\) and therefore \(\mathcal L\) is a \(\sigma\)-algebra. \(_\square\)

We now will state and prove a useful theorem in the construction of probability measures, called the \(\pi \)-\(\lambda\) theorem.

Let \(\mathcal P \) be a \(\pi\)-system and \(\mathcal L\) a \(\lambda\)-system containing \(\mathcal P \). Then \(\sigma(\mathcal P) \subset \mathcal L.\)

Let \(\mathcal L\) be the smallest \(\lambda\)-system containing \(\mathcal P \), then \(\mathcal L \) is a \(\lambda\)-system as the intersection of all classes of the same type preserves the properties of that class. Let \(\sigma(\mathcal P)\) be the smallest \(\sigma\)-algebra containing \(\mathcal P\), then \(\sigma(\mathcal P)\) is closed under all properties of a \(\lambda \)-system, so \(\sigma(\mathcal P)\) is also a \(\lambda\)-system that contains \(\mathcal P\). Thus, \(\mathcal L \subset \sigma(\mathcal P)\), since \(\sigma(\mathcal P)\) is one such collection of the intersection of all \(\lambda \)-systems that contain \(\mathcal P\). Now we must show that \(\sigma(\mathcal P) \subset \mathcal L \). We do so by defining the following collection:

Let \(\mathcal L_A := \{B \subset \Omega : B \cap A \in \mathcal L \} \), that is, if \(A \in \mathcal L \), then \(\mathcal L_A\) is a \(\lambda\)-system. Let \(A \in \mathcal L \), and since \(\Omega \cap A = A \in \mathcal L\), we have \(\Omega \in \mathcal L_A\). Let \(B, C \in \mathcal L_A\), such that \(B \subset C \). Given \(A \in \mathcal L\), we have \( (C-B) \cap A = (C \cap A) - (B \cap A)\). Since \((B \cap A) \subset (C \cap A)\), we have that \((C-B) \cap A \in \mathcal L\), and thus \(C-B \in \mathcal L_A\), so 2-1 holds (which implies that 2 holds). Let \(B_i \in \mathcal L_A \forall i \in \mathbb N\), such that the \(B_i \)'s are disjoint. Since \(A \in \mathcal L\), then each \(B_i \cap A \) is disjoint and since \(\cup_i B_i \in \mathcal L\), we have that \(\cup_i(A \cap B_i) \in \mathcal L\), and thus \(\cup_i B_i \in \mathcal L_A\) and \(\mathcal L_A\) is a \(\lambda\)-system.

Now, if we assume that \(A \in \mathcal P\), \(B \in \mathcal P\), since \(\mathcal P\) is a \(\pi\)-system, then \(A \cap B \in \mathcal P\). Since \(\mathcal P \subset \mathcal L \implies A \cap B \in \mathcal L \implies B \in \mathcal L_A\), and thus \(\mathcal P \subset \mathcal L_A\). Thus, \(A \in \mathcal P \) and \(B \in \mathcal L \implies B \in \mathcal L_A \implies B \in \mathcal L_B \). By using the results from the definition on \(\mathcal L_A\), we see that \(B \in \mathcal L \implies \mathcal P \subset \mathcal L_B\). Again, by the construction of \(\mathcal L_A\), we see that \(\mathcal L_B \) is a \(\lambda\)-system and \(B \in \mathcal L \implies \mathcal L \subset \mathcal L_B\). Take \(B \in \mathcal L\) and \(C \in \mathcal L\) to arrive at either \(C \in \mathcal L \) or \(B \cap C \in \mathcal L\) and by symmetry of the construction, it must be the case that \(A \cap B \in \mathcal L\) and \(\mathcal L\) is both a \(\pi\)-system and a \(\lambda\)-system. Using the previous lemma, we have that \(\mathcal L\) is a \(\sigma\)-algebra that contains \(\mathcal P\). Thus, \(\sigma(\mathcal P) \subset \mathcal L\) and we see that \(\sigma(\mathcal P) = \mathcal L\). \(_\square\)

We now provide a construction that is used to generate probabilities that are defined on an algebra, \(\mathcal A\).

We call \(\mathbb P^*\) an outer probability measure if \( \forall A \in \Omega \), \(\mathbb P^*(A) = \inf \big\{\sum_{n \in \mathbb N} \mathbb P(B_n): B_n \in \mathcal A, A \subset \bigcup_{n \in \mathbb N} B_n \big\}.\)