Independence

Independent Events

$A, B \in \mathcal{B}$ are independent if

$$P(A \cap B) = P(A)P(B)$$

$A_1, \ldots, A_n$ are mutually independent if

$$P\Big(\bigcap_{i \in I} A_i\Big) = \prod_{i \in I} P(A_i) \quad \text{for all nonempty } I \subseteq \{1, \ldots, n\}$$

Independent Classes

Nonempty classes $\mathcal{C}_1, \ldots, \mathcal{C}_n \subseteq \mathcal{B}$ are independent if every choice of $A_i \in \mathcal{C}_i$, $i = 1, \ldots, n$, gives mutually independent events.

For fixed $B \in \mathcal{B}$, $\mathcal{L}_B = \{A \in \mathcal{B} : P(A \cap B) = P(A)P(B)\}$ is a $\lambda$-system. Using Dynkin's theorem, if $\mathcal{C}_1, \ldots, \mathcal{C}_n \subseteq \mathcal{B}$ are independent nonempty $\pi$-systems, then $\sigma(\mathcal{C}_1), \ldots, \sigma(\mathcal{C}_n)$ are independent.

If $\mathcal{F}_1, \mathcal{F}_2, \mathcal{F}_3$ are independent $\sigma$-algebras, then $\mathcal{F}_1 \vee \mathcal{F}_2 := \sigma(\mathcal{F}_1 \cup \mathcal{F}_2)$ is independent of $\mathcal{F}_3$.

Independent Random Variables

$\{X_t : t \in T\}$ are independent if $\{\sigma(X_t) : t \in T\}$ are independent.

Factorization $\{X_t : t \in T\}$ are independent iff for all finite nonempty $J \subseteq T$ and all $x_t \in \mathbb{R}$, $t \in J$,

$$F_J(x_t : t \in J) := P\Big(\bigcap_{t \in J} [X_t \le x_t]\Big) = \prod_{t \in J} P([X_t \le x_t]) = \prod_{t \in J} F_t(x_t)$$

A discrete RV is a real valued RV with range $X(\Omega)$ finite or countably infinite. Discrete $X_1, \ldots, X_n$ are independent iff

$$P\Big(\bigcap_{i=1}^{n} [X_i = x_i]\Big) = \prod_{i=1}^{n} P([X_i = x_i]) \quad \text{for all } x_1, \ldots, x_n \in \mathbb{R}$$

If $X, Y$ are independent real RVs with continuous distribution functions, $P[X = Y] = 0$.

Ranks and Records

Let $X_1, X_2, \ldots$ be iid with a continuous df. The relative rank of $X_n$ among $X_1, \ldots, X_n$ is

$$R_n = \sum_{i=1}^{n} \mathbf{1}\{X_i \ge X_n\}$$

Rényi

1. $R_1, R_2, \ldots$ are independent and $P(R_n = k) = 1/n$ for $k = 1, \ldots, n$.
2. Let $A_n$ be the event that $X_n$ is a new upper record, $A_1 = \Omega$ and $A_n = \{X_n > \max_{1 \le i \le n-1} X_i\}$ for $n \ge 2$. Then $A_1, A_2, \ldots$ are independent and $P(A_n) = 1/n$.

Borel-Cantelli

$$\limsup_{n \to \infty} A_n = \bigcap_{n=1}^{\infty} \bigcup_{k=n}^{\infty} A_k = \{A_n \text{ i.o.}\}$$

If $B_n := \bigcup_{k=n}^{\infty} A_k$, then $B_n \downarrow \limsup_{n \to \infty} A_n$.

First Borel-Cantelli For any events, if $\sum_{n=1}^{\infty} P(A_n) < \infty$ then $P(A_n \text{ i.o.}) = 0$.

Borel's Zero-One Law If $\{A_n : n \ge 1\}$ are independent events, then

$$P(A_n \text{ i.o.}) = \begin{cases} 0 & \text{iff } \sum_{n=1}^{\infty} P(A_n) < \infty \\ 1 & \text{iff } \sum_{n=1}^{\infty} P(A_n) = \infty \end{cases}$$

For example, for independent $X_n \sim \mathrm{Bern}(p_n)$, $X_n \to 0$ a.s. iff $\sum p_n < \infty$. For iid $E_n \sim \mathrm{Exp}(1)$,

$$\limsup_{n \to \infty} \frac{E_n}{\log n} = 1 \quad \text{a.s.}$$

Kolmogorov's Zero-One Law

For RVs $X_n$, $n \ge 1$, set $\mathcal{F}'_n = \sigma(X_{n+1}, X_{n+2}, \ldots)$. The tail $\sigma$-field is

$$\mathcal{T} = \bigcap_{n \ge 1} \mathcal{F}'_n$$

Kolmogorov Let $(X_n)_{n \ge 1}$ be independent RVs with tail $\sigma$-field $\mathcal{T}$. If $\Lambda \in \mathcal{T}$, then $P(\Lambda) = 0$ or $1$.

A $\sigma$-field is almost trivial if all its events have probability 0 or 1. If $\mathcal{G}$ is almost trivial and $X$ is a $\mathcal{G}$-measurable RV, there is a $c$ such that $P(X = c) = 1$.

So for independent $(X_n)_{n \ge 1}$,

1. the probability that $\sum_{n=1}^{\infty} X_n$ converges is 0 or 1,
2. $\limsup X_n$ and $\liminf X_n$ are a.s. constant, possibly different constants,
3. for each $c \in \mathbb{R}$, $P(\bar{X}_n \to c)$ is 0 or 1, where $\bar{X}_n = \frac{1}{n} \sum_{i=1}^{n} X_i$.