Modes of Convergence

Almost Sure Convergence

An event $A \in \mathcal{F}$ holds almost surely (a.s.) if $P(A) = 1$.

$X_n$ converges a.s. to $X$, $X_n \to X$ a.s., if there is a set $N$ with $P(N) = 0$ such that $X_n(\omega) \to X(\omega)$ for all $\omega \notin N$. Equivalently,

$$\limsup_{n} X_n = \liminf_{n} X_n = X \quad \text{a.s.}$$

Example: if $X_n$ are iid with df $F$ and $F(x) < 1$ for all $x$, then $M_n := \max_{i \le n} X_i \to \infty$ a.s.

Convergence in Probability

$X_n$ converges in probability to $X$, $X_n \xrightarrow{P} X$, if for all $\varepsilon > 0$

$$P(|X_n - X| > \varepsilon) \to 0 \quad \text{as } n \to \infty$$

If $X_n \to X$ a.s. then $X_n \xrightarrow{P} X$. The converse is false.

$(X_n)$ is Cauchy in probability if for all $\varepsilon > 0$, $P(|X_n - X_m| > \varepsilon) \to 0$ as $n, m \to \infty$. For real RVs, $X_n$ converges in probability to some RV iff $(X_n)$ is Cauchy in probability.

Subsequence Criterion $X_n \xrightarrow{P} X$ iff every subsequence of $(X_n)$ has a further subsequence that converges to $X$ a.s. In particular, if $X_n \xrightarrow{P} X$, some subsequence converges to $X$ a.s.

Continuous Mapping Let $g : \mathbb{R} \to \mathbb{R}$ be continuous. If $X_n \to X$ a.s. then $g(X_n) \to g(X)$ a.s. If $X_n \xrightarrow{P} X$ then $g(X_n) \xrightarrow{P} g(X)$.

DCT in Probability If $X_n \xrightarrow{P} X$ and $|X_n| \le \xi$ a.s. for all $n$, where $\xi \ge 0$ and $E\xi < \infty$, then $X \in L^1$ and $E X_n \to E X$.

If $X_n \xrightarrow{P} X$ and $Y_n \xrightarrow{P} Y$, then $X_n + Y_n \xrightarrow{P} X + Y$ and $X_n Y_n \xrightarrow{P} XY$.

$L^p$ Convergence

For $1 \le p < \infty$, $L^p$ is the set of RVs with $E|X|^p < \infty$. Its norm is $\|X\|_p = (E|X|^p)^{1/p}$ and its metric is $d(X,Y) = \|X - Y\|_p$, identifying RVs that are equal a.s.

$X_n$ converges in $L^p$ to $X$, $X_n \xrightarrow{L^p} X$, if $E|X_n - X|^p \to 0$. $(X_n)$ is Cauchy in $L^p$ if $\|X_n - X_m\|_p \to 0$ as $n, m \to \infty$.

$L^2$ is a Hilbert space with $\langle X, Y \rangle = E XY$.

If $X_n \xrightarrow{L^p} X$ then $X_n \xrightarrow{P} X$, and $\|X_n\|_p \to \|X\|_p$.

Counterexamples

1. In probability does not imply $L^p$ $X_n = 2^n Y_n$ with $Y_n \sim \mathrm{Bern}(\frac{1}{n})$. Then $X_n \xrightarrow{P} 0$ but $E|X_n|^p = 2^{np}/n \to \infty$.
2. $L^p$ does not imply a.s. For $U \sim \mathrm{Unif}(0,1)$, the indicators $\mathbf{1}\{U \in (\frac{j-1}{k}, \frac{j}{k}]\}$, $j = 1, \ldots, k$, $k = 1, 2, \ldots$, sweeping $(0,1)$. $E|X_n|^p = \frac{1}{k} \to 0$, but $X_n(\omega)$ converges for no $\omega$.
3. a.s. does not imply $L^p$ $X_n = n^{1/p} \mathbf{1}_{(0, 1/n)}$ on $(0,1)$. Then $X_n \to 0$ a.s. but $E|X_n|^p = 1$.

Convergence in Distribution

$X_n \xrightarrow{d} X$ if $\lim_{n \to \infty} F_n(t) = F(t)$ at every continuity point $t$ of $F$.

• If $g : \mathbb{R} \to \mathbb{R}$ is continuous and $X_n \xrightarrow{P} X$, then $g(X_n) \xrightarrow{P} g(X)$.
• If $X_n$ (on the same $\Omega$) converges in distribution to a constant $c$, then $X_n \xrightarrow{P} c$.
• Slutsky If $X_n \xrightarrow{d} X$ and $Y_n \xrightarrow{d} c$ on the same $\Omega$, then $X_n + Y_n \xrightarrow{d} X + c$, $X_n Y_n \xrightarrow{d} cX$, and $X_n / Y_n \xrightarrow{d} X/c$ if $c \neq 0$.

Summary

• a.s. convergence $\Rightarrow$ convergence in probability
• $L^p$ convergence $\Rightarrow$ convergence in probability
• convergence in probability $\Rightarrow$ a.s. convergence along a subsequence
• convergence in probability and ui $\iff$ $L^1$ convergence
• $L^p$ convergence $\Rightarrow$ $L^q$ convergence for $1 \le q \le p$

ui is uniform integrability, see here.