There are three primary modes of convergence for random variables, ranked from strongest to weakest: | Mode of Convergence | Observed in | Implies | | ---------------------------------- | ------------------------------- | --------------------------- | | Convergence Almost Surely (”a.s.”) | Strong [[Law of Large Numbers]] | Convergence in Probability | | Convergence in Probability | Weak [[Law of Large Numbers]] | Convergence in Distribution | | Convergence in Distribution | [[Central Limit Theorem]] | - | ## Convergence Almost Surely A [[Convergence of Series#^7e111e|sequence]] of [[Random Variable|random variables]] $X_n$​ converges almost surely to a r.v. $X$ if, for almost every outcome $\omega$, the sequence $X_n(\omega)$ converges to $X(\omega)$ as $n \to \infty$. This is also referred to as *convergence with probability 1*. $ X_n \xrightarrow[n \to \infty]{a.s.}X \quad \text{ iff: } \mathbf P\Big (\{\omega:X_n(\omega) \xrightarrow[n \to \infty]{}X(\omega)\}\Big)=1 $ In limit notation we simply say that $X_n$ reaches $X$ at the limit. $ \lim_{n \to \infty} X_n =X $ ![[convergence-almost-surely.png|center|350]] ## Convergence in Probability A sequence of random variables $X_n$ converges to $X$ if the probability of $(X_n-X)$ being absolutely greater than some value $\epsilon$ goes to zero as $n \to \infty$. $ X_n \xrightarrow[n \to \infty]{\mathbf P}X \quad \text{ iff: } \mathbf P\Big (\lvert X_n-X \rvert \ge \epsilon\Big)\xrightarrow[n \to \infty]{}0 \quad \text{where } \epsilon>0 $ In limit notation we see that the limit is on the probability, not the r.v. itself. Therefore it is called convergence in probability. $ \lim_{n \to \infty} \mathbf P \big(\lvert X_n-X \rvert > \epsilon \big)=0 $ ![[convergence-in-probability.png|center|400]] **Key Difference:** - *For convergence in probability*, there can be some outliers far away from the limit, but its probability of occurring has to get smaller and smaller. - *For convergence almost surely* we want any possible value of the sequence to get closer and closer to the limit. ## Convergence in Distribution A sequence of random variables $X_n$​ converges in distribution to $X$ if the [[Cumulative Density Function|CDF]] of $X_n$​ converges pointwise to the CDF of $X$. $ X_n \xrightarrow[n \to \infty]{(d)}X \quad \text{ iff: } F_{X_n}(x) \xrightarrow[n \to \infty]{} F_X(x) $ In terms of interval probabilities the following is equivalent: $ \mathbf P(a\le X_n \le b) \xrightarrow[n \to \infty]{} \mathbf P(a\le X \le b) $ This is the weakest form of convergence. Only the distribution of $X_n$ converges to the distribution of $X$, while the actual values of $X_n$ might not converge to $X$.