## Two Random Variables Let $Z=X_1+X_2$ and we want to find out $\mathrm {Var}(Z)$, to see how the variance for a sum of a multiple [[Random Variable|random variables]] behaves. $ \mathrm{Var}(Z) = \mathbb{E}\Big [(Z-\mathbb{E}[Z])^2 \Big ] $ Derivation: $ \begin{align} \mathrm {Var}(Z)&=\mathbb{E}\Big[\big(\overbrace{X_1+X_2}^{Z}-\mathbb{E}[\overbrace{X_1+X_2}^{Z}]\big)^2 \Big] \tag{1} \\ &=\mathbb{E}\Big[\big(X_1+X_2-(\mathbb{E}[X_1]+\mathbb{E}[X_2])\big)^2 \Big] \tag{2}\\ &=\mathbb{E}\Big [\big((X_1-\mathbb{E}[X_1])+(X_2-\mathbb{E}[X_2])\big)^2 \Big] \tag{3}\\ &=\mathbb{E}\Big [(X_1-\mathbb{E}[X_1])^2+(X_2-\mathbb{E}[X_2])^2-2(X_1-\mathbb{E}[X_1])*(X_2-\mathbb{E}[X_2]) \Big] \tag{4}\\[6pt] &= \mathrm{Var}(X_1)+\mathrm{Var}(X_2)-2 * \mathrm{Cov}(X_1,X_2) \tag{5} \end{align} $ where: - (2) Applying [[Linearity of Expectations]] to split up $\mathbb E[X_1+X_2]$. - (3) Re-arrange terms to solve the quadratic. ## Special Case of Independence When $X_1$ and $X_2$ are [[Independence of Random Variables|independent]] r.v's, their [[Covariance]] will be $0$. This makes the variance of the sum equal to the sum of the variances. $ \mathrm{Var}(X+Y)= \mathrm{Var}(X) + \mathrm{Var}(Y), \quad \text{when } X \perp Y$ We can combine this with findings from [[Variance after Linear Transformation]], where $a,b,c$ are constants. $ \mathrm{Var}(aX+bY+c)=a^2\mathrm{Var}(X)+b^2\mathrm{Var}(Y) $ > [!note:] > While independence implies additivity of variance terms, the converse does not need to hold. Thus it can be the case that $\mathrm{Var}(X+Y)=\mathrm{Var}(X)+\mathrm{Var}(Y)$ even if $X,Y$ are not independent. ## Multiple Random Variables Now consider $Z$ to be the sum of $n$ r.v's. $\sum_{i=1}^n X_i$. For simplicity, assume that $\mathbb E[X_i]=0$ for all $X_i$. We can safely assume this, as adding a constant [[Variance after Linear Transformation|does not affect variance]]. Simplified definitions: $ \text{ when } \mathbb E[X_i]=0 \:\forall X_i \begin{cases} \mathrm{Var}(X_i) = \mathbb{E}[X_i^2]\\ \mathrm{Cov}(X_i,X_j) = \mathbb{E}[X_i X_j] \end{cases} $ Derivation: $ \begin{align} \mathrm {Var}(X_1+\dots +X_n)&=\mathbb{E}\left[\Big(\sum_{i=1}^n X_i \Big)^2\right] \tag{1}\\[4pt] &=\mathbb{E}\left[\sum_{i=1}^n X_i^2 +2*\sum_{i<j} X_iX_j \right] \tag{2}\\[4pt] &=\sum_{i=1}^n \mathbb{E} [X_i^2] +2*\sum_{i<j} \mathbb{E}[X_i X_j] \tag{3}\\[4pt] &=\sum_{i=1}^n \mathrm{Var}(X_i)+2*\sum_{i<j} \mathrm{Cov}(X_i,X_j) \tag{4} \end{align} $ where: - (2) Expanding the quadratic of the sum of $X_i$. - (3) Applying [[Linearity of Expectations]]. - (4) Apply simplified definitions of variance and covariance as established above. > [!note:] > As we only sum over half of the covariance matrix $\sum_{(i,j): i<j}$, we need to multiply by 2. Alternatively we could also sum over $\sum_{(i,j): i \neq j}$ and drop this factor of 2.