We want to derivate the distribution of $Z=X+Y$, where $X$ and $Y$ are [[Independence of Random Variables|independent random variables.]] (i.e. $X \perp Y$). The approach depends on whether $X$ and $Y$ are discrete or continuous. ## Discrete Random Variables Let $Z=X+Y$ where $X$ and $Y$ are both discrete [[Random Variable|random variables]]. $ \begin{align} \mathbf P_Z(z)&= \sum_{g(X,Y)=z} \mathbf P_{X,Y}(X=x,Y=y) \tag{1}\\ \mathbf P_Z(z)&= \sum_x \mathbf P_{X,Y}(X=x,Y=z-x) \tag{2}\\ \mathbf P_Z(z)&= \sum_x \mathbf P_X(x)*\mathbf P_Y(z-x) \tag{3} \end{align} $ where: * (1) Sum the [[Joint Probability Mass Function|Joint PMF]] $\mathbf P_{X,Y}(x,y)$ over all pairs $(x,y)$ where $g(X,Y)=Z=x+y$. * (2) Since $Z=X+Y$ we substitute $Y=Z-X$. * (3) The independent of $X$ and $Y$ allows us to express their joint PMF as a product of the marginal PMFs. ## Continuous Random Variables Let $Z=X+Y$ where $X$ and $Y$ are both continuous r.v's.. The PDF $f_Z(z)$ can be derived using the convolution formula. **Conditional PDF:** $ \begin{align} f_{Z\vert X}(z \vert x) &= f_{Y+X \vert X}(z \vert x) \tag{1}\\ f_{Z\vert X}(z \vert x) &= f_{Y+x \vert X}(z \vert x) \tag{2}\\ &= f_{Y+x}(z) \tag{3}\\ &= f_Y(z-x) \tag{4} \end{align} $ where: * (1) Replace $Z$ with $X+Y$. * (2) Consider a specific conditional case where $X=x$. * (3) When $Y \perp X$, the addition of a constant $x$ does not impact this condition. Therefore $Y+x$ is also independent of $X$. Conditioning on an [[Independence of Events#Independence of Events|independent]] $X$ does not alter the distribution $f_{Y+x}$. Thus the conditioning can be removed. * (4) The distribution $f_{Y+x}$ is just a shifted version of $f_Y$. **Joint PDF:** We can obtain the joint distribution of $f_{X,Z}(x,z)$ by applying the [[Probability Multiplication Rule]]. $ f_{X,Z}(x,z)=f_X(x)*f_{Z \vert X}(z \vert x) \tag{5}$ **Marginal PDF:** Finally we get from the joint PDF to the marginal PDF, by integrating over one of the r.v’s.. Since we want $f_Z(z)$ we integrate over all $x$. By plugging in the definitions from above, we obtain a similar equation as in the discrete case. $ \begin{align} f_Z(z) &= \int_{-\infty}^{\infty} f_{X,Z}(x,z)*dx \tag{6}\\ f_Z(z) &= \int_{-\infty}^{\infty} f_X(x)*f_{Z \vert X}(z \vert x)*dx \tag{7}\\ f_Z(z) &= \int_{-\infty}^{\infty} f_X(x)*f_Y(z-x)*dx \tag{8} \end{align} $ where: - (6) Replace the joint distribution with multiplication from (5). - (7) Replace the conditional distribution with (4). We call the final expression [[Convolution]]. ## Gaussian Random Variables This is a special case of the continuous random variables. If $X \sim \mathcal N(\mu_X, \sigma_X^2)$ and $Y \sim \mathcal N(\mu_Y, \sigma_Y^2)$, their sum $Z=X+Y$ is also [[Gaussian Distribution]]. The parameters of $Z$ are: $ \begin{align} \mu_Z &= \mu_X + \mu_Y \\[4pt] \sigma_Z^2 &= \sigma_X^2 + \sigma_Y^2 \end{align} $ Using the convolution formula: $ f_Z(z) = \int_{-\infty}^{\infty} f_X(x)*f_Y(z-x)*dx$ where: $ \begin{align} f_X(x)&=\frac{1}{\sqrt{2\pi}\sigma_x}* e^{-(x-\mu_x)^2/2\sigma_x^2} \\ f_Y(y)&=\frac{1}{\sqrt{2\pi}\sigma_y}* e^{-(y-\mu_y)^2/2\sigma_y^2} \end{align} $ We simply plug-in the PDFs. After some algebra we obtain $f_Z(z)$ in the form of a new Gaussian PDF. $ \begin{align} f_Z(z) &= \int_{-\infty}^{\infty} \frac{1}{\sqrt{2\pi}\sigma_x} \exp \Big \lbrace -\frac{(x-\mu_x)^2}{2\sigma_x^2} \Big \rbrace * \frac{1}{\sqrt{2\pi}\sigma_y} \exp\Big \lbrace - \frac{(z-x-\mu_y)^2}{2\sigma_y^2} \Big \rbrace * dx \\[8pt] f_Z(z) &= \frac{1}{\sqrt{2\pi(\sigma_x^2 +\sigma_y^2)}} * \exp \Big \lbrace{-\frac{{ (z-\mu_x - \mu_y})^2}{2(\sigma_x^2 + \sigma_y^2) }} \Big \rbrace \end{align} $ which can be written as: $ Z \sim \mathcal N(\mu_x+\mu_y, \sigma_x^2+\sigma_Y^2)$ > [!note:] > This result generalized to any number of independent Gaussian r.v's.. ## Mixed Discrete and Continuous Random Variables We want to derive the convolution formula by following the standard approach from [[General Functions of Random Variables]]. Let $Z=X+Y$ where $X$ is a discrete r.v. and $Y$ is a continuous r.v.. *Step 1:* Express the CDF $Z$ in terms of $X,Y$ $ \begin{align} F_Z(z) &= \mathbf P(Z \le z) \tag{1}\\[8pt] &= \mathbf P(X+Y \le z) \tag{2}\\[8pt] &=\sum_x \mathbf P(X+Y \le z \vert X =x)*\mathbf P(X=x) \tag{3}\\ &=\sum_x \mathbf P(x+Y \le z \vert X=x)* p_X(x) \tag{4}\\ &=\sum_x \mathbf P(x+Y \le z) * p_X(x) \tag{5}\\ &=\sum_x \mathbf P(Y \le z-x)*p_X(x) \tag{6}\\ &= \sum_x F_Y(z-x)*p_X(x) \tag{7} \end{align} $ where: * (3) We apply the [[Total Probability Theorem]] to sum over all conditional probabilities. It is easier to set the discrete r.v. as the condition. * (4) Within the summation $X$ is fixed to $x$. * (5) When $Y \perp X$, the addition of a constant $x$ does not impact this condition. Therefore $Y+x$ is also independent of $X$. Conditioning on an [[Independence of Events#Independence of Events|independent]] $X$ does not alter the distribution. Thus the conditioning can be removed. *Step 2:* Differentiate the CDF to obtain the PDF of $Z$ $ \begin{align} f_Z(z) &= \frac{d}{dz} F_Z(z) \tag{1}\\[8pt] &= \frac{d}{dz}\sum_x F_Y(z-x)*p_X(x) \tag{2}\\[4pt] &= \sum_x \frac{d}{dz} F_Y(z-x)*p_X(x) \tag{3}\\[4pt] &= \sum_x f_Y(z-x)*p_X(x) \tag{4} \end{align} $ where: * (3) we can put the derivative into the sum. This is valid when $x$ is a finite set. * (4) We apply the chain rule, but the inner derivative $(z-x)$ is just $1$. Also $p_x(x)$ is just a factor that stays unchanged since we differentiate w.r.t. $z$.