The independence of [[Random Variable|random variables]] extends the concept of [[Independence of Events]]. Two random variables $X$ and $Y$ are independent if knowing the value of one provides no information about the other. This is mathematically expressed through the [[Joint Probability Mass Function|Joint PMF]] or [[Joint Probability Density Function|Joint PDF]]: **Discrete:** $p_{X,Y}(x,y)=p_X(x)*p_Y(y) \quad \forall x,y $ **Continuous:** $ f_{X,Y}(x,y)=f_X(x)*f_Y(y) \quad \forall x,y $ This definition implies that the conditional distribution of $X$ given $Y$ is equal to the marginal distribution of $X$ and vice versa, as shown below for the continuous case. $ f_{X\vert Y}(x \vert y)= \frac{f_{X,Y}(x,y)}{f_Y(y)}= \frac{f_X(x)*f_Y(y)}{f_Y(y)}= f_X(x) $ This property holds symmetrically for $f_{Y\vert X}(y\vert x)=f_Y(y)$. ## From Events to Random Variables To connect the concept of independence of events to independence of random variables, we use indicator variables. For events $A$ and $B$, let: $ \begin{align} I_A : X = \begin{cases} 1 & \text{if } A \text{ occurs} \\ 0 & \text{otherwise} \end{cases} \qquad I_B :Y = \begin{cases} 1 & \text{if } B \text{ occurs} \\ 0 & \text{otherwise} \end{cases} \end{align} $ If $A$ and $B$ are independent, their indicator variables $I_A$ and $I_B$ are also independent. $ p_{I_A,I_B}(x,y)=p_{I_A}(x)*p_{I_B}(y), \quad \forall x,y \in \{0,1\}$ ## Independence and the Joint CDF When the r.v. $X \perp Y$ are independent, the [[Joint Cumulative Density Function|Joint CDF]] is the product of their marginal CDFs. **Derivation:** $ \begin{align} F_{X,Y}(x,y) &= \mathbf P(X\le x, Y\le y) \tag{1}\\[10pt] &= \int_{-\infty }^y \int_{-\infty }^x f_{X,Y}(x,y) \, dx \, dy \tag{2}\\[10pt] &= \int_{-\infty}^x f_X(x) \, dx \int_{-\infty}^y f_Y(y) \, dy \tag{3}\\[14pt] &=F_X(x) * F_Y(y) \tag{4} \end{align} $ (2) Express the joint CDF as a double integral over the joint PDF. (3) Since the double integral is like a nested loop, where the the value of the outer loop is always fixed during the inner loop, we can factor out the outer loop. This results in product of two separate integrals. (4) Each integral represents a CDF.