We assume the distribution of a [[Random Variable]] consists of two sub-populations, where each sub-population follows a separate [[Gaussian Distribution]]: - $X_1 \sim \mathcal{N}(\mu_1, \sigma_1^2)$ - $X_2 \sim \mathcal{N}(\mu_2, \sigma_2^2)$ The mixing proportion $\pi^\star$ determines the contribution of each sub-population in the overall mixture: - $\pi^\star \in [0,1]$: Proportion of first sub-population. - $1-\pi^\star$: Proportion of second sub-population. The [[Probability Density Function|PDF]] of the mixture is a linear combination of the individual Gaussian PDFs: $ f(x)=\pi^\star* \frac{1}{\sigma_1\sqrt{2\pi}}* \exp\Big(-\frac{(x-\mu_1)^2}{2\sigma_1^2}\Big) + (1-\pi^\star)* \frac{1}{\sigma_2\sqrt{2\pi}}* \exp\Big(-\frac{(x-\mu_2)^2}{2\sigma_2^2}\Big) $ Using a compact notation for the individual Gaussian PDFs $f_1(x)$ and $f_2(x)$: $ f(x) = \pi^\star *f_1(x) + (1- \pi^\star)*f_2(x) $ ## Validity of the Mixture To ensure $f(x)$ is a valid PDF, it must satisfy the conditions from the [[Probability Axioms]]: - *Non-negativity:* All densities need to be greater equal to zero → $f(x)\ge0$ - *Normalization:* The integral over the entire domain must equal one → $f(x)=1$. **Proof of Non-negativity:** Since both $f_1(x), f_2(x)$ satisfy the condition of positive density separately, the mixture (a linear combination with positive factors) has to be satisfied as well. $ \begin{rcases} f_1(x)\ge0\\ f_2(x)\ge0 \end{rcases} f(x)\ge0 $ **Proof of Normalization:** Since both densities $f_1(x), f_2(x)$ integrate to $1$, the multiplication with $\pi^\star$ and $(1-\pi^\star)$ has to equal $1$ as well. $ \begin{aligned} \int f(x) \,dx&= \pi^\star* \overbrace{\int f_1(x) \,dx}^{=1} + (1-\pi^\star)* \overbrace{\int f_2(x) \,dx}^{=1} \\ &=\pi^\star_1 + (1-\pi^\star)*1 \\[6pt] &=1 \end{aligned} $ ## Random Weights Representation Instead of fixed weights $\{\pi^\star, 1-\pi^\star \}$, we can also model the weights of a mixture as a [[Bernoulli Distribution|Bernoulli]] r.v. $Z \sim \text{Ber}(\pi)$. $ X=ZX_1+ (1-Z)X_2 $ So if $Z$ returns $1$, $X$ will be sampled only from $X_1 \sim \mathcal{N}(\mu_1, \sigma_1^2)$ and otherwise only from $X_2 \sim \mathcal{N}(\mu_2, \sigma_2^2)$. $ Z = \begin{cases} 1 & \text{w.p. } \pi &&\text{then } X\sim N(\mu_1, \sigma_1^2) \\ 0 & \text{w.p. } (1-\pi) &&\text{then } X\sim N(\mu_2, \sigma_2^2) \end{cases} $ For this representation to give the same result as with a fixed $\pi$, both $X_1, X_2$ have to be independent of $Z$. This means e.g. $f(X_1\vert Z=1)=f(X_1)$.