## Bernoulli Distribution Assume $X_1, \dots, X_n \stackrel{iid}\sim \text{Ber}(p)$ from [[Bernoulli Distribution]]. *Model:* $\Big(\{0,1\}, \{\text{Ber}(p)\}_{p \in (0,1)}\Big) $ *Density:* $ X= \begin{cases} 1 & \text{w.p.} &p\\ 0 & \text{w.p.} &(1-p)\\ \end{cases} $ *Likelihood:* $ \begin{align} L(x_1,\dots,x_n;p) &=\prod_{i=1}^n p_X(x_i) \tag{1}\\[4pt] &=\prod_{i=1}^n p^{x_i}*(1-p)^{1-x_i} \tag{2}\\[6pt] &=p^{\sum_{i=1}^n x_i}*(1-p)^{n-\sum_{i=1}^n x_i} \tag{3} \end{align} $ where: - (2) Either $x_i=1$, and only $p$ gets triggered, or $X_i=0$, where only $(1-p)$ gets triggered. - (3) The product OF the exponent is just the sum IN the exponent. ## Poisson Distribution Assume $X_1, \dots, X_n \stackrel{iid}\sim \text{Pois}(\lambda)$ from [[Poisson Distribution]]. *Model:* $ \Big(\mathbb N, \{\text{Pois}(\lambda)\}_{\lambda \in (0,\infty)}\Big) $ *Density:* $ \mathbf P(x_i)=p_{x_i}(k)=\frac{\lambda^{x_i}}{x_i!}*e^{-\lambda} $ *Likelihood:* $ \begin{align} L(x_1, \dots,x_n;\lambda) &=\prod_{i=1}^n p_X(x_i) \tag{1}\\[6pt] &=\prod_{i=1}^n\frac{\lambda^{x_i}}{x_i!}*e^{-\lambda} \tag{2}\\[6pt] &=e^{-n\lambda}*\prod_{i=1}^n\frac{\lambda^{x_i}}{x_i!} \tag{3}\\[6pt] &=e^{-n\lambda}* \frac{\lambda^{\sum_{i=1}^nx_i}}{x_1!\dots x_n!} \tag{4} \end{align} $ (2) Since $e^{-\lambda}$ is not dependent on $i$, we can take it out of the product by multiplying its exponent by $n$. ## Gaussian Distribution Assume $X_1, \dots, X_n \stackrel{iid}\sim \mathcal N(\mu, \sigma^2)$ from [[Gaussian Distribution]]. *Model:* $ \Big(\mathbb R, \{\mathcal N(\mu, \sigma^2)\}_{(\mu, \sigma^2) \in \mathbb R \times (0, \infty)}\Big) $ *Density:* $ f_{\mu, \sigma^2}(x)=\frac{1}{\sigma \sqrt{2 \pi}}*\exp\Big(-\frac{(x- \mu)^2}{2 \sigma^2} \Big) $ *Likelihood:* $ \begin{align} L(x_1,\dots ,x_n;\mu,\sigma^2) &=\prod_{i=1}^nf(x_i) \tag{1}\\ &=\frac{1}{(\sigma \sqrt{2 \pi})^n} \prod_{i=1}^n \exp\Big(-\frac{(x_i- \mu)^2}{2 \sigma^2} \Big) \tag{2}\\[6pt] &=\frac{1}{(\sigma \sqrt{2 \pi})^n} \exp\Big(\sum_{i=1}^n-\frac{(x_i- \mu)^2}{2 \sigma^2} \Big) \tag{3}\\[6pt] &=\frac{1}{(\sigma \sqrt{2 \pi})^n} \exp\Big(-\frac{1}{2 \sigma^2}\sum_{i=1}^n(x_i- \mu)^2 \Big) \tag{4} \end{align} $ where: - (2) Bring the constant term outside of the product. By doing so we need to set it to be power of $n$. - (4) Bring the constant term outside of the sum. ## Exponential Distribution Assume $X_1, \dots, X_n \stackrel{iid}\sim \text{Exp}(\lambda)$ from [[Exponential Distribution]]. *Model:* $ \Big((0, \infty), \{\text{Exp}(\lambda)\}_{\lambda \in (0,\infty)}\Big) $ *Density:* $ f_X(x) = \begin{cases} \lambda e^{-\lambda x} & \text{when } x \ge 0\\ 0 & \text{when } x <0 \end{cases} $ *Likelihood:* $ \begin{align} L(x_1, \dots ,x_n; \lambda) &=\prod_{i=1}^n \lambda e^{-\lambda x_i} \\ &=\lambda^n \exp\left(\sum_{i=1}^n -\lambda x_i\right) \\ &=\lambda^n \exp \left(-\lambda\sum_{i=1}^n x_i\right) \prod_{i=1}^n \mathbf 1_{(x_i>0)} \end{align} $ Note that the indicator function $\mathbf 1_{x_i>0}$ returns $1$ of $x_1$ is positive and $0$ otherwise. The product of indicators makes sure, that all $x_i$ are positive, and returns $0$ otherwise, which would make the likelihood zero as well. However, at this point we assume that we only deal with a well specified model. This means that the data matches the chosen model, so we can also omit indicators that do not contain the parameters, over which we want to maximize later. ## Uniform Distribution Assume $X_1, \dots, X_n \stackrel{iid}\sim \mathcal U[a,b]$ from [[Continuous Uniform Distribution]]. *Model:* $ \Big(\mathbb R, \{\mathcal U[a,b]\}_{(a,b) \in \mathbb R, \,a<b} \Big) $ *Density:* $ f(x)=\frac{1}{b-a} \quad \text{for } a \le x_i \le b $ *Likelihood:* $ \begin{align} L(x_1,\dots,x_n; b) &= \prod_{i=1}^n f(x_i) *\prod_{i=1}^n \mathbf 1_{(a\le x_i \le b)} \\[6pt] &= \frac{1}{(b-a)^n}*\prod_{i=1}^n \mathbf 1_{(a\le x_i)}*\prod_{i=1}^n \mathbf 1_{(x_i \le b)} \\[6pt] &=\frac{1}{(b-a)^n}*\mathbf 1_{(a \le \min (x_i), \le \max x_i \le b)} \end{align} $