Classical statistics distinguishes itself from Bayesian statistics, mainly in the different perspectives on the quantity $\theta$. **Bayesian Statistics:** - $\Theta$ is a r.v. whose distribution is shaped according to a prior belief. - We work with a single model of [[Conditional Probability]] $p_{X \vert \Theta}(x \vert \theta)$ to estimate $\theta$. - Inference is made by combining the likelihood of the data with the prior belief about $\Theta$ ([[Bayes Rule]]). **Classical Statistics:** - $\theta$ is a unknown but fixed constant. - We work with multiple candidate models $p_X(x; \theta)$, one for each possible value of $\theta$. - Inference involves selecting the model which maximizes the likelihood. | | Bayesian Statistics | Classical Statistics | | ------------------------- | ------------------------------------ | -------------------- | | Data $(X)$ | r.v. | r.v. | | Estimator $(\hat \Theta)$ | r.v. | r.v. | | Unknown $(\Theta)$ | r.v. | - | | Unknown $(\theta)$ | realization of r.v. | unknown constant | | Model | $p_{X \vert \Theta}(x \vert \theta)$ | $p_X(x; \theta)$ | **Arguments for Classical Statistics:** - *Interpretation:* Sometimes it does not make sense to model some unknown $\theta$ as distribution, especially when it is some natural but unknown constant (e.g. the mass of an electron). - *Avoiding bias:* Sometimes you do not want to bias your estimate, by imposing a prior distribution, when you have little knowledge about it.