Bayes' Rule allows us to update our beliefs about the state of a [[Random Variable]] $X$ (or [[Events|event]] $A_i$​) given the observation of another r.v. $Y$ (or event $B$). The rule can be derived from the definition of [[Conditional Probability]]: - Numerator → Expanded by [[Probability Multiplication Rule]] - Denominator → Expanded by [[Total Probability Theorem]] **Events:** $ \mathbf P(A_i \vert B) = \frac{\mathbf P(A_i \cap B)}{\mathbf P(B)} = \frac{\mathbf P(A_i)*\mathbf P(B \vert A_i)}{\mathbf P(B)} = \frac{\mathbf P(A_i)*\mathbf P(B \vert A_i)}{\sum_j \mathbf P(A_j) * \mathbf P(B \vert A_j)} $ **Discrete Random Variables:** If $X$ and $Y$ are discrete, the Bayes' rule is expressed in terms of both of their [[Probability Mass Function|PMFs]]. $ \begin{align} \mathbf P(X=x \vert Y=y) &= \frac{\mathbf P(X=x)*\mathbf P(Y=y \vert X=x)}{\mathbf P(Y=y)} \\[6pt] &= \frac{\mathbf P(X=x)*\mathbf P(Y=y \vert X=x)}{\sum_{x^\prime} \mathbf P(X=x^\prime) * \mathbf P(Y=y \vert X=x^\prime)} \end{align} $ **Continuous Random Variables:** If $X$ and $Y$ are continuous, the Bayes' rule is expressed in terms of both of their [[Probability Density Function|PDFs]]. $ \begin{align} f_{X\vert Y}(x \vert y) &= \frac{f_X(x)*f_{X \vert Y}(x \vert y)}{f_Y(y)} \\[6pt] &= \frac{f_{Y \vert X}(y \vert x)*f_X(x)}{\int_{x^\prime} f_{Y \vert X}(y \vert x^\prime)*f_X(x^\prime)\, dx} \end{align} $ ## Bayesian Inference We have some initial beliefs about each $\mathbf P(A_i)$, where $A_i$ is some scenario (or feature attribute) that should have impact on some output $B$. - *Model:* Given the current scenario $A_1$ our model returns a probability for $B$. $ A_i \xrightarrow[P(B \vert A_i)]{\text{model}}B $ - *Inference:* Once we have observed $B$ and can draw conclusions about the impact of the scenarios on $B$. $ B \xrightarrow[P(A_i \vert B)]{\text{inference}}A_i $