Let us partition the [[Sample Space]] into finite number of [[Events]], e.g. $\{A_1, A_2, A_3\}$. The total probability theorem (”TPT”) states that we can obtain an unconditional probability of $\mathbf P(B)$, by summing the [[Conditional Probability]] of each partition $\mathbf P(B \vert A_i)$ and weighting them by size size of the respective partition $P(A_i)$. ^83d664 $ \begin{aligned} \mathbf P(B) &= \mathbf P(B\vert A_1)*\mathbf P(A_1) \\ &+ \mathbf P(B\vert A_2)*\mathbf P(A_2) \\ &+ \mathbf P(B\vert A_3)*\mathbf P(A_3) \end{aligned} $ Obviously this extends to any $n$ partitions. $ \mathbf P(B) = \sum_{i=1}^n \mathbf P(B\vert A_i)*\mathbf P(A_i) $ We can derive TPT by adding disjoint parts of $B$ from each partition (e.g. $B \cap A_1$). Each of these pieces can be deconstructed via the [[Probability Multiplication Rule]] into $\mathbf P(B \vert A_1)*\mathbf P(A_1)$. ![[total-probability-theorem.png]] ## Extension to Discrete Random Variables To prove that this theorem has to work also for discrete [[Random Variable|random variables]] ("r.v."), we simply say that event $B$ is when r.v. $X$ equals some realization $x$. This lets us translate the probability of an event $\mathbf P (B)$ into a [[Probability Mass Function#^ac7640|PMF]]. $ \begin{align} \mathbf P(B) &:\mathbf P(X=x) =p_X(x) \\ \mathbf P(B\vert A_i) &:\mathbf P(X=x \vert A_i) =p_{X\vert A_i}(x) \end{align} $ We perform the substitution and have established the TPT for PMF’s. $ p_X(x) = \sum_{i=1}^n p_{X \vert A_i}(x)*\mathbf P(A_i) $ ^881fd7 ## Extension to Continuous Random Variables For continuous r.v’s. we say that event $B$ is when $\{X \le x\}$, which lets us translate the probability of event $\mathbf P(B)$ into a [[Cumulative Density Function]]. $ \begin{align} \mathbf P(B)&:\mathbf P(X \le x) =F_X(x) \\ \mathbf P(B\vert A_i)&:\mathbf P(X \le x \vert A_i) =F_{X\vert A_i}(x) \end{align} $ We perform the substitution and have established the TPT for CDF’s. By taking the derivative on both sides we transform CDF’s to PDF’s. $ \begin{align} F_X(x)&=F_{X\vert A_1}(x)_\mathbf P(A_1)+ \dots +F_{X\vert A_n}(x)_\mathbf P(A_n) \\ f_X(x)&=f_{X\vert A_1}(x)_\mathbf P(A_1)+ \dots +f_{X\vert A_n}(x)_\mathbf P(A_n) \\ \end{align} $