## Probability Mass Function The underlying process is a [[Convergence of Series#^7e111e|sequence]] of [[Bernoulli Distribution|Bernoulli]] trials. However, here the outcome is the number of trials until the first successful trial. The [[Sample Space]] $\Omega$ is infinite. $ p_X(k)=\mathbf P(X=k)= \underbrace{(1-p)^{k-1}}_{k-1 \text{ no success}}*\underbrace p_{\text{first success}} $ The [[Probability Mass Function|PMF]] gets closer to $0$ the bigger $k$ gets. $ p_X(\infty)=\underbrace{(1-p)^\infty}_{=0}*p $ ## Conditional Geometric Distribution **Memorylessness:** The geometric [[Random Variable]] has the property of memorylessness, which states that any prior outcomes, that we put as conditions (e.g. results of last 3 trials), resets the PMF as if we start all over again with the trials. More mathematically put: $ p_{X-n \vert X > n}(k) = p_X(k) $ **Example:** Assume you have flipped a coin already two times, and both events where tails. The conditional geometric for getting first head at 5th try is the same as the unconditional 3rd trial. ## Expectation We can obtain the [[Expectation]] of a geometric in two ways. Following the usual definition, we sum over all possible $k$ to multiply them with their respective probabilities. $ \mathbb E[X]= \sum_{k=1}^\infty k*p_X(k) = \sum_{k=1}^\infty k*p*(1-p)^{k-1} $ **Via Memorylessness:** This approach entails working with an infinite sum, which is cumbersome. For a simpler solution, we make use of the memorylessness property of the geometric. $ \begin{align} \mathbb E[X] &=1+ \mathbb E[X-1] \tag{1}\\ &=1+p*\mathbb E[X-1 \vert X=1] + (1-p)* \mathbb E[X-1 \vert X>1] \tag{2}\\ &=1+p_0+(1-p)*\mathbb E[X] \tag{3}\\ \mathbb E[X]-(1-p)*\mathbb E[X]&=1 \tag{4}\\ \mathbb E[X]*(1-(1-p)) &=1 \tag{5}\\ \mathbb E[X] &=1/p \tag{6} \end{align} $ where: - (1) Split up the geometric into the first trial and the expectation of any further trials $\mathbb E[X−1]$. - (2) Split up the expectation into two conditionals (using total expectation theorem). - (3) Use memorylessness to convert $\mathbb E[X-1 \vert X>1]$ to $\mathbb E[X]$. **Via Geometric Series:** We express $(1-p)$ as $q$ and then write $q^{k-1}$ as its derivative. This gives a [[Convergence of Series#Geometric Series|Geometric Series]], which brings it into simpler form. $ \begin{align} \mathbb E[X]&=\sum_{k=0}^\infty k*p*q^{k-1} \tag{1}\\[2pt] &=p*\sum_{k=0}^\infty k*q^{k-1} \tag{2}\\[2pt] &=p*\sum_{k=0}^\infty \frac{dq}{dk} q^k \tag{3}\\[2pt] &=p*\frac{d}{dq} \sum_{k=0}^\infty q^k \tag{4}\\[2pt] &=p*\frac{d}{dq} \frac{1}{1-q} \tag{5}\\[2pt] &=p*\left(\frac{1}{1-q}\right)^2 \tag{6}\\[2pt] &= \frac{1}{p} \tag{7}\\[2pt] \end{align} $ where: - (1) Express $(1-p)$ as $q$. - (3) Recognize that $k*q^{k-1}$ can be written as derivative of $q^k$ w.r.t. $q$. - (4) The sum of derivatives can be written as the derivative of a sum. - (5) Recognize that the remaining sum forms a [[Convergence of Series#Geometric Series|Geometric Series]]. ## Variance We use a similar approach (total expectation theorem) as above to obtain the second moment, and thereby the [[Variance]]. We make use of the memorylessness property by saying: $ \mathbf P(X-n \vert X>n) = \mathbf P(X)\implies \mathbf P(X \vert X>n) = \mathbf P(X+n) $ **Second Moment Derivation Via Memorylessness:** $ \begin{align} \mathbb E[X^2] &= \overbrace{\mathbf P(X=1)*\mathbb E[X^2 \vert X=1]}^{\text{first trial success}} + \overbrace{\mathbf P(X>1)*\mathbb E[X^2 \vert X>1]}^{\text{first trial no success}} \tag{1}\\[6pt] &=p*\mathbb E[X^2 \vert X=1] + (1-p)* \mathbb E[X^2 \vert X>1] \tag{2}\\[6pt] &=p_1+(1-p)*\mathbb E[(X+1)^2] \tag{3}\\[6pt] &=p+(1-p)*(\mathbb E[X^2]+2\mathbb E[X]+1) \tag{4}\\[6pt] &= \frac{2}{p^2}- \frac{1}{p} \tag{5} \end{align} $ where: - (2) The conditional expectation $\mathbb E[X^2 \vert X=1]=1$ with certainty. - (3) We make use of above stated memorylessness property. **Second Moment Derivation Via Geometric Series:** $ \begin{align} \mathbb E[X^2]&= \sum_{k=0}^\infty k^2*p*q^{k-1} \tag{1}\\[2pt] &= \frac{p}{q}*\sum_{k=0}^\infty k^2q^k \tag{2}\\[2pt] &= \frac{p}{q}*\sum_{k=0}^\infty \left(q\frac{d}{dq}\right)^2 q^k \tag{3}\\[2pt] &= \frac{p}{q}*\left(q\frac{d}{dq}\right)^2 \sum_{k=0}^\infty q^k \tag{4}\\[2pt] &= \frac{p}{q}*\left(q\frac{d}{dq}\right)^2 \frac{1}{1-q} \tag{5}\\[2pt] \end{align} $ where: - (2) Factor out $p \over q$ - (3) Express $k^2q^k$ as the second derivative of the operator $(q \frac{d}{dq})$ of $q^k$. - (4) The sum of derivatives is the derivative of the sum. - (5) Express the sum of $q^k$ as geometric series. Now we apply the operator $q \frac{d}{dq}$ on the geometric series: $ q\frac{d}{dq}\frac{1}{1-q}= \frac{q}{(1-q)^2}$ Applying the operator $q \frac{d}{dq}$ for a second time, we need to use the [[Differentiation Rules#Quotient Rule|Quotient Rule]]. Therefore we identify the respective terms. - $u=q$ - $u^\prime=1$ - $v=(1-q)^2$ - $v^\prime = 2*(1-q)*(-1)=2q-2$ Applying the Quotient rule: $ \begin{align} q\frac{d}{dq}\frac{q}{(1-q)^2} &= q*\frac{1*(1-q)^2-(q*(2q-2))}{((1-q)^2)^2} \\[4pt] &= q*\frac{1-2q+q^2-2q^2+2q}{(1-q)^4} \\[4pt] &=q*\frac{1-q^2}{(1-q)^4} \end{align} $ Substituting back into original equation (5): $ \begin{align} \mathbb E[X^2] &=\frac{p}{q} * q \frac{1-q^2}{(1-q)^4}\\[4pt] &=p*\frac{1-(1-p)^2}{p^4}\\[4pt] &=p*\frac{1-1-2p-p^2}{p^4}\\[4pt] &= \frac{2p^2-p^3}{p^4}\\[4pt] &=\frac{2-p}{p^2} \end{align} $ **Derivation of Variance:** Now that we have derived the second moment in two different way, we can use the usual short-form for variance. $ \begin{align} \mathrm{Var}(X) &=\mathbb E[X^2]- (\mathbb E[X])^2 \\[8pt] &=\frac{2}{p^2}- \frac{1}{p}-\frac{1}{p^2} \\[8pt] &=\frac{1-p}{p^2} \end{align} $