## General Link Function
Dependent on what the response variable looks like an appropriate link function has to be chosen, to map the [[Sample Space]] of the response variable into the real line $\mathbb R$.
| Sample Space | Link Function Name | Link Function |
| -------------------------------- | ------------------ | -------------------------------- |
| $\mathbb R \mapsto \mathbb R$ | Identity | $g(\mu)=\mu$ |
| $(0, +\infty) \mapsto \mathbb R$ | Log-link | $g(\mu) = \log(\mu)$ |
| $[0,1] \mapsto \mathbb R$ | Logit | $g(\mu)=\log(\frac{\mu}{1-\mu})$ |
| $[0,1] \mapsto \mathbb R$ | Probit | $g(\mu) = \Phi^{-1}(\mu)$ |
If it is only about mapping from the sample space of $Y$ to the real line $\mathbb R$, there is an arbitrary number of other functions that could be chosen instead of the examples listed above.
## Derivation of Canonical Link Function
It is a special case of the link function. While the generic link function maps the sample space from the response variable to $\mathbb R$, the canonical link function links the [[Conditional Expectation]] $\mu$ to the [[Canonical Exponential Family|canonical parameter]] $\theta$ of the distribution.
$
\begin{aligned}
\text{General Link}&: g(y) \mapsto \mathbb R \\ \text{Canonical Link}&: \, g(\mu) \mapsto \theta
\end{aligned}
$
It does that by equating the canonical parameter $\theta$ to the transformed conditional expectation $\mu= \mathbb E[Y\vert X]$, where $g$ is the transformation function (i.e. canonical link). This makes the canonical link function fit, to the respective type of [[Canonical Exponential Family]].
$
\begin{align}
g(\mu)&=\theta \tag{1}\\[6pt]
g\big(b^\prime(\theta)\big)&= \theta \tag{2}\\[4pt]
\underbrace{g(\mu)}_{\theta} &\equiv \underbrace{(b^\prime)^{-1}(\mu)}_{\theta} \tag{3} \end{align}
$
where:
- (2) We utilize the [[Canonical Exponential Family#Calculate Moments|fact]], that for canonical exponential distributions, the first derivative $b^\prime(\theta)=\mu$.
- (3) Since $b^\prime$ maps $\theta \mapsto \mu$, then the inverse $(b^\prime)^{-1}$ maps $\mu \mapsto \theta$.
**Example:**
When the response is [[Bernoulli Distribution|Bernoulli]], we know that it is from the canonical exponential family. We have derived its log-partition function $b(\theta)$. We need to start by taking its first derivative and then its inverse to get to the canonical link function $g$.
$
\begin{align}
b(\theta)&=\log(1+e^\theta)\\[4pt]
b^\prime(\theta) &= \frac{1}{1+e^\theta}*e^\theta=\mu\\[8pt]
e^\theta&=\mu+\mu e^\theta\\[6pt]
e^\theta - \mu e^\theta&=\mu\\[6pt] e^\theta(1-\mu)&=\mu\\[6pt]
e^\theta &= \frac{\mu}{1-\mu}\\[6pt]
\theta &= \log\Big(\frac{\mu}{1-\mu}\Big) \equiv (b^\prime)^{-1}(\mu) \equiv g
\end{align}
$
>[!note:]
>For the Bernoulli case, the canonical link function $g$ is the logit function.
## Examples of Canonical Link Functions
| Distribution | $b(\theta)$ | $g(\mu)$ |
| ------------ | ----------------------- | ------------------------- |
| Normal | $\theta^2 \over 2$ | $\mu$ |
| Poisson | $\exp(\theta)$ | $\log(\mu)$ |
| Bernoulli | $\log(1+ \exp(\theta))$ | $\log(\frac{\mu}{1-\mu})$ |
| Gamma | $-\log(-\theta))$ | $- 1 \over \mu$ |