| Model Type | Equation |
| ----------------------- | ---------------------------------------- |
| Linear trend | $X_t= \beta_1+\beta_2t+W_t$ |
| Internal regressor (AR) | $X_t = \phi_1+X_{t-1}+\phi_2X_{t-2}+W_t$ |
| External regressor | $X_t = \beta_1X_{t-1}+\beta_2Y_s+W_t$ |
## Cross-Variance
If we decide to include external regressors, then we can look at the cross-covariance, i.e. $\mathrm{cov}(X_t, Y_s)$. Also cross-covariance needs to fulfill the [[Stationarity#Weak Stationarity|Weak Stationarity]] condition. It must be only a function of the gap $h$ between $s,t$ and not a function of absolute time.
$
\begin{align}
\gamma_{XY}(t,s)&=\mathrm{cov}(X_t, Y_s) \\[4pt]
\gamma_{XY}(h)&=\mathrm{cov}(X_t, Y_{t+h})
\end{align}
$
To estimate cross-covariance, we again average over all available lagged pairs along both time series.
$ \hat\gamma_{XY}(h)=\frac{1}{n}\sum_{t=1}^{n-h} (x_{t+h}- \bar x)(y_t - \bar y) $
Cross-correlation factor (CCF):
$ \rho_{XY}(h)=\frac{\gamma_{XY}(h)}{\sigma_X\sigma_Y} $
## Procedure for Time Series Model Fitting
*Step 1: Transformations to make time series stationary*
- Log-transform
- Remove trends & seasonality
- Differentiate successively
*Step 2: Check if transformed series is white-noise*
- If yes, then we are done.
- If not look at ACF and decide if remainder is [[Autoregressive Model|Autoregressive Model]] or a [[Moving Average Model|Moving Average Model]] (this is dependent on how quickly ACF drops).
*Step 3:* $\text{AR}(p)$
- Compute [[Partial Autocorrelation Function|PACF]] to get order (i.e. number of useful lags).
- Estimate $\phi_k$ coefficients and $\sigma_W^2$ of white-noise r.v. $W_i$ via [[Yule-Walker Equations]].
- Compute residuals and test them for white-noise.
*Step 4:* $\text{MA}(q)$:
- Compute ACF to get order(i.e. number of useful lags).
- Estimate $\theta_k$ coefficients and $\sigma_W^2$ of white-noise r.v. $W_i$ via [[Maximum Likelihood Estimation]].
- Compute residuals and test them for white-noise.
*Step 5:* $\text{ARMA}(p,q)$:
- Fit an AR(p) model, and compute residuals.
- Fit an MA(q) model on the residuals (or original data).
- Fit an ARMA(p, q) using the $p,q$ parameters that have been obtained in the previous steps.
- Compute residuals and test them for white-noise.