Suppose we have a function $f(x,y)$ and we want to approximate its value at a point $[x+\delta x, y+\delta y]$ near some reference point $[x,y]$. Since $f$ is multivariate (two input variables), we need a multivariate version of [[Taylor Series]] for the approximation.
**0th order approximation:**
The approximation assumes that the function value remains constant with the same value as it evaluates at the reference point $[x,y]$. Geometrically, this corresponds to a flat surface parallel to the $xy$-plane.
$ f(x+\delta x,y + \delta y)= f(x,y) $
**1st order approximation:**
The approximation adds the slope times the distance away from the reference point in each dimension. Geometrically the approximation is a flat but sloped surface.
$ f(x+\delta x,y + \delta y)= f(x,y)+\left(\frac{\partial f}{\partial x}*\delta x + \frac{\partial f}{\partial y}*\delta y\right) $
Using vector notation:
$ f(x+\delta x,y + \delta y)=f(x,y)+J_f*\delta \,\mathbf x$
where:
- $J_f=\begin{bmatrix} \frac{\partial f}{\partial x} & \frac{\partial f}{\partial y} \end{bmatrix}$ is the [[Jacobian#Jacobian Vector|Jacobian vector]]
- $\delta \mathbf x=\begin{bmatrix} \delta x \\ \delta y \end{bmatrix}$ is the position change vector
**2nd order approximation:**
The approximation includes the effects of curvature using the of second-order partial derivatives. Geometrically, this corresponds to a paraboloid-like surface that accounts for curvature in all directions.
$
f(x+\delta x,y + \delta y) = f(x,y)+
\left(
\frac{\partial f}{\partial x}\delta x +
\frac{\partial f}{\partial y}\delta y
\right) +
\frac{1}{2}
\left(
\frac{\partial f}{\partial xx}\delta x^2 +
\frac{\partial f}{\partial yy}\delta y^2 + 2
\frac{\partial f}{\partial xy}\delta xy
\right)
$
In matrix form this becomes:
$
f(x+\delta x,y + \delta y)
=f(x,y)+
\begin{bmatrix} \frac{\partial f}{\partial x} & \frac{\partial f}{\partial y} \end{bmatrix}
\begin{bmatrix} \delta x \\ \delta y \end{bmatrix} +\frac{1}{2}\left(
\begin{bmatrix} \frac{\partial f}{\partial x} & \frac{\partial f}{\partial y} \end{bmatrix}
\begin{bmatrix} \frac{\partial f}{\partial xx} & \frac{\partial f}{\partial xy} \\[2pt] \frac{\partial f}{\partial yx} & \frac{\partial f}{\partial yy} \end{bmatrix}
\begin{bmatrix} \frac{\partial f}{\partial x} \\[2pt]
\frac{\partial f}{\partial y} \end{bmatrix}\right)
$
Simplifying:
$ f(x+\delta x,y + \delta y) =f(x,y)+J_f*\delta \mathbf x + \frac{1}{2}\delta \mathbf x^t*\mathbf H_f*\delta \mathbf x $
where:
- $\mathbf H_f$ is the [[Hessian]] matrix of function $f$ containing all combinations of second derivatives.
- $J_f$ is the Jacobian vector of first derivatives.
- $\delta x$ is the position change vector.