We have seen how the Taylor approximation allows us to determine an approximate value of a function of any number of variables. We are particularly interested in the evaluation of a function plus a small (but not infinitessimal) increment h. So we can evaluate the Taylor expansion of $u(x + h)$ as
$$u(x+h) = u(x) + h.u'(x) + \frac{1}{2!}h^2.u^{(2)}(x) + … \tag{1}$$
rearranging,
$$u(x+h)\; – \; u(x) = h.u'(x) + \frac{1}{2!}h^2.u^{(2)}(x) + …$$
dividing both sides by $h$
$$\dfrac{u(x+h) \; – \; u(x)}{h} = u'(x) + \frac{1}{2!}h.u^{(2)}(x) + …$$
observe that the second order term is now $h$ and not $h^2$. We ignore this term inducing an error of order $h$. We then obtain the forward difference approximation of the first partial derivative $u'(x)$.
\begin{align}u'(x) \simeq \dfrac{u(x+h) \; – \; u(x)}{h} \end{align}
Solving partial differential equations. The second order central difference.
Another thing learned in high school or in early undergraduate studies is that in order to evaluate the second derivative at point $x=a$ we can evaluate the gradient $dy/dx$ at two points on either side of $x=a$ and observe whether the gradient is increasing, decreasing or even changing sign as $x$ goes from $(x-h)$ to $(x+h)$ which lie on either side of $x=a$. We can therefore introduce the second order central difference
$$u^{(2)}(x) \simeq \dfrac{u(x+h) – 2u(x) + u(x-h) }{h^2} $$
as an approximation of the second partial derivative w.r.t $h$. This then raises the question of what the error is. At the outset, we found the Taylor exansion of $u(x+h)$. In the second derivative above, we have a new term $u(x-h)$. This we can also evaluate using its Taylor expansion. It may be easier to think of $u(x-h)$ as $u(x + [-h])$
$$u(x – h) = u(x) + [-h].u'(x) + \frac{1}{2!}[-h]^2.u^{(2)}(x) + \frac{1}{3!}[-h]^3.u^{(3)}(x) + \frac{1}{4!}[-h]^4.u^{(4)}(x)… \tag{2}$$
We can now use Taylor expansions $(1)$ and $(2)$ to obtain approximations of the second partial derivative $u^{(2)}(x)$, using the second order central difference.
Proof
By observation,
$$u^{(2)}(x) = \dfrac{1}{h^2} \biggl(- 2u(x) + \text{sum of Taylor Series (1) and (2)} \biggr) $$
Note that in Taylor series $(2)$,
$$u(x – h) = u(x)\; – h.u'(x) + \frac{1}{2!}h^2.u^{(2)}(x) \;- \frac{1}{3!}h^3.u^{(3)}(x) + \frac{1}{4!}h^4.u^{(4)}(x)… \tag{2}$$
the signs alternate between odd powered $h$ and even powered $h$ such that by adding $(1)$ and $(2)$, odd power terms of $h$ cancel out leaving only even terms
\begin{align} u^{(2)}(x) &= \dfrac{1}{h^2} \biggl(- 2u(x) + 2u(x) + \frac{2}{2!}h^2.u^{(2)}(x) + \frac{2}{4!}h^4.u^{(4)}(x)… \biggr) \\
u^{(2)}(x) &= \dfrac{2}{h^2} \biggl(\frac{1}{2!}h^2.u^{(2)}(x) + \frac{1}{4!}h^4.u^{(4)}(x)… \biggr)\\
u^{(2)}(x) &= 2\biggl(\frac{1}{2!}u^{(2)}(x) + \frac{1}{4!}h^2.u^{(4)}(x)…\biggr) \end{align}
this simply confirms that the second order central difference
$$u^{(2)}(x) \simeq \dfrac{u(x+h) – 2u(x) + u(x-h) }{h^2} $$
gives us the second derivative $u^{(2)}(x) $ plus an error $Oh^2$.