Solving simple differential equations – What we learned in high school
Given what we know from simple high school calculus
$$\frac{dy}{dx} = \frac{y_2 – y_1}{x_2 – x_1}$$
We could stop and ponder, what happens if what we have to begin with is
$$\frac{dy}{dx} = f'(x)$$
where f'(x) is known, but from which we would like to find an equation for $y$? The answer depends on what our known f'(x) actually is. One simple case covered in high school was for simple problems such as
$$\frac{dy}{dx} = f'(x) = cy$$
For such a problem we can solve for y analytically. For these examples we use the try $e^x$ solution
$$y(x) = y(o) e^{cx}$$
Such solutions would fail if the original problem was
$$\frac{dy}{dx} = f'(x) = cy + {cos(\sqrt{y})}^{9.81} $$
But more on that later. For our $ f'(x) = cy$, or indeed the expression above for $u'(x)$, what we can obtain, and have obtained for $u'(x)$ is an approximation, a numerical approximation of the true solution (if one exists). This numerical approximation of the first order differential equation $u'(x)$ allows us to determine an expression for evaluating $u(x+h)$ given a prior $u(x)$.
In real life, this is often the type of problem we encounter such as when we want to know how interest rates vary with time so we can determine how much we would obtain in future from an interest paying investment. Or, the interest rate required in order to obtain a certain amount in future. This was the kind of problem John Napier had on his mind in the early 17th century when without specifically thinking of the problem as a differential equation problem he proposed the following
$$\frac{dy}{dx} = cy(x) \approx \frac{y(x_k) \; – \; y(x_{k-1})}{x_k \; – \; x_{k-1} } $$
We saw hints of this in high school. We learned that the tangent to a curve at point $x=a$ gives the gradient $dy/dx$ evaluated at point $x=a$ where $a$ is some constant value taken by $x$ (even if only momentarily). So for our differential equation $u'(x)$ we have simply suggested that for $u'(x)$
$$u'(x) \simeq \dfrac{u(x+h) \; – \; u(x)}{h} $$
In the subsequent section finite difference methods, we use the Taylor expansion to show the above approximation is true and also to show what the order of the error is.
A discrete function
If $x$ can only take on discrete values at $h$ intervals then $x = kh$ ($h \gt 0$) is a discrete framework which would naturally give discrete values of $y$. We can therefore recast $dy/dx$ in a discrete form
$$\frac{dy}{dx} \equiv y'(kh) = cy(kh) \approx \frac{y(kh) \; – \; y(k-1)h}{h} $$
as the approximate gradient at $kh$. Rearranging we can say
\begin{align} y_k &\approx y_{k-1} + hcy_{k-1} \\
&= (1+ hc)y_{k-1}
\end{align}
Note that we could also say
\begin{align} y_{k-1} &\approx y_{k-2} + hcy_{k-2} \\
&= (1+ hc)y_{k-2}
\end{align}
Such that,
\begin{align} y_k &= (1+ hc)^2 y_{k-2}
\end{align}
As each subsequent $y_k$ relies on a preceding $y_{k-1}$, we have a recurring series right back to $y_o$. Hence in order to obtain a solution, it is necessary to define an initial value $y_o = y(x_o) = 1$ or other known constant. We can then obtain a discrete solution at final $x$ value $nh = X, \Rightarrow h = X/n$
\begin{align}y_n &= (1 + hc)^n y_o \quad \text{ and substituting for } y_o, \\ \\
y(X) & \simeq (1 + hc)^n \\ \\
y(X) & \simeq \biggl(1 + \frac{Xc}{n} \biggr)^n
\end{align}
It can be shown that
$$ \biggl(1 + \frac{Xc}{n} \biggr)^n \; \longrightarrow \; e^{cX} $$
Differential Equations Requiring Numerical Methods
In the prior example, we chose a very simple problem $dy/dx = cy(x)$ for which we also have an analytical solution. We mentioned the following example for which such analytical methods would fail
$$\frac{dy}{dx} = f'(x) = cy + {cos(\sqrt{y})}^{9.81} $$
Now that we have a numerical method for solving differential equations. We could attempt to solve the above equation. We already have the form of the solution, since for $dy/dx = cy(x)$ we have
$$dy/dx = cy(x)$$
for
$$\frac{dy}{dx} = f'(x) = cy + {cos(\sqrt{y})}^{9.81} $$
we now have
$$dy/dx = cy(x) + h(cy_{k-1} + cos(\sqrt{y_{k-1}})^{9.81}$$
Euler’s Method
The approximate solutions described above have been obtained using a method more generally known as Euler’s method. In general
for
\begin{align}
\frac{dy}{dx} &= f'(x) \qquad \qquad \text{ for }\; y_o = 1, \\ \\ \dfrac{y_k \; – \; y_{k-1} }{h} &= f(x_{k-1}, y_{k-1})\\ \\ \Rightarrow y_k &= y_{k-1} + hf(x_{k-1}, y_{k-1})
\end{align}