First Order Initial Value Problem

First Order Initial Value Problem#

The more general form of a first order Ordinary Differential Equation is:

(1)#

y^{^{'}} = f (t, y) .

This can be solved analytically by integrating both sides but this is not straight forward for most problems. Numerical methods can be used to approximate the solution at discrete points.

Euler method#

The simplest one step numerical method is the Euler Method named after the most prolific of mathematicians Leonhard Euler (15 April 1707 – 18 September 1783) .

The general Euler formula for the first order differential equation

(2)#

y^{^{'}} = f (t, y),

approximates the derivative at time point $t_{i}$ ,

(3)#

y^{^{'}} (t_{i}) \approx \frac{w_{i + 1} - w_{i}}{t_{i + 1} - t_{i}},

where $w_{i}$ is the approximate solution of $y$ at time $t_{i}$ .

This substitution changes the differential equation into a difference equation of the form

(4)#

\frac{w_{i + 1} - w_{i}}{t_{i + 1} - t_{i}} = f (t_{i}, w_{i}) .

Assuming uniform stepsize $t_{i + 1} - t_{i}$ is replaced by $h$ , re-arranging the equation gives

(5)#

w_{i + 1} = w_{i} + h f (t_{i}, w_{i}) .

This can be read as the future $w_{i + 1}$ can be approximated by the present $w_{i}$ and the addition of the input to the system $f (t, y)$ times the time step.

## Library
import numpy as np
import math 
import pandas as pd
%matplotlib inline
import matplotlib.pyplot as plt # side-stepping mpl backend
import matplotlib.gridspec as gridspec # subplots
import warnings

warnings.filterwarnings("ignore")

Population growth#

The general form of the population growth differential equation is:

(6)#

y^{^{'}} = ϵ y

where $ϵ$ is the growth rate. The initial population at time $a$ is

(7)#

y (a) = A,

(8)#

a \leq t \leq b .

Integrating gives the general analytic (exact) solution:

(9)#

y = A e^{ϵ x} .

We will use this equation to illustrate the application of the Euler method.

Discrete Interval#

The continuous time $a \leq t \leq b$ is discretised into $N$ intervals by a constant stepsize

(10)#

h = \frac{b - a}{N} .

Here the interval is $0 \leq t \leq 2$ is discretised into $20$ intervals with stepsize

(11)#

h = \frac{2 - 0}{20} = 0.1,

this gives the 21 discrete points:

(12)#

t_{0} = 0, t_{1} = 0.1, . . . t_{20} = 2.

This is generalised to

(13)#

t_{i} = 0 + i 0.1, i = 0, 1, . . ., 20.

The plot below shows the discrete time steps.

### Setting up time
t_end=2.0
t_start=0
N=20
h=(t_end-t_start)/(N)
time=np.arange(t_start,t_end+0.01,h)
fig = plt.figure(figsize=(10,4))
plt.plot(time,0*time,'o:',color='red')
plt.xlim((0,2))
plt.title('Illustration of discrete time points for h=%s'%(h))
plt.plot();

../_images/faf2071c155aea8715f9a0bc29fcb3816c1bb0b17f7b7eaa1d7f01bc1afd93c8.png

Initial Condition#

To get a specify solution to a first order initial value problem, an initial condition is required.

For our population problem the intial condition is:

(14)#

y (0) = 10.

This gives the analytic solution

(15)#

y = 10 e^{ϵ t} .

Growth rate#

Let the growth rate

(16)#

ϵ = 0.5

giving the analytic solution.

(17)#

y = 10 e^{0.5 t} .

The plot below shows the exact solution on the discrete time steps.

## Analytic Solution y
y=10*np.exp(0.5*time)

fig = plt.figure(figsize=(10,4))
plt.plot(time,y,'o:',color='black')
plt.xlim((0,2))
plt.xlabel('time')
plt.ylabel('y')
plt.title('Analytic (Exact) solution')
plt.plot();

../_images/01796c97c9bc021d34d69d9741c0495b72525ffd7c3e7cd7b0e997b6e410908f.png

Numerical approximation of Population growth#

The differential equation is transformed using the Euler method into a difference equation of the form

(18)#

w_{i + 1} = w_{i} + h ϵ w_{i} .

This approximates a series of of values $w_{0}, w_{1}, . . ., w_{N}$ . For the specific example of the population equation the difference equation is \begin{equation} w_{i+1}=w_{i}+h 0.5 w_i. \end{equation} where $w_{0} = 10$ . From this initial condition the series is approximated. The plot below shows the exact solution $y$ in black circles and Euler approximation $w$ in blue squares.

w=np.zeros(N+1)
w[0]=10
for i in range (0,N):
    w[i+1]=w[i]+h*(0.5)*w[i]

fig = plt.figure(figsize=(10,4))
plt.plot(time,y,'o:',color='black',label='exact')
plt.plot(time,w,'s:',color='blue',label='Euler')
plt.xlim((0,2))
plt.xlabel('time')
plt.legend(loc='best')
plt.title('Analytic and Euler solution')
plt.plot();

../_images/6bb7f99dd703396f424fa02c7fde59b4568830449ae65650a2aa0a98993fad13.png

Numerical Error#

With a numerical solution there are two types of error:

local truncation error at one time step;
global error which is the propagation of local error.

Derivation of Euler Local truncation error#

The left hand side of a initial value problem $\frac{d y}{d t}$ is approximated by Taylors theorem expand about a point $t_{0}$ giving:

(19)#

y (t_{1}) = y (t_{0}) + (t_{1} - t_{0}) y^{^{'}} (t_{0}) + \frac{(t_{1} - t_{0})^{2}}{2!} y^{^{″}} (ξ), ξ \in [t_{0}, t_{1}] .

Rearranging and letting $h = t_{1} - t_{0}$ the equation becomes

(20)#

y^{^{'}} (t_{0}) = \frac{y (t_{1}) - y (t_{0})}{h} - \frac{h}{2} y^{^{″}} (ξ) .

From this the local truncation error is

(21)#

τ \leq \frac{h}{2} M \sim O (h),

where $y^{^{″}} (t) \leq M$ .

Derivation of Euler Local truncation error for the Population Growth#

In most cases $y$ is unknown but in our example problem there is an exact solution which can be used to estimate the local truncation

(22)#

y^{'} (t) = 5 e^{0.5 t},

(23)#

y^{″} (t) = 2.5 e^{0.5 t} .

From this a maximum upper limit can be calculated for $y^{^{″}}$ on the interval $[t_{0}, t_{1}] = [0, 0.1]$

(24)#

y^{″} (0.1) = 2.5 e^{0.1 \times 0.5} = 2.63 = M,

(25)#

τ = \frac{h}{2} 2.63 = 0.1315 .

The plot below shows the exact local truncation error $| y - w |$ (red triangle) and the upper limit of the Truncation error (black v) for the first two time points $t_{0}$ and $t_{1}$ .

fig = plt.figure(figsize=(10,4))
plt.plot(time[0:2],np.abs(w[0:2]-y[0:2]),'^:'
         ,color='red',label='Error |y-w|')
plt.plot(time[0:2],0.1*2.63/2*np.ones(2),'v:'
         ,color='black',label='Upper Local Truncation')
plt.xlim((0,.15))
plt.xlabel('time')
plt.legend(loc='best')
plt.title('Local Truncation Error')
plt.plot();

../_images/d0024517b316515d65d41f20f8ba0b43db8bcf1ae8a5e54d6abf2fcc0cf1e13d.png

Global Error#

The error does not stay constant accross the time this is illustrated in the figure below for the population growth equation. The actual error (red triangles) increases over time while the local truncation error (black v) remains constant.

fig = plt.figure(figsize=(10,4))
plt.plot(time,np.abs(w-y),'^:'
         ,color='red',label='Error |y-w|')
plt.plot(time,0.1*2.63/2*np.ones(N+1),'v:'
         ,color='black',label='Upper Local Truncation')
plt.xlim((0,2))
plt.xlabel('time')
plt.legend(loc='best')
plt.title('Why Local Truncation does not extend to global')
plt.plot();

../_images/187ac9233fdfa1163b27fcb32bbd3c2c640bf7cfa079f5f9ba14edcd9b035b9e.png

Theorems#

The theorem below proves an upper limit of the global truncation error.

Euler Global Error#

Theorem Global Error

Suppose $f$ is continuous and satisfies a Lipschitz Condition with constant L on $D = {(t, y) | a \leq t \leq b, - \infty < y < \infty}$ and that a constant M exists with the property that

(26)#

| y^{^{″}} (t) | \leq M .

Let $y (t)$ denote the unique solution of the Initial Value Problem

(27)#

y^{^{'}} = f (t, y) a \leq t \leq b, y (a) = α,

and $w_{0}, w_{1}, . . ., w_{N}$ be the approx generated by the Euler method for some positive integer N. Then for $i = 0, 1, . . ., N$

(28)#

| y (t_{i}) - w_{i} | \leq \frac{M h}{2 L} | e^{L (t_{i} - a)} - 1 | .

Theorems about Ordinary Differential Equations#

Definition

A function $f (t, y)$ is said to satisfy a Lipschitz Condition in the variable $y$ on the set $D \subset R^{2}$ if a constant $L > 0$ exist with the property that

(29)#

| f (t, y_{1}) - f (t, y_{2}) | < L | y_{1} - y_{2} |

whenever $(t, y_{1}), (t, y_{2}) \in D$ . The constant L is call the Lipschitz Condition of $f$ .

Theorem Suppose $f (t, y)$ is defined on a convex set $D \subset R^{2}$ . If a constant $L > 0$ exists with

(30)#

| \frac{\partial f (t, y)}{\partial y} | \leq L,

then $f$ satisfies a Lipschitz Condition an $D$ in the variable $y$ with Lipschitz constant L.

Global truncation error for the population equation#

For the population equation specific values $L$ and $M$ can be calculated.

In this case $f (t, y) = ϵ y$ is continuous and satisfies a Lipschitz Condition with constant

(31)#

| \frac{\partial f (t, y)}{\partial y} | \leq L,

(32)#

| \frac{\partial ϵ y}{\partial y} | \leq ϵ = 0.5 = L,

on $D = {(t, y) | 0 \leq t \leq 2, 10 < y < 30}$ and that a constant $M$ exists with the property that

(33)#

| y^{^{″}} (t) | \leq M .

(34)#

| y^{^{″}} (t) | = 2.5 e^{0.5 \times 2} \leq 2.5 e = 6.8 .

Specific Theorem Global Error