Lecture 3.1:
Forward, backward and central differences for derivatives

Linear electrical circuits consist of resistors, capacitors, inductors, and voltage and current sources. Let us consider here a simple resistor-inductor (RL) one-port network driven by a current source. When a current I = I(t) is applied to the input port, the voltage V = V(t) develops across the port terminals. The voltage output V(t) can be determined as a sum of the voltage drop across the resistor (which is R I(t) from the Ohm's law) and of the voltage drop across the inductor (which is L I'(t), where I'(t) denotes the derivative of I(t)). Suppose the input current I(t) can be measured (detected) at different times t = t_k and it is generated by the following oscillatory and decaying signal (click to enlarge the image):

The output voltage V(t) = R I(t) + L I'(t) can be computed at the times t = t_k only if the derivative I'(t_k) can be estimated numerically from the given data set (t_k,I_k). This is the problem of numerical differentiation.

Let us denote the current time instance as t = t₀, when the current is I = I₀. Then, the data values in the past are: (t_-1,I_-1), (t_-2,I_-2), and so on. The data values in the future are: (t₁,I₁), (t₂,I₂), and so on. Depending on whether we use data in the future or in the past or both, the numerical derivatives can be approximated by the forward, backward and central differences. The simplest way to approximate the numerical derivatives is to look at the slope of the secant line that passes through two points (linear interpolation).

Forward difference

Consider a linear interpolation between the current data value (t₀,I₀) and the future data value (t₁,I₁). The slope of the secant line between these two points approximates the derivative by the forward (two-point) difference:

I'(t₀) = (I₁-I₀) / (t₁ - t₀)

Forward differences are useful in solving ordinary differential equations by single-step predictor-corrector methods (such as Euler and Runge-Kutta methods). For instance, the forward difference above predicts the value of I₁ from the derivative I'(t₀) and from the value I₀. If the data values are equally spaced with the step size h, the truncation error of the forward difference approximation has the order of O(h).

Backward difference

Consider a linear interpolation between the current data value (t₀,I₀) and the past data value (t_-1,I_-1). The slope of the secant line between these two points approximates the derivative by the backward (two-point) difference:

I'(t₀) = (I₀-I_-1) / (t₀ - t_-1)

Backward differences are useful for approximating the derivatives if data in the future are not yet available. Moreover, data in the future may depend on the derivatives approximated from the data in the past (such as in control problems). If the data values are equally spaced with the step size h, the truncation error of the backward difference approximation has the order of O(h) (as bad as the forward difference approximation).

Central difference

Finally, consider a linear interpolation between the past data value (t_-1,I_-1) and the future data value (t₁,I₁). The slope of the secant line between these two points approximates the derivative by the central (three-point) difference:

I'(t₀) = (I₁-I_-1) / (t₁ - t_-1)

If the data values are equally spaced, the central difference is an average of the forward and backward differences. The truncation error of the central difference approximation is order of O(h²), where h is the step size. It is clear that the central difference gives a much more accurate approximation of the derivative compared to the forward and backward differences. Central differences are useful in solving partial differential equations. If the data values are available both in the past and in the future, the numerical derivative should be approximated by the central difference.

Errors of the difference approximations for derivatives

            Presents graphs of numerical derivatives for the data values above