Numerical Methods

As much as we've talked about different methods of integration, and as many types of functions as we've learned to deal with, the fact remains that there are many functions that simply won't yield to any of the methods that we've discussed. For example, 212ex2 dx\int_2^{12} e^{x^2}\ dx

isn't doable by any of the analytical methods we've got in our toolbox. There are massive tables of integrals that can be consulted for some of these (although that is becoming less and less common in the modern age), but we can also tackle this with numerical methods.

The basic idea hearkens back to what you learned when you first saw integrals: if we want to find the area under a curve, we can approximate this area by adding up the areas of a bunch of rectangles (called a Riemann sum).

Approximating area under a curve using rectangles

The Riemann sum is typically written i=1nf(xi) Δx,\sum_{i=1}^n f(x_i)\ \Delta x, where ii is the index that keeps of the rectangles, nn is the number of rectangles we draw, f(xi)f(x_i) is the height of the ii-th rectangle, and Δx\Delta x is the width of each rectangle (they all have the same width).

Of course, in Calculus I, you probably saw this and then discovered that as you increase the number of rectangles, the approximation to the area under the curve gets better and better, so as nn \to \infty, this sum changes to an integral and becomes the exact area under the curve.

Not so fast: don't go to infinity

If, however, we don't let the number of rectangles increase to infinity (but rather to some large, manageable number), we can stick with an approximation, and there's nothing we can't handle! So if we want to integrate ex2e^{x^2}, we can't do so analytically, but we can get a good approximation for the definite integral between any two limits that we like. Of course, the more rectangles we use, the better the approximation but the more tedious it is to do. Fortunately, computers are perfectly suited to performing simple, repetitive tasks, so we can get approximations that are indistinguishable from the exact values.

This only works for definite integrals, but since we'll be dealing with definite integrals whenever we see any kind of application, it is still a tremendously powerful tool.

We'll look at two variations for approximating integrals; there are others, but these will serve to illustrate the point. We'll do the same example with both of them; we'll approximate 24x2 dx\int_2^4 x^2\ dx using 4 subintervals (our approximation won't be that great, but we want to keep the number small enough that we can do it by hand). Of course, we can also find the exact answer for this one, which will be useful to see how close our approximations are.

Midpoint Rule

The Midpoint Rule is a simple variation on the Riemann sum shown above; instead of using one end of the rectangles to define their height, we can use the middle of each rectangle, as shown in this figure:

Rectangles with height defined at midpoint

Midpoint Rule

Approximate 24x2 dx\displaystyle\int_2^4 x^2\ dx using the Midpoint Rule with 4 subintervals.


Since we're using four subintervals to cover the distance from 22 to 44, the width of our rectangles will be Δx=0.5\Delta x=0.5. Thus, the first rectangle will start at 22 and go to 2.52.5 (with the first midpoint at 2.252.25), and so on.

Example of Midpoint Rule

Add up the areas of these rectangles to approximate the integral: 24x2 dxf(2.25)Δx+f(2.75)Δx+f(3.25)Δx+f(3.75)Δx=(2.25)2(0.5)+(2.75)2(0.5)+(3.25)2(0.5)+(3.75)2(0.5)=18.625\begin{aligned} \int_2^4 x^2 \ dx &\approx f(2.25) \Delta x + f(2.75) \Delta x + f(3.25) \Delta x + f(3.75) \Delta x\ &= (2.25)^2 (0.5) + (2.75)^2 (0.5) + (3.25)^2 (0.5) + (3.75)^2 (0.5)\ &= 18.625 \end{aligned}

Therefore, 24x2 dx18.625\int_2^4 x^2 \ dx \approx \ans{18.625} according to the Midpoint Rule.

How good is this approximation?

Of course, with the example we picked, we know what the exact answer is, which is valuable because it gives us a chance to see how good this approximation is. The exact value is 24x2 dx=13x324=64383=56318.667\int_2^4 x^2 \ dx = \dfrac{1}{3}x^3 \bigg|_2^4 = \dfrac{64}{3} - \dfrac{8}{3} = \dfrac{56}{3} \approx 18.667

First of all, then, we can talk about the absolute error, which is how far our approximation was from the truth. This is the difference between the two numbers, and we take the absolute value of that difference so that the error is a positive number (which matches our intuition about it being a distance): Absolute Error= Exact AnswerApproximate Answer =56318.625=0.042\begin{aligned} \textrm{Absolute Error} &= \bigg|\ \textrm{Exact Answer} - \textrm{Approximate Answer}\ \bigg|\ &= \left|\dfrac{56}{3} - 18.625\right|\ &= 0.042 \end{aligned}

A better measure, though, is the relative error, which puts that number into context by converting it into a percentage. To do this, divide the absolute error by the exact answer and translate that answer into a percentage: Relative Error= Exact AnswerApproximate AnswerExact Answer =56/318.62556/3=0.00223=0.223%\begin{aligned} \textrm{Relative Error} &= \left|\ \dfrac{\textrm{Exact Answer} - \textrm{Approximate Answer}}{\textrm{Exact Answer}}\ \right|\ &= \left|\dfrac{56/3 - 18.625}{56/3}\right|\ &= 0.00223 = \ans{0.223%} \end{aligned}

It turns out that this Midpoint approximation is actually very good, off by less than a quarter of one percent.

Trapezoidal Rule

With the Trapezoidal Rule, as the name suggests, we approximate the area under the curve by adding up the areas of trapezoids instead of rectangles, as shown here.

Trapezoids to approximate area

All we need, then, is the formula for the area of a trapezoid. It can be found by chopping the trapezoid into a rectangle and a triangle, but the easiest way to remember it is simply as the area of a rectangle whose height is the average height of the trapezoid. The average height of the trapezoid is h1+h22\dfrac{h_1 + h_2}{2} where h1h_1 and h2h_2 are the distances from the flat base of the trapezoid to each of its peaks. Therefore, the area of a trapezoid with base bb is given by 12(h1+h2)b.\dfrac{1}{2}(h_1 + h_2)b.

Trapezoidal Rule

Approximate 24x2 dx\displaystyle\int_2^4 x^2\ dx using the Trapezoidal Rule with 4 subintervals.


The width of our trapezoids will still be Δx=0.5\Delta x=0.5. Thus, the first trapezoid will start at 22 and go to 2.52.5, and so on. alt text

Add up the areas of these trapezoids: 24x2 dx=12(f(2)+f(2.5))Δx+12(f(2.5)+f(3))Δx+12(f(3)+f(3.5))Δx+12(f(3.5)+f(4))Δx=12(22+2.52)(0.5)+12(2.52+32)(0.5)+12(32+3.52)(0.5)+12(3,52+42)(0.5)=18.75\begin{aligned} \int_2^4 x^2 \ dx &= \dfrac{1}{2}(f(2)+f(2.5))\Delta x + \dfrac{1}{2}(f(2.5)+f(3))\Delta x + \dfrac{1}{2}(f(3)+f(3.5))\Delta x + \dfrac{1}{2}(f(3.5)+f(4))\Delta x\ &= \dfrac{1}{2}(2^2+2.5^2)(0.5) + \dfrac{1}{2}(2.5^2+3^2)(0.5) + \dfrac{1}{2}(3^2+3.5^2)(0.5) + \dfrac{1}{2}(3,5^2+4^2)(0.5)\ &= 18.75 \end{aligned}

Therefore, 24x2 dx18.75\int_2^4 x^2 \ dx \approx \ans{18.75} according to the Trapezoidal Rule.

How good is this approximation?

Here again we can calculate the absolute and relative errors: Absolute Error= Exact AnswerApproximate Answer =56318.75=0.0833Relative Error= Exact AnswerApproximate AnswerExact Answer =56/318.7556/3=0.00446=0.446%\begin{aligned} \textrm{Absolute Error} &= \bigg|\ \textrm{Exact Answer} - \textrm{Approximate Answer}\ \bigg|\ &= \left|\dfrac{56}{3} - 18.75\right|\ &= 0.0833\ \textrm{Relative Error} &= \left|\ \dfrac{\textrm{Exact Answer} - \textrm{Approximate Answer}}{\textrm{Exact Answer}}\ \right|\ &= \left|\dfrac{56/3 - 18.75}{56/3}\right|\ &= 0.00446 = \ans{0.446%} \end{aligned}

Notice that, although in the picture the approximate area is nearly indistinguishable from the exact area (and the Trapezoidal approximation is still very good), the Midpoint Rule actually ended up getting a better approximation in this case. Without going into detail, it turns out that this is not an unusual result: for simple, smooth functions like x2x^2, the pieces above and below the curve that represent errors in the Midpoint Rule tend to cancel each other out, so the Midpoint Rule outperforms the Trapezoidal Rule.

With more complicated functions, though, the Trapezoidal Rule oftens outperforms the Midpoint Rule, and of course there are other numerical methods that can do better than either of these two, but usually at the sacrifice of simplicity and thus performance.