Numerical integration

by Eugeniy E. Mikhailov and Greg Bentsen

Logistics and Agenda

Midterm 1:

Do it at home part

In class part

Integration problem statement

Suppose we are given function f(x)f(x) our goal is to find ∫abf(x)dx\int_a^b f(x) dx Not all function can be easily integrated analytically in the elementary enough form.

∫0yexp(−x2)x3+cos(ωx)dx\int_0^y \frac{\exp(-x^2)}{x^3+\cos(\omega x)} dx

So we must use numerical methods.

The Riemann sum

Recall the Riemann integral definition our goal is to find

∫abf(x)dx=limN→∞∑i=1N−1f(xi)h\int_a^b f(x) dx = \lim_{N \to \infty} \sum_{i=1}^{N-1} f(x_i) h

where NN is the number of points, h=(b−a)/(N−1)h=(b-a)/(N-1) is the distance between points.

Function under integral
Function under integral

The Rectangle (box) method

Riemann rule is almost direct recipe for the rectangle left end method

∫abf(x)dx≈∑i=1N−1f(xi)h\int_a^b f(x) dx \approx \sum_{i=1}^{N-1} f(x_i) h where h=b−aN−1h=\frac{b-a}{N-1} and xi=a+(i−1)hx_i=a+(i-1)h

We just need to remember about round off errors so hh should not be too small or equivalently NN should not be to big.

Rectangle left end method
Rectangle left end method

Trapezoidal method

∫abf(x)dx≈h×(12f1+f2+f3+⋯+fN−2+fN−1+12fN)=h∑i=1Nf(xi)wi\int_a^b f(x) dx \approx h \times (\frac{1}{2} f_1 + f_2 +f_3 + \cdots +f_{N-2} +f_{N-1} + \frac{1}{2} f_{N}) = h \sum_{i=1}^N f(x_i) w_i

where h=b−aN−1h=\frac{b-a}{N-1} and xi=a+(i−1)hx_i=a+(i-1)h

Trapezoid method
Trapezoid method

Let’s compare implementation of trapezoidal method

  • machine generated with prompt “use python and write integration by trapezoid method, it should take a, b, and Np - number of points”

see file trapezoid_ai.py


# AI generated code

def trapezoidal_ai(func, a, b, Np):
    """
    Numerical integration using the trapezoidal rule.

    Parameters:
    func: function to integrate
    a: lower limit of integration
    b: upper limit of integration
    Np: number of points (intervals = Np - 1)

    Returns:
    Approximation of the integral
    """

# Calculate step size

h = (b - a) / (Np - 1)

# Evaluate function at endpoints

integral = 0.5 * (func(a) + func(b))

# Sum the interior points

for i in range(1, Np - 1):
        x = a + i * h
        integral += func(x)

# Multiply by step size

integral *= h

    return integral
  • human programmed

see file trapezoid_human.py


# human generated code

import numpy as np

def trapezoidal_human(func, a, b, Np):
    """Numerical integration using the trapezoidal rule."""
    h = (b - a) / (Np - 1)

    x = np.linspace(a, b, Np)
    y = func(x)
    y[0] = y[0] / 2
    y[-1] = y[-1] / 2

    return np.sum(y) * h

This code also runs faster by factor of 30

Simpson method: approximation by parabolas.

Main idea take two adjacent points fk−1f_{k-1}, fkf_{k}, and fk+1f_{k+1} and approximate the f(x)f(x) by parabola on this 2h2h interval. Then the integral on this interval is approximated as h3(fk−1+4fk+fk+1)\frac{h}{3} (f_{k-1} + 4f_{k} + f_{k+1})

∫abf(x)dx≈h13×(f1+4f2+2f3+4f4+⋯+2fN−2+4fN−1+fN)=h∑i=1Nf(xi)wi\int_a^b f(x) dx \approx h \frac{1}{3}\times ( f_1 + 4 f_2 + 2 f_3 + 4 f_4 + \cdots +2 f_{N-2} +4 f_{N-1} + f_{N}) = h \sum_{i=1}^N f(x_i) w_i

where h=b−aN−1h=\frac{b-a}{N-1} and xi=a+(i−1)hx_i=a+(i-1)h

Note that N must be in special form N=2k+1, i.e. odd.

Simpson method
Simpson method

Integration error estimate

Rectangle method E=𝐎((b−a)h2f′)=𝐎((b−a)22Nf′)E={\mathbf{O}}\left( \frac{ (b -a) h}{2} f' \right) = {\mathbf{O}}\left( \frac{ (b -a)^2}{2 N} f' \right)

Trapezoidal method E=𝐎((b−a)h212f″)=𝐎((b−a)312N2f″)E={\mathbf{O}}\left( \frac{ (b -a) h^2}{12} f'' \right) = {\mathbf{O}}\left( \frac{ (b -a)^3}{12 N^2} f'' \right)

Simpson method E=𝐎((b−a)h4180f(4))=𝐎((b−a)5180N4f(4))E={\mathbf{O}}\left( \frac{ (b -a) h^4}{180} f^{(4)} \right) = {\mathbf{O}}\left( \frac{ (b -a)^5}{180 N^4} f^{(4)} \right)

Here 𝐎\mathbf{O} represents big O notation for so called “order of approximation”.

f(x)=𝐎(𝐱) f(x) = \mathbf{O(x)} means that |f(x)|≤C|x||f(x)| \le C|x|, where CC is a constant

Library method for numerical integration

use quad from scipy.integrate module

import scipy.integrate as integrate

def f(x): return x*x
int_estimate, error_estimate = integrate.quad(f, 0, 1)

In [2]: int_estimate
Out[2]: 0.33333333333333337

In [3]: error_estimate
Out[3]: 3.700743415417189e-15